Free sample lesson · Agentic Testing: Where AI Agents Fit in Your Testing Stack
Tests enforce journeys, agents verify goals
Lesson 1 of the Wondering course Agentic Testing: Where AI Agents Fit in Your Testing Stack
Two ways to test a journey
A traditional end-to-end test records a fixed sequence of clicks. Run it again and it repeats the exact same path.
An agentic test receives a goal instead. It decides how to reach that goal at runtime.
One enforces a journey. The other verifies an outcome.
- Deterministic E2E
- Agentic test
The 80% finding
Across agent runs, about 80% take a different action path than the previous run.
Yet those runs reach identical outcomes.
So roughly 20% repeat the same path. Path repetition is the exception, not the rule.
- Different path: Runs that take a different action path but reach the same outcome.
- Same path: Runs that repeat the previous action path.
Why paths drift apart
The agent is adaptive. It reads the interface it finds, not the one recorded last week.
A moved button, a new banner, or a different load order can all send it down another route.
The goal stays fixed while the route floats.
- Goal given: The agent receives an outcome to reach.
- Read interface: The agent inspects the interface it actually finds.
- Same as before?
- Repeat path
- Choose new route: A moved button or new banner sends it another way.
- Outcome verified
- re-inspect
A common misconception
Many teams read a changed path as a failure. It is usually not.
If the outcome matches, the run passed. The different route was the agent adapting, exactly as designed.
Asserting on clicks turns adaptation into noise.
Designing for outcomes
Write the assertion where the value lives.
Ask whether the goal state exists at the end, not whether step four happened at 1.2 seconds.
Then a moved button breaks nothing, and a wrong outcome still fails loudly.
- State goal: Describe the outcome the agent must reach, not the clicks.
- Drop steps: Remove assertions tied to a fixed click sequence.
- Assert end state: Check that the goal state exists when the run finishes.
- Allow drift: Treat a different action path as expected adaptation.
When fixed paths still win
Goal-directed testing is not always the right tool.
If the journey is the product requirement, a fixed sequence is exactly what you want to enforce.
Choose by asking what would count as a real defect.
- Deterministic E2E
- Agentic test
What to carry forward
Deterministic tests pin the route. Agentic tests verify the destination.
With roughly 80% of runs taking a new path, outcome assertions are what stay stable.
Match the assertion to the thing you actually promised users.
- Runs taking a different action path: They still reach identical outcomes.
- Pin the route
- Fixed click sequences: Every run repeats the recorded steps.
- Journey is the requirement: Deviation counts as a real defect.
- Verify the destination
- Goal-directed and adaptive: The agent chooses its own route at runtime.
- Assert the end state: A moved button breaks nothing when the outcome holds.
Match the assertion to what you promised users.
Check yourself
An agentic test chooses how to reach its goal during execution rather than replaying one fixed click sequence.
- True
- False
Answer: True
This is correct. An agentic test receives a goal and adapts its route at runtime. Replaying a fixed click sequence describes a traditional end-to-end test.
A deterministic test pins the _____, while an agentic test verifies the _____.
Answer: A deterministic test pins the route, while an agentic test verifies the destination.
A deterministic test enforces a particular route. An agentic test allows the route to vary and checks whether the intended destination or outcome is reached.
Across agent runs, what pattern is most consistent with the reported 80% finding?
- About 80% repeat the same path and reach different outcomes
- About 80% take a different path and reach the same outcome
- About 20% take a different path and reach the same outcome
- All runs take different paths and reach different outcomes
Answer: About 80% take a different path and reach the same outcome
About 80% of runs take a different action path from the previous run, yet reach identical outcomes. Therefore, repeating the same path is the exception rather than the norm.
Keep going
This is lesson 1 of 8 in Agentic Testing: Where AI Agents Fit in Your Testing Stack. Next up:
- The fourth layer of the testing pyramid
- MCP vs. CLI: why the execution environment matters more than the model
- Choosing your agentic testing infrastructure
- Why context transmission — not reasoning — drives cost
- The cheaper model paradox
Wondering turns any topic into short lessons you can read, watch, or listen to, with practice questions and an AI tutor.