Debugging Agent Failures: Trace the Steps, Not the Output

Debugging agent failures means tracing every step and tool call, not re-reading the final output and guessing where the plan diverged.

Debugging agent failures means tracing every step, tool call, and handoff in a run to find exactly where the plan diverged from reality. It does not mean re-reading the final answer and guessing. Most multi-step failures trace back to orchestration design or a missing verification step, not the underlying model, which is why the fix is usually instrumentation before it is a better prompt.

I have sat in reviews where an agent's output looked fine and was wrong anyway. The team's first instinct is always the same: reread the transcript, spot the wrong sentence, rewrite the prompt around it. That fixes the symptom you noticed. It does nothing for the failure mode that produced it, because the transcript is the last five percent of a run nobody watched.

Key takeaways

The debugging unit is the trace, not the transcript. If you cannot see every tool call, retry, and handoff in a run, you are debugging a guess about what happened.
Most multi-step failures trace back to design or verification gaps, not the model. A taxonomy of 1,600+ multi-agent traces found named, reproducible failure categories (kappa = 0.88 inter-annotator agreement), not random bad luck.
The same prompt can pass on Monday and fail on Tuesday. Agent non-determinism means a single passing run proves almost nothing about the next one.
Anthropic's own team could not debug their agents until they built full tracing. "Not finding obvious information" was a symptom with at least three different causes, and only a trace could tell them which one fired.
Retries and checkpoints can mask a root cause as often as they fix one. An agent that retries a flawed plan three times looks resilient in the logs and still ships the wrong answer.

This is the debugging layer that sits under my broader argument in multi-agent systems and when they earn their cost: the architecture decides how many places a run can break, and tracing is how you find out which one did.

What "debugging agent failures" means in a multi-step system

In a single function, a bug has one location: it threw, or returned the wrong value, and a stack trace points at the line. A multi-step agent run has no single location. It has a sequence: a plan, a tool call, a result, a decision to call another tool or hand off, repeated until the run stops.

A failure can enter at any step and stay invisible until a downstream agent trusts a wrong intermediate result and builds on it. Debugging agent failures means reconstructing that sequence until you find the exact point where the plan diverged from reality, then asking whether that was a model error, a tool error, or a design error in how the steps were supposed to fit together.

The three places multi-step failures come from

Every multi-step failure I have traced lands in one of three buckets, and they are not equally common. A taxonomy built from more than 1,600 annotated multi-agent traces across seven frameworks sorts them the same way: system design, inter-agent coordination, and task verification (Cemri et al., 2025).

Design failures. The task was decomposed wrong, a step's boundaries were vague, or the plan assumed a tool would behave in a way it does not.
Coordination failures. One agent or step handed off a result that the next step misread, ignored, or acted on before it was ready.
Verification failures. Nobody, human or automated, checked whether the combined output was correct before it shipped.

Verification is the bucket teams skip most often, and it decides whether the other two ever get caught. A design flaw or a coordination bug produces a wrong intermediate result either way; verification is what stops that from becoming a wrong final answer a customer acts on.

Instrument before you debug: tracing every step, not just the final answer

You cannot debug what you cannot see, and most teams cannot see their agents. Anthropic's own multi-agent research team ran into this directly: "users would report agents 'not finding obvious information,' but we couldn't see why. Were the agents using bad search queries? Choosing poor sources? Hitting tool failures?" (Anthropic engineering, 2025). That is not a skill gap. It is a team without a trace, staring at one symptom with three possible causes and no way to tell which fired.

"Not finding obvious information" is a symptom with three different causes. Only a trace tells you which one fired. A better prompt fixes one of the three, at most, and you will not know which.

Their fix was full production tracing that captures decision patterns and interaction structure, not conversation content, paired with retry logic and checkpoints, because small errors compound fast across steps. The instrumentation came before the fix. You cannot fix what you cannot see.

This is no longer ad hoc logging territory. OpenTelemetry's GenAI semantic conventions define standard span types for agent execution: invoke_agent, execute_tool, create_agent, and invoke_workflow, with attributes like gen_ai.agent.name, per-span token usage, and a conversation ID that threads a run together (OpenTelemetry GenAI spans, 2025). You do not need the full spec to get the value. Every step needs a name, an input, an output, and a status, logged the same way every time, so you can query a hundred failed runs instead of rereading one transcript.

If your team is still deciding what "good enough" tracing looks like before scaling an agent past a demo, that is worth a second set of eyes. Hire an AI developer at ViitorCloud to instrument the trace layer before the failure modes reach production, not after.

Reading a trace: where the failure hides in a multi-agent run

A trace turns a mystery into a search problem. Once you have every step logged, debugging becomes reading the sequence backward from the wrong output until you find the first step where the trace and reality disagree.

Here is an illustrative example, not a named client: a lead-enrichment pipeline where a planner agent lists target companies, a tool-calling agent enriches each with firmographic data, and a scoring agent ranks the list. The final ranking looked plausible. About one in five companies was scored on stale data, and nobody noticed until a sales rep called a company that had been acquired eight months earlier.

# trace excerpt, illustrative, spans simplified from OTel GenAI shape

invoke_agent name=planner status=ok output=42 companies

execute_tool name=enrich_firmographics target=acme_co

cache_hit=true cache_age_days=243 fresh_check=none

invoke_agent name=scorer input=acme_co(stale) status=ok

score=88 confidence=high

The failure was not in the model's reasoning. The enrichment tool had a cache with no freshness check, the tool call reported success, and every downstream agent treated a 243-day-old record as current. Reading the trace top to bottom took four minutes. Reading the final ranking and guessing had already cost a wasted sales call.

That is the pattern in most traces I read: the divergence point is a step that reported success and was still wrong. Status codes and exceptions catch a fraction of agent failures. The rest look exactly like success until someone checks the substance of what came back.

Common multi-step failure modes and how to catch each one

A handful of patterns account for most of what shows up once you start reading traces at scale. Each one leaves a different signature, and each needs a different check, not a better prompt.

Failure mode	Signature in the trace	What catches it
Stale or cached data treated as fresh	Tool call reports success; no timestamp or freshness field checked downstream	Require a freshness assertion on every cached read before it feeds a decision
Silent tool failure	Tool returns a default, empty, or partial value; agent proceeds without noticing	Schema-validate every tool return; treat an empty or default value as a failed status
Ignored or dropped context on handoff	Receiving step's reasoning never references information the sender included	Diff what was sent against what the receiving step's output used
Over-decomposition	More subagents spawned than the query needed; token cost triples with no quality gain	Cap subagent count and compare against a single-agent baseline on the same task
No verification step	Final output ships straight from the last agent with no check against ground truth	Add a scoring or schema-check step before the result reaches a user or a downstream system

Notice what these have in common: not one is a bad prompt. Each is a gap in what the system checks, not in what the model was told to do. That is the honest reason "improve the prompt" so rarely fixes a multi-step failure. The prompt was never the part that broke.

Why the same prompt fails differently on retry

Run the identical prompt against the identical agent twice and you can get two different traces. Model sampling introduces variance in which tool gets called first, which source gets read, and how a borderline decision resolves. In a single-step system that shows up as a slightly different sentence. In a multi-step system it compounds, because step three's input is step two's output, and step two came out different this time.

Take a research agent answering the same question twice in one afternoon. The first run's search returns a well-ranked source, extracts a clean answer, and finishes in three steps. The second run's search returns a slightly different order, the agent picks a lower-quality source, and answers with the same confident tone. Nothing crashed. Nothing logged an error. The second answer is wrong, and looks exactly as finished as the first.

This is the part of the job that is unfamiliar coming from deterministic software: a passing run today is evidence, not proof, that the same prompt passes tomorrow. One green trace tells you the system can succeed. It does not tell you how often it will.

Building an eval harness that predicts these failures before production

A trace tells you what happened in one run. An eval harness tells you how often it happens across many, which is the number that matters before you ship. Three pieces make a harness that catches multi-step failures, not just single-turn ones.

Run the same task many times, not once. A single pass proves the system can succeed, not that it reliably will. Run each scenario enough times to see the failure rate, not just a pass or fail.
Score intermediate steps, not just the final answer. A correct final answer can hide a broken step that got lucky. Check the enrichment step's freshness, the handoff's completeness, and the tool call's schema, independent of whether the last step landed right.
Include failure modes you have already seen in production. Every stale-cache, dropped-context, or silent-failure case you find becomes a regression test. The harness gets stricter with each new failure mode, not just wider.

This is where an eval suite pays for itself against the alternative: finding these failure modes in front of a customer. The trade-off is real. Building this harness costs engineering time you could spend shipping features. Skip it, and that time gets spent later, debugging an incident with less information than a trace would have given you.

What to do when small errors compound across steps

Anthropic names the pattern directly: small errors compound fast across steps, which is why their fix pairs tracing with retry logic and checkpoints. That advice is correct and incomplete on its own, and the trade-off is worth naming plainly.

Retries and checkpoints can mask a root cause instead of fixing one. An agent that retries the same flawed plan three times, with slightly different tool calls each time, looks resilient in the logs. It still ships the wrong answer, on the fourth attempt instead of the first. Checkpointing without re-verification does the same thing differently: it gives a bad intermediate state a safe place to persist, and the next step inherits it with confidence intact.

Tracing tells you where a run broke. It does not tell you whether the recovered output is correct. Retries and checkpoints buy resilience, not correctness, and the two get confused constantly.

A concrete version: a document-processing pipeline parses a date field as month-day instead of day-month at step one. Every downstream step, due-date calculation, reminder scheduling, invoice formatting, inherits the flipped date and produces internally consistent output. Each step's local check passes, because each step correctly transforms the input it received. Nothing errors, so nothing retries. The compounding error ships clean and surfaces three weeks later, when an invoice comes due on the wrong day.

The fix is not more retries. It is a verification step early enough to catch the divergence before four downstream steps build on top of it, plus a harness that includes exactly this case going forward.

Frequently asked questions

Why does my AI agent fail on some runs and not others with the same prompt?

Model sampling introduces run-to-run variance in tool choice, source selection, and borderline decisions. In a multi-step system, an early difference compounds across steps instead of staying contained to one sentence. Treat a single passing run as evidence the system can succeed, not proof it reliably will.

How do I trace a multi-step agent to find exactly where it broke?

Log every step, tool call, and handoff with a consistent structure: name, input, output, status, threaded by a single run ID. Read the trace backward from the wrong output until you find the first step where trace and reality disagree. Look especially at steps that reported success, since most agent failures hide behind a status that looks fine.

What's the difference between an agent orchestration bug and a model reasoning bug?

An orchestration bug is a gap in how steps are decomposed, handed off, or verified, and shows up even when every model call reasoned correctly. A reasoning bug is the model getting a specific judgment wrong given correct inputs. Most traced multi-step failures turn out to be the first kind. I go deeper on the coordination side of this in agent orchestration patterns and agent-to-agent communication.

Do I need OpenTelemetry to debug AI agents, or is logging every step enough?

Logging every step with a consistent structure gets you most of the value. OpenTelemetry's GenAI conventions matter once you need to query across many runs, compare traces across tools, or hand instrumentation to a team that didn't build the agent. Start with structured logs; adopt the standard schema when ad hoc logging stops scaling.

Multi-step agents fail in the gap between what a system did and what a transcript shows. My book Observability for AI Systems goes deeper on the tracing, prompt logging, and replay discipline that closes that gap, before a customer finds the failure for you. If you are past the prototype and need that instrumentation built into a production system, ViitorCloud's AI development team builds the trace layer and the eval harness together, not as an afterthought once something has already broken.