Debugging failures
From alert to root cause: the AI failure taxonomy, the trace-driven debugging order, and turning each fix into a permanent eval case.
Something's wrong — a metric moved, a customer complained, the sampled score dropped. Debugging AI failures is different from debugging code: there's no exception, no line number, just a wrong-but-fluent output. The trace is your line number, and a systematic method beats staring.
The debugging order (cheapest cause first)
- 1Pull the trace. Read what the model actually saw — the full assembled prompt, not your template. Most 'the model is dumb' bugs are 'the model was sent garbage': a truncated context, an empty retrieval, a mangled tool result.
- 2Localize the span. In a multi-step system, which span went wrong — retrieval (wrong chunks), assembly (context malformed), the model call (bad output from good input), or a tool (bad result)? You learned this order in RAG and Agentic AI; the trace makes it a lookup, not a guess.
- 3Classify the failure (taxonomy below) — the type points at the fix.
- 4Reproduce it in your eval harness by adding the failing input as a case. If you can't reproduce it, you can't confirm a fix — and non-determinism means 'ran it once, seems fine' is not a fix.
The failure taxonomy → the fix
- Bad input to the model (truncation, empty retrieval, malformed context) → fix the pipeline stage, not the prompt. The most common and most misdiagnosed.
- Bad output from good input (hallucination, wrong format, missed instruction) → prompt, examples, or a validator.
- Retrieval failure (RAG) → chunking/embedding/ranking — upstream of the prompt.
- Tool/agent failure → tool description, error handling, or a guardrail.
- Drift (was fine, now isn't, code unchanged) → model version or data change; check the trace's model-version field against your last-known-good.
- Environment (timeout, rate limit, transient API error) → the traditional-ops layer; retries and fallbacks.
The payoff of classification is the same as it was for agents: each type has a home for its fix. 'The AI is wrong' is not actionable; 'retrieval returned empty for this query phrasing' is a ticket with an owner.
The last step of fixing any production failure is adding it to the eval set with its correct output. This is the flywheel made concrete: the bug you fixed today is a regression test forever. A fix that doesn't become a test case will be un-fixed by a future change, silently — and you'll debug it twice.