Introduction
The score tells you whether to ship. The failed cases tell you what to fix, and they are the only part of an experiment that changes what you build. Reading them well is a skill, and the first thing it requires is doubting the metric.
Ask whether the failure is real
Every failed case is one of three things, and they need completely different responses:
Kind | Meaning | Fix |
Real failure | The agent got it wrong | The agent |
Bad metric | The output was fine; the metric disagreed | The metric |
Bad test case | The expected output was wrong, ambiguous, or stale | The dataset |
Note: Assume the metric is wrong until you have read the output yourself. An AI-graded metric is a model forming a judgement, and it can be as wrong as the agent it is grading. Teams that skip this step spend weeks fixing agents that were behaving correctly, and end up trusting their evaluation less than when they started.
Working through the failures
Read the metric's reasoning first — Scored results carry the reasoning behind the verdict, not just a number. It is the fastest route to understanding what the metric objected to.
Read the actual output against the expected one — With your own judgement, before accepting the verdict.
Classify it into one of the three kinds above — This is the step that determines everything that follows.
Group the real failures by cause — Twenty failures are rarely twenty problems. Usually two or three causes, repeated.
Fix causes, not cases — Tuning the agent until a specific case passes is overfitting to your test set — the score rises and nothing improves.
Patterns worth recognising
Pattern | Usually means |
All failures share an input type | A gap in the agent's instructions for that case |
All failures are on one metric | Either a systematic weakness, or a badly calibrated metric |
Failures are scattered with no pattern | Often variance rather than a defect. Re-run before investigating |
The output is right but formatted differently | The metric is checking form when you meant substance |
The expected output is itself wrong | A dataset problem. Common in sets generated rather than curated |
"Fixing the agent until the test passes is not improvement. It is memorising the answer key."
Failures improve the dataset too
Reviewing failures is also how a dataset earns its keep over time. Two moves worth making as you go:
Correct expected outputs that turned out to be wrong. A stale expectation fails every future run and trains everyone to ignore that case.
Add the case you just found in production. A dataset can be built from live sessions, so a real failure becomes a permanent regression test — see Import & Manage Test Cases.
Where a case genuinely needs human judgement recorded against it rather than a metric's verdict, the Evaluation module carries an annotation surface for exactly that. It sits outside what these pages cover today.