Error Analysis & Debugging Quality

A score tells you how often the system fails. It does not tell you why. Error analysis is the practice of reading failures closely, grouping them and finding root causes. It is where most real improvement comes from. This lesson shows a method that works for any AI system.

Read before you measure more

Teams often respond to a low score by adding more metrics. A better response is to read 50 failures. Patterns appear quickly. The same cause lies behind many failures. Fixing one cause may lift the score more than a week of tuning.

A simple process

1. SAMPLE      Pull 50 failed cases at random (not only the worst ones).
2. DESCRIBE    For each, write one plain sentence: what went wrong?
3. NAME        Give similar problems the same short label, like "old data" or "ignored limit".
4. COUNT       Tally each label. Sort from most to least common.
5. ROOT CAUSE  For the top two labels, ask why five times until you reach a fixable cause.
6. FIX         Make one targeted change per cause.
7. VERIFY      Re-run. Did that label shrink? Did a new one appear?
8. RECORD      Add the cases to the test set and note the fix in the change log.

Common failure families

• Missing or wrong information: search did not find the fact, or the source was out of date.
• Ignored instructions: the prompt told the model a rule, and it did not follow it.
• Made-up facts: the answer states things no source supports.
• Wrong format: the output cannot be parsed or breaks a length rule.
• Wrong tool or wrong input: the agent called the right tool with a bad argument, or the wrong tool.
• Over- or under-refusal: it refuses safe requests, or answers risky ones.
• Context problems: the key fact was buried in a long input, or got cut off.
• Hostile input: a user or a document tried to override the rules.

Ask why, five times

A shallow fix hides the real cause. Keep asking why. The answer was wrong. Why? The model used an old price. Why? Search returned the old page. Why? Both pages exist in the index. Why? Nobody removed the retired page. The fix is to remove it and set review dates, not to tell the model to prefer new prices.

Slice the results

An overall score hides groups that do badly. Slice your results by type of question, customer group, language, length and difficulty. A system that is right nine times out of ten overall may be right only five times out of ten for one important group.

Fix the cheapest cause first

• Data and sources: update, remove or restructure documents.
• Prompt: clearer rules, better examples, an explicit unsure path.
• Code: add checks, limits or validation outside the model.
• Search: change chunking, filters or ranking.
• Model: switch or upgrade only if the above fail.

Write it down

Keep a short log: date, change, reason, score before and after, and anything that got worse. This log becomes the team's memory. It also protects you when someone asks why a decision was made.