Model Judges & Human Review
When to use people or model judges, how judges go wrong and how to calibrate them.
Some qualities need judgment. People are the gold standard but slow. Model judges are fast but biased by length, position and self-preference. Use clear rubrics and both orders, pin the judge version, and always calibrate a judge against human scores.
- Model judge
- A language model that scores answers against your rubric.
- Agreement
- How often two reviewers give the same score.
- Calibration
- Checking a judge's scores against people's scores.
Some qualities cannot be checked by code. Is this summary faithful? Is this answer helpful? Does this reply sound kind? For these you need judgment. Judgment comes from people, or from a language model asked to act as a judge. This lesson explains when to use each, how model judges go wrong, and how to check them against people.
Human review
People are the gold standard for quality, but they are slow and costly. Use them where the stakes are high, where the rubric is new, and to check your automatic methods.
- Use domain experts for specialist content, such as legal or medical text.
- Give reviewers the rubric, solved examples and a quiet, clear interface.
- Hide which version produced which output, so reviewers are not biased.
- Use two reviewers on a sample, and measure how often they agree.
Measuring agreement
If two reviewers often disagree, the rubric is unclear or the task is subjective. Measure it. Simple percent agreement is a start. A measure such as Cohen's kappa also corrects for agreement by chance. You do not need the formula to use the idea: higher agreement means the rubric is clearer.
Model judges
A model judge is a language model that reads an answer and scores it against your rubric. It is fast and cheap, and it can score thousands of cases. It is not perfect. Treat it as a tool that must be checked.
You are grading an answer from a customer support assistant.
QUESTION: {{question}}
SOURCE ARTICLES: {{articles}}
ANSWER TO GRADE: {{answer}}
Grade each item. Use only the source articles to decide what is true.
1. CORRECT (0, 1 or 2): ...rubric text...
2. GROUNDED (0, 1 or 2): ...rubric text...
3. MUST-NEVER-HAPPEN: yes or no, with the exact words that break the rule.
Return JSON: {"correct": n, "grounded": n, "violation": "yes|no", "reason": "one sentence"}
Give the reason first in your thinking, then the scores.How model judges go wrong
- Length bias: longer answers often get higher scores, even when they are no better.
- Position bias: in a side-by-side comparison, the first answer may be favored.
- Self-preference: a model may favor text written by itself or a similar model.
- Vague rubrics: if you do not define a score, the judge will invent its own meaning.
- Drift: the judge model can change when a provider updates it.
Make judges reliable
- Write a clear rubric with examples inside the prompt.
- Ask for a short reason and a fixed score scale.
- For comparisons, run both orders and keep only consistent results.
- Use a low-randomness setting so scores are steady.
- Pin the judge model version, and re-check when it changes.
Calibrate against people
Before you trust a judge, have people score a sample of 50 to 100 cases. Compare the judge's scores with the people's. If they mostly agree, use the judge for volume and use people for spot checks. If they disagree, fix the rubric or the judge prompt and test again.
A sensible mix
- Code checks on every case, every change.
- Model judge on every case for open qualities.
- Human review on a sample each week, and on every release candidate.
- People on all must-never-happen reports.
- ✓People are the gold standard but slow. Use them for high stakes, new rubrics and checking your automatic methods.
- ✓Measure reviewer agreement. Low agreement means an unclear rubric or a subjective task.
- ✓Model judges are fast but biased by length, position and self-preference. Use clear rubrics, both orders and pinned versions.
- ✓Calibrate every judge against human scores, then mix code, judges and people.
Check a judge against a person
- Write a judge prompt for a rubric you wrote earlier.
- Collect 20 outputs. Score them yourself, then ask a model to judge them.
- Compare the scores. Count how often you agree, and look at each disagreement.
- Change the judge prompt to fix one pattern of disagreement and run it again.
Q1Why run a comparison judge in both orders?show answer
Judges can favor the first answer. Running both orders and keeping consistent results removes that position bias.
Q2What does low reviewer agreement tell you?show answer
The rubric is unclear or the task is subjective. Add examples or sharper rules before trusting the scores.
Q3How do you calibrate a model judge?show answer
Have people score a sample, compare with the judge's scores, and fix the prompt or rubric until they mostly agree.