A judge is a second LLM call that grades the first one. You give it the input, the candidate output, a rubric, and (sometimes) a reference. It returns a label and a critique. That is the whole technique.
It works because a strong frontier model on a well-scoped rubric agrees with human raters somewhere above eighty percent of the time on open-ended generation 1. It does not work when the task is hard correctness: GPT-4o scores barely above chance on JudgeBench 2, and even reasoning judges stop short of an executable check. Most of the failure modes between those two poles trace back to the same three biases.
What an LLM judge actually is
A judge is a classifier you wrote in English instead of training in scikit-learn. The training signal is a small human-labeled set; the inference cost is one extra API call per trace; the output is a binary label plus a free-form critique you can read 3.
# Minimum viable judge.
def judge(query: str, response: str, rubric: str) -> dict:
prompt = f"""You evaluate whether a response satisfies the rubric.
RUBRIC:
{rubric}
QUERY:
{query}
RESPONSE:
{response}
Return JSON: {{"label": "PASS" or "FAIL", "critique": "..."}}"""
raw = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": prompt}],
temperature=0,
response_format={"type": "json_object"},
)
return json.loads(raw.choices[0].message.content)
You measure the judge against a held-out human-labeled set the same way you would measure any classifier: true positive rate, true negative rate, confusion matrix. Agreement percent is a trap on imbalanced data; we cover that on the calibration page.
When to use a judge
Use a judge when the criterion is real but expensive to write code for. The four cases that almost always work:
- Open-ended generation where a reference is impossible (creative writing, brainstorming, suggested replies) 4.
- RAG faithfulness, where the question is "does the response stick to what is in the retrieved context."
- Tone and style adherence (refund-policy compliance, professional register, brand voice).
- Summarization quality on dimensions humans agree on (coverage, conciseness, faithfulness to source) 5.
In all four cases, a calibrated binary judge beats a Likert rubric, beats ROUGE-style reference metrics, and beats the vendor's pre-baked "quality score" you cannot inspect.
When not to use a judge
The hard cases. JudgeBench, the Tan et al. benchmark (ICLR 2025) built to test judges on hard correctness, scores GPT-4o at 56.6 percent overall with the Arena-Hard judge prompt and 50.9 percent with a vanilla prompt, against 50 percent for a coin flip. The pairs cover knowledge, reasoning, math, and coding questions where ground truth exists but is non-trivial 2. The reason is straightforward: if the judge model cannot solve the underlying problem, it cannot reliably tell you whether the candidate solved it.
The same paper shows the other side. Reasoning models judge much better: o3-mini at high effort scores 80.9 percent, o1-preview 75.4, and DeepSeek-R1 73.1, while o3-mini at low effort drops to 70.6 2. Judge accuracy on hard problems tracks how much the judge reasons before it answers. Those numbers come from the April 2025 revision and predate the current model lineup, so score your own judge model on your own gold set before you rely on them.
flowchart TD
A[Task] --> B{Verifiable<br/>by code?}
B -->|yes| C[Executable verification<br/>unit tests, regex, SQL exec, schema]
B -->|no| D{Hard reasoning,<br/>math, or code?}
D -->|yes| E[LLM judge as<br/>secondary signal only<br/>cite JudgeBench]
D -->|no| F{Reference<br/>available?}
F -->|yes| G[Hybrid: ref metric<br/>plus calibrated judge]
F -->|no| H[Calibrated LLM judge<br/>binary + critique]
The rule: never let an LLM judge be the sole verifier on a task where ground truth is hard to compute but a programmatic check exists. Use executable verification first; use the judge as a second signal 2. If you can run a unit test, run the unit test.
The three biases
Every team's first judge hits these in order. The mechanics are covered in detail on the biases page; the headline numbers are below.
Position bias. When you ask the judge to pick between A and B, it favors whichever response sits first. On near-identical answers, Zheng et al. measured GPT-3.5 favoring the first-listed answer in 50.0 percent of cases and Claude v1 in 75.0 percent. GPT-4 was the steadiest, giving the same verdict after a swap in 65.0 percent of cases 1. The mitigation is to swap positions and only count agreement when both orderings agree.
Verbosity bias. The judge prefers longer responses even when concision is closer to the rubric. Zheng et al. built a "repetitive list" attack: take 23 answers that contain a numbered list and pad each one with a rephrased copy of its own list, adding no new information. Claude v1 and GPT-3.5 preferred the padded answer 91.3 percent of the time; GPT-4 fell for it 8.7 percent of the time 1. The mitigation is rubric language that scores conciseness explicitly, or a length-normalized pairwise setup.
Self-preference bias. A judge favors outputs from its own model family. Zheng et al. found GPT-4 favoring its own answers with a 10 percent higher win rate than humans gave it, and Claude v1 with a 25 percent higher win rate; GPT-3.5 did not favor itself 1. The G-Eval authors flagged the same risk for LLM-generated text 5. The mitigation is to pick a judge from a different family than the generator, and to prefer a judge that is bigger than the generator so the capability ceiling does not bound the score. Anthropic's eval guide gives the weaker form of the same rule: use a different model to grade than the one that generated the output 6.
Figure: LLM judge bias rates from MT-Bench (Zheng et al. 2023, Tables 2 and 3). Position bias: on near-identical answers Claude v1 favors the first-listed one in 75.0% of cases (GPT-3.5 50.0%). Verbosity bias: when an answer is padded with a rephrased copy of its own numbered list, Claude v1 and GPT-3.5 prefer the padded version 91.3% of the time, while GPT-4 does so 8.7% of the time.
The bigger-than-generator rule
If your generator is Claude Sonnet 5.5, judge with Claude Opus 5.5 or GPT-6 Astra, not Claude Haiku 4.5 7 8. If your generator is a 70B open-weight model, do not judge with its 8B sibling. The judge has to be at least as capable as the model it grades or it will systematically miscall the hard cases 3. This is a hard rule, not a guideline; teams that violate it almost always blame "judge instability" when the real problem is that the judge cannot solve the task.
Capability now includes how much the judge reasons, as the JudgeBench numbers above show. Treat the reasoning effort setting as part of the judge's identity: pin it with the model ID, calibrate at that setting, and recalibrate when either changes.
Hamel and Shankar's current FAQ draws the line at measurement instead of model family. Start with the most capable model you can get, validate it against held-out human labels, then try smaller and cheaper models and measure how much TPR and TNR you lose. It also calls a same-model judge usually fine for a scoped binary rubric, and suggests switching models only when alignment stalls 9. Read that as permission, not an exemption: a same-model judge is acceptable once it clears TPR and TNR on a test split that includes the generator's own failures.
The exception is a distilled judge fine-tuned for the specific rubric. Prometheus 13B can match GPT-4 on custom rubrics it was trained on 10. We cover the distillation play on the distilled-judges page in the techniques cluster.
Sclar et al. demonstrate up to seventy-six accuracy-point swings on LLaMA-2-13B from meaning-preserving format changes 11. The relevance to judging is that "the rubric stays the same; only the output schema or option labels change" can move the judge's calls by enough to ruin a calibration. Pin the schema. Pin the option labels. Re-run calibration if either changes.
CITATION
Zheng et al. 2023, "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena," NeurIPS 2023 Datasets and Benchmarks. The canonical paper; introduces position, verbosity, and self-preference bias formally, reports GPT-4 reaching above eighty percent agreement with human raters on MT-Bench. arxiv.org/abs/2306.05685
Direct vs pairwise
Two operational modes; pick by task. Direct scoring asks the judge to label one response against a rubric (PASS or FAIL). Pairwise asks the judge to pick between two responses on the same query. Yan's rule is the one we use: pairwise for subjective preference where humans cannot calibrate a scalar, direct for objective binary criteria where the rubric has a clean cutoff 4.
Direct binary is cheaper (one judge call per trace, no position swap needed), gives you a confusion matrix you can report directly, and forces the rubric author to write a falsifiable criterion. Pairwise scales when you have many systems to rank without re-labeling each, but it costs two judge calls (position swap) and produces a ranking, not an absolute score 1. The Arena methodology (Chatbot Arena in the original paper; the platform has since been renamed Arena and lives at arena.ai 12) uses pairwise at crowdsource scale; that is the right call when "which is better" is the only question that matters.
For a typical product team starting from zero, the default is direct binary on a single dimension your business actually cares about (faithfulness, refund-policy compliance, tone). Pairwise comes later when you are comparing two prompt versions head-to-head.
What good looks like
A production judge program has all five of the following, or it is not production:
- A frozen rubric versioned with the code that uses it.
- Human-labeled dev and test sets built from 100 to 200 labeled examples per failure mode 9, with kappa or alpha of at least 0.80 between two raters when more than one person labels.
- TPR and TNR both above 0.90 on the test set, measured before deploy.
- Position-swap discipline on every pairwise comparison.
- A monthly recalibration cadence, a full recalibration whenever the judge model or its effort setting changes, and an alert if the judge-vs-human agreement on a fresh slice drifts.
The cheapest first artifact is a fifty-trace labeled set, a one-page rubric, and a half-day of human labeling. From there you walk the rubric to a hundred labels, split train and dev, write the judge prompt, calibrate, then go look at the test number once.
A note on Likert
The literature uses 1-to-5 and 1-to-10 Likert scales because academic eval traditions inherited them from BLEU-era summarization work. Practitioners should not. Binary plus critique wins on three dimensions Hamel articulated clearly: lower required N for the same statistical power, higher inter-rater agreement, and a forcing function that makes the rubric author specify exactly what failure looks like 9. A judge that returns "4 out of 5" is also a judge that ducks the question; you cannot ship a release-gate decision on the difference between a 3 and a 4.
The exception is multi-dimensional grading where you genuinely need to attribute failures across orthogonal axes (faithfulness, conciseness, helpfulness, tone). Even there, the right shape is multiple independent binaries, not a multi-axis Likert.
The next page shows how to write the prompt that gets you there. After that, the calibration page shows how to measure it.