Five biases show up in every production judge. Four have well-characterized empirical evidence and standard mitigations. The fifth, format sensitivity, is more recently characterized and the mitigation is mostly discipline. This page is the punch list you walk before shipping.
1. Position bias
The judge prefers whichever response sits first (or last) in a pairwise comparison, regardless of content.
Evidence. Zheng et al. measure this directly: for each MT-Bench question they generate two near-identical answers, then swap the order. With the default prompt GPT-3.5 picks the first slot 50 percent of the time and Claude v1 picks it 75 percent of the time, a strong directional skew. GPT-4 is more balanced (30 percent first, 65 percent consistent across the swap) but not immune 1.
Mitigation: position swap. Call the judge twice, once with [A, B] and once with [B, A]. Only count a preference when both orderings agree. If they disagree, mark it a tie. Zheng et al. recommend the same conservative rule 1. This doubles the inference cost; it is non-negotiable on any pairwise harness used to gate a release.
flowchart LR
Q[Trace pair] --> AB[Judge call:<br/>A then B]
Q --> BA[Judge call:<br/>B then A]
AB --> P{Both<br/>agree?}
BA --> P
P -->|yes, A wins| A[Score: A]
P -->|yes, B wins| B[Score: B]
P -->|disagree| T[Score: tie]
A subtler variant: when you sample data for human labels, randomize the visible position too. Human raters carry their own position bias; if your gold set was always "candidate on the left, reference on the right," your calibration is correlated with the layout.
2. Verbosity bias
The judge prefers longer responses even when concision is closer to the rubric.
Evidence. Zheng et al. built a "repetitive list" attack: take 23 MT-Bench answers that contain a numbered list, have GPT-4 rephrase the list, and prepend the rephrased copy so the answer says everything twice. Claude v1 and GPT-3.5 judged the padded answer better in 91.3 percent of cases; GPT-4 fell for it in 8.7 percent 1. Eugene Yan's survey of the evaluator literature repeats the result 2. The strongest judge in that test mostly resisted, so do not assume yours does: run the check below.
Mitigations.
- Score conciseness in the rubric directly. Add "FAIL if the response includes information not required by the query" as a binary clause.
- Length-normalize the pairwise comparison. Truncate both candidates to the shorter one's length before showing them to the judge. The truncation is sometimes ugly; the calibration gain is worth it.
- Control for length statistically when you score many systems. Length-controlled AlpacaEval fits length as a covariate and raised its Spearman correlation with Chatbot Arena from 0.94 to 0.98 3.
- Use a reference-anchored direct rubric instead of pairwise where you can. Prometheus showed reference grounding is the single largest factor in raising agreement with humans on customized rubrics 4.
The cleanest signal that verbosity is biting: your judge says the long response is better, your humans tied them, and the long response was padded with restated context. Sample fifty disagreements and read them.
3. Self-preference bias
The judge favors outputs from its own model family.
Evidence. Liu et al. flag the risk that LLM-based evaluators favor LLM-generated text when scoring with G-Eval 5. Zheng et al. see it on MT-Bench but hedge: GPT-4 gives itself a win rate 10 percent higher than humans do and Claude v1 gives itself 25 percent higher, yet GPT-3.5 does not favor itself and the authors say their data cannot settle the question 1. Panickssery et al. give the sharper result: GPT-4 and Llama 2 can tell their own outputs from other models' and from human text, and the strength of self-preference tracks that self-recognition linearly 6. A judge that recognizes its own style has a reason to reward it.
Mitigations.
- Cross-family judging. Generator is GPT-4o; judge with Claude. Generator is Claude; judge with GPT. The bias does not disappear, but the correlation with your generator's idiosyncratic style drops.
- Bigger than the generator. A judge weaker than the generator cannot reliably detect failure on the hard cases; that interacts badly with self-preference because the judge defers to the generator on anything it cannot solve itself 7.
- Distilled judge from another family. An open judge such as JudgeLM, fine-tuned with swap augmentation, reference support, and reference drop to reduce position, knowledge, and format bias 8, is unlikely to share a family with your generator. None of that training targets self-preference directly. This is more work than swapping the API key; pay the cost only when cross-family judging is not enough.
The judge's labels move when meaning-preserving format changes happen in the prompt or candidate output. This is the bias most teams discover the hard way: they revise their rubric's option labels from "PASS / FAIL" to "1 / 0" for downstream parsing convenience, and their calibration tanks.
Evidence. Sclar et al. report up to seventy-six accuracy-point swings on LLaMA-2-13B from meaning-preserving format changes 9. The effect persists at scale and shows weak correlation across models, which means you cannot extrapolate a calibration done on one schema to another. Anthropic's "Challenges in Evaluating AI Systems" (October 2023) reports that changing MMLU options from (A) to (1), or from (A) to [A], can move accuracy by about 5 percent 10.
Mitigations.
- Pin the schema. JSON keys, option labels, ordering of few-shot examples, and the order of rubric clauses are all part of the calibration. Changing them means rerunning calibration.
- Version the rubric. Treat the prompt as code; check it into git; include the prompt hash in the eval result so a stale calibration is detectable.
- Test the rubric on a known-good slice. When you do change format, run the new prompt on the calibration set first and confirm the metrics did not move before deploying.
5. Sycophancy
The judge tends to agree with whatever framing or claim the prompt itself encodes, especially when the prompt expresses a preference.
Evidence. Sharma et al. show five frontier assistants consistently exhibit sycophancy across four free-form text-generation tasks, and that human preference data is part of the cause: a response that matches the user's stated views is more likely to be preferred, and preference models sometimes favor a convincingly written sycophantic answer over a correct one 11. Practical effect on judges: if your judge prompt contains language like "we generally expect the response to be helpful and detailed," the judge tilts toward calling things PASS to align with the stated expectation.
Mitigations.
- Neutral framing in the rubric. "Evaluate whether the response satisfies the criterion" beats "Evaluate whether the response is helpful and on-topic." The second sentence primes a positive bias.
- No anchor on the candidate's confidence. Strip "I am confident that..." or hedging tokens from the candidate before showing it to the judge if you can; the judge picks up on stated confidence and uses it as a signal.
- Adversarial test set. Build a small slice of traces where the candidate sounds confident but is wrong, and check that the judge catches them. Drop the judge if it does not.
A composite mitigation: panels and ensembles
When the cost budget allows, run two or three judges in parallel and require agreement. Verga et al. report that a panel of smaller models from different families outperforms a single large judge across three judge settings and six datasets, shows less intra-model bias, and costs over seven times less than GPT-4 as a single judge 12. The disagreement set is a free signal for "interesting traces" you should send to human review. A panel is still not a free win: calibrate it against humans, because two judges with the same blind spot can be wrong in the same direction.
CITATION
Zheng et al. 2023, "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena," NeurIPS 2023 D&B. The empirical source for position bias (Claude v1 picks the first slot 75% of the time on near-identical answers), verbosity bias (Claude v1 and GPT-3.5 prefer a padded answer 91.3% of the time, GPT-4 8.7%), and a hedged self-enhancement result. arxiv.org/abs/2306.05685
How to detect each bias on your own data
The mitigations above are operational; before you apply them you have to confirm the bias is biting. Each of the five has a cheap test.
Position bias check. Take fifty pairwise traces from your calibration set. Run each one twice with swapped order. Count how often the judge picks position one across all runs. The expected rate at no bias is around fifty percent. If the judge picks position one materially more than fifty-five percent of the time, position bias is large enough that the position-swap discipline is mandatory, not optional 1.
Verbosity bias check. Pull fifty traces where the two candidates differ by at least 2x in length but agree on substance (you can generate this slice synthetically by truncating one of two semantically equivalent answers). Run the judge. If it picks the longer answer in more than seventy percent of cases, you have a verbosity problem. Mitigate by rubric language or length normalization.
Self-preference check. Generate the same response from two different model families on a hundred queries. Have your judge (call it model X) rate both. If the X-generated responses systematically rate higher than the non-X responses on equivalent traces (a rate gap of more than ten points on a calibrated rubric), self-preference is biting. Switch judge family.
Format sensitivity check. Take your final rubric. Make a meaning-preserving rewrite: change PASS/FAIL to YES/NO, or change the order of few-shot examples, or move the criteria block above the role block. Re-run on the calibration set. If TPR or TNR moves by more than three percentage points, your rubric is fragile to format; pin the original schema and version it 9.
Sycophancy check. Build a slice of fifty traces where the candidate sounds confident but is wrong (you can write these by hand or generate with adversarial prompting). Run the judge. If the judge calls PASS on more than ten percent of these confident-wrong cases, the rubric is biased toward stated confidence; neutralize the framing.
Figure: The five LLM judge biases with their detection red flags and fixes: position bias when the first slot wins over 55 percent of swapped runs, verbosity when the longer answer wins over 70 percent of pairs, self-preference at a 10-point own-family gap, format sensitivity when TPR or TNR moves more than 3 points on a meaning-preserving rewrite, and sycophancy when over 10 percent of confident-but-wrong candidates get a PASS.
Checklist before you ship
Before you deploy a judge to gate releases, confirm you have done all five:
If any box is empty, the judge metrics you computed on the calibration set are an upper bound on what you will see in production, not a fair estimate.
What is next
There is a sixth category of judge failure that no bias mitigation fixes: the cases where the judge cannot solve the underlying task and so cannot grade it. JudgeBench measures this directly, and the result for 2024-era judges is sobering 13. The JudgeBench honest take page covers when not to use a judge at all.