A single accuracy number on a single prompt format on a single option order at a single temperature is an unfalsifiable claim. The model that scored 78% might score 60% on a paraphrased prompt, 72% on a permuted option order, or 81% with stochastic sampling. Robustness testing surfaces the range so the headline number carries the uncertainty it actually has.
For the chapter-length treatment, including the adversarial side, see safety and red-teaming.
Sclar et al. showed that meaning-preserving format changes can move LLaMA-2-13B accuracy by up to seventy-six points on the same task 1. The change is not a different prompt; it is the same prompt with a different separator, a different option label format, a different spacing. The model behaves like a different system.
The result persists at scale. Across the paper's tasks the median spread over 10 sampled formats is 7.5 points, one task in five spreads at least 15 points for every LLaMA-2 setting, and the spread does not shrink with model size (LLaMA-2-70B included), instruction tuning, or more few-shot examples. Ten sampled formats is a lower bound on the true spread. Format also reorders models: when one of LLaMA-2-13B and LLaMA-2-70B beats the other by two points or more under one format, the other wins by two points or more under a different format 14 percent of the time. A ranking built on one format is a property of that format.
Anthropic's "Challenges in Evaluating AI Systems" notes the related case: changing (A) (B) (C) to (1) (2) (3), changing the brackets, or adding a space between option and answer can move MMLU accuracy by about five percent without changing question content 2.
Format is not the only surface. Brittlebench applies 15 kinds of meaning-preserving perturbations (typos, extra spaces, quote and newline padding, paraphrase) to six benchmarks including MMLU and GPQA 3. Scores drop by as much as 12 percent, perturbations explain up to half of a model's score variance, and a single perturbation changes the ranking of the six open-weight models tested in 63 percent of cases. Commercial models vary less in absolute terms, but prompt sensitivity is still above a quarter of their variance. Add a typo-and-padding sweep next to the format sweep, and compare model rankings across variants, not only each model's own range.
FormatSpread's fix: report a range, not a point. Sample several plausible format variants, score each, report min, median, max.
def format_sweep(model_call, prompt_template, variants, n_samples=200):
scores = {}
for name, render in variants.items():
correct = 0
for example in sample_n(n_samples):
response = model_call(render(prompt_template, example))
if grade(response, example): correct += 1
scores[name] = correct / n_samples
vals = sorted(scores.values())
return {"min": vals[0], "median": vals[len(vals)//2], "max": vals[-1], "per_variant": scores}
The variant set is typically four to eight: bullet-list versus numbered, (A) (B) versus 1. 2., Markdown headers versus plain text, with and without a system message, with and without explicit "Answer:" sentinel.
For tool calling you do not have to design the sweep. BFCL V4 ships a format-sensitivity suite of 26 prompt variants, 200 cases each: return syntax in Python, JSON, or two XML styles, function docs in JSON, Python, or XML, with and without a tool-call tag, plaintext versus Markdown, and two instruction styles 4. It runs only for prompt-mode models (no native function calling) and sits outside the leaderboard's overall score, so read it as a separate number.
Figure: Robustness testing with a format sweep: the same prompt rendered in six meaning-preserving format variants scores between 60% and 81% on identical examples, a 21-point spread that a single point estimate hides. FormatSpread-style reporting ships the min, median, max triplet (60 / 72 / 81) instead of the one best number.
Order sensitivity
For multi-option tasks (multiple-choice, pairwise preferences, multiple-document retrieval), order is a second axis of variance. The standard mitigation is to permute and average. For pairwise judging, swap the order and treat the pair as a single observation. For multiple-choice with few enough options (4 options is 24 permutations, tractable), run all permutations and report the per-permutation distribution.
For few-shot prompts, example order matters substantially. Yan's survey of LLM-evaluator work reports position bias in pairwise judging (most judges preferred the first position) and few-shot judge prompts whose performance was unstable when the label, the example order, or the number of examples changed 5. Test multiple orderings before trusting a few-shot number.
Harness details behave the same way. Hugging Face ran three implementations of MMLU, the original 6, HELM, and the EleutherAI harness, which differ in prompt wording and in how they score the answer. They produced widely different numbers and changed the ranking of models on its leaderboard 7. Pin the harness commit next to every score.
Temperature sweeps
For deterministic tasks (judging, classification, schema-filling), temperature is zero and remaining variance is GPU-level nondeterminism. For creative tasks (brainstorming, summarization with reasoning), temperature is non-zero and the score depends on sampling.
Three patterns are common.
Pass@k. Sample k times; report whether any of the k passed. Standard on code benchmarks. Rises with k, so it measures what the model can do at its best, not what it does reliably.
Pass^k. Sample k times; report whether all k passed. The consistency metric. Used on agent benchmarks (tau-bench is canonical). Falls with k.
Self-consistency at temperature T. Sample n times at temperature T; take the majority vote. Reduces individual-sample variance at n times the inference cost.
Reporting just pass@1 at a single temperature is the failure mode. Pass^4 sits between p^4 and p: a model at 80% pass@1 lands somewhere from 41% to 80% at pass^4, and where it lands is the consistency number production cares about.
CheckList: INV, MFT, DIR
Ribeiro et al.'s CheckList is the canonical framework 8.
Minimum Functionality Test (MFT). The model must pass a basic test of the capability ("if I ask a sentiment classifier I love this product, does it return positive").
Invariance Test (INV). The model's output should not change under a meaning-preserving input perturbation ("if I swap excellent for great in a positive review, does the classifier still return positive").
Directional Expectation Test (DIR). The model's output should change in a known direction ("if I add not to a positive review, does sentiment flip negative").
In a user study, the team behind a commercial sentiment-analysis model found new, actionable bugs with CheckList in a model they had already tested extensively. A passing benchmark score does not imply robustness.
| Test type | Property | Example |
|---|
| MFT | minimum capability | "is X positive": positive |
| INV | invariance under paraphrase | "X is great" = "X is excellent" |
| DIR | known-direction change | adding "not" flips polarity |
For LLM judges, INV tests check that meaning-preserving rubric edits do not change the verdict. For generation systems, DIR tests check that adding "in formal English" produces a formal output. IFEval works like a DIR suite for instruction-following: each prompt carries verifiable instructions ("write in more than 400 words") and a program checks that the output obeys them 9.
A practical reporting template
For a release-gate claim:
- Point estimate at the canonical format, order, temperature.
- Wilson 95% interval at that setting.
- Format range: min and max across four to eight variants.
- Order range: min and max across two to four variants.
- Temperature range: pass@1 and pass^k at the production temperature.
The headline triplet most teams ship is (median across formats, min across formats, max across formats).
What not to do
Cherry-picking the best format. A team runs ten formats, reports the best, omits the others. Sclar's data shows a best-to-worst spread of at least 15 points on one task in five across just ten sampled formats, so the chosen headline can overstate the worst format by that much or more 1. Corrective: pre-register the format set.
Reporting one temperature. A team reports pass@1 at temperature 0.7, mentions consistency informally, never quantifies pass^k. Corrective: publish both.
Skipping INV. A team has a high MFT score and never tests paraphrase tolerance. Corrective: at least three INV tests per major capability, derived from real production paraphrase pairs.
What good looks like
A robustness-aware program pre-registers four-to-eight format variants per task, pre-registers order variants for any multi-option task, reports pass^k at production temperature alongside pass@1, and maintains INV and DIR suites for the top three failure categories from error analysis. The format-sweep code runs in CI on every release candidate; min, median, max are the three numbers on the release-gate dashboard.