In reinforcement learning with verifiable rewards, the verifier is the reward function. Every answer it wrongly rejects pushes the policy away from correct behaviour, and every answer it wrongly accepts is a target the policy will learn to hit. This page shows how to measure both errors before training, with a test you can run in five minutes against a real verifier library.
What is a verifier in RLVR?
A verifier is the function that decides whether a rollout succeeded. It extracts the model's final answer or inspects the final state, compares it with a reference or runs checks against it, and returns a reward. In RLVR the verifier usually is the reward, with no separate reward model to absorb its mistakes, so the verifier's error rate becomes the reward's error rate and the policy optimizes against both.
NeMo Gym's docs describe the verifier's job as task completion scoring, one of the four parts of an environment 1. The word covers very different code, from a regex that compares two numbers to a container that runs a hidden test suite to an LLM grading against a rubric.
What kinds of verifiers are there?
Verifiers fall into four families, and each one fails in a characteristic direction. Knowing the family tells you which error to hunt first.
| Verifier | How it scores | Main false negative | Main false positive |
|---|
| Rule-based answer match | Parse the final answer and compare with the reference (Math-Verify, exact match, regex) | Valid formats the parser does not handle | Parsing the wrong number out of free text |
| Execution-based | Run tests or check program behaviour (unit tests, fail-to-pass checks) | Tests that demand one implementation | Tests too weak to catch a wrong solution |
| State checker | Inspect the environment after the episode (files, database rows, page state) | Checking the wrong place or too strictly | Accepting a state reached the wrong way |
| Model-based | An LLM or trained verifier judges the answer, often against a rubric | Rare, but grows with judge strictness | Persuasive or rubric-shaped wrong answers |
Hybrids are common. TinyV keeps a rule-based verifier and adds a lightweight LLM check to rescue answers the rules reject 2.
Why do false negatives and false positives do different damage?
A false negative scores correct work as failure. It wastes the rollout and teaches the policy to avoid a correct behaviour, usually an unusual but valid format. A false positive scores wrong work as success, and that creates a target: once the policy finds a wrong answer the verifier accepts, it can raise its reward by producing more of them. Report the two rates separately, never as one accuracy number.
The literature measures both. Huang et al. found that rule-based verifiers often fail to recognize equivalent answers in different formats, and that these false negatives increasingly hinder RL performance as the policy model gets stronger. The same study found model-based verifiers improve static accuracy but are highly susceptible to reward hacking during RL, misclassifying certain response patterns as correct, particularly after fine-tuning 3. TinyV put a number on the false-negative side: of the responses a rule-based verifier marked incorrect in Big-Math-RL-Verified, 38.5% were actually correct, and adding a small LLM check to rescue them raised pass rates by up to 10% 2.
How do you measure verifier error?
Use metamorphic testing. Build two sets from tasks with known answers: a positive set of correct answers rewritten in every way the task allows, which the verifier should all accept, and a negative set of near-miss wrong answers, which it should all reject. The false-negative rate is the share of positives rejected, the false-positive rate the share of negatives accepted. Count both by rewrite category.
Good rewrites for the positive set: other number formats (fraction, decimal, percent, thousands separators, scientific notation), units, reordered set elements, algebraically equivalent forms, a self-correction before the final answer, and for code, refactors that keep behaviour. Good near-misses for the negative set: off-by-one values, sign flips, rounding, partial answers, a hedge that lists several candidates, the right format around the wrong content, and for code, patches that special-case the visible tests.
Xin's September 2026 audit ran this method at scale: 307,420 verdicts from four verifiers, namely Math-Verify 0.9.0 with its default LaTeX extraction, Math-Verify restricted to plain expressions, a DeepSeek-Math-style string normalizer, and a reference cascade of string, numeric and SymPy checks. Certified-equivalent rewrites make any rejection a provable false negative, and adversarial rewrites such as off-by-one answers make any acceptance a provable false positive. On identical inputs, the verifiers' self-validation rates ranged from 53.8% to 95.2%. Math-Verify's two configurations disagreed with each other on 49.9% of pairs, and whitespace and punctuation caused 93.0% of in-contract failures for the default LaTeX configuration. The reference cascade, which compares numbers with a relative tolerance of 10^-4, accepted off-by-one wrong answers as a step function of magnitude, 0% below 10^4 and 100% at or above 4.
Run the test again whenever the verifier, its configuration, or the prompt format changes, and every so often during training. The policy's outputs drift toward formats the verifier accepts, and Huang et al. found the false-negative cost grows as the policy improves 3.
What does Math-Verify get wrong?
At its default settings, Math-Verify 0.9.0 rejected 3 of 12 correct rewrites in the test below and accepted 4 of 8 wrong answers, and every configuration change traded one kind of error for the other.
Hugging Face's Math-Verify is a widely used rule-based verifier for math answers, released under Apache-2.0 5. The script runs a 20-case metamorphic set (12 correct rewrites, 8 near-misses) against version 0.9.0, the current PyPI release, published January 2026 6, under four configurations. The anchored configuration accepts only a boxed answer or LaTeX that follows an anchor phrase such as "The answer is", and drops free-text extraction. The script ran unchanged in two fresh virtual environments on Python 3.11.
"""
Measure a verifier's false negatives and false positives on a small
metamorphic set: equivalent rewrites of correct answers (should pass)
and near-miss wrong answers (should fail).
pip install "math-verify[antlr4_13_2]==0.9.0"
"""
import re
from math_verify import LatexExtractionConfig, parse, verify
# (case, gold answer, model output, should the verifier accept it?)
CASES = [
("decimal form", r"$\frac{1}{2}$", r"So the probability is \boxed{0.5}.", True),
("thousands comma", r"$1000$", r"\boxed{1,000}", True),
("thin-space digits", r"$1000$", r"\boxed{1\,000}", True),
("number as a word", r"$12$", r"The answer is twelve.", True),
("x10^n notation", r"$1000$", r"\boxed{1 \times 10^{3}}", True),
("e-notation", r"$1000$", r"\boxed{1e3}", True),
("currency sign", r"$1000$", r"\boxed{\$1000}", True),
("trailing unit", r"$5$", r"\boxed{5\text{ cm}}", True),
("self-correction", r"$3$", r"The answer is 2. Wait, that misses a case. \boxed{3}", True),
("negative fraction", r"$-\frac{1}{2}$", r"\boxed{\frac{-1}{2}}", True),
("unboxed answer", r"$7$", r"The answer is 7.", True),
("six-place decimal", r"$\frac{1}{3}$", r"\boxed{0.333333}", True),
("off by one", r"$10000$", r"\boxed{10001}", False),
("sign flip", r"$3$", r"\boxed{-3}", False),
("tiny magnitude", r"$0.0000001$", r"\boxed{0.0000002}", False),
("thin space vs 0", r"$0$", r"\boxed{1\,000}", False),
("abandoned answer", r"$2$", r"The answer is 2. Wait, that misses a case. \boxed{3}", False),
("hedged list", r"$4$", r"It is 2, 3, or 4.", False),
("negated guess", r"$2$", r"I think it is 3, not 2.", False),
("three-place decimal", r"$\frac{1}{3}$", r"\boxed{0.333}", False),
]
# Only a boxed answer or LaTeX after an anchor such as "The answer is".
ANCHORED = [LatexExtractionConfig(boxed_match_priority=0, try_extract_without_anchor=False)]
def thin_space_fix(text):
# "1\,000" is a thousands separator, not a product.
return re.sub(r"(?<=\d)\\,(?=\d)", "", text)
CONFIGS = [ # name, extraction config, float_rounding, pre-normalizer
("default", None, 6, None),
("anchored", ANCHORED, 6, None),
("anchored, rounding 12", ANCHORED, 12, None),
("anchored, rounding 12, thin-space fix", ANCHORED, 12, thin_space_fix),
]
for name, extraction, rounding, prep in CONFIGS:
missed, accepted = [], []
for case, gold, output, should_accept in CASES:
text = prep(output) if prep else output
pred = parse(text) if extraction is None else parse(text, extraction_config=extraction)
ok = verify(parse(gold), pred, float_rounding=rounding)
if should_accept and not ok:
missed.append(case)
if ok and not should_accept:
accepted.append(case)
n_pos = sum(c[3] for c in CASES)
print(f"{name}: false negatives {len(missed)}/{n_pos}, false positives {len(accepted)}/{len(CASES) - n_pos}")
print(f" rejected but correct: {', '.join(missed) or 'none'}")
print(f" accepted but wrong: {', '.join(accepted) or 'none'}")
The output:
default: false negatives 3/12, false positives 4/8
rejected but correct: thin-space digits, number as a word, e-notation
accepted but wrong: tiny magnitude, thin space vs 0, hedged list, negated guess
anchored: false negatives 4/12, false positives 2/8
rejected but correct: thin-space digits, number as a word, e-notation, unboxed answer
accepted but wrong: tiny magnitude, thin space vs 0
anchored, rounding 12: false negatives 5/12, false positives 1/8
rejected but correct: thin-space digits, number as a word, e-notation, unboxed answer, six-place decimal
accepted but wrong: thin space vs 0
anchored, rounding 12, thin-space fix: false negatives 4/12, false positives 0/8
rejected but correct: number as a word, e-notation, unboxed answer, six-place decimal
accepted but wrong: none
Twenty hand-picked cases measure which failure classes exist, not how often they occur in your rollouts; the rates depend entirely on the case mix. Four findings hold regardless.
- One parse bug can cause both errors. At default settings
1\,000, a common LaTeX way to write one thousand, parses as the product 1 times 0. So it is rejected when the answer is 1000 and accepted when the answer is 0.
- Default rounding hides small numbers.
verify() rounds floats to 6 decimal places unless told otherwise, so 0.0000001 and 0.0000002 compare equal and a factor-of-two error passes. Raising float_rounding to 12 fixes that case and rejects six-place answers such as 0.333333 for 1/3, which passed at 6. Set the tolerance from your task contract, and check it against both the smallest answers in your task set and the precision you ask the model for.
- Free-text extraction is a false-positive source. The default configuration pulls a number out of unanchored text, so a hedge like "It is 2, 3, or 4." passes when the answer is 4, and "I think it is 3, not 2." passes when the answer is 2. The anchored configuration removes both, and costs a false negative on a correct plain-text answer. If you require an anchored answer, say so in the prompt and keep a separate format reward so the cost lands on format, not on correctness.
- Every fix moves errors between columns. Only the last configuration reaches zero false positives on this set, and it still rejects 4 of 12 correct answers. The configuration is part of the verifier, which is why Xin found Math-Verify's own two configurations disagreeing on half of all pairs 4. Pin the library version and the full configuration in every result you report.
This run agrees with Xin's audit, which used the same Math-Verify 0.9.0 install: both Math-Verify configurations rejected off-by-one answers at every magnitude, and so did this test at 10^4. The step function came from the reference cascade's relative tolerance, which is the reason to check any numeric tolerance you set against your largest answers as well as your smallest.
How do you test an execution-based verifier?
Generate solutions that should fail and check that they do. For code environments that means patches that special-case the visible tests, delete or weaken assertions, hardcode expected outputs, or edit the test files. If any of them passes, the tests are too weak to train on, however many of them there are.
Rajan's audit generated Docker-verified incorrect patches and found that 28.5% of a 49-task SWE-bench Verified sample, and 25.0% of 20 R2E-Gym tasks, had test suites an incorrect patch could pass. The paper's hardening procedure pairs an LLM judge with a Docker gold-sanity gate 7. EvilGenie built a coding environment where reward hacking was easy and compared three detectors: held-out unit tests, LLM judges, and detection of edits to test files. The LLM judge was highly effective on unambiguous cases, and held-out tests added little 8.
Build-time gates catch the opposite problem, tests that never pass at all. DeepSeek-V3.2 counted a code environment as built only when the gold patch turned at least one failing test into a passing one and broke no passing test 9, and SWE-smith synthesizes task instances by introducing changes that break existing tests in a repository, so every task starts from a failing test 10. Keep the verifier out of the policy's reach too: since Harbor 0.25.0, new tasks default to a separate verifier that runs the tests in a fresh sandbox, one that does not inherit the agent's filesystem changes 11 12.
How do you test a state checker?
Run the episode two ways: once with a correct but unusual path, once with an incorrect path that ends in a similar-looking state. The first should pass and the second should fail. State checkers in browser and desktop environments tend to check one field, one file, or one screen, and an agent that reaches the goal another way gets marked wrong.
An audit of 150 failure-scored computer-use trajectories from five web, enterprise-workflow and desktop benchmarks found 15.3% of FAIL verdicts wrong: evaluator false negatives accounted for 10.7% of the audited verdicts and broken tasks for 4.7% 13. Epoch AI's review of OSWorld 2.1 found errors in 36 of the 50 tasks it audited, with false negatives that could affect up to 25.4% of the audited score and false positives up to 17.6% 14. Sample failed trajectories by hand every time you change the checker, and keep the ones it misjudged as regression cases.
How do you test a model-based or rubric verifier?
Treat it as an LLM judge, because it is one. Calibrate it against human labels and report its true-positive and true-negative rates, then attack it with responses that have the right shape and the wrong content: long and confident, keyword-matched to the rubric, partially complete on compound criteria. Re-run the attack after training starts, since the policy will search for exactly these responses.
Rubric rewards make the attack surface explicit. Rubrics as Rewards showed that per-prompt rubrics can train models in domains without checkable answers 15. Mahmoud et al. then found that weak rubric verifiers in medical and science domains produced large proxy-reward gains that did not transfer to reference verifiers. Exploitation grew over training and concentrated in recurring failures: partial satisfaction of compound criteria, treating implicit content as explicit, and imprecise topical matching. Stronger verifiers substantially reduced the exploitation but did not eliminate it, and the authors propose a verifier-free diagnostic, the self-internalization gap, computed from the policy's log-probabilities 16. The calibration methods in LLM-as-judge calibration apply unchanged.
What should a verifier report contain?
A verifier report states the library, version and full configuration, the probe sets behind the numbers, the false-negative and false-positive rates with intervals by category, and when it was last run. Keep one per verifier version: without it, two training runs that used "the same verifier" are not comparable, and a reward curve cannot be read.
- The library, its version, and the full configuration, including extraction rules, tolerances, and any pre-normalization.
- The probe sets: how many positive rewrites and near-miss negatives, by category, and how they were built.
- The false-negative rate on positives and the false-positive rate on negatives, each with a confidence interval and a per-category breakdown.
- For execution verifiers, the wrong-solution acceptance rate and the gold-patch gate results.
- For model-based verifiers, agreement with human labels and the adversarial attack results.
- The date of the last run and the training step it was run at, so drift is visible.