AI Evals › Papers

Methodology and production

SPADE, RewardBench, MixEval, WildBench, IFEval, InfoBench, FollowBench, plus the 2025 agent-eval and observability work.

Twelve papers covering eval methodology and production observability. SPADE is the assertion-mining pattern that ties prompt versions to invariants. RewardBench, MixEval, and WildBench are the leaderboard and cost-correlation results to read before adopting any meta-benchmark. IFEval, InfoBench, and FollowBench are the instruction-following trilogy: atomic verifiable constraints, DRFR, and stacked difficulty. The 2025 work, the Yehudai survey, IBM's observability paper, and the data-centric position paper, are the references for the production end of the pipeline. ScienceAgentBench and SWE-bench Multimodal are pulled in here because their findings are about eval methodology as much as agent capability.

AI Evals

The most useful, opinionated reference for AI evals.

AI Evals is a curated technical reference for evaluating large language model systems in production. The site covers error analysis, LLM-as-judge calibration, RAG evaluation, agentic eval, statistical rigor, and twenty task-specific eval playbooks. Every claim is cited.