Multimodal

MMBench, MM-Vet, MMMU, BLINK, Vibe-Eval, Video-MME, MMMU-Pro, ChartQA.

Eight papers covering the canonical multimodal benchmarks. MMBench's circular eval is the position-bias control to copy. MMMU and MMMU-Pro are the standard college-level VQA pair, with MMMU-Pro tightening the loop by stripping text-only-solvable questions. BLINK is the result to cite when someone says perception is solved. Video-MME and Vibe-Eval push into video and harder prompts. ChartQA remains the chart-reasoning baseline worth re-verifying before publication.

AI Evals

The most useful, opinionated reference for AI evals.

AI Evals is a curated technical reference for evaluating large language model systems in production. The site covers error analysis, LLM-as-judge calibration, RAG evaluation, agentic eval, statistical rigor, and twenty task-specific eval playbooks. Every claim is cited.

Why evals matter
Your first eval in 30 minutes
LLM-as-Judge introduction
RAG eval in two layers
Trajectory eval for agents
Glossary
Papers