An RL environment is an eval with a gradient attached. It holds a set of tasks, a harness that lets the model act, and a verifier that scores the result, and the score becomes the reward a training run optimizes. Prime Intellect's docs put it plainly: RL environments and agent evals are basically the same thing, a dataset plus a harness plus scoring rules [1]. So every flaw this site teaches you to find in a benchmark (impossible tasks, wrong answer keys, graders that can be gamed) does more damage inside an environment, because the model is trained to find it.
The industry treats these environments as a product now. Epoch AI's January 2026 interviews put contracts at six to seven figures per quarter and tasks at $200 to $2,000 each, and interviewees consistently named robustness against reward hacking as the most important quality criterion [2]. Open hubs and frameworks give you the same parts for free: OpenEnv, Prime Intellect's verifiers and Environments Hub, NVIDIA NeMo Gym, Harbor, and Gymnasium for classic control [3] [4]. Audits keep finding the same problems in environments and their verifiers. One 2026 audit of 1,968 terminal-agent tasks found 16% hackable by frontier models given only the task description [5], and a verifier audit found two configurations of Hugging Face's Math-Verify disagreeing on 49.9% of answer pairs [6].
The frame for this section: before you train on an environment, evaluate it the way you would evaluate a judge. Measure what it gets wrong, then decide whether the error rate is one you can train through.
Chapters:
- What is an RL environment?. Tasks, harness, verifier; how an environment differs from a gym, a sandbox and a benchmark; RLVR vs RLHF; what open models trained on in 2025 and 2026.
- How to evaluate an RL environment. Ten checks before you train on or buy one: solvability, difficulty spread, verifier error, hackability, isolation, reliability, contamination, plus a rollout audit script.
- Testing RL verifiers. Rule-based, execution and model-based verifiers, a metamorphic test set, and measured false negatives and false positives for Math-Verify 0.9.0.
- How to evaluate a reward model. What RewardBench 2, PPE, RM-Bench and the process-reward benchmarks measure, how well they predict RL results, and a length-bias check for your own pairs.
- Evaluating a model after RL training. pass@k against the base model, random-reward controls, contamination tests, seed and backend variance, and the side effects to track.
- RL gyms and environment hubs compared. OpenEnv, verifiers, NeMo Gym, Harbor, Gymnasium, Reasoning Gym and the trainers that consume them, plus where managed RL fine-tuning stands in November 2026.
Reward hacking in deployed agents has its own chapter under Agentic Evals, and pass^k and consistency covers the reliability side of sampling.