MuJoCo benchmarks

MuJoCo is a fast, accurate physics simulator (the name stands for Multi-Joint dynamics with Contact), and the standard set of tasks built on it has become the de facto proving ground for continuous-control algorithms. These are the simulated creatures you will see in every paper: HalfCheetah, Hopper, Walker2d, Ant, Humanoid. Each gives the agent a body with joints, an observation of positions and velocities, and a reward for moving forward without falling over.

Their value is comparability. Because everyone runs the same simulated robots with the same reward definitions, a new algorithm can be measured against DDPG, TD3, SAC, and PPO on identical ground, and the learning curves are directly comparable. They are hard enough to separate good methods from bad — Humanoid in particular is a genuine challenge — yet cheap and fast enough to run thousands of times, which made the rapid progress of deep continuous control possible.

The caveat, worth stating honestly, is that they are simulations with clean observations, no sensor noise, and reward functions hand-tuned to be learnable. Strong MuJoCo scores do not guarantee a method transfers to a real robot, where contact is messy and perception is hard. They measure algorithmic progress on the control problem, not readiness for the physical world — which is why sim-to-real is its own separate concern.