locomotion tasks

Locomotion tasks ask an agent to learn how to move a body through space — to walk, run, hop, or gallop by coordinating the torques at many joints. They are the headline application of continuous control because they are so visceral: a simulated cheetah that starts as a flailing heap and gradually discovers a smooth run is one of RL's most satisfying demonstrations, and the achievement is real — no one wrote the gait, the agent found it.

What makes them hard is coordination under physics. The action is a high-dimensional vector of joint torques applied every few milliseconds; balance is precarious, a single misstep ends the episode, and the right movements emerge only from the interplay of many actuators with momentum, gravity, and ground contact. The reward is usually simple — forward velocity minus a small energy cost — yet from that thin signal the agent must discover the whole intricate rhythm of a gait.

Locomotion is where SAC, TD3, and PPO are routinely benchmarked, and progress here has crossed into the real world: learned controllers now run on physical quadrupeds and bipeds. It also exposes a recurring tension — agents readily find reward-hacking gaits that exploit simulator quirks rather than locomote the way you intended, a reminder that the reward, not your imagination, is what actually gets optimised.