The Loop: Observe, Act, Get Rewarded, Repeat
Imagine teaching someone to ride a bike without ever explaining how. You just let them pedal, watch them wobble, and shout "good!" when they stay upright a little longer. After enough tries, their body figures out the rest. That is the spirit of reinforcement learning (RL): instead of programming a robot step by step, you let it learn from the consequences of its own actions.
The whole process is a tight loop that repeats over and over. The robot observes its situation (where its joints are, how fast its body is tipping), acts by sending commands to its motors, and then gets a reward — a single number that says how good that moment was. Then it observes again, and the loop turns. Each turn is a tiny experiment.
The thing that decides which action to take from each observation is the control policy — think of it as the robot's reflexes. At first the policy is random and clumsy. The reward signal slowly reshapes it: actions that led to higher rewards become more likely, and the rest fade away.
A Worked Example: A Quadruped Learns to Walk
Let's make this concrete with a four-legged robot — a dog-sized machine attempting quadrupedal locomotion. We want it to walk forward smoothly without falling. We will not hand-write a gait; we will let RL grow one. The two ingredients from earlier — reward and policy — now come together.
Everything hinges on the reward function. We give the robot a positive reward for moving forward, a small penalty for jerky motion or wasted energy, and a big penalty for falling over. That single recipe, applied millions of times, is what "teaches" walking — nobody describes a stride.
- Observe: read joint angles, body tilt, and velocity into the policy.
- Act: the policy outputs target torques for all twelve leg joints.
- Score: the reward function adds up forward progress minus penalties for this instant.
- Update: nudge the policy so high-reward actions become more likely; then repeat.
Early on the robot flops and trips constantly — and that is fine. The crashes are data. A central tension shapes the run: the exploration–exploitation tradeoff. The robot must keep trying odd new actions (explore) to discover something better, while also leaning on what already works (exploit). Lean too far either way and learning stalls.
Curriculum Learning: Start Easy, Ramp Up
Here is a trap. If you drop the untrained robot straight onto rocky terrain at high speed, it will fall on every single try. The reward is always terrible, so there is nothing to learn from — every attempt looks equally bad and the policy never gets a foothold. Training stalls before it starts.
The fix is curriculum learning: teach in the same order a good coach would. Start on flat ground at a slow target speed, where even clumsy attempts occasionally earn a little reward. Once the robot is steady, gently raise the difficulty — faster speeds, then slopes, then rubble. Each stage is reachable from the last, so the policy always has a slightly better action within reach.
It is the same reason we teach children to add before we teach algebra. Skills stack. A robot that first masters balance on flat ground carries that balance into rougher worlds — it does not have to relearn how to stand up every time the floor changes.
Why Not Just Learn on the Real Robot?
By now an obvious question looms: if trial and error works, why not run those millions of tries on the actual machine? The honest answer is that pure trial-and-error on real hardware is painfully slow and genuinely risky.
Slow, because the real world runs at one second per second — a few million trials could take months. Risky, because a learning robot fails on purpose: it falls, slams its joints, and tugs at its cables. Real motors overheat and gears wear out, and an early policy might lurch in a way that damages the robot or hurts a bystander. You cannot fast-forward physics, and broken hardware is expensive.
This is exactly why most robot RL today happens inside a robot simulator — a physics video game where a thousand virtual robots can stumble in parallel, faster than real time, with no broken parts. The catch is that simulators are never perfect copies of reality, so a policy that walks beautifully in sim can stagger on the real robot. Closing that gap is the job of sim-to-real transfer, which the next guide takes on.