reward shaping

Reward shaping is the trick of sprinkling in small, helpful rewards along the way so a robot can learn a hard task without flailing in the dark. The trouble with a bare reward function is that for many jobs the robot only earns points at the very end — a maze solved, a peg inserted — and until then everything scores zero. By blind trial and error it could fumble for a very long time before stumbling onto that single payoff. Shaping hands out little hints — a tiny bonus for getting closer to the exit, for lining the peg up straighter — so the robot gets warmer-and-warmer feedback and climbs toward the goal instead of wandering blind.

The delicate part is adding those hints without secretly changing what the robot should ultimately do. Done carelessly, breadcrumbs can backfire: reward a robot just for moving toward a goal and it might learn to jog in place near the finish, milking the proximity bonus forever instead of crossing the line. The safe recipe, called potential-based shaping, only ever rewards genuine progress between states, which mathematically guarantees the best behavior stays exactly the same — the robot just finds it far faster. Shaping changes the speed of learning, not the destination.