A policy is a habit of acting
When a robot learns, the thing it actually learns is a policy: a rule that looks at what it currently senses and decides what to do next. Feed it a situation, and it hands back an action. That is the whole job. You can think of a policy as the robot's habit — not a single clever move, but a standing answer to the question "given what I see right now, what should my motors do?"
Picture a robot told to tidy a child's room. The state is everything it can observe: where the blocks are, where the basket is, where its own gripper sits. The action is the next small move — reach left, close the fingers, drop. A policy is the mapping from each state to the next action, applied over and over until the room is clean. Crucially, the policy does not memorize one room; it learns a habit that works across many messy rooms.
The reward turns a goal into a number
A policy by itself has no opinion about whether its habit is any good. That judgment comes from the reward function: a number, handed to the robot after each move, that says how well things just went. Picking up a block and dropping it in the basket might earn +1; knocking the basket over might earn -5. The robot's entire goal collapses into one instruction — act so that the rewards, added up over time, are as large as possible.
This is where it gets surprisingly hard. The reward is the only place a human gets to say what "good" means, and the robot will chase that number with no common sense whatsoever. Reward the robot for every block placed in the basket, and a clever learner might discover it can take a block out and put it back in forever, racking up points while the room stays a mess. The robot did exactly what you scored — just not what you meant.
It helps to notice that a reward is the learning-world cousin of the error signal in classical control. A controller is told the exact target and pushes the error to zero; a reward only whispers "warmer" or "colder" and lets the robot figure out the target for itself. That freedom is powerful for tasks too messy to write down — and dangerous when the score and the true goal quietly drift apart.
Reward shaping: leaving a trail of crumbs
Suppose you only reward the robot when the room is completely clean. Early on it flails about almost at random, and a fully tidy room may take thousands of lucky moves to stumble into. Until that happens the reward is zero, so there is nothing to learn from. This is the needle-in-a-haystack problem: the goal is so rare that the robot almost never sees the signal that would teach it.
Reward shaping is the fix: you add small, helpful hints along the way so the robot is not learning in the dark. Give a little reward for each block that ends up closer to the basket, a little more for actually dropping one in, and the big prize for finishing. Now every sensible move earns a crumb of feedback, and the robot can follow the trail of crumbs toward the goal instead of waiting for one rare jackpot.
Exploration versus exploitation
Even with a good reward and helpful crumbs, the robot faces a constant dilemma known as the exploration–exploitation tradeoff. Exploitation means doing the thing that has worked best so far — cashing in. Exploration means trying something new and untested, on the chance it works even better. Do too much of either and learning stalls.
Think of choosing where to eat. Exploitation is going back to the one restaurant you already like; you know it is decent. Exploration is trying the new place down the street; it might be your new favorite, or it might be terrible. If you always exploit, you never discover anything better. If you always explore, you waste every meal on gambles and never enjoy the good ones you found.
A common recipe is to explore boldly at the start, when the robot knows almost nothing, then gradually lean toward exploitation as a good policy emerges. This is also why so much robot learning happens in simulation first: a reinforcement learning agent can take wild, exploratory chances in a simulator where a fall costs nothing — then bring the polished policy back to the real, breakable machine.
Putting the pieces together
Step back and the loop is simple. The policy proposes an action, the world responds, the reward scores the result, and the robot nudges its policy to earn more reward next time. Shaping makes the scores informative enough to follow; the exploration–exploitation balance decides whether the robot dares to look for something better. Round and round, the habit improves.
Not every robot learns from reward alone. Sometimes it is faster to simply show the robot what to do and have it copy you — a family of methods covered separately under imitation learning and behavior cloning. But even there, policies and rewards lurk underneath: the demonstration shapes a policy, and somewhere a notion of "good" decides whether the copy is faithful enough. Master these two ideas and the rest of robot learning has a backbone to hang on.