How RL Differs: From Supervised Learning to Planning

Not supervised learning

If you've met machine learning before, it was probably supervised learning: a dataset of inputs paired with the correct labels, and a model trained to copy those labels. RL is a different animal. The contrast — supervised vs reinforcement learning — comes down to what feedback you get. Supervised learning is told the right answer; RL is told only how good its own answer turned out to be.

That single difference ripples outward. A supervised model can learn from a fixed pile of examples that exists before training. An RL agent must generate its own data by acting, and the data it sees depends on the very behaviour it is still learning — a moving target with no fixed answer key.

Learning by consequences, with no teacher

Because there's no answer key, the agent is back to trial and error: the only way to find out whether an action is good is to try it and watch the reward. This is liberating and dangerous at once. Liberating, because the agent can invent moves no one taught it. Dangerous, because it can only learn about actions it actually takes — and it might never take the action that would have been best.

The explore-or-exploit dilemma

That danger has a famous name: the exploration–exploitation trade-off. At every step the agent faces a quiet dilemma. Exploit: do the thing that has worked best so far, and bank a reliable reward. Explore: try something new and unproven, which might be worse — or might be much better and reveal an option you'd otherwise never find. Lean too far toward exploiting and you get stuck in mediocrity; explore too much and you waste your life sampling bad ideas.

There's no free lunch here — every learning agent, and arguably every person, must strike some balance. A common-sense recipe is to explore a lot early on, while you know little, then gradually shift toward exploiting as you grow confident about what works. Whole tracks later in this domain are devoted to doing this well.

RL vs planning: learning the rules or knowing them

RL is also worth contrasting with classical planning. In planning you already have a perfect model of how the world works — the rules of chess, say — and you reason ahead through possible futures to pick a move. In RL versus planning, the defining difference is knowledge of the rules: a pure RL agent does not start with a model. It must learn how the world responds purely from the experience of bumping into it.

The line isn't a wall, though. An agent can learn a model of the world and then plan inside it — a powerful hybrid we'll explore in the model-based tracks. For now, just hold the two poles in mind: planning knows the rules and thinks; learning doesn't know them and experiments.

Episodes vs continuing tasks

One last distinction shapes how we even define success: does the task end? An episodic task has natural finish lines — a game ends, a maze is solved, a robot drops the cup — and then the world resets and a fresh episode begins. A continuing task never stops: a server balancing traffic, a thermostat, a trading system simply runs on and on with no reset in sight.

The discounted return sums future rewards, with the discount factor γ keeping the total finite even over an endless, continuing stream.