model-based reinforcement learning

Imagine a chess player who, before touching a piece, plays the next few moves out in their head. Model-based RL gives an agent that same inner sandbox. Instead of learning only a reflex that maps situations to actions, the agent learns a model of how the world responds — what state likely comes next and what reward it brings — and then uses that model to think ahead before acting.

Concretely the agent fits a function that approximates the environment's transition and reward dynamics from experience. With that model in hand it can do something model-free methods cannot: generate imagined experience and plan, either by simulating many possible futures and picking the best opening action, or by training a policy on cheap synthetic rollouts. Because each real interaction also teaches the model, every sample gets reused many times.

The promise is dramatic sample efficiency, which matters when real data is slow or costly, as on a physical robot. The price is that a flawed model can mislead planning, so the central craft of model-based RL is learning models that are accurate where it counts and staying honest about where they are not.