planning with a model

Planning is what you do once you have a model: you use it to look before you leap. Rather than committing to an action and seeing what happens, you ask the model to play out candidate courses of action in imagination, score the resulting futures by the reward they earn, and then take the action that the best imagined future started with.

Mechanically this turns control into a search or optimisation problem over the model. You might sample many random action sequences and keep the highest-scoring one, refine sequences with an optimiser, or grow a tree of futures with smarter lookahead. In every case the model supplies the predicted next states and rewards that let you compare options without paying the cost of trying them for real.

Planning's strength is that it can react to a freshly learned or changed model immediately — no slow policy retraining required. Its weakness is computation at decision time and total dependence on model quality: plan inside a wrong model and you will confidently choose a wrong action.