learned dynamics model

A learned dynamics model is the agent's best guess at the physics of its world, built from data rather than handed down. Show it where you are and what you do, and it predicts where you will end up and what reward you will get. It is the engine inside every model-based method: nothing can be planned or imagined without something that says what happens next.

In practice it is a function — often a neural network — trained on collected transitions to predict the next state and reward from the current state and action. Training is ordinary supervised learning: minimise the gap between the model's prediction and what actually happened. The model can be deterministic (one predicted next state) or probabilistic (a distribution over next states), and it may predict in the raw observation space or in a compressed latent space.

The hard truth is that a dynamics model is only ever an approximation, accurate near the data it saw and unreliable far from it. Planning tends to seek out exactly those unreliable regions, so a good model must also signal its own uncertainty rather than confidently making things up.