latent dynamics model

When observations are huge and messy — a stream of camera frames, say — predicting the next raw image pixel by pixel is wasteful and brittle. A latent dynamics model sidesteps this by first squeezing each observation into a small vector of essential features, the latent state, and then predicting the future in that compact space instead of in the original pixels.

It is built from an encoder that maps observations to latent states and a transition function that, given the current latent state and an action, predicts the next latent state and reward — all learned jointly. Planning and policy learning then happen entirely on these small vectors, which is far cheaper and forces the model to keep only what matters for control. A decoder may be trained alongside so the latent state must retain enough to reconstruct what was seen.

The payoff is that an agent can imagine thousands of futures quickly, because each step is a tiny vector update rather than a full image prediction. The risk is that the learned features might discard something the task secretly needed, or drift in ways that are hard to inspect because the space is not human-readable.