world models

A world model is an agent's compact, generative imagination of its entire environment — not just one-step physics, but a simulator rich enough to dream whole episodes. The vivid version comes from high-dimensional inputs like pixels: the agent learns to compress what it sees into a small internal code, then learns how that code evolves over time, so it can roll the movie of the future forward inside its own head.

Such a model usually has two pieces. A representation model encodes raw observations into a low-dimensional latent state, and a transition model predicts how that latent state changes given an action, alongside a head that predicts reward. Once trained, the world model becomes a fast private simulator: the agent can generate enormous amounts of imagined experience and learn a policy almost entirely inside it, touching the real environment only to keep the model honest.

World models matter because real interaction is the scarce resource — a learned simulator turns a trickle of real data into a flood of practice. The danger is that the agent can overfit to its own dreams, exploiting flaws in the imagined world that would never pay off in reality.