trajectory and rollout

A trajectory is the story of one run: the agent starts somewhere, takes an action, sees the next state and reward, acts again, and on it goes, a chain of states, actions and rewards stretching from start to finish. A rollout is the act of generating such a story, letting a policy loose in the environment, or in a model of it, and recording everything that happens.

Trajectories are the raw data of reinforcement learning. Almost every algorithm learns by collecting rollouts and squeezing information from them: Monte Carlo methods sum the rewards along a trajectory to estimate value, policy gradients reweight whole trajectories by how much return they earned, and planners imagine rollouts inside a model before committing to a real move. Because the world is often stochastic, a single trajectory is noisy, so we usually average over many.