model predictive control (in RL)

Model predictive control is planning that never fully commits to a plan. At each step the agent looks a short distance into the future with its model, works out the best sequence of actions over that window, executes only the very first action, and then throws the rest away and replans from scratch on the next step. It is plan-act-replan, over and over.

The repeated optimisation is over a finite horizon: maximise the sum of predicted rewards across the next H steps, possibly plus an estimated value at the horizon's end to account for the rest of the future. A common RL flavour is sampling-based MPC, where you draw many random action sequences from the model, evaluate each, and shift the next plan toward the best ones — the cross-entropy method and random shooting are popular variants.

Constant replanning is what makes MPC robust: because only the first action is ever used and the model is re-queried from the true current state every step, prediction errors are corrected before they can compound. The cost is solving a fresh optimisation problem at every single decision.