Monte Carlo control

Prediction tells you how good a policy is; control is about making it better. Monte Carlo control loops between two moves: play full episodes and average their returns to estimate how good each action is, then quietly switch to preferring the actions that look best. Repeat, and the policy ratchets upward — the same generalized policy iteration idea used everywhere in RL, but powered entirely by sampled experience instead of a known model.

One twist matters: to improve a policy without a model, you must estimate action-values Q(s,a), not just state-values, because you need to compare actions directly. And greedily chasing the current best would stop you from ever discovering better options, so control keeps exploring — through exploring starts, or by staying soft with an epsilon-greedy policy — so every action keeps getting sampled and re-evaluated.