dynamic programming (in RL)

When you completely know the rules of a game — the probability of every outcome and every reward — you do not have to play to work out the best strategy. Dynamic programming is the family of methods that compute optimal behaviour by reasoning directly over the known model, breaking one big question ("what is this state worth?") into smaller ones ("what about the states I can reach next?").

In reinforcement learning, dynamic programming means using the Bellman equations as update rules. You sweep over every state and, for each, back up the value of its successors to get a better estimate of its own value. Repeating these sweeps drives the estimates to the true value function; acting greedily on them then gives the optimal policy. Policy iteration and value iteration are the two classic DP algorithms.

The catch is that dynamic programming needs the full model and a pass over every state, so it is more the gold standard you compare against than something you can run on a real robot. Much of the rest of RL is about recovering DP's answers from samples when the model is unknown.