the dynamic programming principle
The dynamic programming principle (DPP) is the single idea that makes sequential decision-making under uncertainty tractable. Bellman stated it as the principle of optimality: an optimal policy has the property that, whatever the initial state and first decision, the remaining decisions must form an optimal policy for the state that results. In plain words — a tail of an optimal plan is itself optimal. This lets you replace one impossibly large optimisation over whole strategies by a sequence of small optimisations over a single next step, each one trusting the value function to summarise everything that comes after.
Formally, for a controlled diffusion with value function V(t,x), the DPP says that for any intermediate (stopping) time theta in [t, T], V(t,x) = inf_u E[ integral_t^theta f(X_s, u_s) ds + V(theta, X_theta) given X_t = x ]. Read it carefully: you optimise the cost accumulated on the short interval [t, theta], and at time theta you stop accounting in detail and simply charge yourself the value function V(theta, X_theta) — because from theta onward you will, by assumption, behave optimally, and V already encodes that best continuation. The proof has two halves: '<=' (any control is no better than acting optimally on the tail) and '>=' (you can paste together near-optimal tail controls measurably, which needs a measurable-selection argument). The DPP holds for very general (Markovian) problems and is the substrate from which the HJB equation is derived by sending theta to t and using Ito's formula.
The DPP is why optimal control is computable at all: backward induction in discrete time, HJB in continuous time, value iteration and Q-learning in reinforcement learning are all incarnations of it. An honest caveat: the abstract DPP is intuitively obvious but technically delicate — the '>=' direction requires that you can select near-optimal controls in a measurable, adapted way (measurable selection theorems), and the conditional-expectation manipulations require integrability. It also presumes the problem is genuinely Markovian (the value depends on the current state, not the full history); for non-Markovian costs or partial observation one must enlarge the state (e.g. to the filter's conditional distribution) before the DPP applies.
Shortest path through a grid of cities with random travel times: instead of enumerating all routes, define V(city) = expected minimum remaining cost to the destination. Then V(c) = min over neighbours c' of [ E(cost c -> c') + V(c') ]. Solving this Bellman equation backward from the destination gives the optimal next move at every city — the DPP turns an exponential search into a linear sweep.
Bellman's principle: the optimal next move only needs the value function of where you'd land, not the whole future plan.
The DPP looks obvious but its rigorous proof (the '>=' half) needs measurable selection of near-optimal controls and integrability; and it requires a Markovian state — partial observation forces you to lift the state to the conditional distribution (the filter) first.