the verification theorem
The verification theorem closes the logical loop in dynamic programming. The HJB equation is a NECESSARY condition: IF the value function is smooth, THEN it solves HJB. But to actually solve a control problem you usually work backwards — you guess or compute a smooth solution W of the HJB equation, derive a candidate feedback control from it, and then you need to PROVE that W really is the value function and that the candidate control really is optimal. The verification theorem is exactly that converse: it gives sufficient conditions under which a smooth solution of HJB, together with the maximiser inside it, certifies optimality.
Here is the statement in spirit. Suppose W(t,x) is a sufficiently smooth (C^{1,2}) function satisfying the HJB equation with the right terminal condition W(T,x) = g(x), and let a(t,x) be a measurable control attaining the sup/inf in HJB at every point. Then: (1) for ANY admissible control u, applying Ito's formula to W(t, X_t) shows W(t,x) <= J(t,x; u) (W is a lower bound on the cost of every strategy — this is where HJB's inequality form -V_t - L^u V - f <= 0 for all u is used, making W(s, X_s) + accumulated cost a submartingale); and (2) for the candidate control a, the same computation holds with equality, so W(t,x) = J(t,x; a). Together these force W = V (the value function) and a = the optimal control. The proof is a clean Ito-plus-martingale argument; the heart is that the drift of W(t, X_t) under any control is nonpositive and is zero under the optimal one.
The verification theorem is what makes the whole HJB programme rigorous rather than heuristic: it is the certificate that turns 'a function that satisfies the equation' into 'the answer'. An honest caveat, and the reason this is its own theorem: it ASSUMES a smooth (C^{1,2}) solution exists, plus integrability / transversality conditions ensuring the local martingale arising from Ito is a true martingale (so that its expectation vanishes — a genuine local martingale here would break the bound). When the value function is not smooth (the common case), the classical verification theorem does not apply and one needs the viscosity-solution machinery or a smooth-fit verification tailored to the free boundary. So a successful verification is also an a-posteriori proof that the value function happened to be smooth.
In the LQG regulator you guess W(t,x) = x^T P(t) x for a symmetric matrix P(t). Plugging into HJB and doing the sup over the control gives the optimal linear feedback u* = -R^{-1} B^T P x and reduces HJB to the matrix Riccati ODE for P(t). The verification theorem then confirms: x^T P(t) x IS the value function and u* IS optimal — because P from the Riccati equation makes the verification inequalities hold with equality along u*.
Verification in action: a quadratic ansatz solves HJB, and the theorem certifies it is genuinely optimal.
Verification ASSUMES a smooth C^{1,2} solution of HJB plus integrability ensuring the Ito local martingale is a true martingale. When the value function is non-smooth (typical), classical verification fails and you need viscosity solutions or smooth-fit arguments.