the stochastic maximum principle
/ PON-tree-AH-gin /
The stochastic maximum principle is the second great route to a stochastic control problem, parallel to and dual with dynamic programming. Where dynamic programming asks for the value function over the WHOLE state space (a PDE in many variables), the maximum principle follows a SINGLE optimal trajectory and characterises the optimal control pointwise along it, via an extra adjoint (costate) process. It is the stochastic generalisation of Pontryagin's deterministic maximum principle, and it is the natural language when the state space is high-dimensional (where solving an HJB PDE is hopeless) or when the noise is control-dependent in ways that break dynamic programming's Markovian assumptions.
The construction introduces a Hamiltonian and an adjoint process. For dX = b(X,u) dt + sigma(X,u) dB with cost integral f + g, define the Hamiltonian H(x,u,p,q) = b(x,u) . p + trace( sigma(x,u)^T q ) + f(x,u). The first-order adjoint process (p_t, q_t) solves a backward stochastic differential equation (a BSDE) running backward from the terminal condition p_T = grad g(X_T): dp_t = - grad_x H(X_t, u_t, p_t, q_t) dt + q_t dB_t. The crucial extra unknown is q_t — it is forced into existence by the requirement that p_t be ADAPTED despite its terminal condition being given at time T, and it is exactly the martingale-representation integrand. The principle then states: along the optimal control u*, the Hamiltonian is minimised (for cost minimisation) pointwise, u*_t = argmin_u H(X_t, u, p_t, q_t), almost surely for almost every t. For convex problems this necessary condition is also sufficient.
The maximum principle matters because it scales where HJB does not, it handles general (non-Markovian, random-coefficient) problems naturally, and it exposes the deep duality: the adjoint p_t is, when V is smooth, exactly the gradient of the value function, p_t = grad_x V(t, X_t), so the two methods are two faces of the same answer. An honest caveat: the second-order term — the q_t process and, in the general (non-convex Hamiltonian, control-dependent diffusion) case, a SECOND adjoint process and a second-order correction term — is essential and is what makes the STOCHASTIC maximum principle genuinely harder than the deterministic one; dropping it gives a wrong condition. Also the principle gives a NECESSARY condition for optimality; sufficiency requires convexity (e.g. H convex in (x,u) and g convex), and existence of an optimal control is a separate question.
For the LQG problem the adjoint becomes linear: with quadratic cost, p_t turns out to be p_t = P(t) X_t for the Riccati matrix P(t), and the Hamiltonian-minimisation step grad_u H = 0 gives R u + B^T p = 0, i.e. u* = -R^{-1} B^T P X — the very same linear feedback the HJB/Riccati route produced. The two methods agree, with p_t = grad_x V making the duality explicit.
In LQG the adjoint is P(t)X_t and the maximum principle reproduces the Riccati feedback — the two routes meet.
The principle is NECESSARY, not automatically sufficient — sufficiency needs convexity. And the second-order (q_t and, in general, a second adjoint) term is essential; omitting it gives a wrong optimality condition, which is what distinguishes the stochastic from the deterministic principle.