the Hamilton-Jacobi-Bellman equation
/ HAM-il-tun ya-KOH-bee BEL-man /
The Hamilton-Jacobi-Bellman (HJB) equation is the differential form of the dynamic programming principle: it is the partial differential equation that the value function of a stochastic optimal control problem must satisfy. Where the DPP relates the value over an interval to the value at the interval's end, HJB shrinks that interval to zero and reads off a pointwise equation in time and state. It converts an optimisation over strategies (an infinite-dimensional object) into a PDE (a local condition you can in principle solve or discretise), and it is the central computational tool of continuous-time control.
For the controlled diffusion dX = b(X,u) dt + sigma(X,u) dB with running cost f and terminal cost g, the value function V(t,x) satisfies, for t < T, the HJB equation -V_t + sup_u { - L^u V - f(x,u) } = 0 (for maximisation; replace sup by inf and signs accordingly for cost minimisation), with terminal condition V(T,x) = g(x). Here L^u is the generator of the diffusion under the fixed control value u: L^u V = b(x,u) . grad_x V + (1/2) trace( sigma sigma^T(x,u) Hess_x V ). The derivation is direct: apply Ito's formula to V(t, X_t), take expectations (the martingale part vanishes), and impose the DPP — the supremum/infimum over u of the resulting infinitesimal drift must be zero along the optimal path. The optimal feedback control is the maximiser/minimiser inside the sup/inf, a(t,x) = argmax_u { -L^u V(t,x) - f(x,u) }, read off pointwise from the gradient and Hessian of V.
HJB is the bridge from probability to nonlinear PDE: it is a second-order, fully nonlinear equation (nonlinear because of the sup over u), and its solution feeds back to define the optimal policy. An honest and important caveat: classical (twice-differentiable) solutions of HJB often do NOT exist — the value function can be merely continuous, with kinks. The correct general notion is the viscosity solution (Crandall-Lions), under which V is THE unique solution of HJB with the given boundary data even when it is not differentiable. So 'V solves HJB' is true in the viscosity sense in general; a classical solution (and the clean verification theorem) is available only under regularity, typically uniform ellipticity (nondegenerate sigma sigma^T) plus smooth coefficients. When the diffusion is absent (deterministic control) HJB reduces to the first-order Hamilton-Jacobi equation, whose shocks are exactly why viscosity solutions were invented.
Merton's problem again: plug a power-utility ansatz V(t,w) = h(t) w^(1-gamma)/(1-gamma) into HJB. The sup over the stock fraction pi is a simple quadratic in pi, maximised at pi* = (mu - r)/(gamma sigma^2). Substituting back leaves an ODE for h(t), solved in closed form. HJB has turned a control problem into 'differentiate a quadratic, then solve one ODE'.
Solving HJB by ansatz: the sup over the control becomes a pointwise maximisation, leaving an ODE for the time factor.
Classical solutions of HJB frequently do not exist; the value function is the unique VISCOSITY solution. A smooth solution requires nondegeneracy (uniform ellipticity) and regularity — only then does the verification theorem apply directly.