the Hamilton-Jacobi-Bellman equation
/ HAM-il-ton ya-KOH-bee BELL-man /
Suppose you are steering something — a rocket, a portfolio, a thermostat — and at every instant you choose a control to keep costs low over the whole journey ahead. How should you choose now, when each choice changes where you end up later? Richard Bellman's principle of optimality says: whatever you do first, the rest of the plan must itself be optimal from wherever you land. Turning that principle into calculus gives the Hamilton-Jacobi-Bellman equation, the master PDE of optimal control and dynamic programming.
The central object is the value function V(x, t): the best possible remaining cost if you are at state x at time t and play optimally from there on. Bellman's principle, made infinitesimal, becomes a PDE for V. For a system dx/dt = f(x, a) with running cost L(x, a) (where a is the control you choose from a set A), the HJB equation is -V_t + max over a in A of [ -f(x, a) . grad V - L(x, a) ] = 0, often written V_t + H(x, grad V) = 0 with H the Hamiltonian formed by optimizing over the control. It is a nonlinear first-order PDE (a Hamilton-Jacobi equation) precisely because of that max/min over controls. Solve it once for V on the whole state space, and you read off the optimal control everywhere: at each (x, t) pick the action a that achieves the maximum — that is the feedback law. The catch is that V is generically NOT smooth; it has corners. So HJB is understood in the VISCOSITY-solution sense, which is exactly the framework built to make such equations well-posed.
Why is it central? HJB unifies optimal control, differential games, and (in a stochastic version with a second-order diffusion term) much of mathematical finance — the Black-Scholes equation and Merton's portfolio problem are HJB equations. It converts a hard infinite-dimensional optimization-over-trajectories into a single PDE for a function on state space. The honest caveats are two: the value function is usually non-smooth, so you genuinely need viscosity solutions for uniqueness; and solving HJB in high dimension suffers the 'curse of dimensionality' (cost grows exponentially with the number of state variables), which is why approximate and learning-based methods matter so much in practice.
To bring a cart to rest at the origin in least time, with bounded acceleration |a| <= 1, the HJB equation says the value function V(x) (minimum time from x) satisfies a first-order PDE whose optimization over a yields 'push full throttle toward the target'. The solution is the famous bang-bang control: always accelerate at the maximum in the right direction, switching sign exactly once. HJB delivers both the minimum time and the feedback rule.
Bellman's optimality principle, made into a PDE for the value function.
The value function is usually NOT classically differentiable, so HJB cannot be solved in the ordinary sense — viscosity solutions are what make it well-posed and unique. Also beware the curse of dimensionality: directly solving HJB becomes infeasible as the number of state variables grows.