Hessian matrix

The gradient captures how a surface tilts; the Hessian captures how it curves. It is the multivariable second derivative: a square matrix of all the second-order partial derivatives, which records the bending of the function in and between every pair of directions. Where one-variable calculus has a single second derivative telling concave-up from concave-down, several variables need a whole matrix because curvature now depends on direction.

For f(x, y) the Hessian is the 2-by-2 matrix [partial^2 f / partial x^2, partial^2 f / (partial x partial y); partial^2 f / (partial y partial x), partial^2 f / partial y^2]. The diagonal entries are the pure curvatures along each axis; the off-diagonal entries are the mixed partials that record how the slope in one direction changes as you move in another. By Clairaut's theorem the mixed partials are equal whenever they are continuous, so the Hessian is symmetric in practice. A symmetric matrix can be classified as positive definite, negative definite, or indefinite, and that classification is exactly what tells a critical point apart as a local minimum (bowl up), local maximum (bowl down), or saddle (up one way, down another).

The Hessian is the engine of the second-derivative test and of second-order Taylor expansion, and it is central to fast optimization: Newton's method for minimization uses the inverse Hessian to leap toward a critical point far faster than gradient descent, and the eigenvalues of the Hessian set the conditioning that determines how hard a problem is to minimize. In physics it appears as the matrix of force constants near equilibrium, whose eigenvalues give the squared normal-mode frequencies of a vibrating system.