the second-order optimality condition

Knowing the ground is flat tells you that you are at a special spot, but not WHICH kind. Are you at the bottom of a bowl, the top of a dome, or on a saddle that curves up one way and down another? To tell, you check how the ground bends around you: bowed upward in every direction means a true valley floor. The second-order optimality condition is exactly this curvature check, and the curvature is captured by the Hessian matrix.

The Hessian H is the matrix of all second partial derivatives, H_ij = d^2 f / (dx_i dx_j); it records how the gradient itself changes as you move, i.e. the curvature in every pair of directions. At a stationary point x* (where grad f = 0): if H is POSITIVE DEFINITE — meaning v^T H v > 0 for every nonzero direction v, equivalently all its eigenvalues are positive — then x* is a strict local minimum (the surface curves up in every direction). If H is NEGATIVE definite (all eigenvalues negative), x* is a local maximum. If H has both positive and negative eigenvalues, x* is a saddle point. The necessary version says a local minimum must have H positive SEMIdefinite (eigenvalues >= 0); the sufficient version needs strict positive definiteness.

This matters because it is the rigorous way to confirm a minimum, and the Hessian's eigenvalues also reveal the problem's CONDITIONING: the ratio of the largest to the smallest eigenvalue is the condition number of the optimization, and a large ratio (a long, thin valley) is what makes plain gradient descent zig-zag and crawl. The honest catch is cost: forming and factoring the full n-by-n Hessian costs about O(n^3) work and O(n^2) storage, infeasible when n is in the millions, which is precisely why quasi-Newton methods approximate curvature from gradients instead of computing H directly. The borderline case — H positive semidefinite but singular (a zero eigenvalue) — is genuinely inconclusive and needs higher derivatives to resolve.