local versus global optimum

Stand in a small dip on a hillside: locally, you are at the bottom — every nearby step goes up. But across the whole mountain range there may be a far deeper valley you cannot see. A local optimum is best only compared to its immediate neighbourhood; a global optimum is the best over the entire feasible region. Telling them apart is one of the central difficulties of optimization.

Formally, a point is a local minimum if f there is less than or equal to f at all nearby points, and a global minimum if f there is less than or equal to f at every point in the domain. The calculus conditions — gradient zero, Hessian positive definite — are inherently local: they examine only an infinitesimal neighbourhood, so they can certify a local minimum but say nothing about whether a lower value exists elsewhere. A function can have many local minima of different depths; finding the global one in general requires either checking all of them, exploiting special structure, or settling for a good-enough local answer.

This gap is where real difficulty lives. Convexity is the great resolver: on a convex function over a convex set, every local minimum is a global minimum, so local methods solve the global problem. Off the convex world — in protein folding, neural-network training, combinatorial design — there is no general efficient guarantee of finding the global optimum, and practitioners use restarts, stochastic search, or simply accept a strong local solution. Being honest about the difference between can certify local and have found global is part of mature optimization practice.