Multivariable Optimization

saddle point

Sit on a horse's saddle and notice its shape: it curves up toward the front and back, but down toward your left and right. A saddle point of a surface is exactly this geometry — a stationary point that is a minimum along some directions and a maximum along others, and therefore neither a true peak nor a true valley. Walk away one way and you climb; walk away another way and you descend.

Formally, a saddle is a point where the gradient vanishes (so it is stationary) but the Hessian is indefinite, having both positive and negative eigenvalues. The cleanest example is f(x, y) = x^2 - y^2 at the origin: along the x-axis the value rises like x^2, along the y-axis it falls like -y^2. The crossing point sits flat at first order yet is unstable — it is an equilibrium that a ball would roll away from. In two variables the second-derivative discriminant D = f_xx f_yy - (f_xy)^2 being negative is the tell-tale sign.

Saddle points matter precisely because they masquerade as optima. Setting the gradient to zero finds them right alongside genuine maxima and minima, which is why the classification step is essential. They are central in saddle-point methods for asymptotic integrals, in the theory of phase transitions, in game theory (a minimax equilibrium is a saddle of the payoff), and notoriously in deep learning, where the loss landscapes of large networks are riddled with saddles that can stall naive gradient descent far more often than local minima do.

A real mountain pass is a saddle of the elevation function: it is the lowest point along the ridge separating two valleys, yet the highest point along the trail that crosses from one valley to the other. Lowest one way, highest the other — that is what neither-max-nor-min means.

A mountain pass is the everyday saddle — minimal along the ridge, maximal along the crossing.

A saddle has a zero gradient just like a maximum or minimum, so the first-order condition cannot detect it — only the Hessian, or the second-derivative discriminant, exposes a saddle.

Also called
minimax point马鞍点馬鞍點