Lagrange multiplier
/ Lagrange: luh-GRAHNZH /
Suppose you want the highest point of a hillside, but you are not free to wander — you must stay on a marked trail. The trail is a constraint, a curve g(x, y) = c you are forbidden to leave. The method of Lagrange multipliers is the elegant calculus for finding the best point of a function f subject to such an equality constraint, without ever solving the constraint for one variable in terms of another.
The geometric heart is a picture of touching level sets. As you walk along the constraint curve, f rises and falls; you stop improving exactly where the constraint curve is tangent to a level curve of f. At that tangency the two gradients point the same way (or opposite), so grad f is a scalar multiple of grad g: grad f = lambda times grad g. That scalar lambda is the Lagrange multiplier. Combined with the constraint g = c, this gives a system of equations whose solutions are the constrained candidates. A tidy bookkeeping device is the Lagrangian L = f - lambda(g - c); setting all its partial derivatives (including the one with respect to lambda) to zero reproduces both the gradient condition and the constraint at once.
Lagrange multipliers are everywhere in applied science. They give the equilibrium of thermodynamics (maximizing entropy at fixed energy), the optimal allocation of a fixed budget in economics, the shape of a hanging chain, and the foundations of constrained machine learning. And the multiplier is not mere scaffolding: lambda equals the rate at which the optimal value of f changes as you loosen the constraint level c — its shadow price, or marginal value, a quantity of deep practical meaning.
Maximize f(x, y) = xy subject to x + y = 10. The condition grad f = lambda grad g reads (y, x) = lambda(1, 1), so x = y; the constraint then gives x = y = 5 and the maximum product is 25. The multiplier here is lambda = 5.
The objective gradient parallel to the constraint gradient pins down the optimal point.
The method finds candidates where the gradients are parallel, but does not by itself say which is a max, a min, or neither — you still compare the values, or use the bordered Hessian, to decide.