a Lagrange multiplier
/ luh-GRAHNZH /
You want to climb to the highest point of a hill, but a fence restricts you to walking along one fixed path. The summit you reach is where the path runs level with the contour lines of the hill — where moving along the path no longer takes you higher. The Lagrange multiplier is the elegant bookkeeping trick that finds exactly such constrained peaks (and valleys), by translating 'stay on the fence' into a clean equation about gradients.
Consider minimizing f(x) subject to a constraint g(x) = 0 (the fence). At a constrained optimum, you cannot reduce f by moving along the constraint surface — which happens precisely when the gradient of f is parallel to the gradient of g: grad f = lambda * grad g for some scalar lambda, the Lagrange multiplier. Equivalently, define the Lagrangian L(x, lambda) = f(x) - lambda * g(x); setting its gradient (in both x and lambda) to zero recovers grad f = lambda * grad g together with the constraint g(x) = 0. So an n-variable constrained problem with one constraint becomes an (n+1)-variable UNconstrained stationarity problem — the multiplier lambda is the extra unknown that enforces the constraint. With several constraints you get one multiplier each.
Beyond the mechanics, the multiplier has a beautiful meaning: lambda is the SHADOW PRICE — it measures how much the optimal value of f would change if you relaxed the constraint by a tiny amount (if g(x) = c, then d(optimal f)/dc = lambda). In economics it is the marginal value of a resource; in physics, forces of constraint; in machine learning, the trade-off weight in regularized problems. The honest caveat: the multiplier condition is necessary at a regular constrained optimum (one where the constraint gradients are linearly independent, a 'constraint qualification'), and it locates stationary points, not guaranteed minima — you still must check second-order conditions, and the method generalizes to inequalities only through the broader KKT conditions, where the multipliers carry sign restrictions.
Maximize f(x, y) = x y subject to x + y = 10. The Lagrangian is x y - lambda (x + y - 10); its stationarity gives y = lambda, x = lambda, and x + y = 10, so x = y = 5, with lambda = 5. The multiplier lambda = 5 also says: loosen the budget to x + y = 11 and the optimum rises by about 5.
At the optimum, grad f aligns with grad g; lambda is the trade-off rate.
The multiplier condition is necessary only at a REGULAR optimum where the active constraint gradients are linearly independent (a constraint qualification). It also just locates stationary points — confirm a minimum with second-order conditions — and for inequality constraints you need the full KKT conditions, where multipliers must be nonnegative.