Multivariable Optimization

inequality constraint

An equality constraint pins you to a curve or surface; an inequality constraint instead fences off a whole region you must stay inside. Conditions like g(x, y) less than or equal to 0, or x greater than or equal to 0, are inequality constraints. Think of optimizing profit where production cannot exceed a capacity, or a quantity that cannot go negative. The feasible set is now a region with a boundary, not a thin curve.

The decisive new idea is that an inequality constraint can be either active or inactive at the optimum. If the best point lies in the strict interior of the allowed region, the fence plays no role — the constraint is inactive, and you optimize as if it were not there. But if the optimum is pushed right up against the boundary, the constraint is active: it holds as an equality g = 0 there, and it genuinely shapes the answer, behaving like a Lagrange constraint. Because you usually do not know in advance which constraints will be active, the analysis must consider the possibilities — and a sign condition appears, because the boundary can only push back, never pull.

Inequality constraints are what make optimization realistic: budgets, capacities, non-negativity of probabilities or concentrations, and physical limits are all one-sided. They are the setting for linear programming, quadratic programming, and the support-vector machines of machine learning. The systematic treatment of which constraints are active, and the sign of their multipliers, is the content of the Karush-Kuhn-Tucker conditions — the generalization of Lagrange multipliers to inequalities.

Maximize f(x, y) = x + y subject to x^2 + y^2 less than or equal to 1. The unconstrained maximum is at infinity, so the constraint must be active: the optimum sits on the boundary circle x^2 + y^2 = 1, at x = y = 1/sqrt(2). Here the fence does the work.

When the free optimum lies outside the region, the inequality is pushed to equality on the boundary.

An inactive inequality contributes nothing — its multiplier is zero. The whole art is figuring out which constraints are active at the optimum, since you cannot simply treat them all as equalities.

Also called
one-sided constraint单边约束單邊約束