Multivariable Optimization

multiple constraints

One trail keeps you on a curve; two trails crossing keep you at a point. When an optimization must satisfy not one but several equality constraints at the same time — g1 = c1, g2 = c2, and so on — the feasible set shrinks. In three dimensions, two constraint surfaces typically intersect in a curve, and you optimize along that intersection. The Lagrange method extends naturally: you simply allow one multiplier per constraint.

The optimality condition becomes grad f = lambda1 times grad g1 + lambda2 times grad g2 + ... — the gradient of the objective is a linear combination of the constraint gradients, each weighted by its own multiplier. Geometrically, grad f must lie in the subspace spanned by the constraint gradients; equivalently, the part of grad f that points along the feasible set (the directions that keep all constraints satisfied) must be zero, since otherwise you could still improve. Together with the constraints themselves, this gives n + m equations in n variables and m multipliers, which you solve simultaneously.

Multiple constraints are the rule, not the exception, in real problems: a chemical equilibrium conserves several elements at once, a portfolio must hit a target return while keeping weights summing to one, a structure must satisfy several load balances. A crucial honest condition underlies the method — the constraint gradients must be linearly independent at the optimum (a constraint qualification). If they are not, the multipliers can fail to exist or to be unique, and the neat picture breaks down.

Find the extrema of f(x, y, z) = x + y + z on the circle where x^2 + y^2 + z^2 = 1 and x + y - z = 0 meet. Two constraints, two multipliers: grad f = lambda1 grad g1 + lambda2 grad g2 gives three equations, solved together with the two constraints for the optimal point.

One multiplier per constraint; the objective gradient is a weighted blend of the constraint gradients.

You cannot have more independent equality constraints than variables: m constraints in n variables need m less than or equal to n, and the constraint gradients must stay linearly independent, or the method gives no valid multipliers.

Also called
several equality constraints多个等式约束多個等式約束