Karush-Kuhn-Tucker ConditionsInequality constraints
Constraint qualificationEquality constraints
Henrik Juel, DTU Management
Karush-Kuhn-Tucker Conditions – p.1/8
Inequality constraints
min. f(x) s.t. gi(x) ≤ 0 ∀ i
Feasible solutionxo
Binding constraintsI = {i : gi(xo) = 0}
Constraint qualification
KKT necessary optimality conditions:If xo is a local minimum,then there exist multipliersui ≥ 0 for i ∈ Isuch thatf ′(xo) +
∑i∈I uig
′i(x
o) = 0
Karush-Kuhn-Tucker Conditions – p.2/8
Inequality example
min. (x − 3)2 + (y − 2)2
s.t.x2 + y2 ≤ 5, x + 2y ≤ 4andx, y ≥ 0
The gradients are:f ′ = (2x − 6, 2y − 4)g′1 = (2x, 2y), g′2 = (1, 2)g′3 = (−1, 0), g′4 = (0,−1)
Consider the point(x, y) = (2, 1)It is feasible withI = {1, 2}
(−2,−2) + u1(4, 2) + u2(1, 2) = (0, 0)u1 = 1/3, u2 = 2/3
Class exercise: Solve max.ln(x + 1) + ys.t.2x + y ≤ 3, x, y ≥ 0
Karush-Kuhn-Tucker Conditions – p.3/8
Geometric interpretation
f ′(x) points in the direction of steepest ascent−f ′(x) points in the direction of steepest descent
In two dimensions:f ′(xo) is perpendicular to a level curve offg′i(x
o) is perpendicular to the level curvegi(x, y) = 0
Constraint qualification example:max.x s.t. y ≤ (1 − x)3, y ≥ 0
Consider the global max:(x, y) = (1, 0) at a cuspAfter reformulation, the gradients are:f ′ = (−1, 0)g′1 = (3(x − 1)2, 1) = (0, 1), g′2 = (0,−1)
No u1, u2 exist with(−1, 0) + u1(0, 1) + u2(0,−1) = (0, 0)
Karush-Kuhn-Tucker Conditions – p.4/8
Constraint qualifications
KKT constraint qualification:g′i(x
o) for i ∈ I are linearly independent
Slater constraint qualification:gi(x) for i ∈ I are convex functionsA nonboundary point exists:gi(x) < 0 for i ∈ I
Class exercise: Solvemin. x2 + y2
s.t.x2 + y2 ≤ 5, x + 2y = 4andx, y ≥ 0
Karush-Kuhn-Tucker Conditions – p.5/8
Sufficient conditionmin. f(x) s.t. gi(x) ≤ 0 ∀ i
If f andgi for i ∈ I are convex functions,then a feasible KKT point is optimal
An equality constraint is equivalent to two inequalityconstraints:h(x) = 0h(x) ≤ 0 and−h(x) ≤ 0
The corresponding two nonnegative multipliers maybe combined to one free one:u+h′(x) + u−(−h′(x)) = vh′(x)
Karush-Kuhn-Tucker Conditions – p.6/8
Equality constraints also
min. f(x) s.t. gi(x) ≤ 0 ∀ i, hj(x) = 0 ∀ j
As before, letxo be a feasible solution, defineI = {i : gi(x
o) = 0},and assume that a constraint qualification holds.
Necessary optimality condition:If xo is a local minimum,then there exist multipliersui ≥ 0 for i ∈ I andvj ∀ jsuch thatf ′(xo) +
∑i∈I uig
′i(x
o) +∑
j vjh′j(x
o) = 0
Sufficient optimality condition:If f andgi for i ∈ I are convex functions,andhj ∀ j are affine (linear),then a feasible KKT point is optimal
Karush-Kuhn-Tucker Conditions – p.7/8
Alternative formulationmin. f(x) s.t. gi(x) ≤ 0 ∀ i, hj(x) = 0 ∀ j
This form of KKT condition is more common:
f ′(xo) +∑
i uig′i(x
o) +∑
j vjh′j(x
o) = 0
uigi(xo) = 0, ui ≥ 0 ∀ i
xo feasible
Or i vector form:min. f(x) s.t.g(x) ≤ 0, h(x) = 0
f ′(xo) + ug′(xo) + vh′(xo) = 0
ug(xo) = 0, u ≥ 0xo feasible
Class exercise: Write the KKT conditions formax.cx s.t.Ax ≤ b, x ≥ 0
Karush-Kuhn-Tucker Conditions – p.8/8