Karush-Kuhn-Tucker ConditionsInequality constraints

Constraint qualificationEquality constraints

Henrik Juel, DTU Management

Karush-Kuhn-Tucker Conditions – p.1/8

Inequality constraints

min. f(x) s.t. gi(x) ≤ 0 ∀ i

Feasible solutionxo

Binding constraintsI = {i : gi(xo) = 0}

Constraint qualification

KKT necessary optimality conditions:If xo is a local minimum,then there exist multipliersui ≥ 0 for i ∈ Isuch thatf ′(xo) +

∑i∈I uig

′i(x

o) = 0

Karush-Kuhn-Tucker Conditions – p.2/8

Inequality example

min. (x − 3)2 + (y − 2)2

s.t.x2 + y2 ≤ 5, x + 2y ≤ 4andx, y ≥ 0

The gradients are:f ′ = (2x − 6, 2y − 4)g′1 = (2x, 2y), g′2 = (1, 2)g′3 = (−1, 0), g′4 = (0,−1)

Consider the point(x, y) = (2, 1)It is feasible withI = {1, 2}

(−2,−2) + u1(4, 2) + u2(1, 2) = (0, 0)u1 = 1/3, u2 = 2/3

Class exercise: Solve max.ln(x + 1) + ys.t.2x + y ≤ 3, x, y ≥ 0

Karush-Kuhn-Tucker Conditions – p.3/8

Geometric interpretation

f ′(x) points in the direction of steepest ascent−f ′(x) points in the direction of steepest descent

In two dimensions:f ′(xo) is perpendicular to a level curve offg′i(x

o) is perpendicular to the level curvegi(x, y) = 0

Constraint qualification example:max.x s.t. y ≤ (1 − x)3, y ≥ 0

Consider the global max:(x, y) = (1, 0) at a cuspAfter reformulation, the gradients are:f ′ = (−1, 0)g′1 = (3(x − 1)2, 1) = (0, 1), g′2 = (0,−1)

No u1, u2 exist with(−1, 0) + u1(0, 1) + u2(0,−1) = (0, 0)

Karush-Kuhn-Tucker Conditions – p.4/8

Constraint qualifications

KKT constraint qualification:g′i(x

o) for i ∈ I are linearly independent

Slater constraint qualification:gi(x) for i ∈ I are convex functionsA nonboundary point exists:gi(x) < 0 for i ∈ I

Class exercise: Solvemin. x2 + y2

s.t.x2 + y2 ≤ 5, x + 2y = 4andx, y ≥ 0

Karush-Kuhn-Tucker Conditions – p.5/8

Sufficient conditionmin. f(x) s.t. gi(x) ≤ 0 ∀ i

If f andgi for i ∈ I are convex functions,then a feasible KKT point is optimal

An equality constraint is equivalent to two inequalityconstraints:h(x) = 0h(x) ≤ 0 and−h(x) ≤ 0

The corresponding two nonnegative multipliers maybe combined to one free one:u+h′(x) + u−(−h′(x)) = vh′(x)

Karush-Kuhn-Tucker Conditions – p.6/8

Equality constraints also

min. f(x) s.t. gi(x) ≤ 0 ∀ i, hj(x) = 0 ∀ j

As before, letxo be a feasible solution, defineI = {i : gi(x

o) = 0},and assume that a constraint qualification holds.

Necessary optimality condition:If xo is a local minimum,then there exist multipliersui ≥ 0 for i ∈ I andvj ∀ jsuch thatf ′(xo) +

∑i∈I uig

′i(x

o) +∑

j vjh′j(x

o) = 0

Sufficient optimality condition:If f andgi for i ∈ I are convex functions,andhj ∀ j are affine (linear),then a feasible KKT point is optimal

Karush-Kuhn-Tucker Conditions – p.7/8

Alternative formulationmin. f(x) s.t. gi(x) ≤ 0 ∀ i, hj(x) = 0 ∀ j

This form of KKT condition is more common:

f ′(xo) +∑

i uig′i(x

o) +∑

j vjh′j(x

o) = 0

uigi(xo) = 0, ui ≥ 0 ∀ i

xo feasible

Or i vector form:min. f(x) s.t.g(x) ≤ 0, h(x) = 0

f ′(xo) + ug′(xo) + vh′(xo) = 0

ug(xo) = 0, u ≥ 0xo feasible

Class exercise: Write the KKT conditions formax.cx s.t.Ax ≤ b, x ≥ 0

Karush-Kuhn-Tucker Conditions – p.8/8