OR II GSLM 52800

1

OR IIOR IIGSLM 52800GSLM 52800

2

OutlineOutline

inequality constraint

3

NLPNLP with Inequality Constraints with Inequality Constraints

min f(x) s.t. hi(x) = 0 for i = 1, …, p gj(x) 0 for j = 1, …, m in matrix form

min f(x) s.t. h(x) = 0 g(x) = 0

4

Binding ConstraintsBinding Constraints

Inequality constraint gj(x) 0 is binding at point x0 if gj(x0) = 0

=-constraint: always binding

5

Regular PointsRegular Points

K: the set of binding (inequality) constraints

x* subject to h(x) = 0 & g(x) 0 is a regular point if Thi(x*), i = 1, …, p, & Tgj(x*), j K, are

linearly independent

6

FONC for Inequality ConstraintsFONC for Inequality Constraints((Karush-Kuhn-Tucker Necessary Conditions; KKT Conditions)

for a regular point x* to be a local min, there exist * p and * m such that

* *** *

1 1

( ) ( )( )0, 1,..., ;

p mi i

i ii ij j j

h gfj n

x x x

x xx

*( ) 0, 1,..., ;ih i p x

*( ) 0, 1,..., ;ig i m x

* *( ) 0, 1,..., ;i ig i m x

* 0, 1,..., .i i m

In Matrix Form:

f(x*) + *Th(x*) + * Tg(x*) = 0 (stationary condition; dual feasibility)

h(x*) = 0, g(x*) 0 (primal feasibility)

* Tg(x*) = 0 (Complementary slackness conditions)

* 0 (Non-negativity of the dual variables)

v1 = (1, 0), v2 = (0, 1)

{(x, y)|v1+v2 , , 0}

7

The Convex Cone Formed by Vectors The Convex Cone Formed by Vectors

(1, 0)

(0, 1)

v1

v2

The convex cone formed by v1 and v2:

{(x, y)|v1+v2 , , 0}

The convex cone formed by (1, 0) and (0, 1): {(x, y)|v1+v2 , , 0}

8

Geometric Interpretation Geometric Interpretation of the of the KKTKKT Condition Condition

constraint 2: g2(x) 0constraint 1: g1(x) 0

x0

g1(x) = 0g1(x0

)

g2(x) = 0

g2(x0

) f (x0)

f (x0)

x0 cannot be a minimum

9



g1(x) = 0

g2(x) = 0

g2(x0

)g1(x0

)x0

f (x0)f (x0)


10



g1(x) = 0

g2(x) = 0

g2(x0

)g1(x0

)x0

f (x0)

f (x0)

region decreasing in f


11



g1(x) = 0

g2(x) = 0

g2(x0

)g1(x0

)x0


f (x

0 )

f (x

0 )


12



g1(x) = 0

g2(x) = 0

g2(x0

)g1(x0

)x0

f

(x0 )

f (x

0 )


x0 can be a minimum

13

A Necessary Condition A Necessary Condition for for xx00 to be a Minimum to be a Minimum

when f in the convex cone of g1 and g2

f(x0) = 1g1(x0) + 2g2(x0), i.e.,

f(x0) + 1g1(x0) + 2g2(x0) = 0

g1(x) = 0

g2(x) = 0

g2(x0)

g1(x0)

x0

f

(x0 )

f (x

0 )

14

Effect of ConvexityEffect of Convexity

convex objective function, convex feasible region set

the KKT condition necessary & sufficient

convex gj & linear hi

15

Example 10.11 of JBExample 10.11 of JB

KKT conditions

2 21 2

2 21 2

1 2

min ( ) 2( 1) 2( 4) ,

. . 9,

2.

f x x

s t x x

x x

x

1 1 1 2

2 1 2 2

4( 1) (2 ) =0

6( 4) (2 ) =0

x x

x x

2 21 2

1 2

9

2

x x

x x

2 21 1 2

2 1 2

( 9)

(2 =0)

x x

x x

1 20, 0

Tf(x) = (4(x1+1), 6(x24))

Tg1(x) = (2x1, 2x2)

Tg2(x) = (-1, -1)

possibilities

16


2

= 0 > 0

1

= 0

> 0

0 the th constraint is binding i i

2 21 1 2

2 1 2

( 9)

(2 =0)

x x

x x

case 1 = 0, 2 = 0 (both constraints not binding)

x1 = -1, x2 = 4, violating

17


2 21 2 9x x

2 21 2

2 21 2

1 2

min ( ) 2( 1) 2( 4) ,

. . 9,

2.

f x x

s t x x

x x

x

case 1 > 0, 2 > 0 (both constraints binding)

solving

18


2 21 2

1 2

9,

2,

x x

x x

7 7 7 7

1 2 2 2 2 2( , ) 1 ,1 or 1 ,1 x x

7 72 2

1 ,1 : T

T1

T2

( ) (15.48331, 29.225)

( ) (5.741657, 1.74166)

( ) (1,1)

f

g

g

x

x

x

g2(x0) g1(x0)f (x0)

7 72 2

1 ,1 : T

T1

T2

( ) (0.516685, 6.77503)

( ) ( 1.74166,5.741657)

( ) (1,1)

f

g

g

x

x

x

g2(x0)

g1(x0)

f (x0)

case 1 = 0, 2 > 0 (the second constraint binding)

solving, 2 < 0

19


2

2

1 2

1 2 1 4

2 2 2 6

2

4( 1) 0 1

6( 4) 0 4

x x

x x

x x

case 1 > 0, 2 = 0 (the first constraint binding)

solving,

20


1

1

2 21 2

21 1 1 1 2

122 1 2 2 3

9

4( 1) 2 0

6( 4) 2 0

x x

x x x

x x x

1 1

2 22 12

1 1 22 39 =1.096084 =-0.64598, =2.929627 x x

T

T1

( 0.64598,2.929627) (1.41608, -6.42224)

( 0.64598,2.929627) ( 1.29196, 5.859254)

f

g

T T1 1( 0.64598,2.929627) ( 0.64598,2.929627) (0,0)f g

*

convex objective function, convex feasible set

is the global minimum x

21

KKTKKT Condition with Condition with Non-negativity ConstraintsNon-negativity Constraints

min f(x),

s.t. gi(x) 0, i = 1, …, m,

x 0

22


define L(x, ) = f(x) + Tg(x)

***

1

( )( )0, 1,..., ;

mi

iij j j

gL fj n

x x x

xx

*( ) 0, 1,..., ;ii

Lg i m

x

* 0, 1,..., ;jj

Lx j n

x

* *( ) 0, 1,..., ;i ig i m x

* 0, 1,..., ;jx j n

* 0, 1,..., .i i m

23


define L(x, ) = f(x) + 1Tg(x) - 2

Tx

* T * T1 2

*

T *1

T *2

1 2

( ) ( )

( )

( )

,

f

x g x 0

g x 0

-x 0

g x 0

x 0

0

* T *1

*

T *1

* T * T *1

1

( ) ( )

( )

( )

( ) ( )

f

f

x g x 0

g x 0

g x 0

x x g x 0

x 0

0

24


L(x, ) = KKT condition

(a) 2x1 8 + 0, 8x2 16 + 0 (b) x1+ x2 5 0 (c) x1(2x1 8 + ) = 0, x2(8x2 16 + ) = 0 (d) (x1+ x2 5) = 0 (e) x1 0, x2 0, 0

2 21 2 1 2

1 2

1 2

min ( ) 4 8 16 32,

. . 5,

0, 0.

f x x x x

s t x x

x x

x

2 21 2 1 2 1 24 8 16 32 ( 5),x x x x x x

25


eight cases, depending on whether x1, x2, and = 0 or > 0

on checking, the case that x1+x2 5 is binding, x1 > 0, x2 > 0 x = (3.2, 1.8)

and = 1.6 satisfying the KKT condition regular point convex objective with linear constraint KKT being

sufficient

2 21 2 1 2

1 2

1 2

min ( ) 4 8 16 32,

. . 5,

0, 0.

f x x x x

s t x x

x x

x

Documents

OR II GSLM 52800