Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity

Discrete Optimization Lecture #3

2008/3/4 1Shi-Chung Chang, NTUEE, GIIE, GICE

Last Time1. Algorithms and Complexity

» Problems, algorithms, and complexity» Polynomial time algorithms» Intractability» NP-complete problems

2. Basic Properties of Linear Programming» Forms of LP» Basic feasible solutions» Geometry of LP

3. The Revised Simplex Method

Today1. The R evised Simplex Algorithm

» Basics» The algorithm» Getting an Initial Feasible Solution

2. Computational Aspects» Updating the basis» Convergence and Degeneracy» The Revised Simplex Method

3. Duality» Dual LP» The Duality Theorem» Complementary Slackness» The Dual Simplex Algorithm


Reading Assignments:1. Sections 3.1-3.3, 4.1-4.2 of [PaS 8 3]2. Sections 4.1-4.5 of [Lue 8 4]

Next Time: sections 4.3,4.4 and Chapter 5 of [PaS 83]


§2.2 The Revised Simplex MethodThe simplex method for LP : G.B. Dantzig 1951Key idea:

Phase1:find a bfs of

Note that an LP may not have a solution

Example min s.t.

0x

bxA

21 xx

0,

42

21

21

xx

xx


Phase 2Allow one of the zero components of the bfs to become positive and force one of the original positive components to become

zero.=> How to pick “entering” and “leaving” component Cost

Traditional form of the simplex method: Tableau=> read by yourself

Here we consider matrix form for conciseness of presentation and later developments.

xcT


SLP: s.t.

Assume P(A)=m.Partition A=[B:D] B: m linearly indep. columns of A (Assume the first m

cols.)

(SLP)

subject to (3.1)

xcmin

0

x

bxA

bx

xDB

A

B

:

mn

D

m

B

Rx

Rx

ZxCxC DDBB min

0,0

DB

DB

xx

bxDxB


If and a basic solutionWhen from (3.1)

(note that may not ≥0 ) (3.2)

Substituting(3.2) into the cost function

(3.3)

Define (3.4) as the relative cost vector=> To minimize Z, we need only adjust

Q: How?

0Dx bBx B

1

,0Dx

BxDB xDBbBx 11

DBDBDDDBxDBCCbBCxCxDBbBCZ 1111

DBccrBDD

1

Dx


Example

subject to =2 + =5

+ =6

In matrix form.

32133min xxx

4321 2 xxxx

32132 xxx 5

x321

22 xxx 6

x6,,1,0 ix

i

6100122

5010321

2001112654321 baaaaaa

000313 c

0B

0

Bc


0

0

0

0

0

0

1

1

1

)( 100BCB

6

5

2

6

5

4

0

x

x

x

x B

0

0

0

3

2

1

0

x

x

x

xD


3130 Dcr D

Define the vector of simplex multipliers

Compute in two steps:

Assume that is non degenerate If is optimal and it does not pay to increase

If then the cost Z can possible be decreased by increasing

1 BcB

'r

Dcr

cB

D

B

D

B

x

xx

xr ,0

,0Dxr 0

Dx

)0)(( iBx

,0 kr

kmx


Dx

Q: (1) any limitations on increasing ? (2) If there are more than one component of r < 0 => which should we change to increase?

Simplex method chooses one entering variable, Normally one with the most negative

Q: which component should leave?the first one to reach 0 => satisfied again. bxA

kr

kmx


If we consider bringing into the basis, i.e. , as the entering variable

=>

=>

1x

362

55

122

161

151

141

xxx

xxx

xxx

4

4

1

4,4,0,1

6

5

1

1

6541

x

x

x

x

xxxx

B 0

4

3

2

1

x

x

x

x D


1a

Mathematically, let be the new value vector of=>

=>

The ith component of is zero when

i.e. when

Bx

Bx

BkkB

kBb

xbBxaBx

bxaxB

11

yxxbBx

xbByxx

kBB

BBB

.

.1

1

Bx iikbByx 1

i

Bi

i

i

k y

x

y

bBx

1


To maintain feasibility, the smallest of these ratios for i where > 0 determines how large can be. Say, it happens at l th components of

The new bfs has and

leaves the basis while enters

iyk

x

i

Bi

i

l

Bl

y

x

ymi

y

x

01

min

x 0

lx 0

kx

la ka

0Bx

R earranging , w e can get x


Revised Simplex Algorithm( one iteration)Step 1: Giving is the basis B such thatStep 2: Solve to get Step 3 : Calculate If r ≥ 0 => optimal solution achieved, STOP

Find K = arg

Step 4 : Solve for y Step 5 : Find

If , => unbounded solution, STOPStep 6 : Update the basic solution

Step 7 : Update the basis Return to step 2

01 bBx B

BcB Dcr D

iri

min

kayB

i

Bi

il

Bl

y

x

ymiy

x

0,1

min

k

iii

x

kiyxx ,

0i

y


How to Update BasisNote that in the simplex algorithm we need to solve

(step 2) (step 4)

Where B’s differ by only one column between any two subsequent iterations.=> How to calculate B’s efficiently?

B

T CB kayB


Recall

(rank one update)

Tllkoldkloldnew eaaBaaBB )( col.col.

11

1

1111 )(

)(1

)(

oldl

Tllkold

lkold

Tl

oldTllkold

oldnew By

eaaBI

aaBe

BeaaBBB

Tll

ll

Tl ee

yyey

IE1


llold

kold

eaB

yaB

1

1Note:

=>

1

0

010

01

001

2

1

l

m

l

l

l

yy

y

yy

yy

11 oldnew

EBB Product Form of the inverse(PFI)


Some Computational AspectsLet be the elementary matrices after K pivot iterations.

=>

• For large-scale problems, store ‘s as vector and update and y as followers:

kEEE ,,,

21

1

011

1

BEEEBkkk

iE

kk

kkB

T

aBEEy

BEEEc1

01

1

011)(


• What if is small in pivoting? Use LU or QR decompositions in computing If B is the current basis

Note that

ly

1B

kmllnew

oldoldoldm

aaaaaaB

BULaaaB

1121

21

newoldBLH 1

koldmll aLuuuuu 1

1121

=> upper Hessenberg


Using a sequence of elimination steps on H:

• Store • QR-decomposition

Do givens transformation on H

• The simplex method is theoretically an exponential algorithm In practice, 2(n + m) iterations, i.e., a linear function of (n + m)

newmoldnew

emnew

UMMLB

HMMU11

1

1

11

1

oldemnew LMML

)( 111 kmeenew aaaaaB

HBQ newT

old

1

1

meoldnew

newTe

Tm

QQQQ

RHQQ

m

n


Convergence and DegeneracyConvergence:

If the objective function value strictly decreases after every iteration, the simplex algorithm never repeats a basis and it converges in a finite number of iterations.

why?

Degeneracy:If for a such that

=> x is a degenerate

ixbfs ,

0)()( 1

iiB bBx


Property:in the pivoting step,

=> i.e. no change in cost.

=> Problem: cycling among degenerate=> no guarantee of convergence

=> need an anti-cycling algorithm

oldioldnewZYZZ

bfses

0)(

i

iB

y

x


How to get an initial feasible solution?

(p)

(PH1)

subject to

0

x

bxAmn

m

iii

zy1

min

0

0

0

z

y

x

bzIyIxAmm

mRzy ,


• (P) has a feasible solution in (PH1)Q: • can solve (PH1) using revised simplex with

as the initial feasible solution

izyii

,0,0

,0x


otherwise.

0 if

0ii

i

bby

otherwise.

0 if

0ii

i

bbz

Solving S-P by the Revised Simplex

Subject to

* Flow balance eq. corresponding to * is dropped due to redundancy.

xxmix

52213312

0

0

0

0

1

111

111

111

111

11

4

3

2

1

)(0

8

7

6

5

4

3

2

1

87654321

x

x

x

x

x

x

x

x

eeeeeeees

8,,1,0 jxj


Chapter 3 Duality§ 3 .1 Dual LP

Consider (SLP)

Subject to


minc x

x

0

,

.

Ax b

x

Dual Viewpoint

• From the optimality condition, if optimal solution to SLP => corresponding partition of A matrix

such that

*x[ *: D*] = B A

(3.5)

where *' (3.6)

' ' 1* *

' 1*

( * ) 0

*

D BD

B

r c c B D

c B

Note that in Simplex Method, we have' (3.7)

where [ : and (

' 1

] ) .

mBc B R

B D A B m

Dual Viewpoint (Cont.)

' '[

[

= [

' 1

' ' 1

: ]

: ]

: ]

B

B B

A B D

c B B D

c c B D

and .Dc D

' [

[ = ' (3.8)

' ' 1

' '

: ]

: ]

B B

B D

A c c B D

c c c

Note that

The ’ we are looking for must satisfy (3.5) for anyThe root of (3.5) comes from So,

Note that ’ in (3.5) and (3.6) satisfies (3.7) and (3.8).

0.x

Dual Objective FunctionNow consider the objective function

From inequality (3.8) that then for any bfs

’ is the one that leads to the maximum value while satisfying (3.7) and (3.8)

' ' (3.9a)

(3.9b)

( = 0) (3.9c)

**' *'

*

*' *' *' *1 1

*' *' *' *

*' *

* [ : ]

* ( * )

( )

BB D

D

B D B D

D D

D

xc x c x c c

x

c B b c c B D x

b c D x

b x

'' cA 0xwe have

xcbxA ''' bxcbxA *'*''*' and from (3.9)

b'

Dual SLP

(SLP)

Subject to


minc x

x

0

,

.

Ax b

x

(DSLP)

Subject to

max b

,

.

mR

A c

Documents

Discrete Optimization Lecture #3 2008/3/41Shi-Chung Chang, NTUEE, GIIE, GICE Last Time 1.Algorithms and Complexity » Problems, algorithms, and complexity