OR-1 2009 1

Chapter 3. Pitfalls

Initialization Ambiguity in an iteration Finite termination?

Resolving ambiguity in each iterationSelection of the entering variable:

Any nonbasic variable with can enter basis.

(However, performance of the algorithm depends on the choice of the entering variable.

examples of the rules : largest coefficient rule, maximum increase rule, steepest ascent rule, …)

Nj jj Nxczz variablesnonbasic ofset : ,*

0jc

OR-1 2009 2

Selection of leaving variable:

a) No restriction in minimum ratio test : can increase the value of the entering variable indefinitely while satisfying nonnegativity, hence problem is unbounded

Ex)

145

1432

143

437

325

5

xxx

xxxx

xxxz

734

523

5

541

4321

431

xxx

xxxx

xxxz

Increase x3 while keeping x1 and x4 at 0, then direction of

movement should be d = ( 0, 2, 1, 0, 0 ) 0 so that the new point satisfies the equations.

Hence, new point x0+d 0 satisfies nonnegativity for any 0. Also objective value increases by 1 ( 1 is coefficient of x3 in z

row).

OR-1 2009 3

0

0

0

1

2

0

13004

01213

Ad

0any for equations

satisfies ( i.e. Hence

bAx

dxbbAdAxdxA ),0)( 000

Also note that

Note that if we want to move from x0 to x0 + d, >0 , then we must have A(x0 + d)=b so that the new point satisfies the equations Ax=b.

Then A(x0 + d)=b, >0 holds if and only if Ad=0. So the direction of movement d must satisfy Ad=0 for the d vector to be a feasible direction of movement ( d is in the null space of A).

OR-1 2009 4

b) In case of ties in the minimum ratio test :

3216

3215

34

321

432

6423

21

820

xxxx

xxxx

xx

xxxz

4216

4215

43

421

230

3420

5.05.0

424

xxxx

xxxx

xx

xxxz

Ties (=1/2)

Basic variables have value 0after pivot.

OR-1 2009 5

4216

4215

43

421

230

3420

5.05.0

424

xxxx

xxxx

xx

xxxz

( x1 entering, x5 leaving )

5426

5421

43

542

5.05.30

5.05.120

5.05.0

34

xxxx

xxxx

xx

xxxz

Though x1 cannot be increased, we perform the pivot as usual, making x1 as basic and x5 as nonbasic. No change in solution, but basic and nonbasic status changes for two variables. Later, we will examine what this means in geometry.

In the next iteration,

OR-1 2009 6

Terminology:

degenerate solution ( 퇴화해 ) : basic feasible solution with one or more basic variables having 0 values.

degenerate iteration (pivot) : simplex iteration that does not change the basic solution values (only basis changes).

Observations:Given a nondegenerate b.f.s., we must have ties in the minimum ratio

test so that we have a degenerate solution after pivot.A degenerate pivot occurs only if we have a degenerate solution, but the

converse is not true (i.e. we may have a nondegenerate pivot although we have a degenerate solution).

OR-1 2009 7

5426

5421

43

542

5.05.30

5.05.120

5.05.0

34

xxxx

xxxx

xx

xxxz

Geometric meaning of a degenerate iteration

4216

4215

43

421

230

3420

5.05.0

424

xxxx

xxxx

xx

xxxz

x1=0

x6=0

x5=0x2=0

x1=0

x6=0

x5=0x2=0

( x1=0, x2=0 used) ( x2=0, x5=0 used)

A A

OR-1 2009 8

(ex- continued)

In the first dictionary, the point A is identified by using the three equations in Ax = b and setting x1 0, x2 0 at equalities. The

nonnegativity constraints for nonbasic variables (here x1, x2) are used

to indentify point A

( provides 5 equalities together with Ax = b).

After pivot, the solution point A is not changed. But it is identified now by using x2 0, x5 0 at equalities together with Ax = b.

Hence degenerate iteration changes the defining equations one at a time, but the solution point is not changed.

OR-1 2009 9

Getting out of degenerate iterations:

5426

5421

423

542

5.05.30

5.05.120

5.05.05.0

34

xxxx

xxxx

xxx

xxxz

x1=0

x6=0

x5=0x2=0

( x3=0, x5=0 used)

A

We changed the data in the previous dictionary a little bit.

Now, x2 enters (takes value 1) and x3 leaves the basis, and it is a

nondegenerate pivot.

Geometrically, we still satisfy the equation x5 = 0, but do not need to

satisfy x2 = 0. As we increase the value of x2 up to 1, x3 becomes 0,

hence the equation x3 = 0 is now used (together with x5 = 0 and

Ax=b) to define a new point.

x30

OR-1 2009 10

Degenerate pivot is the process of identifying the same point using different defining equations (nonnegativity constraints). If we are lucky enough to obtain defining equations that guides the moving direction, we move to a different point with a nondegenerate pivot.

During the degenerate iterations, the algorithm stalls and it may hamper the performance of the algorithm. Such phenomenon is of practical concern and affects the performance of the algorithm (especially, for problems with some special structures and large problems).

Later, we will see how to get out of degenerate iterations systematically.

OR-1 2009 11

If we have a degenerate solution, pivot may continue indefinitely (Example in text p.31, pivoting rule is largest coefficient for entering variable and smallest subscript for leaving in case of ties. Then we have the initial dictionary again after 6 pivots.)

Terminology:Cycling : appearance of the same dictionary again in the simplex

iterations.

Cycling is the only reason that simplex method may fail to terminate (i.e. simplex method terminates in a finite number of iterations as long as cycling is avoided).

OR-1 2009 12

Thm) If simplex method fails to terminate, it must cycle.

Pf) Number of ways to choose basic variables are finite. Hence, if simplex fails to terminate, same basis must appear again.

Show same basis same dictionary, Hence same dictionary appears, i.e. cycles.

(1) and (2) have the same set of solutions (ignore nonnegativity constraints)

Let Consider a solution to (1) given as the following:

(1)

Bjjijii

Bjjj

basisBixabx

xcvz

:,

,

(2) ,

basisBixabx

xcvz

Bjjijii

Bjjj

:

,

**

**

.Bxk

tcvz

Bitabx

RtkjBjxtx

k

ikii

jk

(

,

),,0,

OR-1 2009 13

This solution satisfies (1). So it also satisfies (2), hence

These equalities must hold for any real number t.

Therefore, we have

We can use any nonbasic variable in the above proof, so dictionaries are identical.

The theorem can be proved easily if we use matrices, but the above proof uses only algebraic arguments.

tcvtcv

Bitabtab

kk

ikiiki

**

** ,

**

**

,

,,

kk

ikikii

ccvv

Biaabb

OR-1 2009 14

The proof of the theorem shows that if we have the same basis, then the dictionaries (tableaus) are the same. Since there are only a finite number of ways to choose the basis, the simplex method terminates finitely if the same basis (the same dictionary) does not appear again, i.e. cycling is avoided.

OR-1 2009 15

Avoid cycling

Perturbation method, Lexicographic method Smallest-subscript rule (Bland’s rule)

Smallest-subscript rule (Bland’s rule) : Choose the variable having the smallest index among possible candidates as the entering variable.

Also, if ties occur while choosing the leaving variable, select the smallest indexed variable among the candidates as the leaving variable.

See the proof in the text if time permits.

)0( jc

OR-1 2009 16

Examples of smallest subscript rule:

3216

3215

34

321

432

6423

21

820

xxxx

xxxx

xx

xxxz

4216

4215

43

421

230

3420

5.05.0

424

xxxx

xxxx

xx

xxxz

x3 enters, x4 leaves.

Ties (=1/2)

x1 enters, x5 leaves.

Ties (=0)

OR-1 2009 17

Perturbation method, lexicographic methodIdea: Avoid the appearance of degenerate solution since it is a precondition

of cycling ( If the solution is not degenerate, the objective function value increases strictly in the next iteration, hence the same basis does not appear again because the objective function value never decreases during the simplex iterations). So add very small positive to the r.h.s of

equations so that the solution values are unchanged practically but degeneracy is avoided.

But the ’s added to the r.h.s may cancel out each other during elementary row operations, again causing degeneracy.

Remedy: Add different values of i’s to different r.h.s. so that cancellation

does not occur.

OR-1 2009 18

Add i to the i-th r.h.s., i = 1, …, m with the following property.

Then, it can be shown that the values of the basic variables never become 0 in subsequent simplex iterations, hence no cycling occurs. (In practice, precision can cause problems.)

(In actual implementation, 1= , 2= 2, 3= 3, … may be used or

random numbers in [0, ] for some fixed small are used. )

We may be concerned about the correctness of the optimal solution, but in practice the revised simplex method is usually used instead of dictionary. It has the capability to recover the correct solution values if the current basis is known (Chapter 7). Then we can restart the algorithm again without adding i’s, once we escape

degenerate iterations. Hence the effect of adding i’s to the r.h.s

can be eliminated.

) ( 10 121 mm

OR-1 2009 19

Perturbation method usually refers to the methods actually adding small i‘s to the right hand sides during simplex iterations.

For lexicographic method, the idea is the same as the perturbation method. But we do not actually add i‘s. However, we perform the

pivots as if i‘s are present.

OR-1 2009 20

Lexicographic ordering of numbers :

Consider r = r0 + r1 1 + …. + rm m, s = s0 + s1 1 + …. + sm m

If r s, there is the smallest subscript k such that rk sk .

We say that r is lexicographically smaller than s if rk < sk .

(Similar terminology is used for vectors too)

Then if

r is smaller than s if r is lexicographically smaller than s.

Ex) r = 2 + 211 + 19 2 + 200003

s = 2 + 211 + 202 + 203 + 154 + 145

r is lexicographically smaller than s.

, 10 121 mm

OR-1 2009 21

Lexicographic method : Smart implementation of the perturbation method. We do not actually add epsilons to the r.h.s., but epsilons are treated as symbols (like variables) denoting very small positive numbers.

Moreover, during the simplex iterations, the coefficients of epsilons can be read from the simplex tableau directly without actually expressing the epsilon variables in the tableau

371

264321

154321

1

05.05.15.0

095.25.55.0

xx

xxxxx

xxxxx

02495710 4321 xxxxz

Note that i in the above form may be regarded as additional

variables in the equations like x. Any solution x, (and z) need to satisfy the above equations.

OR-1 2009 22

Lexicographic method

371

264321

154321

1

05.05.15.0

095.25.55.0

xx

xxxxx

xxxxx

0 249 57 10 4321 xxxxz

321765

371

32763

327642

25225~0~0

100~0~

2120~ ~0

181180420300

xxx

xx

xxx

xxxxz

(after two iterations)

(0+21 )

(0+22 )

(1+3 )

OR-1 2009 23

Note that the coefficient matrix for the basic variables x5, x6, x7 and

the coefficient matrix for 1, 2, 3 are the same identity matrices in

the beginning of the lexicographic method.

Since we use the elementary row operations in the simplex iterations, those two coefficient matrices have the same elements in the following iterations. Hence, we can read the coefficients of 1, 2, 3

from the coefficients of x5, x6, x7 . So we do not actually need to add

1, 2, 3 to the tableau.

Usually, lexicographic method means we read coefficients of i’s from

the coefficients of corresponding x variables.

Hence, in the example, we actually do not add i’s. We first perform

ratio test using the entering column and the r.h.s. If ties occur, then perform ratio test (for tied rows) again using the entering column and the column for x5, etc.

OR-1 2009 24

Can cycling be prevented?

Since the elementary row operation matrices are nonsingular, the two matrices for variables and corresponding x variables remain nonsingular after applying elementary row operations (elementary row operation is equivalent to premultiplying the corresponding nonsingular matrices. Since the matrix for variables is identity matrix initially, it is nonsingular. Hence we also obtain nonsingular matrix).

In other words, no row with all 0 elements appear (in the -matrix). Hence the values of basic variables never become 0. (conceptually, after adding small i values.)

OR-1 2009 25

Simplex pivot rule guarantees that all basic variables are nonnegative at any time (treating i’s as small positive values). Hence, combined

with above, it guarantees that the values of basic variables are always positive (conceptually). So strict increase of the objective value is guaranteed. It guarantees that the same tableau (or dictionary) will not appear again.

We can read the real solution value by ignoring the terms in the current dictionary.

It is also observed that the lexicographic method can be started and stopped at any time during the simplex iterations. We just conceptually add or drop the terms.

See the text for more rigorous proof. It proves that rows with all 0 elements never appear in -matrix, hence no solution will have 0 value, i.e. no degeneracy occurs in subsequent iterations.

OR-1 2009 26

Initialization (two-phase method)

We need an initial b.f.s to start the simplex method.

If we have bi < 0 for some constraint i, the slack variable xn+i = bi < 0

is not feasible.

Consider easily obtained solution xi = 0 for all i in (1) and then

subtract some positive number x0 from all l.h.s. so that it becomes

feasible solution to (1),

i.e. xi = 0 for all i, x0 =

Now, if we can find a feasible solution to (1) with x0 = 0, it is a feasible

solution to (1) using original variables.

maximize

n

jjj xc

1

n

jijij mibxa

1,...,1,subject to

njx j ,...,2,1,0

(1)

ibi bi }0:{max

OR-1 2009 27

Hence solve the following problem using the easily obtained initial feasible solution and simplex method to find an optimal solution with x0 = 0. Note that x0 is a nonnegative variable.

(1) has a feasible solution (2) has an optimal solution with optimal value 0 (x0 = 0)

If we find an optimal solution with x0 = 0, we can obtain a feasible

solution to (1) by disregarding x0.

One point to be careful is that we need a b.f.s. to perform the simplex method.

maximize 0x

n

jijij mibxxa

10 ,...,1,subject to

njx j ,...,2,1,0,0

(2)

)(min 0x

OR-1 2009 28

Example

321max xxx

0,,

12

532

422

321

321

321

321

xxx

xxx

xxx

xxx

s.t.

0,,,,,

21

325

224

6543210

03216

03215

03214

,

xxxxxxx

xxxxx

xxxxx

xxxxx

0xw

0max x-

0,,,

12

532

422

3210

0321

0321

0321

xxxx

xxxx

xxxx

xxxx

s.t.

We cannot perform simplex iteration in this dictionary since the basic solution is not feasible (nonnegativity violated)

However a feasible dictionary can be easily obtained by one pivot.

OR-1 2009 29

Current basic solution is not feasible. Let x0 enter basis and the

slack variable with most negative value leaves the basis (It is not a simplex iteration. Just perform the pivot, not considering obj. value. It does not change the solution set.)

53216

5324

53210

3434

29

35

xxxxx

xxxx

xxxxx

2

5321 325 xxxxw

0614

06512

06513

2 3

6.02.04.06.02.2

8.06.02.02.06.1

xxxx

xxxxx

xxxxx

00 xw

0,,,,,

21

325

224

6543210

03216

03215

03214

,

xxxxxxx

xxxxx

xxxxx

xxxxx

0xw

Perform simplex method. After two iterations, we get the optimal dictionary

OR-1 2009 30

We obtained optimal solution with value 0. Hence the current optimal solution gives a b.f.s. to the original problem. Drop x0 (no

more needed) and replace the objective function with the original one. z row is used to read the objective value of a given solution, hence it can be added or dropped without affecting the feasible solution set to the LP. Note that the current b.f.s. is a b.f.s. to the original problem.

0614

06512

06513

23

6.02.04.06.02.2

8.06.02.02.06.1

xxxx

xxxxx

xxxxx

00 xw

614

6512

6513

3

2.04.06.02.2

6.02.02.06.1

xxx

xxxx

xxxx

3210 xxxz

OR-1 2009 31

Express it in dictionary form (only nonbasic variables appear in the r.h.s.) by substituting the basic variables in the objective function.

Now, restart the simplex method with the current dictionary

651

6516511

321

4.02.02.06.0

)6.02.02.06.1()2.04.06.02.2(

xxx

xxxxxxxz

xxxz

614

6512

6513

3

2.04.06.02.2

6.02.02.06.1

xxx

xxxx

xxxx

651 4.02.02.06.0 xxxz

OR-1 2009 32

Algorithm strategy in phase one : choose x0 as leaving variable

in case of ties in the minimum ratio test.

2 possible cases in phase one optimal solution1. w nonzero ( w < 0 ), x0 basic original problem is infeasible

2. w = 0 , x0 nonbasic drop w, express original objective function z

in terms of nonbasic variables, continue the simplex method.

(Note that w = 0, x0 basic can’t happen by our strategy)

Similar idea can be used when the original LP is given in equality form. (Without converting a equality using two inequalities, simplex method can be used directly to solve the equality form. More in Chapter 8.)

OR-1 2009 33

(Fundamental theorem of LP)

Every LP in standard form has the following properties.

1. No optimal solution either unbounded or infeasible

2. feasible solution a basic feasible solution

3. optimal solution a basic optimal solution