The Dual Simplex Algorithm - eaton.math.rpi.edueaton.math.rpi.edu/.../notesMATP4700/lecture13/13C_dualsimplexbeamer.pdf · The simplex algorithm The simplex algorithm Once Phase I

The Dual Simplex Algorithm

John E. Mitchell

Department of Mathematical SciencesRPI, Troy, NY 12180 USA

October 2018

Mitchell The Dual Simplex Algorithm 1 / 25

The simplex algorithm

Outline

1 The simplex algorithm

2 The dual simplex algorithm

3 Example

4 The choice of pivot column

5 Interpretation in the dual

6 The primal iterates in (x3, x5)-space



Optimality conditions

The simplex algorithm is stated in terms of the linear optimizationproblem

minx∈IRn cT x maxy∈IRm bT ysubject to Ax = b (P) dual: subject to AT y ≤ c (D)

x ≥ 0

A pair of points x ∈ IRn and y ∈ IRm is optimal if and only if it satisfiesthe following three conditions:

Primal feasibility: Ax = b, x ≥ 0.Dual feasibility: AT y ≤ c.Complementary slackness: Let w = c − AT y . Then xjwj = 0 forj = 1, . . . ,n.






x ≥ 0








x ≥ 0






Once Phase I is complete, the usual (primal) simplex algorithm alwaysmaintains primal feasibility.

Further, at each iteration, we can construct a dual solution thatsatisfies complementary slackness.

Thus, we’re only lacking dual feasibility. Now, the reduced costs areequal to the dual slacks, when y is constructed in this way. So,iterating to get the dual slacks nonnegative is equivalent to workingtowards dual feasibility.


The dual simplex algorithm

Outline



3 Example






When to use dual simplex

In some settings, we may have dual feasibility and be lacking primalfeasibility, so the entries in c are nonnegative, while b has somenegative entries.

In this case, we can use the dual simplex algorithm, whichmaintains dual feasibility,maintains complementary slackness,and works towards primal feasibility













Dual simplex canonical formA tableau is in dual simplex canonical form if:

1 it is in standard form,2 each reduced cost cj ≥ 0, j = 1, . . . ,n,3 the constraint matrix contains the m columns of the identity matrix,4 and the objective function coefficients corresponding to these m

columns are zero.Note that we make no assumptions about the right hand sideentries bi , i = 1, . . . ,m; on the other hand, we require all the reducedcosts to be nonnegative.

For example, the following tableau is in dual simplex canonical form:

x1 x2 x3 x4 x50 0 0 2 0 1−5 0 1 −1 0 −114 0 0 2 1 3

8 1 0 1 0 2







x1 x2 x3 x4 x50 0 0 2 0 1−5 0 1 −1 0 −114 0 0 2 1 3

8 1 0 1 0 2







x1 x2 x3 x4 x50 0 0 2 0 1−5 0 1 −1 0 −114 0 0 2 1 3

8 1 0 1 0 2







x1 x2 x3 x4 x50 0 0 2 0 1−5 0 1 −1 0 −114 0 0 2 1 3

8 1 0 1 0 2




At each iteration:we first choose a pivot row to be a row with bi < 0,then choose a pivot column to ensure all reduced costs remainnonnegative.

As an alternative, we could use Phase I to get a feasible primalsolution, and then use the regular simplex Phase II to solve theproblem.

Dual simplex is a better approach because it takes advantage of thefact that the reduced costs are already nonnegative.

In contrast, a Phase I approach may well result in many negativereduced costs.




At each iteration:we first choose a pivot row to be a row with bi < 0,then choose a pivot column to ensure all reduced costs remainnonnegative.

As an alternative, we could use Phase I to get a feasible primalsolution, and then use the regular simplex Phase II to solve theproblem.

Dual simplex is a better approach because it takes advantage of thefact that the reduced costs are already nonnegative.

In contrast, a Phase I approach may well result in many negativereduced costs.


Example

Outline



3 Example





Example

An exampleConsider the linear optimization problem

minx∈IR5 2x3 + x5subject to x2 − x3 − x5 = −5

2x3 + x4 + 3x5 = 14x1 + x3 + 2x5 = 8

x1, x2, x3, x4, x5 ≥ 0

This is in dual simplex canonical form. The tableau is:

x1 x2 x3 x4 x50 0 0 2 0 1−5 0 1 −1 0 −114 0 0 2 1 3

8 1 0 1 0 2


Example

PivotingWe first choose the pivot row. We choose the row with the mostnegative bi , so row 1 is the pivot row. We pick the pivot column toensure the reduced costs stay nonnegative. If we choose x3 toenter the basis, we get a tableau that is not in dual canonical form:

x1 x2 x3 x4 x50 0 0 2 0 1−5 0 1 -1© 0 −114 0 0 2 1 3

8 1 0 1 0 2

−R1 then R0 − 2R1

−→

x1 x2 x3 x4 x510 0 2 0 0 −1

5 0 −1 1 0 1. . . . . .. . . . . .


Example

A different entering variable

On the other hand, if we choose x5 to enter the basis, the resultingtableau is in dual canonical form:

x1 x2 x3 x4 x50 0 0 2 0 1−5 0 1 −1 0 -1©14 0 0 2 1 3

8 1 0 1 0 2

−R1 then R0 − R1,R2 − 3R1,R3 − 2R1

−→

x1 x2 x3 x4 x5−5 0 1 1 0 0

5 0 −1 1 0 1−1 0 3 −1 1 0−2 1 2 −1 0 0


Example

The second pivotWe’re still not in optimal form, so we iterate again. Choosing the mostnegative bi leads to row 3 as the pivot row. We want to pivot on anegative entry in the row, so that the updated b3 is positive. So x3enters the basis:

x1 x2 x3 x4 x5−5 0 1 1 0 0

5 0 −1 1 0 1−1 0 3 −1 1 0−2 1 2 -1© 0 0

−R3 then R0 − R3,R1 − R3,R2 + R3

−→

x1 x2 x3 x4 x5−7 1 3 0 0 0

3 1 1 0 0 11 −1 1 0 1 02 −1 −2 1 0 0

This is in optimal form, so the optimal solution is x∗ = (0,0,2,1,3),with optimal value 7 and basic variables x3, x4, and x5.


The choice of pivot column

Outline



3 Example






A minimum ratio test

Given the pivot row i , the general rule is to choose a column j with

aij < 0 and j ∈ argmin{

cj

−aij: aij < 0

}So we require

aij < 0and column j wins the modified minimum ratio test.



A minimum ratio test

Given the pivot row i , the general rule is to choose a column j with

aij < 0 and j ∈ argmin{

cj

−aij: aij < 0

}So we require

aij < 0and column j wins the modified minimum ratio test.


Interpretation in the dual

Outline



3 Example






Dual solutions

The dual of the example problem is

maxy∈IR3 −5y1 + 14y2 + 8y3subject to y3 ≤ 0

y1 ≤ 0−y1 + 2y2 + y3 ≤ 2

y2 ≤ 0−y1 + 3y2 + 2y3 ≤ 1

The initial basic sequence is S = (2,4,1), so the corresponding dualsolution satisfying complementary slackness is y = (0,0,0), which isdual feasible.

We chose to pivot on row 1 in the primal problem, so that correspondsto pivoting on the first column in the dual.



Dual solutions and the minimum ratio

The dual constraints can be written

y3 ≤ 00 ≤ 0− y1

2y2 + y3 ≤ 2 + y1y2 ≤ 0

3y2 + 2y3 ≤ 1 + y1

=⇒need − 1 ≤ y1 ≤ 0to stay feasiblewith y2 = y3 = 0

Currently, we have y1 = 0 and we look to decrease y1, since that willincrease the objective function of the dual maximization problem. Sowe choose y1 = −t for some t ≥ 0.

To stay dual feasible, we need the right hand side values to staynonnegative, and this requirement corresponds to ensuring each dualslack cj + aij t ≥ 0, so we choose the pivot row according to themodified minimum ratio test given above.



Dual steplength and the minimum ratio

Taking y2 = y3 = 0 and y1 = −t , we require the following for dualfeasibility:

0 ≤ 00 ≤ 0 + t0 ≤ 2− t0 ≤ 00 ≤ 1− t

=⇒need 0 ≤ t ≤ 1to stay feasiblewith y2 = y3 = 0, y1 = −t

Thus, the final dual constraint forces t = 1 and y1 = −1 and since thelast dual constraint becomes active we can make x5 basic.

The updated dual solution is y = (−1,0,0) and the updated basicsequence is S = (5,4,1).



The second iteration

In the second iteration, we chose row 3 in the primal problem,corresponding to changing y3 in the dual problem.

The primal basic variable for row 3 is x1, so x1 becomes nonbasic, sofrom complementary slackness we can now allow the first dualconstraint to be satisfied strictly.

The variables x4 and x5 remain basic, so we need to choose anupdated dual solution that keeps the last two dual constraints active.

x1 x2 x3 x4 x5−5 0 1 1 0 0

5 0 −1 1 0 1−1 0 3 −1 1 0−2 1 2 -1© 0 0



The dual directionTo motivate the choice of steplength in the dual, we can consider adirection d = (d1,d2,d3) of change in the dual variables.

The updated dual solution has the form

y ← (−1,0,0) + td = (−1 + td1, td2, td3)

The dual slacks are w = c − AT y , so the last two slacks are

w4 = 0− y2= 0− td2 = −td2

and

w5 = 1 + y1 − 3y2 − 2y3= 1 + (−1 + td1)− 3td2 − 2td3= 0− t(−d1 + 3d2 + 2d3)



The dual directionSo

w4 = −td2

w5 = −t(−d1 + 3d2 + 2d3)

Keeping w4 = w5 = 0 requires a direction d = (2,0,1) soy = (−1,0,0) + t(2,0,1) = (−1 + 2t ,0, t).

Plugging this formula into the dual constraints gives:

t ≤ 0−1 + 2t ≤ 0

1− 2t + 2(0) + t ≤ 2(0) ≤ 0

1− 2t + 3(0) + 2t ≤ 1

=⇒

t ≤ 02t ≤ 1−t ≤ 1

0 ≤ 00 ≤ 0

=⇒

need − 1 ≤ t ≤ 0to stay feasible withy = (−1 + 2t ,0, t)



The choice of steplength t

t ≤ 02t ≤ 1−t ≤ 1

0 ≤ 00 ≤ 0

=⇒need − 1 ≤ t ≤ 0to stay feasible withy = (−1 + 2t ,0, t)

Hence we take t = −1, and the third dual constraint becomes activewhich allows x3 to enter the basis in the primal.

The updated dual solution is y = (−3,0,−1).

It can be checked that this solution is dual feasible, with value 7.


The primal iterates in (x3, x5)-space

Outline



3 Example






Graphing the problem

x3

x5

0

2

4

2 4 6 8x

3 +x

5 =5, x

2 =0

x3 + 2x5 = 8, x1 = 0

2x3 + 3x5 = 14, x4 = 0

optimal contour2x3 + x5 = 7

y = (0,0,0)




x3

x5

0

2

4

2 4 6 8x

3 +x

5 =5, x

2 =0

x3 + 2x5 = 8, x1 = 0

2x3 + 3x5 = 14, x4 = 0


pivot row: x2 becomes nonbasic

x1 x2 x3 x4 x50 0 0 2 0 1−5 0 1 −1 0 -1©14 0 0 −2 1 3

8 1 0 1 0 2

y = (0,0,0)




x3

x5

0

2

4

2 4 6 8x

3 +x

5 =5, x

2 =0

x3 + 2x5 = 8, x1 = 0

2x3 + 3x5 = 14, x4 = 0


bad: overshoot optimal contour

x1 x2 x3 x4 x50 0 0 2 0 1−5 0 1 −1 0 -1©14 0 0 −2 1 3

8 1 0 1 0 2

y = (0,0,0)




x3

x5

0

2

4

2 4 6 8x

3 +x

5 =5, x

2 =0

x3 + 2x5 = 8, x1 = 0

2x3 + 3x5 = 14, x4 = 0

optimal contour2x3 + x5 = 7x1 and x4

go negative

x1 x2 x3 x4 x50 0 0 2 0 1−5 0 1 −1 0 -1©14 0 0 −2 1 3

8 1 0 1 0 2

y = (0,0,0)




x3

x5

0

2

4

2 4 6 8x

3 +x

5 =5, x

2 =0

x3 + 2x5 = 8, x1 = 0

2x3 + 3x5 = 14, x4 = 0


x1 x2 x3 x4 x5−5 0 1 1 0 0

5 0 −1 1 0 1−1 0 3 −1 1 0−2 1 2 -1© 0 0

y = (0,0,0)

y = (−1,0,0)




x3

x5

0

2

4

2 4 6 8x

3 +x

5 =5, x

2 =0

x3 + 2x5 = 8, x1 = 0

2x3 + 3x5 = 14, x4 = 0


pivot row: x1 becomes nonbasic

x1 x2 x3 x4 x5−5 0 1 1 0 0

5 0 −1 1 0 1−1 0 3 −1 1 0−2 1 2 -1© 0 0

y = (0,0,0)

y = (−1,0,0)




x3

x5

0

2

4

2 4 6 8x

3 +x

5 =5, x

2 =0

x3 + 2x5 = 8, x1 = 0

2x3 + 3x5 = 14, x4 = 0


x4 becomes positive also

x1 x2 x3 x4 x5−5 0 1 1 0 0

5 0 −1 1 0 1−1 0 3 −1 1 0−2 1 2 -1© 0 0

y = (0,0,0)

y = (−1,0,0)




x3

x5

0

2

4

2 4 6 8x

3 +x

5 =5, x

2 =0

x3 + 2x5 = 8, x1 = 0

2x3 + 3x5 = 14, x4 = 0


x1 x2 x3 x4 x5−7 1 3 0 0 0

3 1 1 0 0 11 −1 1 0 1 02 −1 −2 1 0 0

y = (0,0,0)

y = (−1,0,0)

x = (0,0,2,1,3)

y = (−3,0,−1)


Documents

The Dual Simplex Algorithm - eaton.math.rpi.edueaton.math.rpi.edu/.../notesMATP4700/lecture13/13C_dualsimplexbeamer.pdf · The simplex algorithm The simplex algorithm Once Phase I