Chapter 1: Linear Programming - Northwestern Universityclark/368/handouts/... · 2016. 12. 6. · Chapter 1: Linear Programming 11. Slack Variables For large number of variables,

Chapter 1: Linear Programming

Math 368

c© Copyright 2013 R Clark Robinson

May 22, 2013

Chapter 1: Linear Programming 1

Max and Min

For f : DDD ⊂ Rn → R, f (DDD) = {f (x) : x ∈ DDD }is set of attainable values of f on DDD , or image of DDD by f .

f has a maximum on DDD at xM ∈ DDD provided that

f (xM) ≥ f (x) for all x ∈ DDD .

max{f (x) : x ∈ DDD } = f (xM), maximum value of f on DDD .

xM is called a maximizer of f on DDD .

arg max{f (x) : x ∈ DDD } = {x ∈ DDD : f (x) = f (xM) }.

f has a minimum on DDD at xm ∈ DDD provided that

f (xm) ≤ f (x) for all x ∈ DDD .

min{f (x) : x ∈ DDD } = f (xm), minimum of f on DDD

arg min{f (x) : x ∈ DDD } = {x ∈ DDD : f (x) = f (xm) }set of minimizers

f has an extremum at x0 p.t. x0 is either a maximizer or minimizer.Chapter 1: Linear Programming 2

Basic Optimization Problem

No maximizer or minimizer of f (x) = x3 on (0, 1),

arg max{ x3 : 0 < x < 1 } = ∅, & arg min{ x3 : 0 < x < 1 } = ∅,

Optimization Problem:

Does f (x) attain a maximum (or minimum) for some x ∈ DDD?

If so, what is the maximum value (or minimum value) on DDD and

what are the points at which f (x) attains a maximum (or minimum)subject to x ∈ DDD?


Notations

v ≥ w in Rn means vi ≥ wi for 1 ≤ i ≤ n

v � w in Rn means vi > wi for 1 ≤ i ≤ n

Rn+ = { x ∈ Rn : x ≥ 0 } = { x ∈ Rn : xi ≥ 0 for 1 ≤ i ≤ n }

Rn++ = { x ∈ Rn : x � 0 } = { x ∈ Rn : xi > 0 for 1 ≤ i ≤ n }


Linear Programming: 1.4.1 Wheat-Corn Example

Up to 100 acres of land can be used to grow wheat and/or corn:

x1 acres used to grow wheat and

x2 acres used to grow corn: x1 + x2 ≤ 100.

Cost or capital constraint: 5x1 + 10x2 ≤ 800.

Labor constraint: 2x1 + x2 ≤ 150.

Profit: f (x1, x2) = 80x1 + 60x2. Objective function

Problem:Maximize: 80x1 + 60x2 (profit),Subject to: x1 + x2 ≤ 100, (land)

5x1 + 10x2 ≤ 800, (capital)2x1 + x2 ≤ 150, (labor)x1 ≥ 0, x2 ≥ 0.

All constraints and objective function are linear: lin. programming prob.


Example, continued

Feasible set FFF is set of all the points satisfying all constraints

x1 + x2 ≤ 100 (land), 5x1 + 10x2 ≤ 800 (capital),2x1 + x2 ≤ 150 (labor) x1 ≥ 0, x2 ≥ 0.

(0, 0)

(50, 50)

(40, 60)

Vertices of the feasible set: (0, 0), (75, 0), (50, 50), (40, 60), (0, 80).

Other points where two constraints are tight(

1403 , 170

3

), (100, 0), etc

lie outside the feasible set, 1403 + 170

3 = 3103 > 100, 2(100) > 150, . . .


Example, continued

Since ∇f = (80, 60)>>> 6= (0, 0)>>>,

maximum must be on boundary of FFF .

f (x1, x2) along an edge is linear combination of values at end points.

If a maximizer were in middle of an edge,

then f would have the same value at two end points of this edge.

Maximizer can be found at one of vertices.

Values at vertices:

f (0, 0) = 0, f (75, 0) = 6000, f (50, 50) = 7000,

f (40, 60) = 6800, f (0, 80) = 4800.

max value of f is 7000, maximizer (50, 50). End of Example


Standard Max Linear Programming Problem (MLP)

Maximize objective function: f (x) = c qx = c1x1 + · · ·+ cnxn,Subject to resource constraints: a11x1 + · · ·+ a1nxn ≤ b1

......

...am1x1 + · · ·+ amnxn ≤ bm

xj ≥ 0 for 1 ≤ j ≤ n.

Given data: c = (c1, . . . , cn)>>>, m × n matrix A = (aij),

b = (b1, . . . , bm)>>> with all bi ≥ 0,

Constraints using matrix notation are Ax ≤ b and x ≥ 0.

Feasible set: FFF = { x ∈ Rn+ : Ax ≤ b }.


Minimization Example

Minimize: 3x1 + 2x2,Subject to: 2x1 + x2 ≥ 4,

x1 + x2 ≥ 3,x1 + 2x2 ≥ 4, x1 ≥ 0, and x2 ≥ 0.

(0, 4)

(1, 2)

(2, 1)

(4, 0)

FFF is unbounded but f (x) ≥ 0 is bded below so f (x) has a minimum.

Vertices: (4, 0), (2, 1), (1, 2), (0, 4)Values: f (4, 0) = 12, f (2, 1) = 8, f (1, 2) = 7, f (0, 4) = 8.

min{f (x) : x ∈FFF } = 7 arg min{f (x) : x ∈FFF } = {(1, 2)}.Chapter 1: Linear Programming 9

Geometric Method of Solving Linear Prog Problem

1 Determine or draw the feasible set FFF .

If FFF = ∅, then problem has no optimal solution,

problem is called infeasible

2 Problem is called unbounded and has no solution p.t.

objective function on FFF has

a. arbitrarily large positive values for a maximization problem, or

b. arbitrarily large negative values for a minimization problem.

3 A problem is called bounded p.t. it is not infeasible nor unbounded;

an optimal solution exists.

Determine all the vertices of FFF and values at vertices.

Choose the vertex of FFF producing the maximum or minimum value

of the objective function.


Rank of a Matrix

Rank of a matrix A is dimension of column space of A.

i.e., largest number of linearly independent columns of A.

Same as number of pivots (in row reduced echelon form of A).

rank(A) = k iff k ≥ 0 is the largest integer s.t. det(Ak) 6= 0,

where Ak is any k × k submatrix of A

formed by selecting any k columns and any k rows.

Ak is submatrix of pivot columns and rows.

Sketch: Let A′ be submatrix with k linearly independent columns

that span column space; rank(A′) = k

dim(row space of A′) = k, so k rows of A′ to get k × k Ak

with rank(Ak) = k, so det(Ak) 6= 0.


Slack Variables

For large number of variables, need a practical algorithm.

Simplex method uses row reduction as solution method.

First step: make all the inequalities of type xi ≥ 0.

Inequality of the form ai1x1 + · · ·+ ainxn ≤ bi for bi ≥ 0 is called

resource constraint.

For resource constraint, introduce slack variable si by

ai1x1 + · · ·+ ainxn + si = bi with si ≥ 0.

si represents unused resource.

Introduction of a slack variable changes a resource constraint into

equality constraint and si ≥ 0.


Introducing Slack Variables into Wheat-Corn Example

x1 + x2 + s1 = 100,5x1 + 10x2 + s2 = 800,2x1 + x2 + s3 = 150, s1, s2, s3 ≥ 0.

In matrix form,

Maximize: (80, 60, 0, 0, 0) q (x1, x2, s1, s2, s3) = 80 x1 + 60 x2

Subj. to:

1 1 1 0 05 10 0 1 02 1 0 0 1

x1

x2

s1s2s3

=

100800150

and

x1 ≥ 0,x2 ≥ 0,s1 ≥ 0,s2 ≥ 0,s3 ≥ 0.

Because of 1’s and 0’s in last 3 columns of matrix, rank is 3.

Initial feasible solution x1 = 0 = x2, s1 = 100, s2 = 800, s3 = 150


Wheat-Corn Example, continued

1 1 1 0 05 10 0 1 02 1 0 0 1

x1

x2

s1s2s3

=

100800150

with xi ≥ 0, and si ≥ 0.

Initial feasible sol’n: p0 = (0, 0, 100, 800, 150)>>> with f (p0) = 0.

Rank is 3, so 5− 3 = 2 free variables. x1, x2.

p0 obtained by setting free variables x1 = x2 = 0

and solving for dependent variables, which are 3 slack variables.

“Pivot” to make a different pair the free variables equal to zero and

a different triple of positive variables.



s1s2

s3

p0 p1

If leave the vertex (x1, x2) = (0, 0), or p0 = (0, 0, 100, 800, 150),

making x1 > 0 entering variable while keeping x2 = 0;

first slack variable to become zero is s3 when x1 = 75.

Arrive at the vertex (x1, x2) = (75, 0), or p1 = (75, 0, 25, 425, 0).

New sol’n has two zero variables and three positive variables.

Move along one edge from p0 to p1.

f (p1) = 80(75) = 6000 > 0 = f (p0).

p1 is a better feasible sol’n than p0.



p1p0

p2

p3

s1s2

s3

p1 = (75, 0, 25, 425, 0).

Repeat, leaving p1 by making x2 > 0 entering variable

while keeping s3 = 0. First other variable to become zero is s1.

Arrive p2 = (50, 50, 0, 50, 0)

f (p2) = 80(50) + 60(50) = 7000 > 6000 = f (p1).

p2 is a better feasible solution than p1.

Have moved along another edge of the feasible set from

(x1, x2) = (75, 0) and arrived at (x1, x2) = (50, 50).



If leave p2 by making s3 > 0 entering variable while keeping s1 = 0,

first variable to become zero is s2, arrive at p3 = (40, 60, 0, 0, 10).

f (p3) = 80(40) + 60(60) = 6800 < 7000 = f (p2).

p3 is worse feasible solution than p2.

Let z ∈FFF r {p2}. v = z− p2, vj = pj − p2.

f (v1) = f (p1)− f (p2) < 0, f (v3) = f (p3)− f (p2) < 0

v1 & v3 are basis of R2, so v = y1 v1 + y3 v3 with y1, y3 ≥ 0.

v 6= 0 points into FFF , so (i) y1, y3 ≥ 0 and (ii) y1 > 0 or y3 > 0.

f (z) = f (p2) + f (v) = f (p2) + y1 f (v1) + y3 f (v3) < f (p2)

Since cannot increase f by moving along either edge going out from p2,

p2 is an optimal feasible solution.



In this example,

p0 = (0, 0, 100, 800, 150), p1 = (75, 0, 25, 425, 0),

p2 = (50, 50, 0, 50, 0), p3 = (40, 60, 0, 0, 10),

p4 = (0, 80, 20, 0, 70)

are called basic solutions

since at most 3 variables are positive,

where 3 is the rank (and number of constraints).

End of Example


Slack Variables added to a Standard Lin Prog Prob, MLP

All resource constraints

Given b = (b1, . . . , bm)>>> ∈ Rm+, c = (c1, . . . , cn)

>>> ∈ Rn,

m × n matrix A = (aij).

Find x ∈ Rn+ and s ∈ Rm

+ that

maximize: f (x) = c qxsubject to: a11x1 + · · ·+ a1nxn + s1 = b1

......

...am1x1 + · · ·+ amnxn + sm = bm

xi ≥ 0, sj ≥ 0 for 1 ≤ i ≤ n, 1 ≤ j ≤ m.

Using matrix notation with I m ×m identity matrix,

maximize f (x) subject to [A, I]

[xs

]= b with x ≥ 0, s ≥ 0.

Indicate partition of augmented matrix by extra vertical lines [A | I |b]


Basic Solutions

Assume A m × (n + m) matrix with rank m (like A = [A, I] )

m dependent variables are called basic variables. (var of pivot col’ns)

n free variables are called non-basic variables. (var of non-pivot col’ns)

A basic solution is a solution p satisfying Ap = b

such that columns corresponding to pi 6= 0 are linearly indep.

≤ rank(A) = m.

If p is also feasible with p ≥ 0, then called a basic feasible solution.

Obtain by setting n free variables = 0, and get basic variables ≥ 0,

allow possibly some basic variables = 0


Linear Algebra Solution of Wheat-Corn Problem

Augmented matrix for the original wheat-corn problem isx1 x2 s1 s2 s31 1 1 0 0 1005 10 0 1 0 8002 1 0 0 1 150

with free variables x1 and x2 and basic variables s1, s2, and s3.

If make x1 > 0 while keeping x2 = 0, x1 becomes a new basic variable

( for new pivot coln ) called entering variable.

(i) s1 will become zero when x1 = 1001 = 100,

(ii) s2 will become zero when x1 = 8005 = 160, and

(iii) s3 will become zero when x1 = 1502 = 75.

Since s3 becomes zero for the smallest value of x1,

s3 is the departing variable and

new pivot is 1st column (for x1) and 3rd row ( old pivot for s3 )Chapter 1: Linear Programming 21

First Pivot

Row reducing to make a pivot in first column third row,x1 x2 s1 s2 s31 1 1 0 0 1005 10 0 1 0 8002 1 0 0 1 150

∼

x1 x2 s1 s2 s30 .5 1 0 .5 250 7.5 0 1 2.5 4251 .5 0 0 .5 75

Setting free variables x2 = s3 = 0,

new basic solution p1 = (75, 0, 25, 425, 0)>>>.

Entries in the right (augmented) column give

values of the new basic variables that are > 0.


Including Objective Function in Matrix

Objective function (or variable) is

f = 80x1 + 60x2, or −80x1 − 60x2 + f = 0.

Adding a row for this equation and a column for variable f

keeps track of the value of f during row reduction.x1 x2 s1 s2 s3 f

1 1 1 0 0 0 1005 10 0 1 0 0 8002 1 0 0 1 0 150

80 60 0 0 0 1 0

This matrix including objective function row is called tableau.

Entries in column for f are 1 in objective function row and 0 elsewhere.

In objection function row of this tableau, entries for xi are negative.


First Pivot, continued

Row reducing tableau by making

first column and third row a new pivot,x1 x2 s1 s2 s3 f

1 1 1 0 0 0 1005 10 0 1 0 0 8002 1 0 0 1 0 150

80 60 0 0 0 1 0

∼

x1 x2 s1 s2 s3 f

0 .5 1 0 .5 0 250 7.5 0 1 2.5 0 4251 .5 0 0 .5 0 75

0 20 0 0 40 1 6000

For x2 = s3 = 0, x1 = 75 > 0, s1 = 25 > 0, s2 = 425 > 0,

Bottom right entry of 6000 is new value of f


Second Pivotx1 x2 s1 s2 s3 f

0 .5 1 0 .5 0 250 7.5 0 1 2.5 0 4251 .5 0 0 .5 0 75

0 20 0 0 40 1 6000

x2 & s3 free (non-basic) variables

If pivot back to make s3 > 0, the value of f becomes smaller,

so select x2 as the next entering variable, keeping s3 = 0.

(i) s1 becomes zero when x2 = 25.5 = 50, and

(ii) s2 becomes zero when x2 = 4257.5 = 56.67.

(iii) x1 becomes zero when x2 = 75.5 = 150,

Since the smallest positive value of x1 comes from s1,

s1 is the departing variable and pivot on 1st row 2nd column.


Second Pivot, contin.

Pivot on 1st row 2nd column,x1 x2 s1 s2 s3 f

0 .5 1 0 .5 0 250 7.5 0 1 2.5 0 4251 .5 0 0 .5 0 75

0 20 0 0 40 1 6000

∼

x1 x2 s1 s2 s3 f

0 1 2 0 1 0 500 0 15 1 5 0 501 0 1 0 1 0 50

0 0 40 0 20 1 7000

Entries in column for f don’t change.

f = 7000 objective function


Third Pivot

Why does the objective function decrease when moving along the edge

making s3 > 0 an entering variable, keeping s1 = 0?

(i) x1 becomes zero when s3 = 501 = 50,

(ii) x2 becomes zero when s3 = 501 = 50, and

(iii) s2 becomes zero when s3 = 505 = 10.

Smallest positive value of s3 comes from s2,and pivot on the 2nd row 5th column.

x1 x2 s1 s2 s3 f

0 1 2 0 1 0 500 0 15 1 5 0 501 0 1 0 1 0 50

0 0 40 0 20 1 7000

∼

x1 x2 s1 s2 s3 f

0 1 1 0 0 0 600 0 3 .2 1 0 101 0 2 2 0 0 40

0 0 100 4 0 1 6800

Value of objective function decreases since the entry is already positive

before pivot in column for s3 of the objective function row


Tableau

Drop the column for the variable f from augmented matrix

since it does not play a role in the row reduction.

(entry in last row stays = 1 and others stay = 0)

Augmented matrix with objective function row

but without column for objective function variable

is called the tableau.


Steps in the Simplex Method for Stand MLP

1 Set up the tableau so that all bi ≥ 0. An initial feasible basicsolution is determined by setting xi = 0 and solving for si .

2 Choose as entering variable any free variable with a negative entry inobjection function row. (often most negative)

3 From column selected in previous step, select row for which ratio ofentry in augmented column divided by entry in column selected issmallest value ≥ 0; departing variable is basic variable for this row.

Row reduce the matrix using selected new pivot position.

4 Objective function has no upper bound and no optimal solutionwhen one column has only nonpositive coefficients above a negativecoefficient in objective function row.

5 Solution is optimal when all entries in objective function row arenonnegative.

6 If optimal tableau has zero entry in objective row for nonbasic variableand all basic variables are positive, then nonunique solution.


General Constraints: Requirement Constraints

All bi ≥ 0 (by multiplying inequalities by 1 if necessary.)

Requirement constraint is given by

ai1x1 + · · ·+ ainxn ≥ bi (occur especially for a min problem.)

Require to have at least a minimum amount to quantity.

Can have a surplus of quantity,

so subtract off a surplus variable to get equality

ai1x1 + · · ·+ ainxn − si = bi with si ≥ 0.

To solve equation initially, also add an artificial variable ri ≥ 0,

ai1x1 + · · ·+ ainxn − si + ri = bi .

(Walker uses ai , but we use ri to distinguish from matrix aij .)

Initial sol’n sets the artificial variable ri = bi while si = 0 = xj ∀ j .


Equality Constraints

For an equality constraint ai1x1 + · · ·+ ainxn = bi ,

add one artificial variable

ai1x1 + · · ·+ ainxn + ri = bi ,

with an initial solution ri = bi ≥ 0 while the xj = 0.

For general constraints (with either requirement or equality constraint)

initial solution has all xi = 0 and all surplus variables zero,

while slack variables and artificial variables ≥ 0.

This sol’n is not feasible if any artificial variables is positive

for a requirement constraint or an equality constraint


Minimization Example

Assume two foods are consumed in amounts x1 and x2

with costs per unit of 15 and 7 respectively, and

yield (5, 3, 5) and (2, 2, 1) units of three vitamins respectively.

Problem is to minimize cost 15 x1 + 7 x2 or

Maximize: 15 x1 7 x2

Subject to: 5 x1 + 2 x2 ≥ 603 x1 + 2 x2 ≥ 40, and5 x1 + 1 x2 ≥ 35.

x = 0 not a feasible sol’n,

initial sol’n involves the artificial variables


Minimization Example, continued

The tableau isx1 x2 s1 s2 s3 r1 r2 r35 2 1 0 0 1 0 0 603 2 0 1 0 0 1 0 405 1 0 0 1 0 0 1 35

15 7 0 0 0 0 0 0 0

To eliminate artificial variables, preliminary steps

to force all artificial variables to be zero.

Use artificial objective function that is

negative sum of equations that contain artificial variables,

13x1 5x2 + s1 + s2 + s3 + ( r1 r2 r3) = 135.

13x1 5x2 + s1 + s2 + s3 + R = 135

R = r1 r2 r3 ≤ 0 new variable, with max of 0.



Tableau with the artificial objective function included(but not a column for the variable R) is

x1 x2 s1 s2 s3 r1 r2 r35 2 1 0 0 1 0 0 603 2 0 1 0 0 1 0 405 1 0 0 1 0 0 1 35

15 7 0 0 0 0 0 0 013 5 1 1 1 0 0 0 135

If artificial objective function can be made equal zero,

then this gives an initial feasible basic solution with ri = 0

and only original and slack variables positive.

Then, artificial variables can be dropped and proceed as before.



13 < 5

x1 x2 s1 s2 s3 r1 r2 r35 2 1 0 0 1 0 0 603 2 0 1 0 0 1 0 405 1 0 0 1 0 0 1 35

15 7 0 0 0 0 0 0 013 5 1 1 1 0 0 0 135

355 = 7 < 40

3 ≈ 13.3, 605 = 12: Pivoting on a31 (making into a pivot)

∼

x1 x2 s1 s2 s3 r1 r2 r30 1 1 0 1 1 0 1 25

0 75 0 1 3

5 0 1 35 19

1 15 0 0 1

5 0 0 15 7

0 4 0 0 3 0 0 3 105

0 125 1 1 8

5 0 0 135 44



Pivoting on a22 (19× 57 ≈ 13.57 < 25 < 7× 5)

x1 x2 s1 s2 s3 r1 r2 r30 1 1 0 1 1 0 1 25

0 75 0 1 3

5 0 1 35 19

1 15 0 0 1

5 0 0 15 7

0 4 0 0 3 0 0 3 105

0 125 1 1 8

5 0 0 135 44

∼

x1 x2 s1 s2 s3 r1 r2 r3

0 0 1 57

47 1 5

747

807

0 1 0 57

37 0 5

737

957

1 0 0 17

27 0 1

727

307

0 0 0 207

97 0 20

797

11157

0 0 1 57

47 0 12

7117

807



Pivoting on a14 yields an initial feasible basic solution:

x1 x2 s1 s2 s3 r1 r2 r3

0 0 1 57

47 1 5

747

807

0 1 0 57

37 0 5

737

957

1 0 0 17

27 0 1

727

307

0 0 0 207

97 0 20

797

11157

0 0 1 57

47 0 12

7117

807

∼

x1 x2 s1 s2 s3 r1 r2 r3

0 0 75 1 4

575 1 4

5 16

0 1 1 0 1 1 0 1 25

1 0 15 0 2

515 0 2

5 2

0 0 4 0 1 4 0 1 2050 0 0 0 0 1 1 1 0



After these two steps, (2, 25, 0, 16, 0) is an initial feasible basic solution

and artificial variables can be dropped.

Pivoting on a15 yields final solution:

x1 x2 s1 s2 s3

0 0 75 1 4

5 16

0 1 1 0 1 25

1 0 15 0 2

5 2

0 0 4 0 1 205

∼

x1 x2 s1 s2 s3

0 0 74

54 1 20

0 1 34

54 0 5

1 0 12

12 0 10

0 0 94

54 0 185

.

All entries in the obj fn are positive, so maximal solution:

(10, 5, 0, 0, 20) with an value of 185.

For original problem, minimal sol’n has value of 185.

End of Example


Steps in Simplex Method w/ General Constraints

1 Make all bi ≥ 0 of constraints by multiplying by 1 if necessary.

2 Add a slack variable for each resource inequality, add a surplusvariable and an artificial variable for each requirement constraint, andadd an artificial variable for each equality constraint.

3 If either requirement constraints or equality constraints are present,then form artificial objective function by taking negative sum of allequations that contain artificial variables,dropping terms involving artificial variables.

Set up tableau matrix. (The row for artificial objective function haszeroes in the columns of the artificial variables.)

An initial solution of equations including artificial variables isdetermined by setting all original variables xj = 0,all slack variables si = bi , all the surplus variables si = 0,and all artificial variables ri = bi .


Steps for General Constraints, continued

4 Apply simplex algorithm using artificial objective function.

a. If it is not possible to make artificial objective function equal to zero,then there is no feasible solution.

b. If the artificial variables can be made equal to zero, then dropartificial variables and artificial objective function from tableau andcontinue using initial feasible basic solution constructed.

5 Apply simplex algorithm to actual objective function.Solution is optimal when all entries in objective function row arenonnegative.


Example with Equality Constraint

Consider the problem ofMaximize: 3 x1 + 4 x2

Subject to: 2 x1 + x2 ≤ 6, and2 x1 + 2 x2 ≥ 24,x1 = 8,x1 ≥ 0, x2 ≥ 0.

With slack, surplus, and artificial variables added the problem becomesMaximize: 3 x1 + 4 x2

Subject to: 2 x1 + x2 + s1 = 62 x1 + 2 x2 − s2 + r2 = 24x1 + r3 = 8.

Artificial obj fn is negative sum of the 2nd and 3rd rows

3 x1 − 2 x2 + s2 + R = 32 where R = r2 − r3


Example, continued

The tableau with variables is

x1 x2 s1 s2 r2 r32 1 1 0 0 0 62 2 0 1 1 0 241 0 0 0 0 1 8

3 4 0 0 0 0 03 2 0 1 0 0 32

Pivoting on a31 and then a22,

∼

x1 x2 s1 s2 r2 r30 1 1 0 0 2 220 2 0 1 1 2 81 0 0 0 0 1 8

0 4 0 0 0 3 240 2 0 1 0 3 8

∼

x1 x2 s1 s2 r2 r3

0 0 1 12

12 3 18

0 1 0 12

12 1 4

1 0 0 0 0 1 8

0 0 0 2 2 1 400 0 0 0 1 1 0

Attained a feasible solution of (x1, x2, s1, s2) = (8, 4, 18, 0).


Example, continued

Can now drop artificial objective function and artificial variables.

Pivoting on a1,4x1 x2 s1 s2

0 0 1 12 18

0 1 0 12 4

1 0 0 0 8

0 0 0 2 40

∼

x1 x2 s1 s20 0 2 1 360 1 1 0 221 0 0 0 8

0 0 4 0 112

Optimal solution of f = 112 for (x1, x2, s1, s2) = (8, 22, 0, 36).

End of Example


Duality: Introductory Example

Return to wheat-corn problem (MLP).

Maximize: z = 80x1 + 60x2

subject to: x1 + x2 ≤ 100, (land)5x1 + 10x2 ≤ 800, (capital)2x1 + x2 ≤ 150, (labor), x1, x2 ≥ 0

Assume excess resources can be rented out or shortfalls rentedfor prices of y1, y2, and y3. (Shadow prices of inputs.)

Profit P = 80x1 + 60x2 + (100− x1 − x2)y1

+(800− 5x1 − 10x2)y2 + (150− 2x1 − x2)y3

If profit by renting outside land, then competitors raise price of landforce farmer 100− x1 − x2 ≥ 0.

If 100− x1 − x2 > 0, then market sets y1 = 0.

(100− x1 − x2)y1 = 0 at optimal

Similar other constraints, Farmer’s perspective yields original MLP


Duality: Introductory Example, contin.

P = 80x1 + 60x2 + (100 x1 x2)y1 + (800 5x1 10x2)y2 + (150 2x1 x2)y3

If a resource is slack, 100− x1 − x2 > 0, then market sets y1 = 0.

Market is minimizing P as function of (y1, y2, y3)

Rearranging profit function from markets perspective yields

P = (80 y1 5y2 2y3)x1 + (60 y1 10y2 y3)x2 + 100y1 + 800y2 + 150y3

Coefficients of xi represents net profit after costs of unit of i th-good

If net profit > 0, the competitors grow wheat/corn. Force ≤ 0,

80− y1 − 5y2 − 2y3 ≤ 0 or 80 ≤ y1 + 5y2 + 2y3

60− y1 − 10y2 − y3 ≤ 0 or 60 ≤ y1 + 10y2 + y3

If a resource is slack, 100− x1 − x2 > 0, then market sets y1 = 0.

0 = (100− x1 − x2)y1 0 = (800− 5x1 − 10x2)y2

0 = (150− 2x1 − x2)y3

Market is minimizing P = 100y1 + 800y2 + 150y3



Market’s perspective results in dual minimization problem:

Minimize: w = 100y1 + 800y2 + 150y3

Subject to: y1 + 5y2 + 2y3 ≥ 80, (wheat)y1 + 10y2 + y3 ≥ 60, (corn)y1, y2, y3 ≥ 0.

MLP: Maximize: z = 80x1 + 60x2

Subject to: x1 + x2 ≤ 100,5x1 + 10x2 ≤ 800,2x1 + x2 ≤ 150, x1, x2 ≥ 0

1. Coefficient matrices of xi and yi are transposes of each other.

2. Coefficients for objective function of MLPbecome constants for inequalities of dual mLP.

3. Constants for inequalities of MLPbecome coefficients for objective function of dual mLP.



For the wheat-corn MLP problem, final tableau

x1 x2 s1 s2 s31 1 1 0 0 1005 10 0 1 0 8002 1 0 0 1 150

80 60 0 0 0 0

∼

x1 x2 s1 s2 s30 1 2 0 1 500 0 15 1 5 501 0 1 0 1 50

0 0 40 0 20 7000

Optimal sol’n MLP is

x1 = 50 and x2 = 50 with a payoff of 7000.

Optimal sol’n dual mLP is (by theorem given later)

y1 = 40, y2 = 0, and y3 = 20 with the same payoff,

where 40, 0, and 20 are entries in bottom row of final tableau

in columns associated with slack variables.End of Example


Example, Bicycle Manufacturing

A bicycle manufacturer manufactures x1 3-speeds and x2 5-speeds.

Maximize profits given by z = 12x1 + 15x2.

Constraints are20 x1 + 30 x2 ≤ 2400 finishing time in minutes,15 x1 + 40 x2 ≤ 3000 assembly time in minutes,x1 + x2 ≤ 100 frames used for assembly,x1 ≥ 0 x2 ≥ 0.

Dual problem is

Minimize: w = 2400 y1 + 3000 y2 + 100 y3,Subject to: 20 y1 + 15 y2 + y3 ≥ 12,

30 y1 + 40 y2 + y3 ≥ 15,y1 ≥ 0, y2 ≥ 0, y3 ≥ 0.


Bicycle Manufacturing, contin.

MLPx1 x2 s1 s2 s320 30 1 0 0 240015 40 0 1 0 30001 1 0 0 1 100

12 15 0 0 0 0

∼

x1 x2 s1 s2 s30 10 1 0 20 4000 25 0 1 15 15001 1 0 0 1 100

0 3 0 0 12 1200

∼

x1 x2 s1 s2 s3

0 1 110 0 2 40

0 0 52 1 35 500

1 0 110 0 3 60

0 0 310 0 6 1320

Optimal sol’n has x1 = 60 3-speeds, x2 = 40 5-speeds,

with a profit of $1320.


Bicycle Manufacturing, continued

x1 x2 s1 s2 s30 1 1

10 0 2 40

0 0 52 1 35 500

1 0 110 0 3 60

0 0 310 0 6 1320

yi are marginal values of corresponding constraint.

Dual problem has a solution of

y1 = 310 profit per finishing minute,

y2 = 0 profit per assembly minute

y3 = 6 profit per frame.

Additional units of the exhausted resources, finishing time and frames,

contribute to the profit but not assembly time.


Rules for Forming Dual LP

Maximization Problem, MLP Minimization Problem, mLP

i th constraint∑

j aijxj ≤ bi i th variable 0 ≤ yi

i th constraint∑

j aijxj ≥ bi i th variable 0 ≥ yi

i th constraint∑

j aijxj = bi i th variable yi unrestricted

j th variable 0 ≤ xj j th constraint∑

i aijyi ≥ cj

j th variable 0 ≥ xj j th constraint∑

i aijyi ≤ cj

j th variable xj unrestricted j th constraint∑

i aijyi = cj

Standard conditions for MLP,∑

j aijxj ≤ bi or 0 ≤ xj , corresp to

standard conditions for mLP, 0 ≤ yi or∑

i aijyi ≥ cj

Nonstandard conditions, ≥ bi or 0 ≥ xj , corresp to nonstand conditions

Equality constraints correspond to unrestricted variables

These rules follow from proof of Duality Theorem (given subsequently)


Example

Minimize : 8y1 + 10y2 + 4y3

Subject to : 4y1 + 2y2 − 3y3 ≥ 202y1 + 3y2 + 5y3 ≤ 1506y1 + 2y2 + 4y3 = 40y1 unrestricted, y2 ≥ 0, y3 ≥ 0

By table, dual maximization problem is

Maximize : 20x1 + 150x2 + 40x3

Subject to : 4x1 + 2x2 + 6x3 = 82x1 + 3x2 + 2x3 ≤ 103x1 + 5x2 + 4x3 ≤ 4

x1 ≥ 0, x2 ≤ 0, x3 unrestricted


Example, continued

Maximize : 20x1 + 150x2 + 40x3

Subject to : 4x1 + 2x2 + 6x3 = 82x1 + 3x2 + 2x3 ≤ 103x1 + 5x2 + 4x3 ≤ 4

x1 ≥ 0, x2 ≤ 0, x3 unrestricted

By making change of variables x2 = v2 and x3 = v3 − w3,

all restrictions on variables are ≥ 0:

Maximize : 20x1 − 150v2 + 40v3 − 40w3

Subject to : 4x1 − 2v2 + 6v3 − 6w3 = 82x1 − 3v2 + 2v3 − 2w3 ≤ 103x1 − 5v2 + 4x3 − 4w3 ≤ 4

x1 ≥ 0, v2 ≥ 0, v3 ≥ 0, w3 ≥ 0


Example, continued

Tableau for maximization problem with variables x1, v2, v3, w3,

with artificial variable r1, and with slack variables s2 and s3 is

x1 v2 v3 w3 s2 s3 r14 2 6 6 0 0 1 82 3 2 2 1 0 0 103 5 4 4 0 1 0 4

20 150 40 40 0 0 0 04 2 6 6 0 0 0 8

∼

x1 v2 v3 w3 s2 s3 r1

1 12

32

32 0 0 1

4 2

0 2 1 1 1 0 12 6

0 132

172

172 0 1 3

4 10

0 140 10 10 0 0 5 400 0 0 0 0 0 1 0

.


Example, continued

Drop artificial objective function but keep artificial variable

to determine value of its dual variable.

x1 v2 v3 w3 s2 s3 r1

1 12

32

32 0 0 1

4 2

0 2 1 1 1 0 12 6

0 132

172

172 0 1 3

4 10

0 140 10 10 0 0 5 40

∼

x1 v2 v3 w3 s2 s3 r1

1 1117 0 0 0 3

17217

417

0 4717 0 0 1 2

17717

12217

0 1317 1 1 0 2

17334

2017

0 225017 0 0 0 20

1710017

88017


Example, continued

x1 v2 v3 w3 s2 s3 r1

1 1117 0 0 0 3

17217

417

0 4717 0 0 1 2

17717

12217

0 1317 1 1 0 2

17334

2017

0 225017 0 0 0 20

1710017

88017

Optimal solution of MLP is

x1 = 417 , x2 = v2 = 0,

x3 = v3 − w3 = 2017 − 0 = 20

17 ,

s2 = 12217 , and s3 = r1 = 0

with a maximal value 20(

417

)+ 150(0) + 40

(2017

)= 880

17 .


Example, continued

Optimal solution for original mLP can be also read off final tableau,

x1 v2 v3 w3 s2 s3 r1

1 1117 0 0 0 3

17217

417

0 4717 0 0 1 2

17717

12217

0 1317 1 1 0 2

17334

2017

0 225017 0 0 0 20

1710017

88017

y1 = 100

17 , y2 = 0, y3 = 2017 ,

and minimal value

8(

10017

)+ 10 (0) + 4

(2017

)= 880

17 .


Example, continued

Alternatively method: First write minimization problem in

with variables ≥ 0 by setting y1 = u1 − z1,

Minimize : 8u1 − 8z1 + 10y2 + 4y3

Subject to : 4u1 − 4z1 + 2y2 − 3y3 ≥ 202u1 − 2z1 + 3y2 + 5y3 ≤ 1506u1 − 6z1 + 2y2 + 4y3 = 40u1 ≥ 0, z1 ≥ 0, y2 ≥ 0, y3 ≥ 0

Dual MLP will now have a different tableau than before

but the same solution.


Remark on Tableau with Surplus/Equal

After artificial objective function is zero

drop this row but keep artificial variables

to determine values of dual variables.

Proceed to make all the entries of row for objective function

≥ 0 in columns of xi , slack and surplus variable,

but allow negative values in artificial variable columns.

Dual variables are entries in row for objective function

in columns of slack variables and artificial variables.

For pair of surplus and artificial variable columns,

value in artificial variable column is ≤ 0 and

1 of value in surplus variable column.


Notation for Duality Theorems

MLP: (primal) maximization linear programming problem

Maximize: f (x) = c qxSubject to:

∑j aijxj ≤ bi , ≥ bi , or = bi for 1 ≤ i ≤ m and

xj ≥ 0, ≤ 0, or unrestricted for 1 ≤ j ≤ nFeasible set: FFFM .

mLP: (dual) minimization linear programming problem

Minimize: g(y) = b qySubject to:

∑i aijyi ≥ cj , ≤ cj , or = cj for 1 ≤ j ≤ n and

yi ≥ 0, ≤ 0, or unrestricted for 1 ≤ i ≤ mFeasible set: FFFm.

Dual of the minimization problem is the maximization problem.


Weak Duality Theorem

Theorem (Weak Duality Theorem)

Let x ∈FFFM for MLP and y ∈FFFm for mLP, any feasible solutions.

a. Then, f (x) = c qx ≤ b qy = g(y).

Thus, optimal value M to either problem satisfiesc qx ≤ M ≤ b qy for any x ∈FFFM and y ∈FFFm.

b. c qx = b qy iff x & y satisfy complementary slackness

0 = yj(bj − aj1x1 − · · · − ajnxn) for 1 ≤ j ≤ m, and

0 = xi (a1iy1 + · · ·+ amiym − ci ) for 1 ≤ i ≤ n.

In matrix notation,

0 = y q (b− Ax) and

0 = x q (A>>>y − c).


Complementary Slackness

For linear programming usually solve by simplex method, row reduction

In nonlinear programming with inequalities often solve

complementary slackness equations,

Karush-Kuhn-Tucker equations


Proof of Weak Duality Theorem

(1) If∑

j aijxj ≤ bi then yi ≥ 0 and yi (Ax)i = yi∑

j aijxj ≤ yi bi .

If∑

j aijxj ≥ bi then yi ≤ 0 and yi (Ax)i = yi∑

j aijxj ≤ yi bi .

If∑

j aijxj = bi then yi is arb and yi (Ax)i = yi∑

j aijxj = yi bi .

Summing over i

y qAx =∑

i yi (Ax)i ≤∑

i yi bi = y qb or y>>> (b− Ax) ≥ 0.

(2) By same type of argument as (1),

c qx ≤ x q (A>>>y)

=(A>>>y

)>>>x = y>>> (Ax) = y qAx(

A>>>y − c)>>>

x ≥ 0.

c qx ≤ y qAx ≤ y qb (a)

(b) y qb− c qx = y>>> (b− Ax) +(A>>>y − c

)>>>x = 0 iff

0 =(A>>>y − c

) qx and 0 = y q (b− Ax). QED


Feasibility/Boundedness

Corollary

Assume that MLP and mLP both have feasible solutions.

Then MLP is bounded above and has an optimal solution.

Also, mLP is bounded below and has an optimal solution.

Proof.

If y0 ∈FFFm and x ∈FFFM , then

f (x) = c qx ≤ b qy0

so f is bounded above, and has an optimal solution.

Similarly, if x0 ∈FFFM , and y ∈FFFm then

g(y) = b qy ≥ c qx0

so g is bounded below, and has an optimal solution.


Necessary Conditions for Optimal Solution

Proposition

If x is an optimal solution for MLP,

then there is a feasible solution y ∈ FFFm of the dual mLP

that satisfies complementary slackness equations,

1. y q (b− Ax) = 0,

2.(A>>>y − c

) q x = 0.

Similarly, if y ∈ FFFm is optimal solution of mLP,

then there is a feasible solution x ∈FFFM that satisfy 1-2.

Proof is longest of duality arguments.

Prove similar necessary conditions for nonlinear situation.


Proof of Necessary Conditions

Let E be set of i such that bi =∑

j aij xj , i.e. is tight or effective.

Gradient of this constraint is transpose of i th-row of A, R>>>i ,

Let E′ be set of i such that xi = 0 is tight.

ei = (0, . . . , 1, . . . , 0)>>> is negative of gradient

Assume nondegenerate, so gradients of constraints

{RTi }i∈E ∪ { ei}i∈E′ = {wi}i∈E′′ are linearly independent.

(Otherwise take an appropriate subset in following argument.)

f has a maximum at x on level set for constraints i ∈ E ∪ E′.

By Lagrange multipliers,

∇f (x) = c =∑

i∈E yi RTj −

∑i∈E′ zi ei

By setting yi = 0 for i /∈ E and 1 ≤ i ≤ m and

zi = 0 for i /∈ E′ and 1 ≤ i ≤ n,

c =∑

1≤i≤m yi RTi −

∑1≤i≤m zi ei = A>>>y − z. (*)


Proof of Necessary Conditions, contin.

Since yi = 0 for bi −∑

j aij xj 6= 0

0 = yi (bi −∑

j aij xj) for 1 ≤ i ≤ m.

Since zj = 0 for xj 6= 0,

0 = xj zj 1 ≤ j ≤ n.

In vector-matrix form using (*),

(1) 0 = y q (b− Ax)

(2) 0 = x q z =(A>>>y − c

) q xStill need (4): (i) yi ≥ 0 for resource constraint,

(ii) yi ≤ 0 for requirement constraint,

(iii) yi is unrestricted for equality constraint,

(iv) zj =∑

i aij yi − cj ≥ 0 for xj ≥ 0, (v) zj ≤ 0 for xj ≤ 0, and

(vi) zj = 0 for xj unrestricted.



{RTi }i∈E ∪ { ei}i∈E′ = {wk}k∈E′′ are linearly independent

Complete to a basis of Rn using vectors perp to these first vectors.

W = (w1, . . . ,wn) and V = (v1, . . . , vn) s.t. W>>>V = I.

Column vk perp to wi except for i = k.

c =∑

1≤i≤m yi RTi −

∑1≤i≤n zi ei =

∑j pj wj where pj = yi , zi , or 0

c qvk = (∑

j pj wj) qvk = pk

Take i ∈ E. Gradient of this constraint is R>>>i = wk for some k ∈ E′′.

Set δ = 1 for resource and δ = +1 for requirement constraints

δ R>>>i points into FFFM (unless = constraint).

For small t ≥ 0, x + t δ vk ∈FFFM

0 ≤ f (x)− f (x + t δ vk) = t δ c qvk = t δ pk ,

δ pk ≥ 0, or yi ≥ 0 for resource and ≤ 0 for requirement constraint

Can’t move off for equality constraint, so yi unrestricted



Take i ∈ E′. ei = wk for some k ∈ E′′.

Set δ = 1 if xi ≥ 0 and δ = 1 if xi ≤ 0

δ wk = δ ei points into FFFM (unless xi unrestricted).

By argument as before, δ pk = δ zi ≥ 0.

Therefore zi ≥ 0 if xi ≥ 0 and zi ≤ 0 if xi ≤ 0

If xi is unrestricted, then the equation is not tight and zi = 0.

This proves y ∈FFFm and satisfies complementary slackness (1) and (2)

QED


Optimality and Complementary Slackness

Corollary

Assume that x ∈FFFM is a feasible solution for primal MLP and y ∈FFFm

is a feasible solution of dual mLP. Then the following are equivalent.

a. x is an optimal solution of MLP and y is an optimal solution of mLP.

b. c q x = b q y.

c. 0 = x q (c− A>>>y) and 0 = (b− Ax) q y.

Proof.

(b ⇔ c) Restatement of Weak Duality Theorem.

(a ⇒ c) By proposition, ∃ y′ that satisfies complementary slackness

By Weak Duality Theorem, c q x = b q y′.So, c q x = b q y′ ≥ b q y ≥ c q x.By Weak Duality Theorem, y satisfies complementary slackness.


Proof of Corollary, contin.

Proof.

(b ⇒ a) If x and y satisfy c q x = b q y, then

for any x ∈FFFM & y ∈FFFm,

c qx ≤ b q y = c q x ≤ b qyx & y must be optimal solutions.


Duality Theorem

Theorem

Consider two dual problems MLP and mLP.

Then, MLP has an optimal sol’n iff dual mLP has an optimal sol’n.

Proof.

If MLP has an optimal sol’n x,

then mLP has feasible sol’n y that satisfies complementary slackness.

By Corollary, y is optimal sol’n of mLP.

Converse is similar


Duality and Tableau

Theorem

If either MLP or mLP is solved for an optimal sol’n by simplex method,then sol’n of its dual LP is displayed in bottom row of final optimal tableauin the columns associated with slack and artificial variables. (not surplus)

Proof:

Start with MLP. To solve by tableau, need x ≥ 0.

Group equations into resource, requirement, and equality constraints.

so tableau for MLPA1 I1 0 0 0 b1

A2 0 I2 I2 0 b2

A3 0 0 0 I3 b3

c>>> 0 0 0 0 0


Proof continued

Row operations to final tableau realized by by matrix multiplication

[M1 M2 M3 0

y>>>1 y>>>2 y>>>2 1

] A1 I1 0 0 0 b1

A2 0 I2 I2 0 b2

A3 0 0 0 I3 b3

c>>> 0 0 0 0 0

=

[M1A1 + M2A2 + M3A3 M1 M2 M2 M3 Mb

y>>>1 A1 + y>>>2 A2 + y>>>3 A3 − c>>> y>>>1 y>>>2 y>>>2 y>>>3 y>>>b

]

Obj fn row is not added to the other rows so last column = (0, 1)>>>,

In final tableau, entries in objective function row ≥ 0,

except for artificial variable columns, so

A>>>y − c =(y>>>A− c>>>

)>>> ≥ 0, y1 ≥ 0, y2 ≤ 0, so y ∈FFFm

c qxmax = y>>>b = b q y so y is minimizer by Optimality Corollary.


Proof continued

Note:(A>>>y

)i= Li

q y Li i th-column of A

If xi ≤ 0, set ξi = xi ≥ 0.

Column in tableau, 1 original column, and new obj fn coef ci .

Now have 0 ≤ ( Li ) q y − ( ci ),

Liq y ≤ ci resource constraint of dual.

If xi arbitrary, set xi = ξi − ηi .

Then get both Liq y ≥ ci & Li

q y ≤ ci ,

Liq y = ci , equality constraint of dual.

QED


Sensitivity Analysis

Sensitive analysis concerns the extent to which more of a resource

would increase the maximum value of a MLP.

Example. In short run,

in stock product 1 per item product 2 per itemprofit $40 $10units of paint 1020 15 10fasteners 400 10 2hours of labor 420 3 5x1 x2 s1 s2 s315 10 1 0 0 102010 2 0 1 0 4003 5 0 0 1 420

40 10 0 0 0 0

∼

x1 x2 s1 s2 s30 7 1 1.5 0 4201 .2 0 .1 0 400 4.4 0 .3 1 300

0 2 0 4 0 1600


Sensitivity Analysis, continued

x1 x2 s1 s2 s30 7 1 1.5 0 4201 .2 0 .1 0 400 4.4 0 .3 1 300

0 2 0 4 0 1600

∼

x1 x2 s1 s2 s3

0 1 17

314 0 60

1 0 135

17 0 28

0 0 2235

914 1 36

0 0 27

257 0 1720

Optimal solution is

x1 = 28, x2 = 60, s1 = 0, s2 = 0, s3 = 36,

with optimal profit of 1720.



x1 x2 s1 s2 s3

0 1 17

314 0 60

1 0 135

17 0 28

0 0 2235

914 1 36

0 0 27

257 0 1720

Values of an increase of constrained quantities are

y1 = 27 , y2 = 25

7 ,

y3 = 0 for quantity that is not tight.

Increase is largest for second constraint, limitation on fasteners.

Next consider range that b2 can be increased

while keeping same basic variables with s1 = s2 = 0.



Let δ2 be change in 2nd resource (fasteners):

10x1 + 2x2 + s2 = 400 + δ2 starting form of constraint.

s2 and δ2 play similar roles (and have similar units),

so the new final tableau adds

δ2 times s2-column to right side of equalities.

Still need x1, x2, s3 ≥ 0,

0 ≤ x2 = 60− 3 δ214 or δ2 ≤ 60 q 14

3 = 280,

0 ≤ x1 = 28 + δ27 or δ2 ≥ 28 q7 = 196,

0 ≤ s3 = 36 + 9 δ214 or δ2 ≥ 36 q 14

9 = 56;

56 ≤ δ2 ≤ 280.



Resource can be incr at most 280 units and decr at most 56 units, or

56 ≤ δ2 ≤ 280

344 = 400− 56 ≤ b2 ≤ 400 + 280 = 680.

For this range, x1, x2, and s3 are still basic variables.

Resource = 344Resource = 400Resource = 680

(20,72)

(28,60)

(68,0)(40,0)


Sensitivity for Change of Constraint Constants

For δ2 = 280,

x1 = 28 + 280 · 17 = 68,

x2 = 60− 280 · 314 = 0,

s3 = 36 + 280 · 914 = 216,

z = 1720 + 280 · 257 = 2720 is optimal value.

For δ2 = 56,

x1 = 28− 56 · 17 = 20,

x2 = 60 + 56 · 314 = 72,

s3 = 36− 56 · 914 = 0,

z = 1720− 56 · 257 = 1520 is optimal value.


General Changes in Tight Constraint

Use optimal (final) tableau that gives maximum

b′i entry of i th-row of constants in right hand column

c ′j ≥ 0 entry in j th-column of objective row

a′ij entry in i th-row and j th-column

exclude right side constants and any artificial variable columns.

C′j j th-column of A′ (note capital and bold, not cj)

Ri i th-row of A′


General Changes in Constraint, contin.

Change in tight r th- resource constraint, br + δr

Assume that sr is in kth-column srA er b + δrer

−cT 0 0

∼ sr

A′ C′k b′ + δrC′k

c′T c ′k M + δrc′k

zi basic variable with pivot in i th-row, need 0 ≤ zi = b′i + δra

′ik .

For a′ik < 0, need δra′ik ≤ b′i , so

δr ≤ min i

{b′

ia′ik

: a′ik < 0}

, kth-column for sr

For a′ik > 0, need b′i ≤ δra′ik , so

min i

{b′

ia′ik

: a′ik > 0}≤ δr , kth-column for sr .

Change in optimal value for δr in allowable range: δrc′k


Change in Slack Constraint Constant

Let sr be for a pivot column in optimal tableau for a slack r th-resource.

To keep same basic variables, need changed amount b′r + δr ≥ 0

δr ≥ b′r .

br can be increased by an arbitrary amount.

For δr is this range, optimal value is unchanged.



x1 x2 s1 s2 s3

0 1 17

314 0 60

1 0 135

17 0 28

0 0 2235

914 1 36

0 0 27

257 0 1720

Allowable δ1 for first resource,

δ1 ≤ min{

28 · 351 , 36 · 35

22

}= min { 980, 57.27 } = 57.27.

δ1 ≥ min{

60 · 71

}= 420.

Change of optimal value 1720 + 27 · δ1

Allowable δ3

δ3 ≥ 36


Changes in Objective Function Coefficients

For a change from c1 to c1 + ∆1, changes in tableauxx1 x2 s1 s2 s315 10 1 0 0 102010 2 0 1 0 4003 5 0 0 1 420

40 ∆1 10 0 0 0 0

∼

x1 x2 s1 s2 s3

0 1 17

314 0 60

1 0 135

17 0 28

0 0 2235

914 1 36

∆1 0 27

257 0 1720

∼

x1 x2 s1 s2 s3∗ ∗ ∗ ∗ ∗ ∗0 0 2

7 −135∆1

257 + 1

7∆1 0 1720 + 28 ∆1


Changes in Coefficients, contin.

To keep objective function row ≥ 0, need

27 −

135∆1 ≥ 0 & 25

7 + 17∆1 ≥ 0

∆1 ≤ 27

351 = 10 & ∆1 ≥ 25

7 ·71 = 25

15 = 40− 25 ≤ c1 + ∆1 ≤ 40 + 10 = 50.

Corresponding optimal value of objective function is 1720 + 28 ∆1


General Changes in Objective Function Coefficients

Consider change ∆k in coefficient ck of basic xk in obj fn,

where xk is a basic in optimal solution and its pivot is in r th row.

a′rj denote the entries in r th row and j th column,

excluding right side constants and any artificial variable columns.

c ′j ≥ 0 denote entries in the objective row of optimal tableau

With change, original entry in the Obj Rn row becomes ck −∆k .

Entry in optimal tableau changes from 0 to ∆k

To keep xk basic, need to add ∆k R′r to Obj Fn row.

Entry in xk -column is now 0 and j th-column is c ′j + ∆ka′rj

For all j , need c ′j + ∆ka′rj ≥ 0.


Changes in Coefficients, contin.

ck + ∆k for basic variable with r th-pivot row, a′rk = 1 pivot.

For a′rj > 0 in r th-pivot row, need ∆k a′rj ≥ c ′j or ∆k ≥c ′ja′rj

,

∆k ≥ min j

{c ′j

a′rj

: a′rj > 0, j 6= k}

, maximal decrease of ck .

For a′rj < 0 in r th-pivot row, need c ′j ≥ a′rj ∆k orc ′ja′rj

≥ ∆k ,

∆k ≤ min j

{c ′j

a′rj

: a′rj < 0}

, maximal increase of ck .

If c ′j = 0 for a′rj < 0, then need ∆k ≤ 0.

If c ′j = 0 for a′rj > 0 & j 6= k, then need ∆k ≥ 0.

Change in optimal value is ∆kb′r


Sensitivity Analysis for Non-basic Variable

Our example does not have any non-basic variables, xk .

If xk were a non-basic variable in optimal solution, xk = 0,

then c ′k + ∆k ≥ 0 insures that xr is a non-basic variable,

∆k ≥ c ′k .

i.e., c ′k ≤ 0 is min decrease needed to make xk a basic variable

and a positive contribution to optimal solution.


Theory: Convex Combinations

Weighted averages in Rn:

For three vectors a1, a2, and a3,a1 + a2 + a3

3is average of each component, average of these vectors.

a1 + a2 + a2 + a3 + a3 + a3

6=

a1 + 2a2 + 3a3

6= 1

6a1 + 26a2 + 3

6a3

is a weighted average of these vectors with weights 16 , 2

6 , and 36 .

For vectors {ai}ki=1 and numbers

∑ki=1 ti = 1 with ti ≥ 0∑k

i=1 ti ai

is a weighted average, and is called a convex combination of {ai}.


Convex Sets

Definition

A set S ⊂ Rn is convex provided that

if x0 and x1 are any two points in S then convex combination

xt = (1− t)x0 + tx1 is also in S for all 0 ≤ t ≤ 1,

i.e., line segment from x0 to x1 in S.

convex convex convex

not convex not convex


Convex Sets, contin.

Each constraint, ai1x1 + · · ·+ ainxn ≤ bi or ≥ bi ,

or xi ≥ 0, defines a closed half-space in Rn.

ai1x1 + · · ·+ ainxn = bi is a hyperplane (n − 1 dimensional).

Definition

Any intersection of a finite number of closed half-spaces

and possibly some hyperplanes is called a polyhedron.



Theorem

a. Intersection of convex sets,⋂

j Sj , is convex.

b. A polyhedron is convex. So feasible set of any LP is convex.

Proof.

(a) If x0, x1 ∈ Sj & 0 ≤ t ≤ 1,

then (1− t) x0 + t x1 ∈ Sj ∀ j , and

(1− t) x0 + t x1 ∈⋂

j Sj .

(b) Each closed half space & hyperplane is convex,

so intersection is convex.



Theorem

If S is a convex set, and pi ∈ S for 1 ≤ i ≤ k,

then any convex combination∑k

i=1 ti pi ∈ S.

Proof.

Proof is by induction.

For k = 2, it follows from the def’n of convex set.

Assume true for k − 1 ≥ 2.

If tk = 1 & tj = 0 for 1 ≤ j < k, then clear.

If tk < 1, then∑k−1

i=1 ti = 1− tk > 0, &∑k−1

i=1ti

1−tk= 1.∑k−1

i=1ti

1−tkpi ∈ S by induction hypothesis

So,∑k

i=1 ti pi = (1− tk)∑k−1

i=1ti

1−tkpi + tk pk ∈ S.


Extreme Points and Vertices

Definition

A point p in a nonempty convex set S is called an extreme point p.t.

if p = (1− t)x0 + tx1 with x0, x1 in S and 0 < t < 1,

then p = x0 = x1.

An extreme point in a polyhedron is called a vertex.

An extreme point of S must be a boundary point of S.

Disk D = { x ∈ R2 : ‖x‖ ≤ 1 } is convex.

Each point on its boundary is an extreme point.

For next few theorems, consider feasible set

FFF = { x ∈ Rn+m+ : Ax = b }

with slack and surplus variables included in x’s and constraints.


Vertices

Theorem

Let FFF = { x ∈ Rn+m+ : Ax = b }.

x ∈FFF is a vertex of FFF if and only if basic feasible solution,

i.e., columns of A with xj > 0 linearly independent set of vectors.

Proof:By reindexing columns and variables, can assume that

x1 > 0, . . . , xr > 0 xr+1 = · · · = xn+m = 0.

(a) Assume {Aj }rj=1 linearly dependent: ∃ (β1, . . . , βr ) 6= 0

β1 A1 + · · ·+ βr Ar = 0.

For βββ>>> = (β1, . . . , βr , 0, . . . , 0), Aβββ = 0,

For small λ, w1 = x + λβββ ≥ 0, & w2 = x− λβββ ≥ 0, Awi = Ax = b.

so w1, w2 ∈FFF & x = 12w1 + 1

2w2, so not vertex.


Proof

(b) Conversely, assume that x ∈FFF is not vertex,

x = t y + (1− t) z for 0 < t < 1,

with y 6= z in FFF .

For r < j ,

0 = xj = t yj + (1− t) zj .

Since both yj ≥ 0 and zj ≥ 0, both must be zero for j > r .

Because y 6= z are both in FFF ,

b = Ay = y1 A1 + · · ·+ yr Ar ,

b = Az = z1 A1 + · · ·+ zr Ar ,

0 = (y1 − z1)A1 + · · ·+ (yr − zr )Ar ,

and columns {Aj }rj=1 are linearly dependent. QED


Some Vertex is an Optimal Solution

For any convex combination

f (∑

tjxj) = c q ∑ tjxj =∑

tjc qxj =∑

tj f (xj).

Theorem

Assume that FFF = { x ∈ Rn+m+ : Ax = b } 6= ∅ for bounded MLP.

Then following hold.

a. If x0 ∈FFF , then there exists a basic feasible xb ∈FFF s.t.

f (xb) = c qxb ≥ c qx0 = f (x0).

b. There is at least one optimal basic solution.

c. If two or more basic solutions are optimal, then

any convex combination of them is also an optimal solution.


Proof a.

If x0 is already a basic feasible sol’n then done.

Otherwise, columns A for x0i > 0 are lin. depen.

Let A′ be matrix with only these columns.

∃ y′ 6= 0 such that A′y′ = 0.

Adding 0 in other entries, get y 6= 0 s.t. Ay = 0.

A( y) = 0, so can assume that c qy ≥ 0.

A [x0 + t y] = Ax0 = b,

If yi 6= 0, then x0i > 0, so x0 + t y ≥ 0 for small t is in FFF .


Proof a, contin.

Case 1. Assume that c qy > 0 and some component yi < 0.

x0i > 0 and x0

i + t yi = 0 for ti =x0i

yi> 0.

As t increases from 0 to ti , objective function increases from

c qx0 to c q [x0 + tiy].

If more than one yi < 0, then select one with smallest ti .

Have constructed point in FFF with one more zero component of x0,

fewer components yi < 0,

and a greater value of objective function.

Can continue until either columns are linearly independent or

all yi ≥ 0.


Proof a, continued

Case 2. If c qy > 0 and y ≥ 0,

then x0 + t y ∈FFF for all t > 0,

c q [x0 + t y] = c qx0 + t c qy is arbitrarily large.

MLP is unbounded and has no maximum, contradiction.

Case 3. If c qy = 0: f (x0 + t y) = c qx0 unchanged

Some yi 6= 0. Considering y & y can assume some yi < 0.

∃ first ti > 0, to make another x0i + tiyi = 0.

Eventually, get corresponding columns linearly independent, and

a basic solution as claimed in part (a).


Proof b,c

(b) Only finitely many basic feasible solutions, {pj}Nj=1.

f (x) ≤ max1≤j≤N f (pj) for x ∈FFF by part (a)

Maximum can be found among f (pj).

(c) Assume f (pji ) = M = max{ f (x) : x ∈FFF } for i = 1, . . . , ` > 1.∑ì=1 tji pji ∈FFF

f(∑`

i=1 tji pji

)=

∑ì=1 tji f (pji ) =

∑ì=1 tji M = M. optimal

QED


Validation of Simplex Method

If ∃ degen. basic feasible sol’ns with fewer than m positive basic var,

then simplex method can cycle by row reduction to matrices

with same positive basic variables but different sets of pivots:

interchange one zero basic variable with zero free variable.

Same vertex of feasible set

Need to insure don’t repeat same set of basic variables at a vertex (cycle)

Theorem

If a maximum solution exists for a linear programming problem

and simplex algorithm does not cycle among degen basic feasible sol’ns,

then simplex algorithm locates a maximum solution in finitely many steps.


Sketch of Proof of Theorem

Assume never reach a degenerate basic solution.

Then reach p0 with all pivoting to p1,. . . ,pk have f (pj) ≤ f (p0)

Complete to set of all basic feasible sol’ns (vertices) p0,p1,. . . ,p`

Set of all convex combinations, convex hull, is bounded polyhedron

H = {∑`

i=0 tipi : ti ≥ 0,∑`

i=0 ti = 1 } ⊂FFF .

Edge of H from pi to pj corresponds to pivoting

(as in proof of Theorem 3.4.2(a)) where

one constraint becomes 6= bi and another become = bj ,

Positive cone out from p0 determined by {pi − p0 }ki=1

C = {p0 +∑k

i=1 yi (pi − p0) : yi ≥ 0 } ⊃ H. (geometrically)

Let q be any vertex of H (basic solution), q ∈ H ⊂ C.

q− p0 =∑k

i=1 yi (pi − p0) with all yi ≥ 0

f (q)− f (p) =∑k

i=1 yi [f (pi )− f (p∗)] ≤ 0. End Proof


Degenerate Basic Solutions

Homework Prob gives example of a degenerate basic solution.

A basic variable is = 0 in addition to free (non-pivot) variables.

When leaving, variable which is becomes > 0 must be

free (non-pivot) variable and not a basic (pivot) variable = 0

First pivoting at the degenerate solution

interchanges a basic variable = 0 and a free variable,

so new free variable can be made > 0 with next pivoting

when the value of objection function is increased;

all same values of variables, so the same point in FFF .

Matter of how row reduction relates to movement on feasible set.


Documents

Chapter 1: Linear Programming - Northwestern Universityclark/368/handouts/... · 2016. 12. 6. · Chapter 1: Linear Programming 11. Slack Variables For large number of variables,