MATH2070 - Linear programming · The standard Linear Programming (LP) Problem Graphical method of solving LP problem ... A feasible solution lies inside a closed region inside the

MATH2070 Optimisation

Linear Programming

Semester 2, 2012Lecturer: I.W. Guo

Lecture slides courtesy of J.R. Wishart

Standard Problem Graphical Method Algebraic Simplex Method Extensions Dual

Review

The standard Linear Programming (LP) Problem

Graphical method of solving LP problem

The simplex procedural algorithm

Extensions to LP Algorithm to include other conditions

Dual Problem






Dual Problem


Motivation

Where do Linear Programming problems occur?

History and Applications

Became a field of interest during World War II.

Applications

I Allocation Problems

I Scheduling

I Blending of raw materials


Initial Example

A pharmaceutical company can produce two types of drug usingthree different resources. Each resource is in very limited supply.Production data are listed in the table below:

Resource usage/unit ResourceDrug 1 Drug 2 Availability

Resource 1 1 0 42 3 2 183 0 2 12

Profit/unit 3 5 × $1000


Formulation


Resource 1 1 0 42 3 2 183 0 2 12


Define x1, x2 be the number produced for each type of drug.

Profit : Z = 3x1 + 5x2

Optimisation

We want to find the optimal x∗1 and x∗2 to maximise the totalprofit, Z∗ .


Standard LP problem

In full generality, the problem is given by,

Maximise:Z = c1x1 + c2x2 + · · ·+ cnxn

such thata11x1 + a12x2 + · · · + a1nxn ≤ b1

a21x1 + a22x2 + · · · + a2nxn ≤ b2

......

...

am1x1 + am2x2 + · · · + amnxn ≤ bm

and x1 ≥ 0, x2 ≥ 0, . . . , xn ≥ 0.

where aij , ci are constants and bi > 0 are constants.


Using matrix notation (slightly abusing notation),

Maximise: Z = cTx

such that: Ax ≤ b and x ≥ 0

where the vectors and matrices are defined,

c =

c1

c2...cn

A =

a11 a12 . . . a1n

a21 a22 . . . a2n...

.... . .

...am1 an2 . . . amn

x =

x1

x2...xn

b =

b1

b2...bm


Objective Function and Decision Variables

Decision Variables

xT = (x1, x2, . . . , xn)

where xi are the decision variables and it is assumed that xi ≥ 0.

Objective Function

Z = Z(x) = cTx = c1x1 + c2x2 + · · ·+ cnxn,

where ci are constants and are referred to as the cost coefficients.

ci shows the linear increase/decrease in Z for unit increase in xi.


Constraints

Constraint system of equations

Ax ≤ b ←→

a11x1 + a12x2 . . . + a1nxn ≤ b1

a21x1 + a22x2 . . . + a2nxn ≤ b2...

...am1x1 + an2x2 . . . + amnxn ≤ bm

System of m different constraints described by the m equationswith n decision variables.

b is referred to the resource vector and it is assumed that bi > 0are constants.

Positivity condition

Note that the resource vector b must have positive elements.


Feasible Solutions

Feasible Solution

Any point x satisfying Ax ≤ b and x ≥ 0 is called a feasiblesolution.

Infeasible Solution

Conversely, if a point x does not satisfy the above equations it isan infeasible solution.

A feasible solution lies inside a closed region inside then−dimensional decision space.

For the initial example that is a two dimensional region (x1, x2)plane.


Feasible Regions

Empty Region

It is possible for the feasible region to be empty. That is no pointcan be found to satisfy the constraints.

This will be called an ill-posed problem.

Feasible region is empty if constraint equations are inconsistent.

Example

x1

x2

R1

R2

R1 ∩R2 = ∅


Optimal solution

Optimal solution

A feasible solution that maximises the objective function Z is theoptimal solution.

The optimal solution is usually denoted x∗ = (x∗1, x∗2, . . . , x

∗n).

The maximised objective function is denoted

Zmax = Z∗ = Z(x∗).


Location of Optimal Solutions

The multivariate objective function Z = cTx is linear.

From introductory lectures, we know that Z∗ must lie on theboundary of the feasible region. Intro Lectures Here

Optimal solution x∗ will lie at either a corner point or an edge ofthe feasible polygon.


Feasible Regions

The feasible region is actually a polygon in n−dimensional space.

Each equation in the constraint describes a plane.

All equations (planes) together describe a region in space. Forexample,


Number of Solutions

No solution

Feasible region is empty.

One optimal solution

Optimal solution at a corner point.

*

Many optimal solutions

Optimal solutions along an edge.


Unbounded feasible region

Another scenario exists where the feasible region could beunbounded. In an example similar to the empty region.

Example

Unbounded feasible region

x1

x2

x1 − x2 ≤ 1−x1 + x2 ≤ 2






Dual Problem


Return to example problem


Resource 1 1 0 42 3 2 183 0 2 12


Bivariate problem (two dimensional) so can easily be solvedgraphically.


Feasible Region

Feasible region is bounded by the five straight lines:x1 = 0 , x2 = 0 , x1 = 4 , 3x1 + 2x2 = 18 , 2x2 = 12 .

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

Feasible region

3x1+2x2=18

2x2 = 12

x1 = 4


Find optimal solution(s)

Objective function Z = 3x1 + 5x2 yields a single optimal solution.

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

(0,6)

(0,0)

(4,0)

(4,3)

(2,6)

0

5

10

20

36

Direction of increasing Z(= 3x1 + 5x2)

Optimal solution

(x∗

1, x∗

2) = (2, 6)

Z∗ = 36


Optimal Solutions

Consider new objective function Z = 6x1 + 4x2

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

C2(4,3)

C1(2,6)

8

16

24

32 36


Note: Z = 6x1 + 4x2 isparallel to the boundary

line 3x1 + 2x2 = 18


Limitations of graphical method

Graphical Limitations

Graphical method is only practical in two dimensions.

Alternative?

The algebraic simplex algorithm.

I Iterative procedure.

I Historically created by George Dantzig in 1947.






Dual Problem


Simplex Algorithm

General overview of method

I Start at an initial feasible corner point.

Usually x = 0.

I Move to an adjacent feasible corner point.

Do this by finding corner with best potential increase in Z.

I Stop when at the optimal solution, Z∗.

Determine optimality when other adjacent corner points resultin decrease in Z.


Start at initial feasible corner point

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

(0,6)

(0,0)

(4,0)

(4,3)

(2,6)

0



First corner point iteration

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

(0,6)

(0,0)

(4,0)

(4,3)

(2,6)

12



Second corner point iteration

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

(0,6)

(0,0)

(4,0)

(4,3)

(2,6)

27



Third and final corner point iteration

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

(0,6)

(0,0)

(4,0)

(4,3)

(2,6)36



Alternative direction: Start at initial point

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

(0,6)

(0,0)

(4,0)

(4,3)

(2,6)

0



Alternative direction: First corner point

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

(0,6)

(0,0)

(4,0)

(4,3)

(2,6)

30



Alternative direction: Second and final corner point

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

(0,6)

(0,0)

(4,0)

(4,3)

(2,6)36

30



System of equations

Need to move from inequality constraints to equality constraints.

Introduce new variables to reduce the inequalities to equalities.

Example

2x1 − 3x2 ≤ 5 ⇐⇒ 2x1 − 3x2 + x3 = 5

for some x3 ≥ 0.

Definition

Slack variables: Each introduced variable that reduces a (less than)inequality constraint to an equality constraint is called a slackvariable.


Slack Variables

For each constraint introduce a new variable equal to thedifference between the RHS and LHS of the constraint.

Ax ≤ b −→ Ax = b

That is we have Ax ≤ b,

a11x1 + a12x2 + · · · + a1nxn ≤ b1

a21x1 + a22x2 + · · · + a2nxn ≤ b2...

......

...am1x1 + am2x2 + · · · + amnxn ≤ bm

with: x1 ≥ 0 , x2 ≥ 0 , · · · , xn ≥ 0.


New constraint matrix

After adding slack variables for each constraint

xs = (xn+1, xn+2, . . . , xn+m) ,

a11x1 + a12x2 + · · · + a1nxn + xn+1 = b1

a21x1 + a22x2 + · · · + a2nxn + xn+2 = b2

......

.... . .

am1x1 + am2x2 + · · · + amnxn + xn+m = bm

with: x1 ≥ 0, x2 ≥ 0, . . . , xn ≥ 0,

and: xn+1 ≥ 0, xn+2 ≥ 0, . . . , xn+m ≥ 0.


Return to example

Initial constraints,

x1 ≤ 4; 3x1 + 2x2 ≤ 18; 2x2 ≤ 12

Introduce a new variable equal to the difference between the RHSand LHS of the constraint.

x3 = 4− x1; x4 = 18− 3x1 − 2x2 ; x5 = 12− 2x2 .

New system of constraints,

x1 ≤ 4; 3x1 + 2x2 ≤ 18; 2x2 ≤ 12.↓ ↓ ↓

x1 + x3 = 4; 3x1 + 2x2 + x4 = 18; 2x2 + x5 = 12.


Algebraic representation

Turn attention to algebraic representation of corner point method.

Full problem with slack variables,

Maximise:Z = c1x1 + c2x2 + · · ·+ cnxn

such thata11x1 + a12x2 + · · · + a1nxn + xn+1 = b1

a21x1 + a22x2 + · · · + a2nxn + xn+2 = b2

......

.... . .

am1x1 + am2x2 + · · · + amnxn + xn+m = bm

andx1 ≥ 0, x2 ≥ 0, . . . , xn ≥ 0, xn+1 ≥ 0, . . . , xn+m ≥ 0.


In tableau form

Z x1 x2 . . . xn xn+1 xn+2 . . . xn+m RHS

1 −c1 −c2 . . . −cn 0 0 . . . 0 00 a11 a12 . . . a1n 1 0 . . . 0 b1

0 a21 a22 . . . a2n 0 1 . . . 0 b2...

......

. . ....

......

. . ....

...0 am1 am2 . . . amn 0 0 . . . 1 bm

I Choose initial corner point solution at (x1, x2, . . . , xn) = 0

I Slack variables take the values of resource vector

xs = (xn+1, xn+2, . . . , xn+m)T = b

I Initially, slack variables are the basic variables.

I Initially, decision variables are the non-basic variables.


For the example

Maximise: Z − 3x1 − 5x2 = 0

such that: x1 + x3 = 4

3x1 + 2x2 + x4 = 18

+ 2x2 + x5 = 12

and x1 ≥ 0, x2 ≥ 0, x3 ≥ 0, x4 ≥ 0, x5 ≥ 0.


Geometric interpretation of slack variables

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

I5

I1 I4

I2

I3 I5

I4

x1=

0

x2 = 0

x3=

0

x4=0

x5 = 0

F5

F1 F2

F3

F4


In tableau form

Step 1: Setup problem with initial corner point solution.

Z x1 x2 x3 x4 x5 RHS

1 −3 −5 0 0 0 00 1 0 1 0 0 40 3 2 0 1 0 180 0 2 0 0 1 12

Slack variables are the initial basic variables,

x3 = 4 , x4 = 18 , x5 = 12 .

Decision variables are set to zero,

x = 0


Step 2: Move to adjacent corner point.

Which direction do we move? There are two choices....

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

(0,6)

(0,0)

(4,0)

(4,3)

(2,6)

0



Choice of movement

I From Objective function, Z − 3x1 − 5x2 = 0I Choose x2 to enter the basis.

I Which basic variable must leave the basis: x3, x4 or x5 ?

I Fix x1 = 0, Increase x2 and see which basic variable reacheszero first.

Constraints equations reduce to

x3 = 42x2 + x4 = 182x2 + x5 = 12

which imply

x3 = 4, x4 = 18− 2x2, x5 = 12− 2x2 .

Thus x3 is unaffected as x2 increases, x4 → 0 as x2 → 9 andx5 → 0 as x2 → 6. Thus x5 must leave the basis, as it goes tozero first.


Algebraic rule

Variable to enter the basis

Choose variable that has the most negative coefficient in the firstrow of the Tableau.

Variable to leave the basis

Given variable xj is to enter the basis.

Find

k = arg mini=1,...,m

biaij

, where aij > 0.

Then variable xk is to leave the basis.


Algebraic Rule in practice

Return to example,

Basis Z x1 x2 x3 x4 x5 RHS Ratio bi/ai2Z 1 −3 −5 0 0 0 0 –x3 0 1 0 1 0 0 4 –x4 0 3 2 0 1 0 18 9x5 0 0 2 0 0 1 12 6 ← min

Pivot on highlighted element.

Basis Z x1 x2 x3 x4 x5 RHSZ 1 −3 0 0 0 5/2 30x3 0 1 0 1 0 0 4x4 0 3 0 0 1 −1 6x2 0 0 1 0 0 1/2 6


Continue the algorithm

Basis Z x1 x2 x3 x4 x5 RHS Ratio bi/ai1Z 1 −3 0 0 0 5/2 30 –x3 0 1 0 1 0 0 4 4x4 0 3 0 0 1 −1 6 2 ← minx2 0 0 1 0 0 1/2 6 –

x1 needs to enter the basis while x4 leaves the basis.

Basis Z x1 x2 x3 x4 x5 RHS Ratio

Z 1 0 0 0 1 3/2 36x3 0 0 0 1 −1/3 1/3 2x1 0 1 0 0 1/3 −1/3 2x2 0 0 1 0 0 1/2 6


Step 3: Stop when at Optimal solution.

How do we know when we reach an optimal solution?


Z 1 0 0 0 1 3/2 36x3 0 0 0 1 −1/3 1/3 2x1 0 1 0 0 1/3 −1/3 2x2 0 0 1 0 0 1/2 6

Stop when all the coefficients in the first row are non-negative.

This is the optimal solution.


Interpret Tableau of Optimal solution


Z 1 0 0 0 1 3/2 36x3 0 0 0 1 −1/3 1/3 2x1 0 1 0 0 1/3 −1/3 2x2 0 0 1 0 0 1/2 6

Any move from here will cause a decrease in Z.

Objective function is Z = 36− x4 − 32x5.

Optimal solution at x∗ = (2, 6) with Z∗ = 36.

Slack variables are x∗s = (2, 0, 0).


Tie Breakers: Variable entering basis

It is possible that there is no unique variable to choose for the rules.

Entering Variable

There may be a tied situation when there are two variables withthe most negative coefficient. E.g.

Basis Z x1 x2 x3 x4 x5 RHS

Z 1 0 −a −a 0 0 C

where a > 0 and C > 0.

Which variable should be chosen to enter the basis? x2 or x3?


Tie Breakers: Variable leaving basis

It is also possible to have a tie for the variable leaving the basis.For example, consider the drug problem, with the secondconstraint modified to 3x1 + 2x2 ≤ 12.

Maximise: Z = 3x1 + 5x2

subject to: x1 ≤ 43x1 + 2x2 ≤ 12

2x2 ≤ 12with: x1, x2 ≥ 0 .

Initial simplex tableau:

Basis Z x1 x2 x3 x4 x5 RHS RatioZ 1 −3 −5 0 0 0 0 –x3 0 1 0 1 0 0 4 –x4 0 3 2 0 1 0 12 6 ← choosex5 0 0 2 0 0 1 12 6 ← either


Tie breaker (cont.)

Basis Z x1 x2 x3 x4 x5 RHS RatioZ 1 −3 −5 0 0 0 0 –x3 0 1 0 1 0 0 4 –x4 0 3 2 0 1 0 12 6x5 0 0 2 0 0 1 12 6Z 1 -3 0 0 0 5/2 30x3 0 1 0 1 0 0 4x4 0 3 0 0 1 -1 0 ← non-positivex2 0 0 1 0 0 1/2 6

I Notice the resource element is zero.

Assumptions of simplex framework violated


Tie breaker (cont.)

Basis Z x1 x2 x3 x4 x5 RHS RatioZ 1 −3 −5 0 0 0 0 –x3 0 1 0 1 0 0 4 –x4 0 3 2 0 1 0 12 6x5 0 0 2 0 0 1 12 6Z∗ 1 9/2 0 0 5/2 0 30 optimalx∗3 0 1 0 1 0 0 4x∗2 0 3/2 1 0 1/2 0 6x∗5 0 −3 0 0 −1 1 0 ← degeneracy

I Degenerate point in the basis with x5 = 0I Can cause stalling.I Can cause an endless cycle.


Degenerate points

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

Feasibleregion

Corner points(0, 6) and (4, 0) are

degenerate.


Solution upon an edge

Return to the previous pharmeutical example with Z = 6x1 + 4x2

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

C2(4,3)

C1(2,6)

8

16

24

32 36


Note: Z = 6x1 + 4x2 isparallel to the boundary

line 3x1 + 2x2 = 18


Solution upon an edge (cont)

Final tableau form.

Basis Z x1 x2 x3 x4 x5 RHS RatioZ 1 −6 −4 0 0 0 0 –x3 0 1 0 1 0 0 4 4x4 0 3 2 0 1 0 18 6x5 0 0 2 0 0 1 12 –Z 1 0 -4 0 0 0 24 –x1 0 1 0 1 0 0 4 –x4 0 0 2 -3 1 0 6 2x5 0 0 2 0 0 1 12 6Z∗ 1 0 0 0 2 0 36 optimalx∗1 0 1 0 1 0 0 4x∗2 0 0 1 -3/2 1/2 0 3x∗5 0 0 0 3 −1 1 6



Final tableau form.

Basis Z x1 x2 x3 x4 x5 RHS RatioZ∗ 1 0 0 0 2 0 36 optimalx∗1 0 1 0 1 0 0 4x∗2 0 0 1 -3/2 1/2 0 3x∗5 0 0 0 3 −1 1 6

I x1, x2, x5 are in the basis

I x3 and x4 are out of the basis.

I However, x3 has a zero coefficient in the objective row.

I Optimal objective function

Z∗ = 36− 2x∗4



Final tableau form.

Basis Z x1 x2 x3 x4 x5 RHS RatioZ∗ 1 0 0 0 2 0 36 optimalx∗1 0 1 0 1 0 0 4x∗2 0 0 1 -3/2 1/2 0 3x∗5 0 0 0 3 −1 1 6

Can parametrise the solution to locate the edge.

I System of equations. Let x∗3 = t ≥ 0.

I

x∗1 = 4− t ≥ 0 ⇒ t ≤ 4.x∗2 = 3 + 3/2t ≥ 0 ⇒ t ≥ −2.x∗5 = 6− 3t ≥ 0 ⇒ t ≤ 2.

I t ∈ [0, 2]






Dual Problem


Minimising the Objective Function

Consider finding the minimum

To minimizeZ = c1x1 + c2x2 + · · ·+ cnxn.

Define a new objective function Z = −Z. Then

minZ ≡ −max Z .

f(x)

x∗−f(x)

x∗


Greater than or equal to constraints

If the problem has a constraint of the form

aTx = a1x1 + a2x2 + · · ·+ anxn ≥ b > 0 ,

introduce a surplus variable xn+1 such that

a1x1 + a2x2 + · · ·+ anxn − xn+1 = b , xn+1 ≥ 0 .

As an example, consider


subject to: x1 ≤ 43x1 + 2x2 ≥ 18

2x2 ≤ 12with: x1, x2 ≥ 0.


Greater than constraint example

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

Feasible

region

Z∗ = 42


Negative Resource Elements

In the standard problem all resource elements bj (i ≤ j ≤ m) arenon-negative. Suppose bj = −b < 0. Then,

aTx = a1x1 + a2x2 + · · ·+ anxn ≤ −b ,

is equivalent to

−aTx = −a1x1 − a2x2 − · · · − anxn ≥ b .


Negative Decision Variable

If xk ≤ 0 introduce a new variable xk = −xk. Then xk ≤ 0 isequivalent to xk ≥ 0.


Unrestricted Decision Variable

If xk is unrestricted in sign, introduce two new variables, xk ≥ 0and xk ≥ 0, and let xk = xk − xk.


Two-Phase Method

Return to the problem,


subject to: x1 ≤ 43x1 + 2x2 ≥ 18

2x2 ≤ 12with: x1, x2 ≥ 0.

Introduce slack and surplus variables,

x1 + x3 = 43x1 + 2x2 − x4 = 18

2x2 + x5 = 12


Finding an initial feasible corner point

I In regular framework.I Set decision variables to zero.I Slack variables set to resource elements.

x1 + x3 = 43x1 + 2x2 − x4 = 18

2x2 + x5 = 12

If done here, implies that,

x1 = 0 , x2 = 0 , x3 = 4 , x4 = −18 , x5 = 12

Surplus variable violates the positivity constraint!


Artificial Variables

I Introduce an artificial variable for any constraint that involvesan equality or greater than or equal to condition.

I For example, introduce x6 ≥ 0 in second constraint equation.

The constraint equations become

x1 + x3 = 43x1 + 2x2 − x4 + x6 = 18

2x2 + x5 = 12

I Set decision and surplus variables to be zero.

I Set slack and artificial variables to resource elements (RHS)

Initial corner point

(x1, x2, x3, x4, x5, x6) = (0, 0, 4, 0, 12, 18) .


Not a corner point of the actual problem.

The initial corner point (x1, x2, x3, x4, x5, x6) = (0, 0, 4, 0, 12, 18)is not in the feasible set.

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6x1

x2

Feasible

region

Z∗ = 42


Reconcile the initial corner point

Feasible solution

A feasible solution of the problem with artificial variables is afeasible solution of the original problem if and only if the artificialvariables are zero.

In the example, the second constraint gives

3x1 + 2x2 − x4 = 18− x6

RHS is smaller than 18 if x6 > 0.

3x1 + 2x2 − x4 = 18

if and only if x6 = 0. Thus we must force the artificial variables tozero.


Find initial feasible solution

To find a basic feasible solution to the original problem maximize,

W = −∑

artificial

xk ≤ 0 .

If maxW = 0, then all artificial variables are zero.

Then proceed with regular simplex algorithm.


First Phase: Maximise W

Basis x1 x2 x3 x4 x5 x6 RHS

W 0 0 0 0 0 1 0Z −3 −5 0 0 0 0 0x3 1 0 1 0 0 0 4– 3 2 0 −1 0 1 18x5 0 2 0 0 1 0 12

W −3 −2 0 1 0 0 −18Z −3 −5 0 0 0 0 0x3 1 0 1 0 0 0 4x6 3 2 0 −1 0 1 18x5 0 2 0 0 1 0 12

After Maximisation on W is complete, remove W and all artificialvariables.



Basis x1 x2 x3 x4 x5 x6 RHS Ratio

W −3 −2 0 1 0 0 −18 –Z −3 −5 0 0 0 0 0 –x3 1 0 1 0 0 0 4 4x6 3 2 0 −1 0 1 18 6x5 0 2 0 0 1 0 12 –

W 0 −2 3 1 0 0 −6 –Z 0 −5 3 0 0 0 12 –x1 1 0 1 0 0 0 4 –x6 0 2 −3 −1 0 1 6 3x5 0 2 0 0 1 0 12 6

W 0 0 0 0 0 1 0 –Z 0 0 −9/2 −5/2 0 5/2 27 –x1 1 0 1 0 0 0 4 4x2 0 1 −3/2 −1/2 0 1/2 3 –x5 0 0 3 1 1 −1 6 2


Second Phase: Maximise Z with new starting point.

Basis x1 x2 x3 x4 x5 RHS Ratio

Z 0 0 −9/2 −5/2 0 27 –x1 1 0 1 0 0 4 4x2 0 1 −3/2 −1/2 0 3 –x5 0 0 3 1 1 6 2

Z 0 0 0 −1 3/2 36 –x1 1 0 0 −1/3 −1/3 2 –x2 0 1 0 0 1/2 6 –x3 0 0 1 1/3 −1/3 2 6

Z 0 0 3 0 5/2 42 Optimalx1 1 0 1 0 0 4x2 0 1 0 0 1/2 6x4 0 0 3 1 1 6


Big M-method

I An alternative to the Two-Phase

Maximise: Z = −x1 + 2x2

subject to: x1 + x2 ≥ 2−x1 + x2 ≥ 1

x2 ≤ 3with: x1, x2 ≥ 0.

I Introduce slack, surplus and artificial variables as before.

I Modify objective function to penalise the artificial variables.

Z = Z ±M∑

k∈artificial

xk.

I NB: Sign infront of M penalises against the optimatisation.


Big M-method

I Constraint equations

x1 + x2 − x3 + x6 = 2−x1 + x2 − x4 + x7 = 1

x2 + x5 = 3

I Objective function is Maximisation:

Max: Z = −x1 + 2x2−M(x6 + x7)

I NB: If Minimisation was desired consider,

Min: Z = −x1 + 2x2+M(x6 + x7)

I M is considered to be an arbitrarily large number.


Big M - implementation:

Basis x1 x2 x3 x4 x5 x6 x7 RHS Ratio

Z 1 -2 0 0 0 M M 0 –– 1 1 −1 0 0 1 0 2 –– -1 1 0 −1 0 0 1 1 –x5 0 1 0 0 1 0 0 3 –

Z 1 -2M-2 M M 0 0 0 -3M –x6 1 1 −1 0 0 1 0 2 2x7 -1 1 0 −1 0 0 1 1 1x5 0 1 0 0 1 0 0 3 3

I Initialise tableau to ensure artificial variables are basic.

I Continue regular simplex algorithm.




Z 1 -2M-2 M M 0 0 0 -3M –x6 1 1 −1 0 0 1 0 2 2x7 −1 1 0 −1 0 0 1 1 1x5 0 1 0 0 1 0 0 3 3

Z -2M-1 0 M -M−2 0 0 2M+2 -M+2 –x6 2 0 −1 1 0 1 -1 1 1

2x2 −1 1 0 −1 0 0 1 1 –x5 1 0 0 1 1 0 -1 2 2

Z 0 0 −0.5 −1.5 0 M+0.5 M+1.5 2.5 –x1 1 0 −0.5 0.5 0 0.5 -0.5 0.5 1x2 0 1 −0.5 −0.5 0 0.5 0.5 1.5 –x5 0 0 0.5 0.5 1 -0.5 -0.5 1.5 3




Z 0 0 −0.5 −1.5 0 M+0.5 M+1.5 2.5 –x1 1 0 −0.5 0.5 0 0.5 -0.5 0.5 1x2 0 1 −0.5 −0.5 0 0.5 0.5 1.5 –x5 0 0 0.5 0.5 1 −0.5 −0.5 1.5 3

Z 3 0 −2 0 0 M+2 M 4 –x4 2 0 −1 1 0 1 -1 1 –x2 1 1 −1 0 0 1 0 2 –x5 −1 0 1 0 1 −1 0 1 1

Z 1 0 0 0 2 M M 6 optx4 1 0 0 1 1 0 −1 2 –x2 0 1 0 0 1 0 0 3 –x5 −1 0 1 0 1 −1 0 1 –






Dual Problem


Dual Problem

Recall the regular problem in matrix notation,

PrimalMaximise: Z = cTxsubject to: Ax ≤ b

with: x ≥ 0

Refer to it as the Primal problem.


Definition of dual problem

Primal DualMaximise: Z = cTx Minimise: v = yTbsubject to: Ax ≤ b subject to: yTA ≥ cT

with: x ≥ 0 with: y ≥ 0

Return to initial pharmaceutical example:

Maximise : Z = 3x1 + 5x2

such that: x1 ≤ 4

3x1 + 2x2 ≤ 18

2x2 ≤ 12

and x1 ≥ 0, x2 ≥ 0, x3 ≥ 0, x4 ≥ 0, x5 ≥ 0.


Example: Dual construction

I Find a bound for optimal solution.I From constraints,

Z = 3x1 + 5x2 ≤ 3× 4 + 5× 6 = 42

I Consider a general linear combination of constraints,

(x1)y1 + (3x1 + 2x2) y2 + (2x2)y3 ≤ 4y1 + 18y2 + 12y3

I Define a new objective function v := 4y1 + 18y2 + 12y3

Notice,

Z := 3x1 + 5x2

≤ (y1 + 3y2)x1 + (2y2 + 2y3)x2

≤ 4y1 + 18y2 + 12y3 =: v


Example: continued

I Last statement true if y ≥ 0 and,

y1 + 3y2 ≥ 3 and 2y2 + 2y3 ≥ 5

I Dual problem is,

Minimise : v = 4y1 + 18y2 + 12y3

such that: y1 + 3y2 ≥ 3

2y2 + 2y3 ≥ 5

and y1 ≥ 0, y2 ≥ 0, y3 ≥ 0.


Useful applications

I Dimension of decision variables and constraint equations swap.

I Optimal values of Max: Z and Min: v coincide under certainconditions.

I Can sometimes simplify the problem to be solved.


Example

Example

Maximise: Z = 8x1 + 30x2 + 7x3

subject to: x1 + 5x2 + 3x3 ≤ 104x1 + 6x2 + x3 ≤ 15

with: x1, x2, x3 ≥ 0.

The corresponding dual problem is:

Minimise: v = 10y1 + 15y2

subject to: y1 + 4y2 ≥ 85y1 + 6y2 ≥ 303y1 + y2 ≥ 7

with: y1, y2 ≥ 0 ,

which can be solved graphically.


Graphical solution

Minimise: v = 10y1 + 15y2

subject to: y1 + 4y2 ≥ 85y1 + 6y2 ≥ 303y1 + y2 ≥ 7

with: y1, y2 ≥ 0 ,

0

1

2

3

4

5

6

7

8

9

0 1 2 3 4 5 6 7 8 9 10 11y1

y2

Optimal solution,

v = 435

7, (y1, y2) = (36

7,5

7)

5y1 + 6y2 = 30

y1 + 4y2 = 8

2y1 + y2 = 8


Second Example

Return to the Pharmaceutical problem. It has the dual problem,

Minimise : v = 4y1 + 18y2 + 12y3

such that: y1 + 3y2 ≥ 3

2y2 + 2y3 ≥ 5

and y1 ≥ 0, y2 ≥ 0, y3 ≥ 0.Standardise the problem for the simplex framework.

Maximise: v = −4y1 − 18y2 − 12y3

subject to:y1 + 3y2 − y4 + y6 = 3

2y2 + 2y3 − y5 + y7 = 5

with: y1 ≥ 0, y2 ≥ 0, y3 ≥ 0.


Solve using Two-Phase


Basis y1 y2 y3 y4 y5 y6 y7 RHS

W 0 0 0 0 0 1 1 0v 4 18 12 0 0 0 0 0– 1 3 0 -1 0 1 0 3– 0 2 2 0 -1 0 1 5

W −1 −5 -2 1 1 0 0 −8v 4 18 12 0 0 0 0 0y6 1 3 0 -1 0 1 0 3y7 0 2 2 0 -1 0 1 5


Maximise W

Basis y1 y2 y3 y4 y5 y6 y7 RHS Ratio

W −1 −5 -2 1 1 0 0 −8 –v 4 18 12 0 0 0 0 0 –y6 1 3 0 -1 0 1 0 3 1y7 0 2 2 0 -1 0 1 5 5/2

W 2/3 0 -2 -2/3 1 5/3 0 −3 –v −2 0 12 6 0 -6 0 -18 –y2 1/3 1 0 -1/3 0 1/3 0 1 –y7 -2/3 0 2 2/3 -1 -2/3 1 3 3/2

W 0 0 0 0 0 1 1 0 –v 2 0 0 2 6 -2 -6 -36 –y2 1/3 1 0 -1/3 0 1/3 0 1 –y3 -1/3 0 1 1/3 -1/2 -1/3 1/2 3/2 –


Phase Two: Discard W

Basis y1 y2 y3 y4 y5 RHS

v 2 0 0 2 6 -36y2 1/3 1 0 -1/3 0 1y3 -1/3 0 1 1/3 -1/2 3/2

I Notice that,

Max Z = 3x1 + 5x2 = 36 = −Max v

I y∗ = (y∗1, y∗2, y∗3) = (0, 1, 3/2) and x∗ = (2, 6)

I The optimal x∗ can be found as the cost coefficients of thesurplus variables y4 and y5 in the objective row.

I Agrees with graphical and simplex methods used before

Documents

MATH2070 - Linear programming · The standard Linear Programming (LP) Problem Graphical method of solving LP problem ... A feasible solution lies inside a closed region inside the