IEOR 4004 Final Review part II.

April 30, 2014

Integer Programming

• Standard problems– Knapsack– Fixed-charge Problem (Facility location)– Cutting Stock Problem– Set cover– Travelling Salesman Problem– Job/Machine Scheduling Problems

max cxAx = b x ≥ 0x integer

pure IP = all variables integermixed IP = some variables integer

IP formulations• fixed charge (penalty) for a failing a constraint

• alternative constraints (logical or)

• if-then constraints

• piece-wise linear objective

f ≤ 0 f ≤ My add+py

to the objectivey {0,1}∈

constraint penaltyfor failingthe constraint

constraint

f ≤ 0 g ≤ M(1-y)

y {0,1}∈g ≤ 0 f ≤ My

if f > 0 , g ≤ M(1-y)

y {0,1}∈then g ≤ 0 f ≤ My

Branch-and-Bound

• solve the LP relaxationoptimal solution to the LP relaxation gives an upper bound• branch on fractional variables (if exist)• how do we (re)-solve the subproblems quickly ?

– Upper-bounded Dual Simplex– special subproblem (Knapsack, Transportation Problem)fractional solution = solution to the LP relaxation

max 12x1+ 4x2 + 5x3

s.t. 5x1+ 2x2 + 3x3 ≤ 13x1,x2,x3 ≥ 0 and integer

max 12x1+ 4x2 + 5x3

s.t. 5x1+ 2x2 + 3x3 ≤ 13x1,x2,x3 ≥ 0

Integer Program Linear Programming(LP) relaxation

drop theintegrality

Upper-bounded Dual Simplex

given a dually feasible basic solution (all coeffs in z negative)• check if all basic variables satisfy their upper bounds

if some xi fails, if xi should be ≤ B but is > B, then substitute xi = B – xi’

• check if all basic variables satisfy their lower boundsif some xi < 0, then use dual ratio test to find xj such that the ratio b/a is smallest possible where:– a is the coefficient of xj in the equation for xi

– (-b) is the coefficient of xj in the equation for z– a is positive

pivot xi out and xj into the basis

Dually feasible solution

Optimal ?1. check upper bounds2. check lower bounds

1. substitute xi = B – xi’ where 0 ≤ xi ≤ B2. pivot (dual ratio test) xi out, xj in the basis

x1 = -2 + 2x2 – 4x3 + 3x4

x5 = 1 + x2 + 2x3 – 2x4

z = 5 – 3x2 – 5x3 – 4x4

x1 leaves, ratio test:x2: 3 / 2 = 1.5x4: 4 / 3 = 1.333 x4 enters

Branching tree

candidatesolution

x1 = 1x2 = 2.4z = 18

x1 = 1.25x2 = 2

z = 17.5

x1 = 1x2 = 2z = 16

x1 = 2x2 = 0.8z = 16

x1 = 0.25x2 = 3

z = 16.5

x1 = 0x2 = 3.2z = 16

infeasible

don’t need to branch (no solution here is better than z=16and we already know a candidate solution of value 16)

max 6x1 + 5x2

s.t. 4x1 + 5x2 ≤ 168x1 + 5x2 ≤ 20

x1,x2 ≥ 0 & integer x2 ≤ 2 x2 ≥ 3

LP relaxation optimal fractional solutionnot an integer solution

branch x2 ≤ 2.4 = 2

x2 ≥ 2.4 = 3

x1 ≤ 1 x1 ≥ 2 x1 ≤ 0 x1 ≥ 1

optimalsolution

Examplemax 12x1+ 4x2 + 5x3

s.t. 5x1+ 2x2 + 3x3 ≤ 13x1,x2,x3 ≥ 0

max 12x1+ 4x2 + 5x3

s.t. 5x1+ 2x2 + 3x3 + x4 = 13x1,x2,x3,x4 ≥ 0

x1 = 2.6 – 0.4x2 – 0.6x3 – 0.2x4

z = 31.2 – 0.8x2 – 2.2x3 – 2.4x4

Initial solution

x1 ≤ 2 x1 ≥ 3

infeasible

x1 = 2.6x2 = 0x3 = 0

z = 31.2

x1 ≥ 3 substitute x1 = 3 + x5

x5 = -0.4 – 0.4x2 – 0.6x3 – 0.2x4

z = 31.2 – 0.8x2 – 2.2x3 – 2.4x4

dictionary is not feasible anymoreSubproblem: x1 ≥ 3

Upper-bounded Dual Simplex methodHow to quickly recover an optimum (without starting over) ?

• check upper bounds: OK• check lower bounds: -0.4 = x5 ≥ 0

x5 leaves, ratio test:x2: no constraintx3: no constraintx4: no constraint

infeasible LP

x1 ≤ 2 substitute x1 = 2 – x1’

x1’ = -0.6 + 0.4x2 + 0.6x3 + 0.2x4

z = 31.2 – 0.8x2 – 2.2x3 – 2.4x4

• check upper bounds: OK• check lower bounds: -0.6 = x1’ ≥ 0

x1’ leaves, ratio test:x2: 0.8/0.4 = 2x3: 2.2/0.6 = 3⅔x4: 2.4/0.2 = 12

x2 enters

x2 = 1.5 + 2.5x1’ – 1.5x3 – 0.5x4

z = 30 – 2x1’ – x3 – 2x4

optimal solution recoveredx1’ = 0 x1 = 2–x1’ = 2x2 = 1.5 x3 = 0 z = 30

x2 ≤ 1 x2 ≥ 2

x1 = 2x2 = 1.5x3 = 0z = 30

not an integer solutionbranch on x2 ≤ 1 and x2 ≥ 2

x1 = 2x2 = 1

x3 = 1/3z = 29⅔

x1 = 2x2 = 1x3 = 0z = 28

x1 = 2x2 = 0x3 = 1z = 29

x1 = 1x2 = 4x3 = 0z = 28

infeasible

x3 ≥ 1 x1 = 2x1 ≤ 1

x1 = 1.8x2 = 2x3 = 0

z = 29.6

candidate solution candidate solution optimal

Repeat

x3 = 0

Knapsackmax 12x1+ 4x2 + 5x3

s.t. 5x1+ 2x2 + 3x3 ≤ 13x1,x2,x3 ≥ 0 and integer

x1 = 2x2 = 1

x3 = 1/3z = 29⅔

x1 = 2x2 = 1x3 = 0z = 28

x1 = 2x2 = 0x3 = 1z = 29

x1 = 1x2 = 4x3 = 0z = 28

infeasible

x1 ≤ 2 x1 ≥ 3

x2 ≤ 1 x2 ≥ 2

x3 = 0 x3 ≥ 1 x1 = 2x1 ≤ 1

x1 = 2.6x2 = 0x3 = 0

z = 31.2

x1 = 2x2 = 1.5x3 = 0z = 30

x1 = 1.8x2 = 2x3 = 0

z = 29.6

infeasible

optimal greedy strategy (for LP relaxation)• pick items by value/weight, highest first• take as many items of

the same type as possible

Initial solution

candidate solution

value/weight

2.4 2 1⅔

max 12x4+ 4x2 + 5x3 + 15s.t. 5x4+ 2x2 + 3x3 ≤ -2

x1 ≥ 3 substitute x1 = 3 + x4

x1 ≤ 2 pick 2 units of item #1and then 1.5 of item #2(as much as possible)

max 12x1+ 4x5 + 5x3 + 8s.t. 5x1+ 2x5 + 3x3 ≤ 9

x2 ≥ 2 substitute x2 = 2 + x5

x1 ≤ 1 pick 1 units of item #1then 2 of item #5(i.e. 4 units of item #2)

candidate solution

pick 1.8 units of item #1

max 4x5 + 5x3 + 32s.t. 2x5 + 3x3 ≤ -1

x1 = 2 substitute x1 = 2

x2 ≤ 1 pick 2 units of item #1then 1 unit of item #2and then 1/3 of item #3

x3 = 0 do not pick item #3(1 unit unused in the bag)

max 12x1+ 4x2 + 5x7 + 5s.t. 5x1+ 2x2 + 3x7 ≤ 10

x3 ≥ 1 substitute x3 = 1 + x7

pick 2 units of item #1

optimal

Cutting Planes

• only for pure IPs• solve LP relaxation• if not an integer optimum, make the feasible

region smaller by cutting out fractional points (without removing any integer points)

• how to generate cuts ?

1 2 3 40 5

1

2

3

4max 6x1 + 5x2

s.t. 4x1 + 5x2 ≤ 168x1 + 5x2 ≤ 20

x1,x2 ≥ 0 & integer

optimalfractionalsolution

newfractionaloptimum

Gomory cuts

x1 + x2 ≤ 3x1,x2 ≥ 0 & integer

Gomory cuts

1 2 3 40 5

1

2

3

4max 6x1 + 5x2

s.t. 4x1 + 5x2 ≤ 168x1 + 5x2 ≤ 20

x1 + x2 ≤ 3x1,x2 ≥ 0 & integer

optimalfractionalsolution

max 6x1 + 5x2

s.t. 4x1 + 5x2 + x3 = 168x1 + 5x2 + x4 = 20

x1 + x2 +x5 = 3

x1,x2,x3,x4,x5 ≥ 0 & integerx1 = 1⅔ – ⅓x4 + 1⅔x5

x2 = 1⅓ + ⅓x4 – 2⅔x5

x3 = 2⅔ – ⅓x4 + 6⅔x5

z = 16⅔ – ⅓x4 – 3⅓x5 x1 = 1⅔ – ⅓x4 + 1⅔x5

x1 – 2x5 – 1 = ⅔ – ⅓x4 – ⅓x5

1+⅔ 0+⅓ 2–⅓

split to whole and fractional part

fractional partwhole part positiveconstant

negativecoeffients< 1≤ 0

cutUse Dual Simplex torecover the optimum

optimalsolution

x1 = 1 – x6 + 2x5

x2 = 2 + x6 – 3x5

x3 = 2 – x6 + 7x5

x4 = 2 + 3x6 – x5

z = 16 – x6 – 3x5

x6 = – ⅔ + ⅓x4 + ⅓x5

z = 16⅔ – ⅓x4 – 3⅓x5

Dynamic Programming

• Break down the problem into stages– each stage has possible states– next stage depends only on the previous stage– identify possible decisions/transitions at each stage– find a formula to evaluate (by recursion)

• Evaluate stage by stage

stage 1 stage 2 stage 3

fi ( j ) = optimum valuefor the first i stages thatends in state j of stage i

e.g. cost of cheapest production plan forfirst i months where we end up with

j items in inventory at the end of month i

(inventory in month i+1 only depends onproduction and inventory level in month i)

1

2

3

1

2

3

1

3

2

time(months, years)

budget(available resources)

Goal: – value of the formula for a target state (in the last stage) – save on calculation by storing the intermediate results fi(j)

optimal substructure property

Dynamic Programming

Investments:#1. x1 > 0 thousand $ yields c1(x1)=7x1+2 thousand $#2. x2 > 0 thousand $ yields c2(x2)=3x2+7 thousand $#3. x3 > 0 thousand $ yields c3(x3)=4x3+5 thousand $

c1(0)=c2(0)=c3(0)=0We can invest up to $6,000.

decision in stage i = how much we invest into #i

fi( j ) = maximum profit we can get by investing j thousand dollars into first i investments

(all we need to know is how much money we have available for investing)

fi( j ) =k {0,1,...,j}∈

how muchwe invest in #i

leftover cash

best possible profitwith the leftover cash

profit from #if0( j ) = 0

6

0

1

2

3

4

5

6

0 0

1

2

3

4

5

6

#1 #2 #3

cash 0 1 2 3 4 5 6

start 0

#1 0 9 16 23 30 37 44

#2 0 10 19 26 33 40 47

#3 49

max { }ci(k) + fi-1( )j – k

green arrow = best choice of k

Traveling Salesman ProblemFind a shortest route that goes through all cities

and comes back where it starts

Map of cities Shortest route

Traveling Salesman Problem

a shortest route starting at s ending at v and visiting all nodes in S

v

Sf(S,v) = length of

f(S,v) = min f(S\{x}, x) + cxvx ∈ S(x,v) ∈ E

Work backwards – which is the last vertex visited before v ?

x

S\{x}

optimalsubstructure

property

initial condition: f(Ø,s) = 0

solution: f(V\{s},s)

s

complexity: O(n22n) time and space brute force (trying all routes)is O(n!) time and O(1) space

Good luck on the final exam.