Decision Aid Methodologies In Transportation

CIVIL-557

Decision Aid Methodologies

In Transportation

Transport and Mobility Laboratory TRANSP-OR

École Polytechnique Fédérale de Lausanne EPFL

Virginie Lurkin

Lecture 4:

Integer programming I: Branch and Bound

On the previous lecture…

Container Storage Inside a Container Terminal

o Two-phase simplex method

o Duality theory

Suppose 2 new containers are expected to arrive for storage in the next planning

period of a terminal. Suppose there are only 2 blocks in the terminal, each with 20

storage spaces. For the moment, there are 6 containers in block 1 and 12 containers

in block 2.

𝑵 = 𝟐

B = 𝟐

A = 𝟐𝟎

𝒂𝟏 = 𝟏𝟎

𝒂𝟐 = 𝟗

𝑭 =𝟏𝟎+𝟗+𝟐

𝟐×𝟐𝟎= 𝟎. 𝟓𝟐𝟓

𝑀𝑖𝑛𝑖𝑚𝑖𝑧𝑒 𝑢1+ + 𝑢1

− + 𝑢2+ + 𝑢2

−

𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜𝑥1−𝑢1

+ + 𝑢1− = 0.5

𝑥2−𝑢2+ + 𝑢2

− = 1.5𝑥1+𝑥2 = 2𝑥1, 𝑥2, 𝑢1

+, 𝑢1−, 𝑢2

+, 𝑢2− ≥ 0

What if the situation is different…

Linear programming

𝟏𝟎 𝟗

Optimal solution: (𝟎. 𝟓, 𝟏. 𝟓, 0, 0, 0, 0)

𝑥1 and 𝑥2 represent the number of containers to be assigned, respectively, to

block 1 and 2

𝑥1 and 𝑥2, in reality, should be integer

Linear (mixed) integer programming

Linear integer programming

𝐴𝑥 ≤ 𝑏

𝑥 ≥ 0,

subject to

where 𝐴 ∈ ℝ𝑚×𝑛, b ∈ ℝ𝑚, c ∈ ℝ𝑛

𝑚𝑖𝑛 𝑐𝑇𝑥𝑥 ∈

𝑛

Pure integer programming (IP):

A mixed-integer programming (MIP) problem is one where only some of

the decision variables are constrained to be integer values


How to solve integer programming problem?

Integer solutions

Objective function

Optimum of LP

relaxation

Nearest integer

(not feasible)

IP optimum

Intuitive idea: remove the integer constraints from the IP formulation (LP relaxation)

o Solve the LP relaxation and round to the nearest integer

Not the best idea!


How to solve integer programming problem?

There is no optimality conditions for integer programming problem

Explicit enumeration is normally impossible due to the exponentially increasing

number of potential solutions

Exact methods can be used to find the optimal solution:

o Two main methods:

o Computing a true optimal solution might be very costly

Heuristic methods can be used to find a good solution:

o Does not guarantee optimality

Branch and Bound (B&B) and Cutting planes

Heuristics

Heuristics

Algorithms and complexity measures

An algorithm is a finite set of instructions that when executed on an input can

produce an output after finitely many steps.

The input encodes an instance of the problem that the algorithm is supposed to

solve, and the output encodes a solution for the given problem instance.

The computation of an algorithm on an input is the sequence of steps it performs.

Algorithms do not always terminate; they may perform an infinite computation for

some, or any, input.

Heuristics


There are problems that can be easily and precisely defined in mathematical terms,

yet that provably cannot be solved by any algorithm (undecidable/ uncomputable

problems.

In practice, we cannot expect to be able to solve real world problem instances of

arbitrary size to optimality.

Depending on the size of an instance or depending on the available CPU time we will

often have to be satisfied with computing approximate solutions.

Heuristics


The term heuristic is used for algorithms which find solutions among all possible

ones ,but they do not guarantee that the best will be found.

A good heuristic algorithm should fulfil the following properties:

o A solution can be obtained with reasonable computational effort.

o The solution should be near optimal (with high probability).

o The likelihood for obtaining a bad solution (far from optimal) should be low.

How can we evaluate the performance of an heuristic algorithm ?

1. Rearrange blocks so that 𝑎𝑖 increases with 𝑖

2. Determine 𝑥𝑖 in increasing order of 𝑖 using:

𝑥1 = min{𝑁,max 0, 𝐴 × F − 𝑎1 }

𝑥𝑖 = m𝑖𝑛{max 0, 𝐴 × F − 𝑎𝑖 , 𝑁 −𝑟=1

𝑖−1

𝑥𝑟} ∀𝑖 ≥ 2

Heuristics

A concrete example for our container storage problem

Branch and Bound algorithm


General idea

Implicit enumeration of the feasible set, by ruling out large sets of feasible

points

How ?

o By partitioning the feasible set into smaller subsets

o By using bounds to identify subsets that are guaranteed not to contain an

optimal solution

𝑺𝑺𝟏

𝑺𝟐

𝑺𝟏𝟏 𝑺𝟏𝟐

𝑺𝟐𝟏 𝑺𝟐𝟐

Theorem 1:

𝑃𝑘is an optimization problem (minimization)

𝑆 is the feasible set of the optimization problem 𝑃

𝑆 is partitioned into 𝐾 subsets: 𝑆1 ∪ … ∪ Sm ∪ Sm+1 ∪ … ∪ SK

For each 𝑘 = 1,2, … , 𝐾, let 𝑙𝑒𝑡 𝑥𝑘∗ be an optimal solution of the

optimization problem 𝑃𝑘(minimization). Let 𝑖 be such that:

𝑓 𝑥𝑖∗ ≤ 𝑓 𝑥𝑘

∗ , 𝑘 = 1,… , 𝐾

Then 𝑥𝑖∗ is an optimal solution of the optimization problem 𝑃

𝑺𝟏𝟏 𝑺𝟏𝟐

𝑺𝟐𝟏 𝑺𝟐𝟐



Partitioning the original problem into smaller subproblems

For each feasible subproblem, we will calculate a lower bound (best possible value)

Lower bounds are obtained by solving relaxations of the original subproblems

Calculate lower bounds is much easier (simplex algorithm can be used)

Partitioning:

Partitioning the feasible set into smaller subsets

Using bounds:

Any feasible solution provides an upper bound on the optimal solution (minimization)

A feasible solution is a solution that satisfies all the constraints

𝑺𝟏𝑺𝟐

𝑃

𝑃1 𝑃2

𝑃11 𝑃12


𝐿𝐵𝑃

𝐿𝐵𝑃1 𝐿𝐵𝑃2

Some candidates are excluded based on the value of the lower bound

Using bounds:

𝑈𝐵

𝑺 If 𝐿𝐵𝑃2 ≥ 𝑈𝐵If 𝐿𝐵𝑃1 < 𝑈𝐵

Branching refers to exploring the subtree of an active node

Bounding refers to estimating a lower bound at an active node

For each 𝑘 = 1,2, … ,𝑚, let 𝑙𝑒𝑡 𝑥𝑘∗ be an optimal solution of 𝑃𝑘.

For each 𝑘 = 𝑚 + 1,… , 𝐾, let 𝑙(𝑃𝑘) be a lower bound on 𝑃𝑘:

𝑙 𝑃𝑘 ≤ 𝑓 𝑥𝑘 ∀𝑘 ∈ 𝑆𝑘

Let 𝑖 be such that:

𝑓 𝑥𝑖∗ ≤ 𝑓 𝑥𝑘

∗ , 𝑘 = 1,… ,𝑚

and

𝑓 𝑥𝑖∗ ≤ 𝑙 𝑃𝑘 , 𝑘 = 𝑚 + 1,… , 𝐾

Then 𝑥𝑖∗ is an optimal solution of the optimization problem 𝑃


Theorem 2:


Branching:

𝑃

𝑃1 𝑃2

𝐿𝐵𝑃

𝑈𝐵

Select a variable 𝑥𝑗 that has a fractional value:

𝑘 < 𝑥𝑗 < 𝑘 + 1, with 𝑘 integer

Create two new LP-subproblems:

1. Previous LP-subproblem + constraint 𝑥𝑗 ≤ 𝑘 (𝑘 = 𝑥𝑗 )

2. Previous LP-subproblem + constraint 𝑥𝑗 ≥ 𝑘 + 1 (𝑘 + 1 = 𝑥𝑗 )

How to branch?

Algorithm

Solve the relaxation 𝑅 𝑃𝑘 and denote 𝑥𝑅∗ the optimal value

𝑙 𝑃𝑘 ≔ 𝑓(𝑥𝑅∗ )

𝑢 𝑃 > 𝑙 𝑃𝑘 ?

𝑥𝑅∗ integer?

no

𝑃 ≔ 𝑃0𝑢 𝑃 ≔ +∞

𝑢 𝑃 , Upper Bound for 𝑃

no

Discard 𝑃𝑘 from 𝑃yes

𝑙 𝑃𝑘 ,Lower Bound for 𝑃𝑘

no

yes𝑓 𝑥𝑅

∗ < 𝑢(𝑃)?

𝑃 empty?

no

𝑥∗ = (𝑥𝑅∗ )

𝑢 𝑃 ≔ 𝑓 𝑥𝑅∗

yes

end

𝒙∗, 𝒇∗= 𝒖 𝑷

yes

Select 𝑖 such that (𝑥𝑅∗ )𝑖 is not integer

Create 𝑃𝑘𝑙 by adding the constraint 𝑥𝑖 ≤ (𝑥𝑅

∗ )𝑖 to 𝑃𝑘Create 𝑃𝑘

𝑟 by adding the constraint 𝑥𝑖 ≥ (𝑥𝑅∗ )𝑖 to 𝑃𝑘

𝑃 ≔ 𝑃 ∪ 𝑃𝑘𝑙 , 𝑃𝑘

𝑟 \P𝑘

Select a problem 𝑃𝑘 in 𝑃. 𝑃𝑘 infeasible?

yes

no


Exploration of the search tree:

𝑃

𝑃1 𝑃2

𝑃11 𝑃12

𝐿𝐵𝑃


𝑈𝐵

If 𝐿𝐵𝑃2 ≥ 𝑈𝐵If 𝐿𝐵𝑃1 < 𝑈𝐵

𝑃122 𝑃123

𝑃1232𝑃1231

𝑃121

Depth-first search (DFS): explores as far as possible along each branch before backtracking


Exploration of the search tree:

𝑃

𝑃1 𝑃2

𝑃11 𝑃12

𝐿𝐵𝑃


𝑈𝐵

If 𝐿𝐵𝑃2 ≥ 𝑈𝐵If 𝐿𝐵𝑃1 < 𝑈𝐵

𝑃122 𝑃123

Breadth-first search (BFS): explores nodes level by level

𝑃121

Depending on the problem at hand, either DFS or BFS could be advantageous


Back to our case:



− + 𝑢2+ + 𝑢2

−


+ + 𝑢1− = 0.5

𝑥2−𝑢2+ + 𝑢2

− = 1.5𝑥1+𝑥2 = 2𝑥1, 𝑥2, 𝑢1

+, 𝑢1−, 𝑢2

+, 𝑢2− ≥ 0

𝑥𝑅∗ = 0.5, 1.5, 0, 0, 0, 0 and 𝑓 𝑥𝑅

∗ = 0

𝑃0

𝑢 𝑃 = +∞

𝑃0𝑟𝑃0

𝑙

𝑥1 ≤ 0 𝑥1 ≥ 1 𝑢 𝑃 > 𝑙 𝑃0

𝑙 𝑃0 = 0

𝑥1, 𝑥2 are not integers

Select 𝑥1 for branching

𝑙(𝑃0) = 0

𝑹(𝑷𝟎)

Back to our case:


𝑥𝑅∗ = 0, 2, 0, 0.5, 0.5, 0 and 𝑓 𝑥𝑅

∗ = 1

𝑃0

𝑢 𝑃 = +∞

𝑃0𝑟𝑃0

𝑙

𝑥1 ≤ 0 𝑥1 ≥ 1 𝑢 𝑃 > 𝑙 𝑃0

𝑙 𝑃0𝑙 = 1

𝑥1, 𝑥2 are integers 𝑢 𝑃 = 1

Discard 𝑃0𝑙

𝑙(𝑃0) = 0


− + 𝑢2+ + 𝑢2

−


+ + 𝑢1− = 0.5

𝑥2−𝑢2+ + 𝑢2

− = 1.5𝑥1+𝑥2 = 2𝑥1≤ 0𝑥1, 𝑥2, 𝑢1

+, 𝑢1−, 𝑢2

+, 𝑢2− ≥ 0

𝑹(𝑷𝟎𝒍 )

𝑙(𝑃0𝑙) = 1

𝑢 𝑃 = 1

Back to our case:


𝑥𝑅∗ = 1, 1, 0.5, 0, 0, 0.5 and 𝑓 𝑥𝑅

∗ = 1

𝑃0

𝑢 𝑃 = +∞

𝑃0𝑟𝑃0

𝑙

𝑥1 ≤ 0 𝑥1 ≥ 1 𝑢 𝑃 > 𝑙 𝑃0

𝑙 𝑃0𝑟 = 1

𝑥1, 𝑥2 are integers

Discard 𝑃0𝑟

𝑙(𝑃0) = 0


− + 𝑢2+ + 𝑢2

−


+ + 𝑢1− = 0.5

𝑥2−𝑢2+ + 𝑢2

− = 1.5𝑥1+𝑥2 = 2𝑥1≥ 1𝑥1, 𝑥2, 𝑢1

+, 𝑢1−, 𝑢2

+, 𝑢2− ≥ 0

𝑙(𝑃0𝑙) = 1

𝑢 𝑃 = 1

𝑓 𝑥𝑅∗ = 1

end

𝑥𝑅∗ = 0, 2, 0, 0.5, 0.5, 0 or 𝑥𝑅

∗ = 1, 1, 0.5, 0, 0, 0.5

𝑹(𝑷𝟎𝒓)


Some integer programming tricks

Integer programming tricks

Example 1

o The value of variable 𝑥 must be either zero or between particular positive bounds

Suppose that a supplier of some item requires that if an item is ordered, then its batch

size must be between a particular minimum and maximum value

Discontinuous values

o 𝑥 is the number of ordered units

o Algebraic notation: 𝑥 = 0 ∣ 𝑙 ≤ 𝑥 ≤ 𝑢


We introduce an indicator variable

An indicator variable is a binary variable (0/1) that indicates a certain state in a model

A set of constraints is used to create the desired properties:

𝑥 ≤ 𝑢𝑦𝑥 ≥ 𝑙𝑦𝑦 ∈ {0,1}

Discontinuous values

𝑦 = ቊ01

for 𝑥 = 0

for 𝑙 ≤ 𝑥 ≤ 𝑢


Example 2

Fixed costs

Suppose that a manager must decide which of 𝑛 warehouses to use for meeting the

demands of 𝑚 customers for a good. The decisions to be made are which warehouses

to operate and how much to ship from any warehouse to any customer

o There is a fixed operating cost associated to each warehouse

o Decisions must be made about tradeoffs between transportation costs and costs

for operating distribution centers

o Very common issue in modeling distribution systems



Data:

Fixed costs

𝑦𝑖 = ቊ10

if warehouse 𝑖 is opened

if warehouse 𝑖 is not opened

𝑥𝑖𝑗 = amount to be sent from warehouse 𝑖 to customer 𝑗

Other decisions variables:

𝑓𝑖 = fixed operating cost for warehouse 𝑖

𝑝𝑖 = per-unit operating cost at warehouse 𝑖

𝑐𝑖𝑗 = transportation cost for shipping from warehouse 𝑖 to customer 𝑗


Model:

Fixed costs

min

𝑖=1

𝑚

𝑗=1

𝑛

(𝑝𝑖+𝑐𝑖𝑗)𝑥𝑖𝑗 +

𝑖=1

𝑚

𝑓𝑖𝑦𝑖

s. t.

𝑖=1

𝑚

𝑥𝑖𝑗 = 𝑑𝑗 , ∀𝑗 = 1,2, … , 𝑛

𝑗=1

𝑛

𝑥𝑖𝑗 ≤ 𝑴𝑦𝑖 , ∀𝑖 = 1,2, … ,𝑚 𝑴 is a big number

𝑥𝑖𝑗 ≥ 0, ∀𝑖 = 1,2, … ,𝑚, ∀𝑗 = 1,2, … , 𝑛

𝑦𝑖 ∈ 0,1 , ∀𝑖 = 1,2, … ,𝑚


Example 3

Suppose that a company has to choose among two modes of operation for the

manufacturing process of its products on a single machine

o Very common issue in modeling manufacturing process

o Each mode has its own performance (number of units produced by minute)

Either… or …

o A mode of operation might better suit to some products



Data:

Either… or …

𝑦𝑗 = ቊ10

if process 1 is chosen for product 𝑗

if process 2 is not chosen for product 𝑗

𝑥𝑗 = number of units of product 𝑗 produced by the company

Other decisions variables:

𝑎𝑖𝑗 = per-unit time needed (in minutes) to produce product 𝑗 using process 𝑖

𝑏𝑖 = maximum amount of time the machine can be used if process 𝑖 is used


Constraints:

Either… or …

s. t.

𝑗=1

𝑛

𝑎1𝑗𝑥𝑗 ≤ 𝑏1 if process 1 is chosen for product 𝑗, i.e. if 𝑦𝑗 = 1

or

s. t.

𝑗=1

𝑛

𝑎2𝑗𝑥𝑗 ≤ 𝑏2 if process 2 is chosen for product 𝑗, i.e. if 𝑦𝑗 = 0

𝑥𝑗 ≥ 0, ∀𝑗 = 1,2, … , 𝑛

𝑦𝑗 ∈ {0,1}

s. t.

𝑗=1

𝑛

𝑎1𝑗𝑥𝑗 ≤ 𝑏1 +𝑀(1 − 𝑦𝑗)

𝑗=1

𝑛

𝑎2𝑗𝑥𝑗 ≤ 𝑏2 +𝑀𝑦𝑗


Constraints:

If… then …

𝑥𝑗 ≥ 0, ∀𝑗 = 1,2, … , 𝑛

𝑦𝑗 ∈ {0,1}

s. t.

𝑗=1

𝑛

𝑎1𝑗𝑥𝑗 ≤ 𝑏1 +𝑀(1 − 𝑦𝑗)

𝑗=1

𝑛

𝑎2𝑗𝑥𝑗 ≤ 𝑏2 +𝑀𝑦𝑗

o If process 1 is applied to product 1, it has to be applied to product 2

if 𝑦1 = 1, then 𝑦2 = 1

𝑦2 ≥ 𝑦1


Data:

𝑐𝑖𝑗 = per-unit cost to produce product 𝑗 using process 𝑖

Elimination of products of variables

Objective function:

min

𝑗=1

𝑛

(𝑐1𝑗𝑥𝑗 𝑦𝑗 + c2j𝑥𝑗(1 − 𝑦𝑗))

nonlinear

𝑗=1

𝑛

((𝑐1𝑗−𝑐2𝑗)𝑥𝑗 𝑦𝑗 + c2j𝑥𝑗)

min

𝑗=1

𝑛

((𝑐1𝑗−𝑐2𝑗)𝑥𝑗 𝑦𝑗 + c2j𝑥𝑗)


Elimination of products of variables

o In general, a product of two variables can be replaced by one variable,

o A number of new constraints is imposed on the new variable

continuous variables

binary variables

Product of a binary variable and a continuous variable

𝑧𝑗 ≤ 𝑢𝑦𝑗

𝑧𝑗 ≤ 𝑥𝑗

𝑧𝑗 ≥ 𝑥𝑗 − u(1 − yj)

𝑧𝑗 ≥ 0

0 ≤ 𝑥𝑗 ≤ 𝑢Upper bound on 𝑥𝑗

New variable 𝑧𝑗 𝑧𝑗= 𝑥𝑗 × 𝑦𝑗


Product of two binary variables

𝑥𝑗𝑦𝑦

binary variables binary variables

Product of two continuous variables

New variable 𝑧𝑗 𝑧𝑗= 𝑥𝑗 × 𝑦𝑗

𝑧𝑗 ≤ 𝑥𝑗

𝑧𝑗 ≤ 𝑦𝑗

𝑧𝑗 ≥ 𝑥𝑗 + 𝑦𝑗 − 1

𝑧𝑗 ∈ {0,1}

𝑥𝑗𝑦𝑦

continuous variables continuous variables

𝑧𝑗 =1

2(𝑥𝑗 + 𝑦𝑗)

𝑤𝑗 =1

2(𝑥𝑗 − 𝑦𝑗)

New variables 𝑧𝑗 and 𝑤𝑗 𝑧𝑗2−𝑤𝑗

2 = 𝑥𝑗 × 𝑦𝑗

Quadratic function


IP and MIP have extremely wide applications in practice

Knapsack problem

Which containers should be loaded in the vessel?

Knapsack problem

𝑤𝑖: weight if container 𝑖

𝑝𝑖: price of container 𝑖

o Data

If container 𝑖 is selected

Otherwise𝑥𝑖 = ቊ

10

o Decision variables

𝐦𝐚𝐱

𝑖

𝑝𝑖𝑥𝑖

o Objective function

𝑖

𝑤𝑖𝑥𝑖 ≤ 𝐶

o Constraints

𝐶: capacity of the vessel

Which boxes should be loaded in the container in order to maximize the revenue?

Related problems deal with the loading of containers in trains or aircrafts

Knapsack problems

Facility location problem

Where to locate inland container depots?

Inland container depots: storage area for empty shipping containers situated at

inland points away from sea ports


Where to locate inland container depots?

Inland container depots: storage area for empty shipping containers situated at

inland points away from sea ports

𝐼: set of 𝑚 alternative facility locations

J: set of 𝑛 customer zones

o Data

𝑐𝑖𝑗: per-unit cost for supplying customer zone 𝑗 by using the facility at 𝑖


𝑓𝑖: fixed cost of establishing a facility location 𝑖

𝑏𝑖 : capacity of facility location 𝑖

If a facility is established at location 𝑖

Otherwise𝑦𝑖 = ቊ

10


𝑥𝑖𝑗 = Units of customer zone 𝑗’s demand satisfied by facility at location 𝑖

𝑑𝑗: total demand for customer zone 𝑗


𝐦𝐚𝐱

𝑖=1

𝑚

𝑗=1

𝑛

𝑐𝑖𝑗𝑥𝑖𝑗 +

𝑖=1

𝑚

𝑓𝑖𝑦𝑖


𝑖=1

𝑚

𝑥𝑖𝑗 = 𝑑𝑗 , ∀𝑗 = 1,… , 𝑛

o Constraints

𝑗=1

𝑛

𝑥𝑖𝑗 ≤ 𝑏𝑖 , ∀𝑖 = 1,… ,𝑚

𝑗=1

𝑛

𝑥𝑖𝑗 ≤ (

𝑗=1

𝑛

𝑑𝑗) 𝑦𝑖 , ∀𝑖 = 1, … ,𝑚

𝑥𝑖𝑗 ≥ 0, ∀𝑖 = 1,2, … ,𝑚; ∀𝑗 = 1,2, … , 𝑛

𝑦𝑖 ∈ 0,1 , ∀𝑖 = 1,2, … ,𝑚

Assignment problem

Berth allocation problem

The berth allocation problem consists of assigning incoming ships to berthing

position

𝐵: set of 𝑚 berths

𝑉: set of 𝑛 vessels

o Data

ℎ𝑖𝑗: handling time of vessel 𝑖 at berth 𝑗

If a vessel 𝑗 is allocated to berth 𝑖

Otherwise𝑦𝑖𝑗 = ቊ

10


Assignment problem

𝐦𝒊𝒏

𝑖=1

𝑚

𝑗=1

𝑛

ℎ𝑖𝑗𝑦𝑖𝑗


o Constraints

𝑖=1

𝑚

𝑦𝑖𝑗 = 1, ∀𝑗 = 1,… , 𝑛

𝑗=1

𝑛

𝑦𝑖𝑗 = 1, ∀𝑖 = 1,… ,𝑚

𝑦𝑖𝑗 ∈ 0,1 , ∀𝑖 = 1,2, … ,𝑚

Assignment problem

Summary

Making Branch and Bound work well in practice requires lots of good ideas

Heuristics are often used to find good initial solutions

An idea for speeding up Branch and Bound is to add valid inequalities (next

lecture)

« Divide ut imperes »

Main references

• Murty, K. G. (2015). Intelligent Modeling Essential to Get Good Results: Container Storage Inside a

ContainerTerminal. In Case Studies in Operations Research (pp. 1-15). Springer NewYork.

• Bierlaire, M. (2015).Optimization: principles and algorithms. EPFL Press, Lausanne, Switzerland.

Documents

Decision Aid Methodologies In Transportation