MATH3902 Operations Research II Integer Programming pweb.hku.hk/~schu/IntegerProgramming.pdf · MATH3902 Operations Research II Integer Programming p.5 If it is known that 0 • xj

MATH3902 Operations Research II Integer Programming p.0

MATH3902 Operations Research II: Integer Programming Topics

1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Some examples of ILP and MILP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Example 1.1: Capital Budgeting Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Example 1.2: Dichotomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

Example 1.3: k-fold Alternatives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Example 1.4: If-Then Constraints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Example 1.5: Piecewise-Linear Continuous Function . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

Example 1.6: Fixed Charge Problem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.3 Some basic properties of ILP and MILP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .5

Example 1.7: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.4 Methods of Integer Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

1.5 Cutting plane methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Method of Integer Forms for ILP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

Example 1.8: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

Mixed Cut for MILP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

Example 1.9: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

1.6 Branch-and-Bound method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

Example 1.10: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

Branch-and-Bound Algorithm for ILP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

[Source of material: Integer Programming by R.S. Garfinkel and G.L. Nemhauser]

0


LN/MATH3902/TGY/CKC/MS/2004-05

Chapter 1 Integer Programming

(Source: Integer Programming by R.S. Garfinkel and G.L. Nemhauser)

1.1 Introduction

Integer programming is a branch of mathematical programming . A general mathe-

matical programming problem can be stated abstractly as

MP : max f(x) , x ∈ S ⊆ Rn

where the set S is called the constraint set and f is called the objective function. Every

x ∈ S is called a feasible solution to (MP). If there is an x∗ ∈ S satisfying

∞ > f(x∗) ≥ f(x) for all x ∈ S

then x∗ is called an optimal solution to (MP). Note that linear programming is simply a

special case of (MP) with f(x) being a linear function and S being the set of vectors x

satisfying a system of linear equations Ax = b and x ≥ 0.

An integer programming problem is a mathematical programming problem in which

S ⊆ Zn ⊆ Rn

where Zn is the set of all n-dimensional integer vectors (i.e. all the components of x are

restricted to integer values). A mixed integer programming problem is a mathematical

programming problem in which at least one, but not all, of the components of x ∈ S are

required to be integers.

In this chapter, we consider integer programming and mixed integer programming

problems which can be reduced to linear programming problems by dropping the integer

restrictions on the variables. Specifically, we shall discuss:

(1) Integer linear programming problem (ILP)

Max (or Min) z = c x

subject to Ax = b , x ≥ 0 integer .

(2) Mixed integer linear programming problem (MILP)

Max (or Min) z = c1x + c2v

subject to A1x + A2v = b

x ≥ 0 integer

v ≥ 0 .


The LP obtained by dropping the integrality constraints from the ILP or the MILP

will be referred to as the corresponding LP (or its LP relaxation).

1.2. Some examples of ILP and MILP

Let us first give an example of an ILP, to be followed by several examples of MILP:

Example 1.1: Capital Budgeting Problem

A firm has n projects that it would like to undertake but, because of budget limi-

tations, not all can be selected. In particular, project j has a present value of cj , and

requires an investment of aij dollars in the time period i, i = 1, 2, · · · ,m. The capital

available in time period i is bi. The problem of maximizing the total present value subject

to the budget constraints can be written as

Max z =n∑

j=1

cjxj

subject ton∑

j=1

aijxj ≤ bi , i = 1, 2, · · · ,m

xj = 0, 1 j = 1, 2, · · · , n

where xj = 1 if project j is selected.

There are many problems such as Example 1.1 above where the integrality constraint

arises quite naturally. Besides these obvious applications, we shall show that linear con-

straint with integer or binary variables can be used to model a variety of complex relations

that may be difficult to tackle otherwise.

Example 1.2: Dichotomy Consider the problem

Max z = f(x)

subject to g(x) ≥ 0 or h(x) ≥ 0 or both .

This is a mathematical programming problem where the (either-or) constraint is called

a dichotomy and is not easily handled by any standard algorithm. However, if it is known

that g and h have finite lower bounds g and h, then the dichotomy is equivalent to

g(x) ≥ δg

h(x) ≥ (1− δ)h

δ = 0, 1 .

If f, g and h are linear, the resulting problem is an MILP.


Example 1.3: k-fold Alternatives A generalization of Example 1.2 is to replace the

dichotomy constraint by the following condition:

At least k of the m constraints

gi(x) ≥ 0 i = 1, · · · ,m

must hold, where 1 ≤ k < m. The condition can be replaced by the constraint set:

gi(x) ≥ δigi, i = 1, · · · , m

m∑

i=1

δi ≤ m− k

δi = 0, 1 i = 1, · · · ,m

where each giis a known finite lower bound on gi(x).

Example 1.4: If-Then Constraint To transform the conditional constraint

g(x) > 0 ⇒ h(x) ≥ 0

into an equivalent set of constraints without the conditional relation, we can use the

following formulation:

g(x) ≤ M(1− δ)

h(x) ≥ −Mδ

δ = 0, 1

where M is arbitrarily large.

Example 1.5. Piecewise-Linear Continuous Function

Let f(t) be a piecewise-linear continuous function given by a set of line segments

determined by the points (s1, f1), · · · , (sp, fp). Figure 1.1 below illustrats the case of f(t)

being a piecewise-linear approximation for a non-linear curve, with p = 5.

��H

HH

H��H

HH

HH

uj

xj

fj(xj)

(s1j , f1j)

(s2j , f2j )

(s3j , f3j )

(s4j , f4j)

(s5j , f5j)

Figure 1.1


The function f(t), 0 ≤ t ≤ u, can be represented as follows:

f(t) = Minp∑

k=1

λkfk

subject to t =p∑

k=1

λksk

p∑

k=1

λk = 1

λk ≥ 0 k = 1, · · · , p .

Furthermore, no more than two of the λk can be positive and these must be of the form

λk, λk+1. This condition can be achieved by the addition of adjacency constraints:

λ1 ≤ δ1

λk ≤ δk−1 + δk , k = 2, · · · , p− 1

λp ≤ δp−1

p−1∑

k=1

δk = 1

δk = 0, 1 k = 1, · · · , p− 1 .

Remark: The above representation of f(t) is called a mixed-integer minimization model

(MIMM) for the function f(t), and is used to convert f(t), a nonlinear function, into a

more manageable linear model. Consider the following problem:

Minn∑

j=1

fj(xj)

subject to Ax = b , x ≥ 0 ,

where each fj(xj) is a piecewise linear continuous function. Then using the above MIMM

representation for each fj(xj), this nonlinear problem can be converted into a MILP.

Example 1.6: Fixed Charge Problem

In a typical production planning problem involving N products, the production cost

for product j may consist of a fixed cost dj independent of the amount produced and a

variable cost cj per unit. Thus if xj is the production level of product j, its production

cost function may be written as

fj(xj) ={

dj + cjxj , xj > 00 xj = 0 .

This is nonlinear in xj because of the discontinuity of fj(xj) at the origin. Consequently,

the following minimum cost problem is also nonlinear:

Min z =N∑

j=1

fj(xj)

subject to Ax = b , x ≥ 0 .


If it is known that 0 ≤ xj ≤ uj and dj > 0 then the nonlinear function fj(xj) can be

transformed into a linear minimization model as follows:

fj(xj) = Min cjxj + djyj

subject to xj − ujyj ≤ 0

yj = 0, 1

It means that for a given value of xj , the value fj(xj) is equal to the optimal value of

the ILP and therefore can be represented by the ILP. Rewriting all the fj(xj) in terms of

their MIMM in the minimum cost problem, we get

Min z =N∑

j=1

(cjxj + djyj)

subject to 0 ≤ xj ≤ ujyj , j = 1, · · · , N

yj = 0, 1 j = 1, · · · , N

Ax = b .

1.3. Some basic properties of ILP and MILP

Before explaining how methods of IP work in general, we make the following elemen-

tary but important observation: If we solve the LP relaxation of a pure IP and obtain a

solution in which all variables are integers, then the optimal solution to the LP relaxation

is also an optimal solution to the IP.

To see why this observation is true, consider the following IP:

Max z = 3x1 + 2x2

subject to 2x1 + x2 ≤ 6x1, x2 ≥ 0 integer

The optimal solution to the LP relaxation of this pure IP is x1 = 0, x2 = 6, z = 12. Because

this solution gives integer values to all variables, the preceding observation implies that

it is also the optimal solution to the IP. Observe that the feasible region for the IP is a

subset of the points in the LP relaxation’s feasible region. Thus, the optimal z-value for

the IP cannot be larger than the optimal z-value (= 12) for the LP relaxation. Thus the

point x1 = 0, x2 = 6, with z = 12, must be optimal for the IP.

From the computational standpoint, ILP problems are much more difficult than LP

problems. One convenient way to solve an ILP is to solve the corresponding LP and then

round the solution to the closest feasible integers. However, this approach is obviously


inadequate in many cases. For instance, in the Capital Budgeting problem, the binary

variable xj is used to denote whether a project is accepted or rejected. In this case, it

is nonsensical to deal with fractional values of xj , and the use of rounding is logically

unacceptable. Also consider the following example.

Example 1.7:

Max z = 2x1 + x2

subject to x1 + x2 ≤ 5−x1 + x2 ≤ 06x1 + 2x2 ≤ 21x1, x2 ≥ 0 integer

Figure 1.2

From the above figure, it is clear that there are eight feasible solutions to the ILP and that

the optimal solution is x1 = 3, x2 = 1. However, the optimal solution to the corresponding

LP is x1 = 11/4, x2 = 9/4. In this case, rounding the optimal LP solution does not give

us the optimal integer solution.

1.4. Methods of Integer Programming

Even though a bounded ILP has only a finite number of feasible solution, the integer

nature of the variables makes it difficult to devise an effective algorithm that searches

directly among the feasible integer points of the solution space. In view of this difficulty,

researchers have developed a solution procedure that is based on exploiting the tremendous

success in solving LP problems. The strategy for this procedure can be summarized in

three steps:


(1) Relax the integer constraints of the ILP so that the ILP is converted into a regular

LP.

(2) Solve the resulting “relaxed” LP model and identify its (continuous) optimum point.

(3) Starting from the continuous optimum point, add special constraints that will itera-

tively force the optimum extreme point of the resulting LP model toward the desired

integer restrictions.

There are two widely used methods for generating the special constraints that will

force the optimum point of the relaxed LP problem toward the desired integer solution:

1. Cutting plane.

2. Branch-and-bound.

In both methods, the added constraint will eliminate portions of the relaxed solution

space, but never any of the feasible integer points. Neither of the two methods can be

claimed to be uniformly more effective in solving ILP’s. Nevertheless, branch-and-bound

methods are far more successful computationally than are the cutting-plane methods.

For this reason most commercial codes are based on the use of the branch-and-bound

procedure.

1.5. Cutting plane methods

The basic idea of the cutting plane method is to cut off parts of the feasible region of

the corresponding LP, so that the optimal integer solution becomes an extreme point and

therefore can be found by the simplex algorithm. For instance, if we add two additional

constraints (cutting planes) to Figure 1.2 of Example 1.7 as in the following figure.

Figure 1.3

Then the optimal integer solution (3, 1) can be found by solving the corresponding LP.

One of the methods to generate these cuts is as follows.


Method of Integer Forms for ILP

We assume all problems are comprised of rational data (notice that all numbers stored in

digital computers are rational) and since rational data can be transformed into integral

data by multiplying by a suitable constant, we shall only consider problems comprised of

integral data.

Let an ILP be given. Assume that a non-integer optimal solution x∗ of the corre-

sponding LP has been found, and that the optimal tableau is written as

· · ·xj · · · b

z z∗

xr · · · yrj · · · x∗r

where {xr : r ∈ B} are the basic variables and {xj : j ∈ N} are the non-basic variables.

Suppose that x∗ has a non-integer component x∗r . Then for any feasible integer solution

x, we have

xr +∑

j∈N

yrjxj = x∗r . (1)

Since x ≥ 0,

xr +∑

j∈N

[yrj ]xj ≤ x∗r (2)

where [u] = largest integer ≤ u. Since x has integer components, the L.H.S. of (2) must

be an integer. Thus

xr +∑

j∈N

[yrj ]xj ≤ [x∗r ] . (3)

Subtracting (3) from (1), we get∑

j∈N

(yrj − [yrj ])xj ≥ x∗r − [x∗r ] .

Letting yrj = [yrj ] + frj and x∗r = [x∗r ] + fr, we obtain (a Gomory cut)∑

j∈N

frjxj ≥ fr (4)

or

S = −fr +∑

j∈N

frjxj , S ≥ 0 . (5)

Further S must also be integral since

xr = −(− fr +

∑

j∈N

frjxj

)+

([x∗r ]−

∑

j∈N

[yrj ]xj

)

and both xr and the expression inside the second parenthesis on the R.H.S. are integral.

An important property of (5) is that fr > 0. Thus (5) can be used as a constraint

(cutting plane) to exclude the optimal soluton x∗ of the corresponding LP having fr > 0.

Furthermore (5) does not exclude any feasible integer solution.


Example 1.8: Consider Example 1.7. Rewrite the problem in standard form:

Max z = 2x + x2

subject to x1 + x2 + x3 = 5−x1 + x2 + x4 = 06x1 + 2x2 + x5 = 21

x1, · · · , x5 ≥ 0 integer

and solve the corresponding LP. This yields the following optimal tableau

x3 x5 b

z 1/2 1/4 31/4

x1 −1/2 1/4 11/4x2 3/2 −1/4 9/4x4 −2 1/2 1/2

Table 1

Using (4) to generate a cut yields

x3/2 + x5/4 ≥ 3/4 (x1-row)

x3/2 + 3x5/4 ≥ 1/4 (x2-row)

x5/2 ≥ 1/2 (x4-row)

Each row could serve as the source of a cut, or simply as a source row . Suppose we add

the x4-row cut, or

−x5/2 + S1 = −1/2 , S1 ≥ 0

to Table 1. This yields Table 2 with S1 = −1/2 as a new basic variable in the dual feasible

solution.

x3 x5 b

z 1/2 1/4 31/4

x1 −1/2 1/4 11/4x2 3/2 −1/4 9/4

Source x4 −2 1/2 1/2Cut S1 0 −1/2 −1/2

Table 2

Whenever additional constraint is added, dual feasibility but not primal feasibility is re-

tained. Thus it is convenient to reoptimize using the dual simplex algorithm. Following

the dual simplex algorithm, S1 leaves and x5 enters the basis. After pivoting we obtain


Table 3 (excluding the S2 row)

x3 S1 b

z 1/2 1/2 15/2

Source x1 −1/2 1/2 5/2x2 3/2 −1/2 5/2x4 −2 1 0x5 0 −2 1

Cut S2 −1/2 −1/2 −1/2

Table 3

Here we can choose either x1 or x2 row as a source row. Arbitrarily selecting the x1 row,

we obtain

x3/2 + S1/2 ≥ 1/2 .

Thus we add the constraint

−x3/2− S1/2 + S2 = −1/2

to Table 3 and reoptimize. The departing variable is S2. Either x3 or S1 can enter, and

we choose x3. After pivoting, we obtain Table 4.

S2 S1 b

z 1 0 7

x1 −1 1 3x2 3 −2 1x4 −4 3 2x5 0 −2 1x3 −2 1 1

Table 4

The solution is primal and dual feasible, and integral; therefore it is optimal to the ILP.

Remarks:

(1) If the source rows are chosen according to a scheme developed by Gomory, then it

can be proved that the algorithm must terminate after a number of iterations.

(2) The methods of integer forms has two disadvantages

(i) When this algorithm is implemented on digital computer, round-off errors may

cause serious problem, especially in distinguishing integers from non-integers.

(ii) The solution of the problem remains infeasible until the optimal integer solution

is reached. This means that there will be no “good” integer solution on hand, if

the calculations are stopped prematurely owing to budget or time limitations.


(3) To overcome the difficulties stated above, various cutting plane methods, such as dual

all-integer algorithm, primal all-integer algorithm · · · etc. have been developed. But

each algorithm has its own disadvantages.

(4) For all cutting plane methods, there is no upper bound on the number of cuts required.

In practice, very often the number of cuts required is so large that the problem

becomes insolvable computationally.

Mixed Cut for MILP

Let xr be an integer variable of the MILP. Again, as in the pure integer case, consider

the xr-equation in the optimal continuous solution. This is given by

xr +∑

j∈N

yrjxj = x∗r (Source row)

and xr − [x∗r ] = fr −∑

j∈N

yrjxj .

Because some of the nonbasic variables may not be restricted to integer values in this case,

it is incorrect to use the cut developed in the preceding section. But a new cut can be

devised based on the same general idea.

For xr to be integer, either xr ≤ [x∗r ] or xr ≥ [x∗r ] + 1 must be satisfied. From the

source row, these conditions are equivalent to:

∑

j∈N

yrjxj ≥ fr (6)

∑

j∈N

yrjxj ≤ fr − 1 . (7)

Let J+ = set of subscripts j in N for which yrj ≥ 0

J− = set of subscripts j in N for which yrj < 0.

Then from (6) and (7) one gets

∑

j∈J+

yrjxj ≥ fr (8)

( fr

fr − 1

) ∑

j∈J−yrjxj ≥ fr . (9)

Since (6) or (7), and hence (8) or (9), must hold, and the left hand sides of both (8) and

(9) are non-negative, it follows that (8) and (9) can be combined into one constraint of

the form

S −

∑

j∈J+

yrjxj +( fr

fr − 1

) ∑

j∈J−yrjxj

= −fr (Mixed cut)


where S ≥ 0 is a nonnegative slack variable. The last equation is the required mixed cut

and it represents a necessary condition for xr to be integer. When this cut is added to the

optimal tableau, the tableau becomes primal infeasible but retains dual feasibility. The

dual simplex method is used to clear the infeasibility.

Example 1.9: Consider Example 1.8. Suppose that only x1 is restricted to integer

values. From the x1-row in Table 1, the mixed cut is

S1 −{

14x5 +

( 34

34 − 1

)(− 1

2

)x3

}= −3

4,

or S1 − 14x5 − 3

2x3 = −3

4.

Adding this to the tableau in Table 1 gives

x3 x5 b

z 1/2 1/4 31/4

x1 −1/2 1/4 11/4x2 3/2 −1/4 9/4x4 −2 1/2 1/2S1 −3/2 −1/4 −3/4

Table 5

After pivoting, we obtain

S1 x5 b

z 1/3 1/6 15/2

x1 −1/3 1/3 3x2 1 −1/2 3/2x4 −4/3 5/6 3/2x3 −2/3 1/6 1/2

Table 6

The solution is primal and dual feasible, and x1 is integral; therefore it is optimal to the

MILP.

1.6 Branch-and-Bound Method

Consider the ILP: max z = c · x subject to x ∈ S0 where S0 = {x|Ax = b, x ≥0 integer}. The general idea of the branch-and-bound method is first to solve the problem

as a continuous model, i.e. solve its corresponding LP:

max z = c · x subject to x ∈ T0 = {x|Ax = b, x ≥ 0} .


Suppose xr is an integer-constrained variable whose optimum continuous value x∗r is frac-

tional. The range [x∗r ] < xr < [x∗r ] + 1 cannot include any feasible integer solution.

Consequently, a feasible integer value of xr must satisfy one of the two conditions, namely

xr ≤ [x∗r ] or xr ≥ [x∗r ] + 1

These two conditions when applied to the continuous model result in two mutually exclu-

sive problems with constraint sets:

(i) T1 = {x|Ax = b, xr ≤ [x∗r ], x ≥ 0}(ii) T2 = {x|Ax = b, xr ≥ [x∗r ] + 1, x ≥ 0}.And when the integrality constraints are also included, the sets

S1 = {x|Ax = b, xr ≤ [x∗r ], x ≥ 0 integer} and

S2 = {x|Ax = b, xr ≥ [x∗r ] + 1, x ≥ 0 integer}

actually form a separation of S0, i.e. S1 ∪ S2 = S0, S1 ∩ S2 = φ. The optimal solution x∗

of the given problem must lie either in S1 or S2 and must also be the optimal solution of

one of the subproblems:

(i) max z = c · x subject to x ∈ S1

(ii) max z = c · x subject to x ∈ S2.

These subproblems can again be solved by repeating the above process of relaxing the

integrality constraint and branching when the optimal solution has a component with

fractional value. This branching process will generate a rooted tree, with each node k of

the tree corresponding to a subproblem: max z = c · x subject to x ∈ Sk. If the optimal

solution of its corresponding LP is feasible with respect to the integrality constraints

(i.e. has integer components), it will be recorded and its objective value will constitute a

lower bound for the optimum. In this case it will be unnecessary to further ‘branch’ this

subproblem and the node is said to have ‘fathomed’. Nodes not yet fathomed are called

‘live’. At any node the optimum value (or the greatest integer less than or equal to the

optimum value if the objective function has integer coefficients) zk of its corresponding

LP is an upper bound for the optimum value of all its children. If the upper bound is less

than the best lower bound available, then it is also unnecessary to branch this subproblem.

The process of branch and bound continues until each subproblem terminates either with

an integer solution or its upper bound is less than the current lower bound. In this case

the best feasible solution at hand, if any, is the optimum.

Example 1.10:

max z = −7x1 − 3x2 − 4x3

subject to x1 + 2x2 + 3x3 − x4 = 83x1 + x2 + x3 − x5 = 5

x1, x2, · · · , x5 ≥ 0 integer


We begin by solving the corresponding LP. The optimal solution is x1 = 25 , x2 = 19

5 ,

x3 = x4 = x5 = 0 with z0 = [− 715 ] = −15.

We must now partition S0 based on x1 or x2 and we arbitrarily choose x2. The

resulting tree is shown in Figure 1.4.

��

��

��

��

@@

@@

@@

@

��

0

1 2

z = −15

z = −∞

z = −15 z = −15

x2 ≤ 3 x2 ≥ 4

Figure 1.4

The optimal solution of the LP at node 1 is x1 = 12 , x2 = 3, x3 = 1

2 , x4 = x5 = 0 with

z1 = [− 292 ] = −15. There is no change in the bounds. We choose to branch node 1 using

x1 as in Figure 1.5.

��

��

��

��

@@

@@

@@

@

��

0

1 2

��

��

3 4

��

��

��

�

@@

@@

@@

@

z = −15

z = −∞

z = −15

z = −15 z = −15

x2 ≤ 3

x1 ≤ 0 x1 ≥ 1

x2 ≥ 4

Figure 1.5

Taking the left branch and solving the LP, we obtain an all-integer optimal solution:

x1 = 0, x2 = 3, x3 = 2, x4 = 4, x5 = 0 with z3 = −17. Thus the new lower bound

z0 = max{−∞, z3} = −17. Also node 3 is fathomed and we backtrack to node 4. It

is found that z4 = −18 < z0 and node 4 is fathomed. We now backtrack to node 2.

The optimal solution of its LP is x1 = 13 , x2 = 4, x3 = 0, x4 = 1/3, x5 = 0 with

z2 = [− 433 ] = −15. We choose to branch on x1 and obtain Figure 1.6.

��

��

��

��

@@

@@

@@

@

��

0

1 2

��

��

��

��

3 4 5 6

��

��

��

�

@@

@@

@@

@

z = −15

z = −17

z = −17 z = −18 z = −15 z = −15

x2 ≤ 3

x1 ≤ 0 x1 ≤ 0 x1 ≥ 1x1 ≥ 1

x2 ≥ 4

Figure 1.6


At node 5, the optimal solution of its LP is all-integer and z5 = −15. Thus z0 =

max{z0, z5} = −15 and nodes 5 and 6 are fathomed. An optimal solution is given by

the solution at node 5: x1 = 0, x2 = 5, x3 = 0, x4 = 2, x5 = 0 and z = −15.

Branch-and-Bound Algorithm for ILP (assuming that an upper bound uj is known for

each variable xj and the problem is a maximization problem.)

Step 1: (Initialization) Begin with one live node 0, where z0 = ∞, z0 = −∞. Go to

Step 2.

Step 2: (Branching) If no live nodes exist, go to Step 7; otherwise select a live node k

(usually according to the backtracking algorithm). If the corresponding LP has

been solved, go to Step 3; otherwise go to Step 4.

Step 3: (Separation) Choose a variable with fractional value, i.e. xr = [x∗r ] + fr, 0 <

fr < 1. Then a separation of the constraint set Sk at node k is constructed as

follow:

Sk ∩ {x|xr ≤ [x∗r ]} and

Sk ∩ {x|xr ≥ [x∗r ] + 1} .

Go to Step 2.

Step 4: (Solving the LP) Solve the corresponding LP. If the LP has no feasible solution,

node k is fathomed and we go to Step 2. If the LP has an optimal solution x∗,

let zk = [z∗] and go to Step 5. (z∗ is the value of the objective function at x∗.)

Step 5: (Fathoming by integrality) If x∗ is not all-integer, go to Step 6. If x∗ is integer,

let z0 = max{z0, z∗}. Go to Step 6.

Step 6: (Fathoming by bounds) Any node k such that zk ≤ z0 is fathomed. Go to Step

2.

Step 7: (Termination) Terminate. If z0 = −∞ there is no feasible solution. If z0 > −∞,

that feasible solution which yielded z0 is optimal.

Remark:

(1) If each variable xj is bounded above, then it can be easily proved that the above

algorithm must terminate after a finite number of steps.

(2) The efficiency of the above algorithm depends directly on

(i) the way that the different subproblems are generated, i.e. which variable with

fractional value is used to generate a separation of the constraint set.

(ii) the way that the nodes of the tree are scanned. In the above example, we have

used the backtracking algorithm. But, scanning the nodes by some other rules

may be faster sometimes.


Unfortunately, there is no definite “best” way for handling this problem, only

empirical rules exist which do enhance the process. These rules are usually

implemented in most of the commercial branch-and-bound codes.

(3) The corresponding LP can be solved by either the primal or dual simplex procedure.

For Example 1.10, the dual approach is readily applicable when the LP is rewritten

as:

max z = −7x1 − 3x2 − 4x3

subject to −x1 − 2x2 − 3x3 ≤ −8−3x1 − x2 − x3 ≤ −5

x1, x2, x3 ≥ 0 integer

(4) After obtaining an optimal solution of the corresponding LP at a node, the addition

of a constraint xr ≤ [x∗r ] or xr ≥ [x∗r ] + 1 will destroy the primal feasibility of the

solution. But since dual feasibility is retained we can reoptimize using the dual

simplex algorithm. For example, at node 0 of Example 1.10, an optimal LP tableau

is obtained as in the following table:

x3 x4 x5 b

z 3/5 2/5 11/5 −71/5

x1 −1/5 1/5 −2/5 2/5x2 8/5 −3/5 1/5 19/5

Table 7

When the constraint x2 ≤ 3 is added, we can rewrite it as S = 3 − x2 = 3 − ( 195 −

85x3 − (− 3

5 )x4 − 15x5), or S − 8

5x3 + 35x4 − 1

5x5 = − 45 , and add it to the tableau:

x3 x4 x5 b

z 3/5 2/5 11/5 −71/5

x1 −1/5 1/5 −2/5 2/5x2 8/5 −3/5 1/5 19/5S −8/5 3/5 −1/5 −4/5

Table 8

Using the dual simplex iteration once we get

S x4 x5 b

z 3/8 5/8 17/8 −29/2

x1 −1/8 1/8 −3/8 1/2x2 1 0 0 3x3 −5/8 −3/8 1/8 1/2

Table 9


which is both optimal and primal feasible. Hence we obtain the (non-integer) optimal

solution x1 = 12 , x2 = 3, x3 = 1

2 , x4 = x5 = 0 at node 1.

(5) As stated before, branch-and-bound methods are empirically observed to be far more

successful computationally than are the cutting-plane methods. Unfortunately, an

unsuspecting comparison could easily lead to a naive opposite conclusion incorrectly.

For example, consider adding a Gomory cut generated from the x2-row

−3x3/5− 2x4/5− x5/5 + x6 = −4/5

to Table 7 and reoptimize. The departing variable is x6. Either x3 or x4 can enter,

and suppose we choose x4. After one pivot, we obtain Table 10. It yields the same

optimal integer solution given at node 5 of the branch-and-bound tree in Example

1.10.

x3 x6 x5 b

z 0 1 2 −15

x1 −1/2 1/2 −1/2 0x2 5/2 −3/2 1/2 5x4 3/2 −5/2 1/2 2

Table 10

However, had we chosen x3 as the entering variable instead, we would get an optimal

(continuous) solution with all basic variables taking on fractional values as in the

following table:

x6 x4 x5 b

z 1 0 2 −15

x1 −1/3 1/3 −1/3 2/3x2 8/3 −5/3 −1/3 5/3x3 −5/3 −2/3 1/3 4/3

Table 11

(6) The branch and bound algorithm can also be applied to MILP. For MILP, only those

variables with integrality constraint are examined and if any of these variables takes

fractional value, further separation will be performed. For example, if in Example

1.10, only x2 is required to be integer, then the branch-and-bound tree will consist

of three nodes 0, 1 and 2, as given by Figure 1.4. (The bounds are z0 = −71/5, z1 =

−29/2 and z2 = −43/3 instead.) Both nodes 1 and 2 are fathomed by integrality,

with the optimal MILP solution given by the solution at node 2.

Documents

MATH3902 Operations Research II Integer Programming pweb.hku.hk/~schu/IntegerProgramming.pdf · MATH3902 Operations Research II Integer Programming p.5 If it is known that 0 • xj