Embed Size (px)
Citation preview
Separation Algorithms for 0-1 Knapsack Polytopes
Konstantinos Kaparis∗ Adam N. Letchford†
February 2008, Corrected May 2008
Abstract
Valid inequalities for 0-1 knapsack polytopes often prove usefulwhen tackling hard 0-1 Linear Programming problems. To use suchinequalities effectively, one needs separation algorithms for them, i.e.,routines for detecting when they are violated. We show that the sepa-ration problems for the so-called extended cover and weight inequalitiescan be solved exactly in O(nb) time and O((n+ amax)b) time, respec-tively, where n is the number of items, b is the knapsack capacity andamax is the largest item weight. We also present fast and effective sep-aration heuristics for the extended cover and lifted cover inequalities.Finally, we present a new exact separation algorithm for the 0-1 knap-sack polytope itself, which is faster than existing methods. Extensivecomputational results are also given.
Keywords: integer programming – branch-and-cut – knapsack prob-lems
1 Introduction
A 0-1 knapsack problem (KP) is a problem of the form
max{cTx : aTx ≤ b, x ∈ {0, 1}n
},
where c ∈ Zn+, a ∈ Zn
+ and b ∈ Z+. It is well-known that the KP is NP -hard,but can be solved in O(nb) time by dynamic programming.
A 0-1 knapsack polytope is the convex hull of feasible solutions to a KP,i.e., a polytope (bounded polyhedron) of the form:
conv{x ∈ {0, 1}n : aTx ≤ b
}.
Such polytopes have been widely studied and many classes of valid inequal-ities are known, such as the lifted cover inequalities (LCIs) of Balas [3] andWolsey [31], and the weight inequalities (WIs) of Weismantel [29].∗University of Southampton†Lancaster University
1
One motive for studying these polytopes, as pointed out by Crowder etal. [11], is that valid inequalities for them can be used as cutting planes forgeneral 0-1 Linear Programs (0-1 LPs). The idea is to consider each indi-vidual constraint of a 0-1 LP as a 0-1 knapsack constraint (complementingvariables if necessary to obtain non-negative coefficients) and derive cuttingplanes from the associated polytope.
The separation problem for 0-1 knapsack polytopes is this: given n, a,b and a vector x∗ ∈ [0, 1]n, either find an inequality that is valid for theknapsack polytope and violated by x∗, or prove that none exists. One candefine the separation problem for a given class of inequalities (such as LCIsor WIs) analogously. Separation algorithms, whether exact or heuristic, areneeded if one wishes to use valid inequalities as cutting planes (see, e.g.,Grotschel et al. [15] or Nemhauser & Wolsey [24]).
Crowder et al. [11] and Gu et al. [16] proposed some simple but effec-tive heuristics for LCI separation, and used them to obtain good resultsfor sparse, unstructured 0-1 LPs. LCIs are now incorporated into severalsoftware packages for integer programming. WI separation, on the otherhand, has received less attention, although there exists a pseudo-polynomialtime exact algorithm (Weismantel [29]) and a simple heuristic (Helmberg &Weismantel [20]).
In some cases, one can get better results by using general valid inequal-ities for knapsack polytopes as cutting planes. The polynomial equivalenceof separation and optimisation (Grotschel et al. [15]) implies that the sepa-ration problem for knapsack polytopes can be solved in pseudo-polynomialtime. Algorithms for this problem have been proposed by Boyd [7, 8, 9] andBoccia [6]. (For extensions to the mixed-integer case, see Yan & Boyd [33]and Fukasawa & Goycoolea [13].)
In this paper, we present and test several new exact and heuristic sepa-ration algorithms. The structure of the paper is as follows:
• In Section 2, we briefly review the literature on 0-1 knapsack polytopes,separation algorithms, and their application to general 0-1 LPs.
• Section 3 is concerned with LCI separation. We consider first theso-called extended cover inequalities (ECIs), a special case of LCIs in-troduced by Balas [3]. We present a fairly simple exact algorithm thatruns in O(n2b) time, and then a more complex exact O(nb) algorithm.We also present a new and effective heuristic. Finally, we show how allthree of these ECI algorithms can be easily modified to yield heuristicsfor general LCIs.
• Section 4 is concerned with WI separation. We first present an en-hanced version of Weismantel’s exact algorithm, which runs inO(nbamax)time, where amax is the largest item weight. We then present a morecomplex exact algorithm that runs in O((n+ amax)b) time.
2
• In Section 5, we present an enhanced version of Boccia’s exact sepa-ration algorithm.
• In Section 6, we give extensive computational results, conducted ontwo sets of benchmark instances: 12 sparse unstructured 0-1 LPs takenfrom MIPLIB [1], and 180 instances of the 0-1 Multi-DimensionalKnapsack Problem, created by Chu & Beasley [10].
• Finally, concluding remarks are made in Section 7.
Throughout the paper, we use the notation N = {1, . . . , n} and amax =maxj∈N aj .
2 Literature Review
2.1 Lifted cover inequalities
Consider a 0-1 knapsack polytope of the form
Q := conv{x ∈ {0, 1}n : aTx ≤ b
}.
A set C ⊆ N is called a cover if it satisfies∑
j∈C aj > b. Given any coverC, the cover inequality (CI)
∑j∈C xj ≤ |C| − 1 is clearly valid for Q. The
strongest CIs are obtained when the cover C is minimal, in the sense thatno proper subset of C is also a cover.
Given a minimal cover C, there exists at least one inequality of the form∑j∈C
xj +∑
j∈N\C
αjxj ≤ |C| − 1 (1)
that induces a facet of Q. Computing the coefficients αj is called lifting andthe inequalities are called lifted cover inequalities (LCIs). The LCIs werediscovered independently by Balas [3] and Wolsey [31].
Lifting is usually done sequentially, i.e., one variable at a time (thoughnot always — see, e.g., Gu et al. [18]). At first sight, it seems that one mustsolve a 0-1 KP just to compute a single lifting coefficient. However, Zemel[34] showed that, given a fixed cover C and a fixed lifting sequence, one cancompute all lifting coefficients in O(n|C|) time. Zemel’s algorithm can bemade even faster using results of Balas & Zemel [4] and Gu et al. [18], whichenable one to determine most of the lifting coefficients in O(n+ |C| log |C|)time.
As pointed out by Balas [3], a fast but naive way of lifting a CI is asfollows. We compute a∗ := maxj∈C aj and define the extension of C asE(C) := C ∪ {j ∈ N \ C : aj ≥ a∗}. Then the extended cover inequality(ECI)
∑j∈E(C) xj ≤ |C| − 1 is valid for Q, though it is not guaranteed to
be facet-inducing.
3
Nemhauser & Wolsey [24] noted that one can derive more general LCIsof the form:∑
j∈C\D
xj +∑
j∈N\C
αjxj +∑j∈D
βjxj ≤ |C \D|+∑j∈D
βj − 1,
where C is a cover and D ⊂ C. (This follows from a more general result ofWolsey [32].) We will follow Gu et al. [16] in referring to the computationof the α and β coefficients as up-lifting and down-lifting, respectively, andin referring to the inequalities (1) as ‘simple’ LCIs. The general LCIs in-clude, not only the simple LCIs, but also the so-called ‘(1, k)-configuration’inequalities of Padberg [25] as special cases.
Gu et al. [18] remark that Zemel’s algorithm can be adapted to computeall down-lifting coefficients in O(|C|n3) time. Although this is polynomial,it is time-consuming. To save time, one can instead compute approximatedown-lifting coefficients by solving a sequence of |D| continuous 0-1 KPs.This idea, of solving LPs to compute approximate lifting coefficients, isalso due to Wolsey [32]. We have used it ourselves in the past (Kaparis &Letchford [22]), and it works well in practice.
2.2 Separation algorithms for LCIs
Several exact and heuristic separation techniques have been suggested forthe LCIs and their special cases.
Crowder et al. [11] noted that the CIs can be written in the form∑
j∈C(1−xj) ≥ 1. This enabled them to formulate the separation problem for CIsitself as a 0-1 KP. Let x∗ ∈ [0, 1]n be the fractional solution to be separated.Define for each j ∈ N the 0-1 variable yj , taking the value 1 if and only if jis in the cover, and solve the following 0-1 KP:
min∑
j∈N (1− x∗j )yj (2)s.t.
∑j∈N ajyj ≥ b+ 1 (3)yj ∈ {0, 1} (j ∈ N). (4)
To save time, Crowder et al. solved this 0-1 KP heuristically, simply insertingitems into C in non-decreasing order of (1−x∗j )/aj . They lifted any violatedCI found, to yield a violated simple LCI.
Later, it was proven that the separation problems for CIs, LCIs andsimple LCIs are all NP -hard (Ferreira et al. [12], Gu et al. [17], Klabjan etal. [23]). The same seems likely for ECIs.
Gu et al. [16] suggested an alternative greedy heuristic for building thecover: simply insert items into C in non-increasing order of x∗j . This heuristictends to put items of smaller weight into C. As a result, more items in N \Ctend to be lifted, and the resulting LCI tends to be violated by more. Guet al. also recommend using general LCIs rather than simple LCIs. They
4
obtained their best computational results by setting D = {j ∈ C : x∗j = 1},and using the following lifting sequence: first up-lift the variables with x∗j >0, then down-lift the variables in D, then up-lift the variables with x∗j = 0.
In Kaparis & Letchford [22], we proposed some simple refinements to thelifting scheme of Gu et al. [16]. The most important of these is the following.If there exists a k ∈ N \ C such that x∗k > 0 and ak > b −
∑j∈D aj , then
the up-lifting coefficient of xk will be undefined. To resolve this problem,we remove one or more items from D if necessary.
Finally, we mention the paper by Gabrel & Minoux [14], which gives anexact separation algorithm for ECIs, based on the solution of at least n 0-1KPs. Since our own algorithms (Subsections 3.1 and 3.2) are both simplerand faster, we do not go into details.
2.3 Weight inequalities
The weight inequalities (WIs for short) were introduced by Weismantel [29].Given any set of items W ⊂ N satisfying
∑j∈W aj < b, we define the
residual capacity r = b−∑
j∈W aj . The WI then takes the following form:∑j∈W
ajxj +∑
j∈N\W
max{0, aj − r}xj ≤∑j∈W
aj = b− r. (5)
In general, the WIs do not induce facets of Q, although they do under somerestrictive conditions [29]. Atamturk [2] presented a procedure, based on su-peradditive functions, for strengthening the WIs under certain conditions.Weismantel [29] also defined a related class of inequalities called extendedweight inequalities. For the sake of brevity, we do not explain these devel-opments in detail.
To our knowledge, only two papers have considered the separation prob-lem for WIs. Weismantel himself noted that the problem can be solvedexactly in pseudo-polynomial time. The idea is to write the WI (5) in thefollowing form:∑
j∈W :aj≤r
ajxj +∑
j∈W :aj>r
rxj ≤ b− r −∑
j∈N :aj>r
(aj − r)xj . (6)
Now the right-hand side of (6) depends on r, but not on W . Thus, for fixedr, the separation problem amounts to finding the set W that maximises theleft-hand side of (6). This can be formulated as an equality-constrained 0-1KP, which can be solved in pseudo-polynomial time. Solving one equality-constrained 0-1 KP for each possible value of r yields the desired pseudo-polynomial separation algorithm for WIs.
The other paper dealing with WI separation is Helmberg & Weismantel[20]. They present a greedy separation heuristic, that simply inserts items
5
into W in non-increasing order of x∗ value, until no more items can beinserted.
To our knowledge, it has not been shown that WI separation is NP -hard.
2.4 Separation algorithms for 0-1 knapsack polytopes
We now turn our attention to the separation problem for Q itself. Four mainapproaches have been proposed:
• Boyd [7] formulated the problem as an LP with O(nb) variables andconstraints. The formulation is based on the classical dynamic pro-gramming representation of 0-1 KP solutions as paths in a networkwith O(nb) nodes and arcs.
• Boyd [9] proposed a subgradient method for finding so-called Fenchelcuts, which are valid inequalities that maximise the distance (accordingto some desired norm) from the point to be separated. Fenchel cutsin general are not facet-inducing.
• Boyd [8] also proposed a primal simplex approach to generate Fenchelcuts, which seemed to perform better than the subgradient method.
• Using the idea of polarity (e.g., Nemhauser & Wolsey [24]), one can for-mulate the separation problem as an LP with an exponential numberof constraints. Boccia [6] recently proposed solving this LP directlyvia a cutting plane method.
We give some details of Boccia’s method, since we will propose ways toimprove it in Section 5.
We seek a valid inequality αTx ≤ α0 for Q which is violated by a givenvector x∗ ∈ [0, 1]n. For simplicity, we normalise the inequality so that α0 =1. The separation LP is then as follows:
max∑
j∈N x∗jαj∑j∈S αj ≤ 1 (∀S ⊂ N :
∑j∈S
aj ≤ b) (7)
αj ≥ 0 (j ∈ N).
The constraints (7) ensure that the inequality αTx ≤ 1 is satisfied by everyfeasible solution to the given 0-1 KP instance. If the solution α∗ has a profitlarger than 1, then the inequality (α∗)Tx ≤ 1 is violated by x∗.
To check if a constraint (7) is violated by a given α∗ ∈ [0, 1]n, it sufficesto solve the following 0-1 KP:
max∑
j∈N α∗jyj
s.t.∑
j∈N ajyj ≤ byj ∈ {0, 1} (j ∈ N).
6
If the solution y∗ to this 0-1 KP has a profit larger than 1, and we setS = {j ∈ N : y∗j = 1}, then the inequality
∑j∈S αj ≤ 1 is violated by α∗.
This observation enables one to solve the separation LP itself via a cuttingplane method, in which an 0-1 KP is solved in each iteration.
As Boccia points out, this naive approach suffers from very long comput-ing times, and significant problems with rounding errors. To address theseproblems, Boccia proposes a three-stage procedure. In the first stage, vari-ables that satisfy x∗j ∈ {0, 1} are temporarily eliminated from the problem,so that the separation LP can be solved in the subspace of the fractionalvariables. If a violated inequality is found, one proceeds to the second stage,in which the inequality is scaled and rounded to integers. Boccia does thisby solving a small Integer Linear Program. In the third and final stage, theviolated inequality is lifted to make it valid and facet-inducing for Q. Todo this, Boccia solves a sequence of 0-1 KPs. As in Gu et al. [16], Boc-cia recommends down-lifting the variables with x∗j = 1 before up-lifting thevariables with x∗j = 0.
2.5 Applications to general 0-1 LPs
As mentioned above, inequalities for 0-1 knapsack polytopes can be used ascutting planes for general 0-1 LPs. Given a 0-1 LP in the form
max{cTx : Ax ≤ b, x ∈ {0, 1}n},
where A ∈ Zm×n and b ∈ Zm, we define the following two polytopes:
P := {x ∈ [0, 1]n : Ax ≤ b} (the feasible region of the LP relaxation)PI := conv{x ∈ {0, 1}n : Ax ≤ b} (the integral hull).
Furthermore, let us denote by Qi the 0-1 knapsack polytope associated withthe ith constraint; i.e.,
Qi := conv
x ∈ {0, 1}n :∑j∈N
aijxj ≤ bi
.
(Negative aij coefficients are easily handled by complementing variables.)Then, we have by definition:
PI ⊆m⋂
i=1
Qi ⊆ P.
Optimising over⋂m
i=1Qi with a method such as Lagrangian decomposi-tion (Guignard & Kim [19]) is extremely time-consuming. Instead, Crowderet al. [11] proposed using valid inequalities for the Qi as cutting planes,to strengthen an initial formulation (an approach now known as cut-and-branch). This works best when the constraint matrix A is sparse and has
7
no special structure. Using their separation heuristic for simple LCIs, theymanaged to solve ten 0-1 LPs that were previously considered intractable.This approach was developed further by Hoffman & Padberg [21] and Gu etal. [16].
Boyd [8, 9] performed some experiments on the same 10 instances asCrowder et al. [11], using his routines for generating Fenchel cuts. Theresulting bounds were very strong, but running times were rather large.
Cover inequalities and their variants have also been applied, with moreor less success, to various 0-1 LPs with special structure. See, e.g., Bektas& Oguz [5], Gabrel & Minoux [14] and our own paper [22] for applicationsto multidimensional knapsack problems and Ferreira et al. [12] for an appli-cation to multiple knapsack problems.
We are not aware of any application of WIs to general 0-1 LPs, but anapplication to the quadratic knapsack problem can be found in Helmberg &Weismantel [20].
Finally, we remark that one need not restrict oneself to deriving knapsackfacets from the rows of the given system Ax ≤ b. Indeed, any surrogateconstraint obtained as a non-negative linear combination of the originalconstraints can be used as a ‘source row’ for generating knapsack facets.This is the approach taken by Boccia himself [6] when tackling certain facilitylocation problems, and by Uchoa et al. [27] when tackling the capacitatedminimum spanning tree problem.
3 New Separation Algorithms for ECIs and LCIs
In this section, we present two exact algorithms and a heuristic for ECIseparation, and then show how all three algorithms can be modified to yieldheuristics for the separation of the (stronger and more general) LCIs.
3.1 A simple O(n2b) exact algorithm for ECIs
We begin by presenting a fairly simple exact algorithm for ECI separation,that involves the solution of a sequence of 0-1 KPs.
Recall from Subsection 2.2 that Crowder et al. [11] formulated the sep-aration problem for CIs as the 0-1 KP (2)-(4). We will need the followinglemma:
Lemma 1 Let C∗ ⊂ N be the cover obtained by solving the 0-1 KP (2)-(4),and let a∗ := maxj∈C∗ aj. If the ECI with C = C∗ is not violated, then aviolated ECI must satisfy C ⊂ {j ∈ N : aj ≤ a∗}.
Proof. The violation of an ECI with cover C is equal to the sum of twoterms:
1.∑
j∈C x∗j − |C|+ 1 (the violation of the CI with cover C)
8
2.∑
j∈N\C:aj≥a∗ x∗j .
By definition, C∗ is the cover that maximises the first term. So, if we replaceC∗ with a different cover C, the first term cannot increase. Moreover, ifmaxj∈C aj > a∗, then the second term cannot increase either. �
If one could make the inequality in Lemma 1 strict, an iterative algorithmfor ECI separation would immediately follow: solve the 0-1 KP (2)-(4),eliminate the items with aj ≥ a∗, and repeat until the total weight of theremaining items no longer exceeds the knapsack capacity. However, wewere unable to determine whether the lemma is valid with strict inequality.Fortunately, we have the following refinement of Lemma 1:
Proposition 1 Let C∗ and a∗ be defined as in Lemma 1, let S∗ = {j ∈ C∗ : aj = a∗}and let k∗ be an item in S∗ of minimum x∗ value. If the ECI with C = C∗ isnot violated, then a violated ECI must satisfy C ⊂ {j ∈ N \ {k∗} : aj ≤ a∗}.
Proof. Suppose that the ECI corresponding to C∗ is not violated, butthere exists a cover C that yields a violated ECI. By Lemma 1, we canassume C ⊂ {j ∈ N : aj ≤ a∗}. Now recall again the two components ofthe violation mentioned in the proof of Lemma 1. If C contained at leastas many items of weight a∗ as C∗ did, then the second term would be nolarger for C than it was for C∗, and the ECI would not be violated. Thus,C must contain fewer items of weight a∗ than C∗ did. Since k∗ is the itemwith smallest x∗ value in S∗, we can assume that k∗ does not belong to C.�
The following separation algorithm follows immediately from Proposition1:
0. Input: a knapsack constraint∑
j∈N ajxj ≤ b and a vector x∗ ∈ [0, 1]n.1. Set B := {j ∈ N : x∗j = 0}.2. Solve the 0-1 KP:
min∑
j∈N\B(1− x∗j )yj
s.t.∑
j∈N\B ajyj ≥ b+ 1
yj ∈ {0, 1} (j ∈ N \B).
3. Let C∗ be the resulting cover. If the corresponding ECI is violated, output it.
4. Compute a∗ and k∗ as above. Insert into B the item k∗ and all items in N \Bwith weight larger than a∗.
5. If∑
j∈N\B aj ≤ b, stop. Otherwise return to 2.
The running time of this algorithm is clearly dominated by the solution ofat most n 0-1 KPs. We will see in the next subsection that, even thoughthese 0-1 KPs are of greater-than type, they can be solved in O(nb) time
9
just like the standard 0-1 KP. So, the algorithm has the rather large runningtime of O(n2b). However, in practice only a small number of 0-1 KPs, ofrapidly decreasing size, typically need to be solved. They can often be solvedquickly by branch-and-bound. Moreover, the algorithm often returns severalviolated inequalities rather than just one.
3.2 An O(nb) exact algorithm for ECIs
In this section, we present a faster exact algorithm for ECI separation. First,we show explicitly how to solve the separation problem for the weaker CIs(i.e., the 0-1 KP (2)-(4)) in O(nb) time by dynamic programming. Then,we will show how to modify the dynamic program in order to solve theseparation problem for the ECIs themselves in O(nb time.
For k = 1, . . . , n and r = 0, . . . , b, define:
f(k, r) := min
k∑
j=1
(1− x∗j )yj :k∑
j=1
ajyj = r, y ∈ {0, 1}k .
Also, for k = 1, . . . , n, define:
g(k) := min
k∑
j=1
(1− x∗j )yj :k∑
j=1
ajyj ≥ b+ 1, y ∈ {0, 1}k .
The following algorithm computes all of the f(k, r) and g(k) values, andthereby solves the CI separation problem:
Set f(k, r) :=∞ for k = 1, . . . , n and r = 0, . . . , b. Set f(0, 0) := 0.Set g(k) :=∞ for k = 1, . . . , n.
For k = 1, . . . , n
For r = 0, . . . , bIf f(k − 1, r) < f(k, r)
Set f(k, r) := f(k − 1, r).
For r = 0, . . . , b− ak
If f(k − 1, r) + (1− x∗k) < f(k, r + ak)Set f(k, r + ak) := f(k − 1, r) + (1− x∗k).
For r = b− ak + 1, . . . , bIf f(k − 1, r) + (1− x∗k) < g(k)
Set g(k) := f(k − 1, r) + (1− x∗k).
If g(k) < 1, output the violated CI.
It is easy to see that this algorithm runs in O(nb) time.Suppose now that the items in N have been sorted in non-decreasing
order of aj . Under this assumption, we have a violated ECI if and only
10
if g(k) −∑n
j=k+1 x∗j < 1 for some k ∈ N . Thus, to convert the exact CI
separation algorithm into an exact ECI separation algorithm, it suffices tomake the following two minor changes:
i) Sort the items in non-decreasing order of aj before running the algorithm.ii) Change the last ‘if’ statement to:
If g(k)−∑n
j=k+1 x∗j < 1, output the violated ECI.
Note that the initial sorting can be performed in O(n+amax) time, usingbucket sort with amax buckets. This leads to the following result:
Theorem 1 The separation problem for ECIs can be solved exactly in O(nb)time.
Proof. The time taken to sort the items, O(n + amax), is dominated bythe time taken to solve the dynamic program, O(nb). �
We remark that this algorithm may in fact return several violated ECIsin a single call. Moreover, it can be made faster in practice by excludingvariables with x∗j = 0 from the DP, although, each time a violated ECI isfound, one should check whether any such variables can be inserted into theextension E(C).
3.3 A fast heuristic for ECIs
The ECI separation algorithm presented in Subsection 3.1 can be easilymodified to form a fast heuristic for ECI separation: one simply solves thesubproblems in step 2 heuristically instead of exactly. Following Crowder etal. [11], we simply insert items into C in non-decreasing order of (1−x∗j )/aj .This leads to the following algorithm:
1. Sort the items in N in non-decreasing order of (1− x∗j )/aj , and store them in alist L. Initialise the cover C as the empty set and initialise a∗ = b.
2. Remove an item from the head of the sorted list L. If its weight is larger thana∗, ignore it, otherwise insert it into C. If C is now a cover, go to step 4.
3. If L is empty, stop. Otherwise, return to step 2.
4. If the ECI corresponding to C is violated by x∗, output it.
5. Find k∗, the heaviest item in C, decrease a∗ accordingly, and delete k∗ from C.Return to step 2.
This heuristic can easily be implemented so that it runs in O(n2) time.Moreover, O(n2) seems to be necessary, since outputting a single violatedECI itself takes O(n) time. However, one can consider a variant of theheuristic in which, in step 4, one stops as soon as one violated ECI has
11
been output. With some care, this variant can be implemented to run inonly O(n log n) time. We skip the details, partly for the sake of brevity,but also because the O(n2) version already works very well in practice (seeSubsection 6.3).
3.4 Heuristics for LCIs
Recall that ECIs are a special case of, and dominated by, the LCIs. Any ofthe algorithms for ECI separation presented in the previous three subsec-tions can be easily converted into a heuristic for LCI separation. Namely,each time we generate an ECI, we convert it into an LCI that is violated byat least as much. To do this, we use the same lifting order as Gu et al. [16].The resulting algorithm is as follows:
0. Input: a knapsack constraint∑
j∈N ajxj ≤ b, a vector x∗ ∈ [0, 1]n and a coverC found by any of the ECI separation algorithms.
1. Let N0 = {j ∈ N : x∗j = 0}, N1 = {j ∈ N : x∗j = 1} and F = N \ (N0 ∪N1).
2. Set D := C ∩N1.
3. If there exists a k ∈ N \ (C ∪N0) such that ak > b−∑
j∈D aj , then remove oneor more items from D.
4. Up-lift variables in F \ C. If the resulting inequality is not violated, stop.
5. Down-lift variables in D and up-lift variables in N0 to produce the final LCI.
Note that the variables in F \ C have to be up-lifted before one checks theLCI for violation. This, and the need to perform down-lifting, makes LCIseparation more time-consuming than ECI separation. Thus, the bounds onrunning times given for ECIs no longer hold.
4 Exact Separation Algorithms for WIs
As we mentioned in Subsection 2.3, the complexity of the separation problemfor WIs is unknown. We conjecture that it is weakly NP -hard. In thissection, we present an enhanced version of Weismantel’s pseudo-polynomialexact algorithm, and then present a faster exact algorithm.
4.1 Enhancements to Weismantel’s algorithm
We begin this subsection by analysing the running time of Weismantel’salgorithm. Recall that it involves the solution of one equality-constrained0-1 KP for each possible value of the residual capacity r = b −
∑j∈W aj .
We will need the following lemma:
Lemma 2 Suppose that x∗ ∈ [0, 1]n and aTx∗ ≤ b. A necessary conditionfor a WI to be violated is that 0 < r < amax.
12
Proof. When r = 0, the WI reduces to the original knapsack constraintaTx ≤ b. When r ≥ amax, the WI is implied by the upper bounds xj ≤ 1for all j ∈W . �
The assumption that x∗ ∈ [0, 1]n and aTx∗ ≤ b is of course reasonable.So, only amax − 1 equality-constrained 0-1 KPs need to be solved. If we
solve an equality-constrained 0-1 LP with a standard dynamic programmingalgorithm, it takes O(nb) time. Thus:
Proposition 2 Weismantel’s algorithm can be implemented to run in O(nbamax)time.
This running time is excessive, in both theory and practice. We have foundthree simple ways to speed up the algorithm, which make a dramatic differ-ence in practice:
1. It can be shown that, if x∗k = 0, then one can assume that k /∈ Wwithout losing any violated WIs. Similarly, if x∗k = 1, then one canassume that k ∈ W . This reduces the number of variables in theequality-constrained 0-1 KPs.
2. For a given value of r, it may be that no W ⊂ N exists satisfying∑j∈W aj = b − r. As a result, the equality-constrained 0-1 KP will
be infeasible. To avoid wasting time trying to solve such infeasibleproblems, it helps to compute the possible values that
∑j∈W aj can
take before moving on to solve the equality-constrained 0-1 KPs. Thiscan easily be done in O(nb) time and O(b) space, by dynamic pro-gramming.
3. If one solves the remaining equality-constrained 0-1 KPs via branch-and-bound, one can abort the branch-and-bound run as soon as theupper bound is less than or equal to the right-hand side of (6).
For the sake of completeness, we include formal proofs of the two resultsmentioned above.
Proposition 3 Assume (as before) that x∗ ∈ [0, 1]n and aTx∗ ≤ b. Ifx∗k = 0, then one can assume that k /∈W without losing any violated WIs.
Proof. Suppose that the WI (5) is violated by x∗ and that there exists ak ∈ W such that x∗k = 0. If we delete k from W , we obtain the followingmodified WI:∑
j∈W\{k}
ajxj +∑
j∈N\W
max{0, aj − r − ak}xj ≤ b− r − ak. (8)
Now, if we sum together:
13
• the original knapsack constraint aTx ≤ b multiplied by ak/(r + ak),
• the modified WI (8) multiplied by r/(r + ak),
• and a suitable non-negative linear combination of non-negativity in-equalities,
we obtain the valid inequality:
∑j∈W\{k}
ajxj +∑
j∈N\W
max{0, aj − r}xj +a2
k
r + akxk ≤ b− r.
This inequality is identical to the original WI, apart from the fact that theleft-hand-side coefficient of xk is smaller. Thus, it must be violated by x∗ bythe same amount as the original WI. Since x∗ satisfies the original knapsackconstraint and the non-negativity inequalities, the modified WI (8) must beviolated by at least as much as the original. �
Proposition 4 Assume (as before) that x∗ ∈ [0, 1]n and aTx∗ ≤ b. Ifx∗k = 1, then one can assume that k ∈W without losing any violated WIs.
Proof. Suppose that the WI (5) is violated by x∗ and that there exists ak /∈W such that x∗k = 1. If ak ≤ r, then inserting k into W yields a WI thatis violated by at least as much as the original. So suppose that ak > r. If wetake the inequality aTx∗ ≤ b and subtract from it r copies of the equationx∗k = 1, we obtain: ∑
j∈N\{k}
ajx∗j + (ak − r)x∗k ≤ b− r.
So the original WI cannot be violated, a contradiction. �
4.2 An O((n + amax)b) exact algorithm
Improving on the running time bound of O(nbamax) turns out to be a non-trivial exercise. We found it helpful to treat the two terms on the left-handside of (6), and the right-hand side of (6), separately. Thus, we make thefollowing three definitions.
Definition 1 For given integers r, s, with 0 < r < amax and 0 ≤ s ≤ b− r,let f(r, s) denote
max
∑j∈N :aj≤r
ajx∗jyj :
∑j∈N :aj≤r
ajyj = s, yj ∈ {0, 1} (j ∈ N : aj ≤ r)
.
14
Definition 2 For given integers r, s, with 0 < r < amax and 0 ≤ s ≤ b− r,let g(r, s) denote
max
∑j∈N :aj>r
x∗jyj :∑
j∈N :aj>r
ajyj = s, yj ∈ {0, 1} (j ∈ N : aj > r)
.
Definition 3 For a given integer r, with 0 < r < amax, let h(r) denote theright-hand side of (6) computed with respect to x∗. That is:
h(r) = b− r −∑
j∈N :aj>r
(aj − r)x∗j .
Armed with these definitions, we propose the following exact separationalgorithm for WIs:
0. Input: a knapsack constraint∑
j∈N ajxj ≤ b and a vector x∗ ∈ [0, 1]n to beseparated.1. Sort the items in N in non-decreasing order of weight.2. Compute f(r, s) for r = 1, . . . , amax − 1 and for s = 0, . . . , b− r.3. Compute g(r, s) for r = 1, . . . , amax − 1 and for s = 0, . . . , b− r.4. Compute h(r) for r = 1, . . . , amax − 1.5. For r = 1, . . . , amax − 1, compute the maximum possible violation of a WI with∑
j∈W aj = b− r. That is, compute:
max0≤s≤b−r
f(r, s) + rg(r, b− r − s)− h(r).
If this quantity is positive, output the violated WI.
We have the following theorem.
Theorem 2 The separation problem for WIs can be solved exactly in O((n+amax)b) time.
Proof. As noted in Subsection 3.2, Step 1 can be performed in O(n+amax)time. Step 2 can be performed in O(nb) time: just use the standard dynamicprogramming algorithm for the 0-1 KP, but introduce new items in non-decreasing order of weight. Similarly, Step 3 can be performed in O(nb)time: use the standard dynamic programming algorithm for the 0-1 KP,but introduce new items in non-increasing order of weight. Step 4 can beperformed in O(namax) time. Finally, step 5 takes O(bamax) time. Theterms O(n+ amax) and O(namax) are dominated by O(nb). �
5 An Enhanced Version of Boccia’s Algorithm
We now turn our attention to the separation problem for the 0-1 knapsackpolytope itself. We have found several effective ways to enhance Boccia’salgorithm. They are explained in the following subsections.
15
5.1 Eliminating rows from consideration
If x∗ can be quickly proven to lie in Q, there is no need to search for violatedinequalities that induce facets of Q. The following propositions, which areeasy to prove, give easily checkable sufficient conditions for this to be thecase.
Proposition 5 Assume (as before) that x∗ ∈ [0, 1]n and aTx∗ ≤ b. If∑j∈N :x∗j >0 aj ≤ b, then x∗ ∈ Q.
Proposition 6 Assume (as before) that x∗ ∈ [0, 1]n and aTx∗ ≤ b. Ifa ∈ {0, 1}n, then x∗ ∈ Q.
5.2 Looking for violated LCIs first
Our best algorithm for LCI separation (see Subsection 6.5) is very fast andeffective. Moreover, the LCIs are typically less dense and better behavednumerically than general facet-inducing inequalities. Thus, we recommendcalling the LCI separation algorithm first, and resorting to exact knapsackseparation only when one does not find a violated LCI.
5.3 Solving knapsack subproblems heuristically
Our goal when solving the knapsack subproblem is to find a violated con-straint of the form (7). Note, however, that we need not necessarily findthe most-violated constraint. Thus, one can solve the knapsack subproblemheuristically, and only call an exact routine when the heuristic fails. We runa simple greedy heuristic, based on profit-to-weight ratio, and then improvethe solution obtained using a simple form of local search. A ‘move’ in ourlocal search algorithm consists in deleting one item from the knapsack andinserting one or more other items. We abort the local search if we find aninequality (7) violated by more than 0.25, otherwise we continue until a localoptimum is reached.
5.4 Re-using old constraints
Initialising each separation LP with the trivial bounds (0 ≤ αj ≤ 1 for allj ∈ N) is not a good idea. One can achieve more rapid convergence, andimprove running times substantially, by ‘re-cycling’ constraints (7) that havealready been generated.
We found the following idea effective: each time the separation LP hasbeen solved to optimality, store in memory all of the constraints (7) thatwere generated. In the next call to the knapsack separator, use these con-straints to initialise the separation LP. To prevent the separation LPs fromgrowing too large, solve the initialised separation LP and throw away the
16
constraints that are non-binding (have zero dual price). Then commencesolving knapsack subproblems as usual until the separation LP is solved tooptimality.
Although this seems like a simple idea, there is a significant complication:the separation LP is solved in the space of the fractional variables, butthe set F changes from one iteration to the next. To deal with this, weproceed as follows. Any item that was previously in F , but is now integer,is deleted from all constraints (7) in which it appears. Next, we checkeach remaining constraint (7) and delete further items (taken from F ), ifnecessary, to obtain valid sets S. Finally, we check if the resulting constraints(7) can be strengthened by inserting into S some items that were previouslyintegral but are now in F .
In practice, the set F does not change radically from one separation callto the next.
5.5 A fast scaling heuristic
Recall that, once a violated inequality is found, it is then necessary to scaleit to integers to avoid problems with rounding errors. Boccia formulatesthis problem itself as a (small) integer linear program. We have found thatthe following simple heuristic works in over 90% of cases: let αmin be thesmallest fractional α∗ value. Divide the entire inequality by αmin, and checkif the resulting inequality is integral (within some desired tolerance).
Note that this heuristic works if and only if there is at least one variablein the desired facet-inducing inequality that has a left-hand-side coefficientequal to 1, which is of course the case for all LCIs.
5.6 One other minor consideration
We mentioned in Subsection 2.2 that, when deriving LCIs, one or moreitems may have to be removed from the down-lifting set D if there existsan item k ∈ N \ (C ∪ N0) whose weight ak exceeds the residual knapsackcapacity b −
∑j∈D aj . An analogous problem arises in Boccia’s method if
there exists an item k ∈ F whose weight ak exceeds the residual knapsackcapacity b′ = b−
∑j∈N1 aj . When this problem arises, we iteratively move
items from N1 to F until no such item exists.
6 Computational Experiments
In this section, we present the results of some computational experiments,to assess the usefulness of our separation algorithms when applied to 0-1
17
LPs. We use the following very simple cutting plane algorithm:
1. Solve the initial LP relaxation of the problem.
2. If the solution is integer, stop.
3. For each row of the problem, call one or more separation algorithms. If anyviolated inequalities are found, add all of them to the LP, re-optimise and go tostep 2.
4. Output the final upper bound and stop.
We use the primal simplex algorithm to solve the initial LP in step 1 anddual simplex to re-optimise after cutting planes have been added.
The algorithm was implemented in Microsoft Visual Studio.NET 2003and called on functions from version 10.0 of the ILOG CPLEX CallableLibrary. All experiments were performed using a PC with an Intel PentiumIV, 2.8 GHz processor and 512 MB of RAM.
6.1 Twelve instances from MIPLIB
We will present comprehensive results for some instances taken from theMIPLIB. If one looks at all of the versions of the MIPLIB, one finds a totalof 16 pure 0-1 instances. We have decided to omit four of them, for reasonsexplained below. In all of the remaining 12 instances apart from one, theobjective is to minimise cost. Table 1 displays some relevant information forthese instances, including: instance name, number of variables n, number ofconstraints m, density (percentage of entries in the constraint matrix A thatare non-zero), number of ‘genuine’ knapsack constraints (i.e., constraintswhose left-hand-side coefficients are not all 0, 1 or -1), cost of the integeroptimum, lower bound obtained by solving the initial LP relaxation (beforecuts are added), and integrality gap, i.e., the gap between the initial lowerbound and the optimum, expressed as a percentage of the optimum.
We remark that the seven P instances were used by both Crowder et al.[11] and Boyd [8, 9] as benchmarks. All twelve instances were used as bench-marks by Gu et al. [16]. The four instances that we excluded were: 1152lav,lp41 and mod010, because they each have only one ‘genuine’ knapsack-typeconstraint; and P6000, partly because it is very large, but also because itsintegrality gap is only 0.01%.
In a few cases, we observed knapsack constraints for which one or moreleft-hand-side coefficients exceeded the right-hand side. This phenomenonindicates the presence of one or more variables that can be fixed permanentlyat zero or one without losing the integer optimum. We performed this fixingright at the start, to avoid problems with our separation routines.
18
name n m %Dens. KPs Opt. Init. LB IG (%)
P0033 33 16 17.4 11 3089.0 2520.6 18.40P0040 40 24 11.3 13 62027.0 61796.5 0.37P0201 201 134 5.0 107 7615.0 6875.0 9.72P0282 282 242 2.0 44 258411.0 176867.5 31.56P0291 291 253 1.9 14 5223.75 1705.1 67.36P0548 548 177 1.8 92 8691.0 315.3 96.37P2756 2756 756 0.4 382 3124.0 2688.7 13.93bm23 27 20 88.5 20 34.0 20.6 39.41lseu 89 28 12.4 11 1120 834.7 25.47mod008 319 6 64.9 6 307 290.9 5.24pipex 48 25 16.0 9 788.263 773.8 1.83sentoy 60 30 100.0 30 -7772 -7839.3 0.87
Table 1: Description of the MIPLIB instances.
6.2 Application of CIs to MIPLIB instances
Table 2 reports the results obtained when applying CIs to the MIPLIBinstances. The columns headed ‘% gap closed’ report the percentage of theintegrality gap closed by the inequalities, and the columns headed ‘time (s)’report the total running time of the cutting plane algorithm, in seconds.This includes the time taken to solve the initial LP, the time spent callingthe separation algorithms and the time taken to re-optimise after adding thecutting planes. The columns headed ‘CJP’ were obtained using the greedyheuristic of Crowder et al. [11]. The column headed ‘Exact’ was obtained bysolving the separation problem exactly. The column headed ‘Ex1’ reportsthe time taken when using the exact separation algorithm in its original form.The column headed ‘Ex2’ reports the time taken by a hybrid approach, inwhich the exact algorithm is called only when the Crowder et al. heuristicfails.
Although the CIs are theoretically very weak cutting planes, we see thatthey close half of the integrality gap on average. Moreover, the Crowderet al. heuristic does a remarkably good job. All of the running times arenegligible (fractions of a second).
6.3 Application of ECIs to MIPLIB instances
Table 3 reports the results obtained when applying ECIs to the MIPLIB in-stances. The columns headed ‘CJP’ were again obtained using the Crowderet al. heuristic. The columns headed ‘GNS’ were obtained using the alter-native greedy heuristic of Gu et al. [16]. The columns headed ‘N.H.’ wereobtained using the new heuristic presented in Subsection 3.3. The columnheaded ‘Exact’ was obtained by solving the separation problem exactly. The
19
%gap closed time (s)
name CJP Exact CJP Ex1 Ex2
P0033 63.55 63.55 0.016 0.015 0.016P0040 76.82 76.82 0.000 0.000 0.000P0201 33.78 33.78 0.016 0.015 0.016P0282 94.19 94.27 0.047 0.468 0.235P0291 95.19 95.23 0.031 0.094 0.063P0548 67.55 67.67 0.063 1.047 0.282P2756 86.26 86.26 0.187 0.546 0.203bm23 5.56 5.56 0.000 0.110 0.047lseu 39.87 39.87 0.000 0.047 0.032mod008 3.36 3.75 0.000 0.266 0.047pipex 16.68 16.68 0.000 0.015 0.015sentoy 16.33 16.58 0.000 0.407 0.297
Aver. 49.93 50.00 0.030 0.253 0.104
Table 2: Results obtained when applying CIs to MIPLIB instances.
column headed ‘Ex1’ corresponds to the O(n2b) exact algorithm describedin Subsection 3.1. The column headed ‘Ex2’ corresponds to the O(nb) ex-act algorithm described in Subsection 3.2. The missing entries are due tothe fact that the algorithm ran into time and memory difficulties for theinstance mod008. Finally, the column headed ‘Ex3’ corresponds to a hybridapproach, in which the O(n2b) exact algorithm is called only when the newheuristic fails.
We found these results rather suprising, for several reasons. First, theECIs do not usually close substantially more of the gap than the CIs, al-though they do for a few instances. Second, the Gu et al. heuristic performsno better than the Crowder et al. heuristic. (In fact, it performs slightlyworse, but we doubt that the difference is statistically significant.) Third,the O(nb) exact algorithm turned out to be much slower in practice thanthe O(n2b) algorithm. Finally, we were also pleasantly surprised to find thatour new separation heuristic closes more of the gap, on average, than theother two heuristics.
All things considered, the new ECI heuristic and the new hybrid exactECI algorithm look very promising.
6.4 Application of simple LCIs to MIPLIB instances
Next, we converted the ECI algorithms into simple LCI algorithms, as ex-plained in Subsection 3.4. Remarkably, the simple LCIs closed no more ofthe gap than the ECIs, for any of the twelve instances, despite the factthat they are stronger in theory. Further investigation revealed that up-
20
%gap closed time (s)
name CJP GNS N.H. Exact CJP GNS N.H. Ex1 Ex2 Ex3
P0033 66.50 66.50 71.51 71.93 0.00 0.02 0.02 0.06 0.09 0.03P0040 76.82 76.82 100.00 100.00 0.00 0.02 0.00 0.00 0.19 0.00P0201 33.78 33.78 33.78 33.78 0.02 0.00 0.02 0.02 0.19 0.00P0282 94.19 93.48 92.72 94.35 0.03 0.03 0.02 0.44 0.39 0.22P0291 95.19 95.15 94.96 95.23 0.03 0.02 0.02 0.09 5.71 0.08P0548 67.55 66.48 67.37 67.68 0.06 0.08 0.06 1.05 16.9 0.30P2756 86.26 78.16 86.26 86.26 0.17 0.22 0.16 0.52 70.6 0.20bm23 15.36 15.36 15.82 15.82 0.00 0.00 0.00 0.08 0.02 0.05lseu 39.87 39.85 61.01 61.36 0.00 0.00 0.02 0.13 3.16 0.06mod008 4.58 4.42 6.31 17.59 0.00 0.02 0.20 0.44 — 0.34pipex 16.68 16.68 23.82 27.96 0.00 0.00 0.00 0.06 0.03 0.09sentoy 16.53 15.99 17.27 21.57 0.02 0.00 0.02 0.94 8.06 0.70
Aver. 51.11 50.22 55.90 57.80 0.03 0.03 0.04 0.32 — 0.17
Table 3: Results obtained when applying ECIs to MIPLIB instances.
lifting coefficients larger than 1 occurred very rarely when simple LCIs weregenerated.
As expected, the running times were slightly higher for simple LCIs thanfor ECIs. (For brevity, we do not present the times in tables.) All thingsconsidered, the simple LCIs exhibit disappointing performance for theseinstances.
6.5 Application of general LCIs to MIPLIB instances
Table 4 reports the results obtained when applying general LCIs to the MI-PLIB instances. Four different separation heuristics are compared. Thecolumns headed ‘CJP’ and ‘GNS’ were again obtained using the Crowderet al. and Gu et al. heuristics to generate the covers. The columns headed‘N.H.1’ were obtained using our best exact ECI separation algorithm (la-belled ‘Ex3’ in Subsection 6.3) to generate the covers. The columns headed‘N.H.2’ were obtained using a hybrid method, in which our best ECI algo-rithm is called only when the Gu et al. heuristic fails to yield a violated LCI.In all four cases, the lifting sequence of Gu et al. [16] was used to derive thegeneral LCIs (see Subsections 2.2 and 3.4).
The amount of gap closed by the general LCIs is significantly better thanthat closed by the ECIs on some instances, and especially so on pipex. Thisconfirms the value of down-lifting. Moreover, in our view, the running timesare extremely promising. All things considered, we recommend using thestrategy represented by ‘N.H.2’.
21
%gap closed time (s)
name CJP GNS N.H.1 N.H.2 CJP GNS N.H.1 N.H.2
P0033 70.36 80.62 80.62 80.62 0.000 0.015 0.031 0.031P0040 100.00 100.00 100.00 100.00 0.000 0.000 0.000 0.000P0201 33.78 33.78 33.78 33.78 0.015 0.016 0.016 0.016P0282 95.97 94.84 96.15 96.21 0.046 0.031 0.625 0.437P0291 95.19 96.39 95.41 96.40 0.015 0.031 0.110 0.047P0548 67.56 66.81 67.71 67.71 0.062 0.078 0.437 0.297P2756 86.26 80.52 86.26 86.26 0.187 0.219 0.265 0.250bm23 16.11 16.00 16.11 16.11 0.000 0.000 0.047 0.047lseu 56.99 59.28 61.56 66.20 0.000 0.016 0.062 0.062mod008 17.81 17.55 18.24 18.24 0.047 0.031 0.359 0.094pipex 72.78 72.62 73.34 73.34 0.000 0.000 0.062 0.032sentoy 16.53 16.50 22.69 22.72 0.000 0.016 0.641 0.813
Aver. 60.78 61.24 62.66 63.13 0.031 0.038 0.221 0.177
Table 4: Results obtained when applying general LCIs to MIPLIB instances.
6.6 Application of WIs to MIPLIB instances
Table 5 reports results obtained for the WIs, using three algorithms: theheuristic of Helmberg & Weismantel [20], the enhanced version of Weisman-tel’s exact algorithm presented in Subsection 4.1, and a hybrid algorithm,in which the exact algorithm is called only when the heuristic fails. Wedo not report results for the O((n + amax)b) exact algorithm presented inSubsection 4.2, because it ran into serious time and memory problems formost of the instances.
We see that, for these instances, the WIs are less effective than the LCIsin terms of percentage gap closed. Moreover, our exact separation algorithmfor WIs is excessively time-consuming on some of the instances, even ifwe call the Helmberg-Weismantel heuristic first. On the other hand, theHelmberg-Weismantel heuristic itself does remarkably well: it is much fasterthan the exact algorithm, yet closes the same percentage of the integralitygap as the exact algorithm on several instances.
We also experimented with several variants of our cutting plane algo-rithm in which LCIs and WIs were used in combination. Using our bestscheme, the LCIs and WIs were able to close 64.54% of the gap on aver-age. Considering that our best scheme for LCIs (see Table 4) already closes63.13% of the gap on average, this suggests that the WIs are of little use forthese instances. (We do not report detailed results, for the sake of brevity.)
22
%gap closed time (s)
name HW Exact HW Ex1 Ex2
P0033 6.44 6.44 0.000 0.203 0.000P0040 29.24 29.24 0.016 0.031 0.016P0201 12.50 12.50 0.016 0.078 0.016P0282 76.46 89.15 0.032 19.859 7.422P0291 88.72 88.79 0.032 0.453 1.078P0548 46.03 73.53 0.016 100.704 97.953P2756 24.71 63.63 0.156 2.750 1.859bm23 1.49 1.81 0.000 0.078 0.000lseu 15.18 15.25 0.031 0.937 0.671mod008 2.30 2.62 0.015 0.344 0.093pipex 4.56 4.56 0.015 0.125 0.015sentoy 3.02 7.32 0.015 342.64 106.844
Aver. 25.89 32.90 0.029 39.017 17.997
Table 5: Results obtained when applying WIs to MIPLIB instances.
6.7 Application of general knapsack facets to MIPLIB in-stances
Table 6 reports results obtained using general facets of the knapsack poly-tope. The percentage gap closed is shown, along with the running timesfor three variants of Boccia’s separation scheme. The columns headed ‘T1’and ‘T2’ correspond to Boccia’s original scheme and our enhanced scheme,respectively. The column headed ‘T3’ was obtained with a scheme thatincorporates all of our enhancements apart from the one mentioned in Sub-section 5.2, i.e., a scheme that does not use LCIs.
Several things are apparent from the table. First, we see that the generalknapsack facets close significantly more of the integrality gap than the LCIsand WIs. Indeed, they close most of the integrality gap on all instances,apart from P0201, bm23 and sentoy. Second, our enhanced scheme is aroundfour times faster, on average, than the original one. Third, it definitely paysoff to call LCI separation first, provided of course that one has a fast andeffective separation heuristic for LCIs. Fourth, our best exact separationalgorithm for general knapsack facets is (bizarrely) less time-consuming thanour best exact separation algorithm for WIs.
A couple of further comments are in order. First, we were using a general-purpose branch-and-bound solver to solve the knapsack subproblems. Onecould probably obtain a substantial speed-up using a specialised algorithmfor the 0-1 KP (Boccia himself uses a modified version of the MINKNAP algo-rithm of Pisinger [26]). Second, we wish to point out that, for the P instances,the percentage gaps closed by the general knapsack facets are exactly the
23
name %gap T1 T2 T3
P0033 87.42 0.594 0.109 0.265P0040 100.0 0.015 0.000 0.016P0201 33.78 1.031 0.031 0.718P0282 98.59 29.188 5.172 7.734P0291 99.43 9.454 1.875 3.579P0548 84.34 37.250 2.250 16.391P2756 86.36 38.469 1.000 24.547bm23 20.08 2.734 0.578 0.703lseu 76.09 3.297 0.687 1.625mod008 89.23 68.906 41.438 57.687pipex 86.55 4.640 0.797 1.188sentoy 30.97 38.844 3.829 11.141
Aver. 74.40 19.535 4.814 10.466
Table 6: Results obtained when applying general knapsack facets to MIPLIBinstances.
same as those reported by Boyd [8, 9], with one exception: for P2756, Boydobtained a slightly smaller value of 86.16%. Indeed, this instance containsa few very dense constraints that, according to Boyd, caused his algorithmto run into difficulties.
6.8 Multi-dimensional knapsack instances
To gain further insight into the relative performance of the different inequal-ities, and also to explore the limits of our approaches, we also performed ex-periments on the 0-1 multi-dimensional knapsack problem (MKP). We usedthe library of MKP instances created by Chu & Beasley [10]. In these in-stances, the constraint matrix A consists entirely of positive integers — andis therefore 100% dense. Although the library contains 270 instances, thelargest instances have not yet been solved to proven optimality (see, e.g.,Vimont et al. [28]). Thus, we restrict our attention to 180 of the instances,for which the optima are known.
The 180 instances are arranged in blocks of ten. Table 7 reports the fol-lowing information for each block. The first three columns show the numberof variables n, the number of constraints m and the so-called tightness ra-tio α. The next column shows the percentage integrality gap of the initialLP relaxation (before the addition of cutting planes). The remaining fivecolumns show the percentage of the integrality gap closed by each of fiveclasses of inequalities. Each figure in the last six columns is averaged overthe given ten instances.
Before interpreting the results, there are three points that we need to
24
n m α IG (%) CIs ECIs LCIs WIs All
100 5 25% 0.99 2.39 3.66 4.20 1.75 17.96100 5 50% 0.45 2.88 5.13 5.81 2.26 21.65100 5 75% 0.32 3.48 5.41 6.77 2.63 22.88100 10 25% 1.59 0.34 0.62 0.79 0.06 5.55100 10 50% 0.80 0.51 1.59 1.73 0.47 7.33100 10 75% 0.48 0.17 0.72 1.01 0.15 7.23100 30 25% 2.97 0.00 0.00 0.00 0.00 0.16100 30 50% 1.34 0.00 0.01 0.01 0.00 0.49100 30 75% 0.83 0.00 0.01 0.01 0.00 0.52250 5 25% 0.25 0.93 1.68 2.02 0.85 14.56250 5 50% 0.11 1.44 2.39 3.04 0.91 15.68250 5 75% 0.08 1.21 2.82 3.94 0.65 17.48250 10 25% 0.46 0.10 0.37 0.58 0.08 4.53250 10 50% 0.23 0.21 0.40 0.47 0.07 4.48250 10 75% 0.14 0.14 0.39 0.58 0.06 5.03500 5 25% 0.07 1.28 1.63 2.14 0.67 13.80500 5 50% 0.04 0.88 1.61 1.90 0.63 11.91500 5 75% 0.02 0.87 1.64 1.98 0.53 13.70
Aver. 0.94 1.67 2.06 0.65 10.28
Table 7: Percentage gap closed by inequalities on 0-1 MKP instances.
make. First, just as with the MIPLIB instances, the simple LCIs gaveexactly the same results as the ECIs, and therefore there is no separatecolumn for them. Second, the column headed ‘WIs’ was obtained usingthe Helmberg–Weismantel separation heuristic, since our exact separationalgorithms ran into time and memory problems for most of these instances.Third, some of the results in the table have been reported elsewhere in theliterature (Bektas & Oguz [5], Gabrel & Minoux [14], Kaparis & Letchford[22]), but only for n = 100 and only for one or two classes of inequalities.
Compared to the results obtained with the MIPLIB instances, there aretwo striking differences. The first is that the general knapsack facets closea much larger proportion of the gap than the LCIs and WIs. This suggeststhat, for constraints involving many variables, the LCIs and WIs give arather poor partial description of the knapsack polytope. The second isthat even the general knapsack facets close only a small proportion of thegap when m > 5. This demonstrates that the intersection of the individualQi gives a poor approximation to PI for problems with more than a fewdense constraints.
For the sake of brevity, we do not report the running times in detail. Wejust want to say that the average running time was a fraction of a second forthe CIs, ECIs, LCIs and WIs, but almost 10 minutes for the general knapsackfacets. This latter figure may seem excessive, but it should be borne in mindthat the separation algorithms of Boyd [7, 8, 9] run into serious difficulties
25
as soon as a constraint has more than about 30 non-zeroes on the left-handside, rather than hundreds of non-zeroes as we have in this case.
7 Concluding Remarks
In this paper, we have presented new separation algorithms for some specificclasses of valid inequalities for the 0-1 knapsack polytope — namely, theECIs, LCIs and WIs — and also for the 0-1 knapsack polytope itself. Wehave also presented extensive computational results.
Our computational results with LCIs confirm the claim in [11] that theLCIs make very useful cutting planes for sparse unstructured 0-1 LPs. Theyalso confirm the claim in [16] that general LCIs are more useful than simpleLCIs. One surprising result, however, is that the simple LCIs generatedin practice tend to be ‘mere’ ECIs. Thus, if one does not wish to botherimplementing down-lifting, then one might as well not bother implementingup-lifting either, and instead work directly with ECIs.
As for the WIs, we were quite disappointed by their performance. Al-though they typically closed a non-trivial portion of the integrality gap, theywere clearly out-performed (in every sense) by the LCIs. Nevertheless, webelieve that further research on WIs is warranted. In the first place, it wouldbe nice to determine whether or not WI separation is NP -hard. It wouldalso be good to examine techniques for strengthening WIs (see Atamturk[2]) in more detail, both theoretically and computationally.
Finally, we recall that, on some instances, the general knapsack facetsclose considerably more of the integrality gap than the LCIs and WIs. Thisindicates that the LCIs and WIs give a poor approximation of the 0-1 knap-sack polytope, especially when the number of variables in the knapsackconstraint is large. Since separation of general knapsack facets is still rathertime-consuming, it might be worthwhile devising and testing separation al-gorithms for other known classes of valid inequalities (such as the extendedweight inequalities [29]). In fact, we believe that the 0-1 knapsack polytopeitself deserves further study.
References
[1] Achterberg T., Koch, T., Martin, A.: MIPLIB 2003. Oper. Res. Lett.34, 361–372.
[2] Atamturk, A.: Cover and pack inequalities for (mixed) integer pro-gramming. Ann. of Oper. Res. 139, 21-38 (2005)
[3] Balas, E.: Facets of the knapsack polytope. Math. Program. 8, 146–164(1975)
26
[4] Balas, E., Zemel, E.: Facets of the knapsack polytope from minimalcovers. SIAM J. Appl. Math. 34, 119–148 (1978)
[5] Bektas, T., Oguz, O.: On separating cover inequalities for the mul-tidimensional knapsack problem. Comp. & Oper. Res. 34, 1771–1776(2007)
[6] Boccia, M.: Using exact knapsack separation for the single-source ca-pacitated facility location problem. Working paper, Department of En-gineering, University of Sannio (2006)
[7] Boyd, E.A.: A pseudo-polynomial network flow formulation for exactknapsack separation. Networks 22, 503–514 (1992)
[8] Boyd, E.A.: Generating Fenchel cutting planes for knapsack polyhedra.SIAM J. on Optimization 3, 734–750 (1993)
[9] Boyd, E.A.: Fenchel cutting planes for integer programs. Oper. Res.42, 53-64 (1994)
[10] Chu, P.C., Beasley, J.E.: A genetic algorithm for the multidimensionalknapsack problem. J. of Heur. 4, 63–86 (1998)
[11] Crowder, H., Johnson, E., Padberg, M.: Solving large-scale 0-1 linearprogramming programs. Oper. Res. 31, 803–834 (1983)
[12] Ferreira, C.E., Martin, A., Weismantel, R.: Solving multiple knapsackproblems by cutting planes. SIAM J. Opt. 6, 858–877 (1996)
[13] Fukasawa, R., Goycoolea, M.: On the exact separation of mixed integerknapsack cuts. In M. Fischetti & D.P. Williamson (eds.) Integer Pro-gramming and Combinatorial Optimization: Proceedings of IPCO XII,Lecture Notes in Computer Science vol. 4513, Springer, Heidelberg, pp.225–239 (2007)
[14] Gabrel, V., Minoux, M.: A scheme for exact separation of extendedcover inequalities and application to multidimensional knapsack prob-lems. Oper. Res. Lett. 30, 252-264 (2002)
[15] Grotschel, M., Lovasz, L., Schrijver, A.J.: Geometric Algorithms andCombinatorial Optimization. Wiley, New York (1988)
[16] Gu, Z., Nemhauser, G.L., Savelsbergh, M.W.P.: Lifted cover inequal-ities for 0-1 integer programs: computation. INFORMS J. Computing10, 427–437 (1998)
[17] Gu, Z., Nemhauser, G.L., Savelsbergh, M.W.P.: Lifted cover inequali-ties for 0-1 integer programs: complexity. INFORMS J. Computing 11,117–123 (1999)
27
[18] Gu, Z., Nemhauser, G.L., Savelsbergh, M.W.P.: Sequence independentlifting in mixed integer programming. J. Comb. Opt. 4, 109–129 (2000)
[19] Guignard, M., Kim, S.: Lagrangean decomposition: a model yieldingstrong Lagrangean bounds. Math. Program. 39, 215–228 (1987)
[20] Helmberg, C., Weismantel, R.: Cutting plane algorithms for semidefi-nite relaxations. In P.M. Pardalos & H. Wolkowicz (eds.) Fields InstituteCommunications, vol. 18, pp. 197–213.
[21] Hoffman, K.L., Padberg, M.W.: Improving LP-representations of zero-one linear programs for branch-and-cut. ORSA J. on Comput. 3, 121–134 (1991)
[22] Kaparis, K., Letchford, A.N.: Local and global lifted cover inequalitiesfor the multidimensional knapsack problem. Eur. J. Opl Res. 186, 91–103 (2008)
[23] Klabjan, D., Nemhauser, G.L., Tovey, C.: The complexity of coverinequality separation. Oper. Res. Lett. 23, 35–40 (1998)
[24] Nemhauser, G.L., Wolsey, L.A.: Integer and Combinatorial Optimiza-tion. Wiley, New York (1988)
[25] Padberg,M.W.: (1, k)-configurations and facets for packing problems.Math. Program. 18, 94-99 (1980)
[26] Pisinger, D.: A minimal algorithm for the 0-1 knapsack problem. Oper.Res. 46, 758–767 (1995)
[27] Uchoa, E., Fukasawa, R., Lysgaard, J., Pessoa, A., Poggi de Aragao,M., Andrade, D.: Robust branch-cut-and-price for the capacitated min-imum spanning tree problem over a large extended formulation. Work-ing Paper, Department of Business Studies, Aarhus School of Business,Denmark (2006)
[28] Vimont, Y., Boussier, S., Vasquez, M.: Reduced costs propagation inan efficient implicit enumeration for the 0-1 multidimensional knapsackproblem. J. Comb. Opt. 15, 165–178 (2008)
[29] Weismantel, R.: On the 0-1 knapsack polytope. Math. Program. 77,49-68 (1997)
[30] Williams, J.W.J.: Algorithm 232 - Heapsort. Communications of theACM 7, 347348 (1964)
[31] Wolsey, L.A.: Faces for a linear inequality in 0-1 variables. Math. Pro-gram. 8, 165-178 (1975)
28
[32] Wolsey, L.A.: Facets and strong valid inequalities for integer programs.Oper. Res. 24, 367-372 (1976)
[33] Yan, X.-Q., Boyd, E.A.: Cutting planes for mixed-integer knapsackpolyhedra. Math. Program. 81, 257–262 (1998)
[34] Zemel, E.: Easily computable facets of the knapsack polytope. Math.Oper. Res. 14, 760–765 (1989)
29
