A Computational Study of Dantzig-Wolfe Decomposition · additional novel ideas. We use DecOptto evaluate several computational strategies in a consistent computational framework,

A Computational Study

of Dantzig-Wolfe Decomposition

James Richard Tebboth

Thesis submitted for the degree ofDoctor of Philosophy

in theUniversity of Buckingham

16 December 2001

AbstractA Computational Study of Dantzig-Wolfe Decomposition

James Richard Tebboth

This thesis evaluates the computational merits of the Dantzig-Wolfe decompositionalgorithm. We use modern computer hardware and software, and, in particular,we have developed an efficient parallel implementation of Dantzig-Wolfe decompo-sition. We were able to solve problems of up to 83,500 rows, 83,700 columns, and622,000 non-zero elements, and one important instance was solved in 63% of thetime taken by a commercial implementation of the simplex method. If we were con-tent with a solution guaranteed to be within 0.1% of optimality, then the solutiontime can be cut by a further 32%. Using parallel hardware we can cut the solutiontime further: by 59% on a four processor PC (equivalent to a speed-up of 2.42) or81% on seven processors of a Silicon Graphics workstation (a speed-up of 5.22).

Dantzig-Wolfe decomposition is an optimisation technique for solving largescale, block structured, linear programming (LP) problems. Problems from manydifferent fields such as production planning, refinery optimisation, and resource al-location may be formulated as LP problems. Where there is some structure arisingfrom repeated components in the problem, such as the handling of multi-periods,multi-locations, or multi-products, the problem may potentially be solved usingDantzig-Wolfe decomposition.

As preparation for our practical work, we investigate how suitable block struc-tures can be identified in practical LP problems. We develop the decompositionalgorithm from first principles, to delineate the theoretical requirements and showwhich aspects are open for experimentation in a practical implementation. We il-lustrate geometrically the transformation from the original problem to the Dantzig-Wolfe master problem and establish precisely how solutions obtained from the de-composition algorithm correspond to solutions of the original problem. We criticallyreview previous practical work.

The major products of this work are two software tools. DecOpt is a combinedserial and parallel implementation of Dantzig-Wolfe decomposition capable of run-ning on a variety of common platforms. It incorporates many different computa-tional strategies, drawing on the best practice of previous work together with someadditional novel ideas. We use DecOpt to evaluate several computational strategiesin a consistent computational framework, using a collection of real world problemsand recognised statistical techniques. We derive a good overall strategy for generalapplication, and compare the performance of Dantzig-Wolfe decomposition and thesimplex method.

DecMod is a new tool for partitioning LP problems. It takes an algebraic modelof the complete problem, together with a concise description of the block structurein a similar notation, and outputs the block matrices required for the applicationof Dantzig-Wolfe decomposition. It permits the extraction of any type of blockstructure to a variety of output formats, and greatly facilitates the application ofany type of block decomposition technique.

Our work shows that if a reasonable block structure can be found, the de-composition method is worth trying. Dantzig-Wolfe decomposition will not rivalmainstream techniques as an optimisation method for all LP problems. But wedo show that Dantzig-Wolfe decomposition has some niche areas of application:certain large scale classes of primal block angular structured problems, and in par-ticular where the context demands rapid results using parallel optimisation, or nearoptimal solutions with a guaranteed quality.

ii

Acknowledgements

I would like to thank Bob Daniel for his patient support; Dash Associates,and in particular Robert Ashford, Stephen Clark, and Richard Laundy, fortheir practical assistance with the experimental work; the European UnionESPRIT project ‘MEMIPS’ for funding my position at Buckingham; theMEMIPS participants, and in particular PowerGen and BASF, for providingtest problems and for working with the software tools I produced; EddieShoesmith, for his advice and references to factorial designs; and finallyClare for all her encouragement.

iii

iv

Contents

Abbreviations xiii

1 Introduction 11.1 Linear Programming in Practice . . . . . . . . . . . . . . . . 11.2 Linear Programming Problems . . . . . . . . . . . . . . . . . 21.3 Dantzig-Wolfe Decomposition . . . . . . . . . . . . . . . . . . 31.4 Structure of Thesis . . . . . . . . . . . . . . . . . . . . . . . . 41.5 Contribution to Knowledge . . . . . . . . . . . . . . . . . . . 5

2 Block Structures 92.1 Sparsity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92.2 Block Structures in a Matrix . . . . . . . . . . . . . . . . . . 10

2.2.1 Primal Block Angular Structure . . . . . . . . . . . . 112.2.2 Dual Block Angular Structure . . . . . . . . . . . . . . 122.2.3 Block Staircase Structure . . . . . . . . . . . . . . . . 122.2.4 Lower Block Triangular Structure . . . . . . . . . . . 13

2.3 Block Structure in an LP Problem . . . . . . . . . . . . . . . 142.4 Identifying Block Structures in LP Problems . . . . . . . . . 162.5 Other Methods of Identifying Block Structure . . . . . . . . . 192.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

3 Dantzig-Wolfe Decomposition 213.1 Basic Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.1.1 Polyhedra, Extreme Points and Extreme Rays . . . . 223.1.2 Linear Programming Duality . . . . . . . . . . . . . . 23

3.2 Overview of Dantzig-Wolfe Decomposition . . . . . . . . . . . 253.3 The Master Problem . . . . . . . . . . . . . . . . . . . . . . . 26

3.3.1 Minkowski’s Representation Theorem . . . . . . . . . 263.3.2 Examples . . . . . . . . . . . . . . . . . . . . . . . . . 283.3.3 Application to Primal Block Angular LP Problems . . 303.3.4 The Correspondence between Solutions of the Master

Problem and Solutions of the Original Problem . . . . 333.3.5 The Polyhedral Structure of the Master Problem . . . 38

v

vi CONTENTS

3.4 The Column Generation Algorithm . . . . . . . . . . . . . . . 413.4.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . 423.4.2 The Restricted Master Problem . . . . . . . . . . . . . 433.4.3 The Pricing Problems . . . . . . . . . . . . . . . . . . 443.4.4 A Dual Bound . . . . . . . . . . . . . . . . . . . . . . 463.4.5 The Algorithm . . . . . . . . . . . . . . . . . . . . . . 473.4.6 Finite Convergence . . . . . . . . . . . . . . . . . . . . 49

3.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

4 Survey of Computational Experience 534.1 First Experiences . . . . . . . . . . . . . . . . . . . . . . . . . 544.2 The Advanced Implementations of Ho and Loute . . . . . . . 584.3 Parallel Implementations . . . . . . . . . . . . . . . . . . . . . 614.4 The Use of Interior Point Methods . . . . . . . . . . . . . . . 634.5 IP Decomposition: Branch and Price . . . . . . . . . . . . . . 664.6 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

5 Computational Evaluation 735.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . 74

5.1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . 745.1.2 Proposal Generation . . . . . . . . . . . . . . . . . . . 755.1.3 Proposal Management . . . . . . . . . . . . . . . . . . 775.1.4 Proposal Initialisation . . . . . . . . . . . . . . . . . . 785.1.5 Termination . . . . . . . . . . . . . . . . . . . . . . . . 795.1.6 Proposals Computation . . . . . . . . . . . . . . . . . 805.1.7 Price Computation . . . . . . . . . . . . . . . . . . . . 805.1.8 LP Optimisation . . . . . . . . . . . . . . . . . . . . . 815.1.9 Parallel Implementation . . . . . . . . . . . . . . . . . 825.1.10 User Interface . . . . . . . . . . . . . . . . . . . . . . . 83

5.2 Experimental Problem Suite . . . . . . . . . . . . . . . . . . . 845.3 Experimental Platforms and Software . . . . . . . . . . . . . 895.4 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 915.5 Evaluation of Computational Strategies . . . . . . . . . . . . 945.6 Evaluation of Parallel Performance . . . . . . . . . . . . . . . 1015.7 Comparison with Standard LP Software . . . . . . . . . . . . 1085.8 Numerical Robustness . . . . . . . . . . . . . . . . . . . . . . 1105.9 The Performance of the PowerGen Problems . . . . . . . . . 1125.10 Practical Merits . . . . . . . . . . . . . . . . . . . . . . . . . . 117

6 A Decomposition Modeller 1196.1 The Role of Modelling Software . . . . . . . . . . . . . . . . . 1206.2 Modelling Support for Decomposition . . . . . . . . . . . . . 1236.3 DecMod: A Decomposition Modeller . . . . . . . . . . . . . . 1266.4 Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

CONTENTS vii

6.4.1 A Simple Production Planning Model . . . . . . . . . 1276.4.2 Decomposition by Facility . . . . . . . . . . . . . . . . 1276.4.3 Decomposition by Time Period . . . . . . . . . . . . . 130

6.5 The Design and Implementation of DecMod . . . . . . . . . . 1326.5.1 Inputs . . . . . . . . . . . . . . . . . . . . . . . . . . . 1326.5.2 Outputs . . . . . . . . . . . . . . . . . . . . . . . . . . 1336.5.3 Decomposition Language . . . . . . . . . . . . . . . . 1346.5.4 Modules . . . . . . . . . . . . . . . . . . . . . . . . . . 1346.5.5 Data Structures . . . . . . . . . . . . . . . . . . . . . . 136

6.6 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

7 Conclusion 139

A Experimental Platforms and Software 141A.1 HPPVM Options . . . . . . . . . . . . . . . . . . . . . . . . . 141

B Experimental Designs and Results 143B.1 Validation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143B.2 Introduction to Fractional Factorial Designs . . . . . . . . . . 147B.3 The 210−3

V Fractional Factorial Design . . . . . . . . . . . . . 151B.3.1 Design Generators . . . . . . . . . . . . . . . . . . . . 151B.3.2 Defining Relation . . . . . . . . . . . . . . . . . . . . . 151B.3.3 Design Matrix . . . . . . . . . . . . . . . . . . . . . . 151B.3.4 Alias Structure . . . . . . . . . . . . . . . . . . . . . . 154

B.4 Evaluation of Computational Strategies . . . . . . . . . . . . 157B.4.1 Experimental Results . . . . . . . . . . . . . . . . . . 157B.4.2 Analysis of Variance . . . . . . . . . . . . . . . . . . . 159B.4.3 Regression Model . . . . . . . . . . . . . . . . . . . . . 162B.4.4 Model Adequacy Checking . . . . . . . . . . . . . . . 162

B.5 Evaluation of Parallel Performance . . . . . . . . . . . . . . . 166B.6 Comparison with Standard LP Software . . . . . . . . . . . . 175B.7 Comparison of Initial and Chosen Strategies. . . . . . . . . . 177

C DecOpt User Manual 179C.1 What is DecOpt? . . . . . . . . . . . . . . . . . . . . . . . . . 179C.2 Tutorial . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 180

C.2.1 A Production Planning Problem . . . . . . . . . . . . 180C.2.2 Identifying a Block Structure . . . . . . . . . . . . . . 181C.2.3 Generating the Matrices using DecMod . . . . . . . . 183C.2.4 The Problem Initialisation File . . . . . . . . . . . . . 184C.2.5 Applying DecOpt . . . . . . . . . . . . . . . . . . . . . 184C.2.6 Different Block Structures . . . . . . . . . . . . . . . . 184

C.3 Requirements and Installation . . . . . . . . . . . . . . . . . . 185C.4 Overview of Dantzig-Wolfe Decomposition . . . . . . . . . . . 187

viii CONTENTS

C.5 Identifying a Suitable Block Structure . . . . . . . . . . . . . 188C.6 Preparing the Problem Data for DecOpt . . . . . . . . . . . . 189C.7 Starting DecOpt . . . . . . . . . . . . . . . . . . . . . . . . . 190C.8 Controlling DecOpt . . . . . . . . . . . . . . . . . . . . . . . . 191C.9 Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

C.9.1 Proposal Generation . . . . . . . . . . . . . . . . . . . 192C.9.2 Proposal Buffers . . . . . . . . . . . . . . . . . . . . . 193C.9.3 Proposal Management . . . . . . . . . . . . . . . . . . 194C.9.4 Algorithm Termination . . . . . . . . . . . . . . . . . 194C.9.5 Solution Output . . . . . . . . . . . . . . . . . . . . . 195C.9.6 Message Output . . . . . . . . . . . . . . . . . . . . . 195C.9.7 Parallel DecOpt . . . . . . . . . . . . . . . . . . . . . 196

C.10 Directory of Controls . . . . . . . . . . . . . . . . . . . . . . . 197C.11 Error Messages . . . . . . . . . . . . . . . . . . . . . . . . . . 201C.12 Matrix File Names . . . . . . . . . . . . . . . . . . . . . . . . 201

D DecMod User Manual 203D.1 Introduction and Background . . . . . . . . . . . . . . . . . . 203D.2 Tutorial: Dantzig-Wolfe Decomposition . . . . . . . . . . . . 204

D.2.1 Theoretical Background . . . . . . . . . . . . . . . . . 205D.2.2 A Production Planning Problem . . . . . . . . . . . . 206D.2.3 Decomposition by Facility . . . . . . . . . . . . . . . . 208D.2.4 Block Structure Specification for Facility Decomposition210D.2.5 Different Block Structures . . . . . . . . . . . . . . . . 212

D.3 Requirements and Installation . . . . . . . . . . . . . . . . . . 213D.4 Starting DecMod . . . . . . . . . . . . . . . . . . . . . . . . . 214D.5 The Block Structure Specification Language . . . . . . . . . . 215

D.5.1 Sections and Cards . . . . . . . . . . . . . . . . . . . . 215D.5.2 Expressions . . . . . . . . . . . . . . . . . . . . . . . . 215D.5.3 Model Entities . . . . . . . . . . . . . . . . . . . . . . 216D.5.4 The SETTINGS Section . . . . . . . . . . . . . . . . . 216D.5.5 The BLOCKS Section . . . . . . . . . . . . . . . . . . 217D.5.6 The CONSTRAINTS Section . . . . . . . . . . . . . . 219D.5.7 The VARIABLES Section . . . . . . . . . . . . . . . . 220D.5.8 The END Section . . . . . . . . . . . . . . . . . . . . . 221

D.6 Exporting the Partitioned Problem . . . . . . . . . . . . . . . 221D.6.1 File Naming Conventions . . . . . . . . . . . . . . . . 222

D.7 Files Used by DecMod . . . . . . . . . . . . . . . . . . . . . . 224D.8 Control Variables . . . . . . . . . . . . . . . . . . . . . . . . . 224

Bibliography 227

List of Tables

2.1 An algebraic model of the production planning problem. . . . 152.2 A partition of the variables and constraints generated by the

set of products in the production planning problem. . . . . . 15

3.1 The correspondence between primal bounds and dual con-straints, and primal constraints and dual bounds. . . . . . . . 23

5.1 Dimensions of the Ho & Loute problem set. . . . . . . . . . . 855.2 Dimensions of the Netlib problem set. . . . . . . . . . . . . . 855.3 Dimensions of the Gondzio problem set. . . . . . . . . . . . . 855.4 Dimensions of the PowerGen problem set. . . . . . . . . . . . 865.5 Dimensions of the Hydrothermal problem set. . . . . . . . . . 875.6 Dimensions of the PDS problem set. . . . . . . . . . . . . . . 875.7 Initial control settings. . . . . . . . . . . . . . . . . . . . . . . 925.8 Control settings evaluated. . . . . . . . . . . . . . . . . . . . . 945.9 Problems used to evaluate computational strategies. . . . . . 955.10 Significant Effects in the evaluation of computational strategies. 975.11 Control settings chosen following the evaluation. . . . . . . . 995.12 Predicted parallel efficiency. . . . . . . . . . . . . . . . . . . . 1025.13 Observed parallel efficiency. . . . . . . . . . . . . . . . . . . . 1035.14 Comparison of the predicted and observed parallel efficiency. 1075.15 Comparison of Dantzig-Wolfe and simplex methods. . . . . . 1095.16 Shortest run times obtained for the PowerGen problems. . . . 1135.17 Time taken to solve the PowerGen problems to ε-optimality. . 116

6.1 An xpress-mp model of the production planning problem. . 1286.2 DecMod specification for the facility decomposition. . . . . . . 1296.3 DecMod specification for the 1 time period decomposition. . . 1306.4 DecMod specification for the 7 time period decomposition. . . 131

B.1 Validation results (part 1). . . . . . . . . . . . . . . . . . . . . 143B.2 Validation results (part 2). . . . . . . . . . . . . . . . . . . . . 145B.3 Design matrix. . . . . . . . . . . . . . . . . . . . . . . . . . . 151B.4 Evaluation of computational strategies. . . . . . . . . . . . . . 157B.5 Analysis of variance. . . . . . . . . . . . . . . . . . . . . . . . 160

ix

x LIST OF TABLES

B.6 Effect estimates. . . . . . . . . . . . . . . . . . . . . . . . . . 161B.7 Problem mean estimates. . . . . . . . . . . . . . . . . . . . . 162B.8 Parallel DecOpt performance on platform 2. . . . . . . . . . . 166B.9 Performance statistics on platform 2: mean time. . . . . . . . 168B.10 Performance statistics on platform 2: standard deviation. . . 168B.11 Performance statistics on platform 2: speed-up. . . . . . . . . 169B.12 Performance statistics on platform 2: efficiency. . . . . . . . . 169B.13 Parallel DecOpt performance on platform 3. . . . . . . . . . . 170B.14 Performance statistics on platform 3: mean time. . . . . . . . 171B.15 Performance statistics on platform 3: standard deviation. . . 171B.16 Performance statistics on platform 3: speed-up. . . . . . . . . 172B.17 Performance statistics on platform 3: efficiency. . . . . . . . . 172B.18 Parallel DecOpt performance on platform 4. . . . . . . . . . . 173B.19 Performance statistics on platform 4: mean time. . . . . . . . 173B.20 Performance statistics on platform 4: standard deviation. . . 174B.21 Performance statistics on platform 4: speed-up. . . . . . . . . 174B.22 Performance statistics on platform 4: efficiency. . . . . . . . . 174B.23 Comparison of Dantzig-Wolfe and simplex methods. . . . . . 175B.24 Comparison of initial and chosen strategies. . . . . . . . . . . 177

C.1 An algebraic model of the production planning problem. . . . 181C.2 An xpress-mp model of the production planning problem. . 182C.3 A partition of the variables and constraints generated by the

set of facilities in the production planning problem. . . . . . . 182C.4 DecMod specification for the facility decomposition. . . . . . . 183C.5 Optimal solution found by DecOpt. . . . . . . . . . . . . . . . 185C.6 A partition of the variables and constraints generated by the

set of products in the production planning problem. . . . . . 185C.7 A partition of the variables and constraints generated by the

set of time periods in the production planning problem. . . . 186

D.1 An algebraic model of the production planning problem. . . . 207D.2 An xpress-mp model of the production planning problem. . 208D.3 DecMod specification for the facility decomposition. . . . . . . 211D.4 DecMod specification for the product decomposition. . . . . . 212D.5 DecMod specification for the 1 time period decomposition. . . 213

List of Figures

2.1 Block structure in a matrix. . . . . . . . . . . . . . . . . . . . 112.2 Primal block angular structure. . . . . . . . . . . . . . . . . . 112.3 Dual block angular structure. . . . . . . . . . . . . . . . . . . 122.4 Block staircase structure. . . . . . . . . . . . . . . . . . . . . 132.5 Lower block triangular structure. . . . . . . . . . . . . . . . . 132.6 Activities and material flows in the production planning model. 14

3.1 A simple bounded polyhedron. . . . . . . . . . . . . . . . . . 283.2 A simple unbounded polyhedron. . . . . . . . . . . . . . . . . 303.3 The original problem. . . . . . . . . . . . . . . . . . . . . . . 383.4 The pricing problem. . . . . . . . . . . . . . . . . . . . . . . . 393.5 The Minkowski Representation of the pricing problem. . . . . 403.6 The master problem. . . . . . . . . . . . . . . . . . . . . . . . 41

5.1 Parallel DecOpt: Running times on platform 2. . . . . . . . . 1045.2 Parallel DecOpt: Speed-up on platform 2. . . . . . . . . . . . 1045.3 Parallel DecOpt: Running times on platform 3. . . . . . . . . 1055.4 Parallel DecOpt: Speed-up on platform 3. . . . . . . . . . . . 1055.5 Parallel DecOpt: Running times on platform 4. . . . . . . . . 1065.6 Parallel DecOpt: Speed-up on platform 4. . . . . . . . . . . . 1065.7 Convergence of two PowerGen 7-day problems. . . . . . . . . 1125.8 Convergence of the PowerGen 84-day problems. . . . . . . . . 1145.9 Convergence of the PowerGen 364-day problems. . . . . . . . 1145.10 The relative duality gap observed solving the PowerGen 84-

day problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . 1155.11 The relative duality gap observed solving the PowerGen 364-

day problems. . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

6.1 Sequential operation of modeller and optimiser software. . . . 1216.2 The modeller as a front-end to the optimiser. . . . . . . . . . 1226.3 The design of DecMod. . . . . . . . . . . . . . . . . . . . . . . 132

B.1 Normal probability plot of the effects. . . . . . . . . . . . . . 161B.2 Normal probability plot of the residuals. . . . . . . . . . . . . 163B.3 Residuals against the fitted values. . . . . . . . . . . . . . . . 164

xi

xii LIST OF FIGURES

B.4 Residuals by problem: 84-t14, 7-10-t, 7-10-s. . . . . . . . . . 164B.5 Residuals by problem: meat12, large, 84-t7. . . . . . . . . . . 165B.6 Residuals by problem: pds003, mrp5, minsil7. . . . . . . . . . 165B.7 Residuals by problem: sctap2. . . . . . . . . . . . . . . . . . . 165

C.1 Activities and material flows in the production planning model.180C.2 Primal block angular structure in an LP problem. . . . . . . 188

D.1 DecMod input and output. . . . . . . . . . . . . . . . . . . . . 204D.2 Primal block angular structure in an LP problem. . . . . . . 205D.3 Activities and material flows in the production planning model.207D.4 Facility based block structure. . . . . . . . . . . . . . . . . . . 209D.5 Initial master problem matrix. . . . . . . . . . . . . . . . . . 209D.6 Pricing problem matrix. . . . . . . . . . . . . . . . . . . . . . 210

Abbreviations

The meaning of some abbreviations as used in this work and the page wherethey are first defined or introduced.

LP Linear Programming. A linear programming problem is a decision prob-lem with continuous decision variables, constraints that are linear func-tions of the decision variables, a single linear objective function, andfixed data (p. 2).

MP Mathematical Programming. The collective name for linear program-ming, integer programming, and other related types of decision prob-lems.

IP Integer Programming. We use this term in its more general sense: anextension of linear programming in which some of the variables arerestricted to discrete values, such as integers, or just zero and one.Occasionally we also use the term mixed integer programming (MIP)in the same sense.

MIP Mixed Integer Programming. See integer programming.

RHS Right hand side. The constant term in a constraint in a mathematicalprogramming problem (p. 2).

IPM Interior Point Methods. A class of solution methods for linear pro-gramming problems (p. 3).

PVM Parallel Virtual Machine. A protocol for communicating betweenprocesses in a parallel application, in which the underlying platformmay be a single computer or a set of computers linked by a networkconnection (p. 83).

MIMD Multiple Instructions Multiple Data. A class of parallel applica-tions consisting of separate processes following different instructions(‘multiple instructions’) on different data (‘multiple data’) (p. 61).

xiii

xiv ABBREVIATIONS

Chapter 1

Introduction

1.1 Linear Programming in Practice

This thesis concerns the application and evaluation of a solution method forsolving linear programming problems. Linear programming (LP) problemsare decision problems, in which the objective is to choose values for a setof decision variables in order to optimise (maximise or minimise) a statedobjective function, subject to a set of constraints. A formal definition ofthe class of LP problems is given below; but first, as this thesis is primarilyabout solving LP problems in practice, let us briefly consider the context inwhich such problems arise, and the importance of being able to solve them.

A diverse range of real world problems can be approximated and for-mulated as LP problems, and there is often great economic or other valueattached to finding an optimal solution. The field of LP was originally de-veloped to plan military logistics operations in the Second World War (theword ‘programming’ in LP means ‘planning’), and since then the range ofapplications has flourished. Examples include industrial diamond blending,management of hire car fleets, distribution warehousing and supply chainmanagement, oil refining, and gas pipeline flow to name a few (see [GPS00,Wil93] for many other applications). The value of being able to find anoptimum solution, as opposed simply to a feasible solution, or sometimesno solution at all, can run into the order of many millions of pounds. Forinstance, a difference of 1% in the objective value in the year long PowerGenproblem treated in chapter 5 represents a difference in cost of £520 millionper annum [Pow98].

The growth of LP as a practical technique would not have been possiblewithout the simultaneous growth in the power and availability of comput-ing. Today, software for LP optimisation is highly sophisticated, with sev-eral commercial codes being actively developed and marketed. A symbioticrelationship exists between the capacity of the codes and the growth of ap-plications, with solutions to larger problems being demanded in less time as

1

2 CHAPTER 1. INTRODUCTION

codes improve. This thesis investigates a well known—but not well used—solution method, which has the potential to solve large problems quickly byexploiting the problem structure.

1.2 Linear Programming Problems

A linear programming problem may be defined as:

maximise (or minimise) z = cx

subject to Ax T b` ≤ x ≤ u;

(1.1)

where x, c, ` and u are n-vectors, b is an m-vector, and A is an m×n-matrix.x is the vector of decision variables, or unknowns, over which we wish tosolve (1.1); and z is the objective value. The data comprise: c, the objectivevector (the cost vector when it is to be minimised); A, the constraint matrix ;b, the right hand side (RHS) vector; and ` and u, the vectors of lower andupper bounds respectively. The constraint types may be any combination of{≤,=,≥}. The lower and upper bounds may take the values plus or minusinfinity.

The form (1.1) is actually a little cumbersome, so now that it has beenintroduced, we switch to a more concise form:

z∗ = max{cx : Ax ./ b, x ∈ Rn3}. (1.2)

The direction of optimisation will be indicated by ‘min’ or ‘max’, and wework with maximisation problems except where indicated.1 The symbol ‘./’represents an arbitrary set of inequality and equality symbols—so that eachconstraint Ai·x ./i bi is one of Ai·x ≤ bi, Ai·x = bi, or Ai·x ≥ bi. Withthe exception of simple bounds xj ≤ 0, xj = 0, xj ≥ 0 and xj unbounded(free), collectively represented by the symbol ‘3’ in ‘x ∈ Rn3 ’, the bounds` ≤ x ≤ u have been incorporated into the constraints, augmenting A, ./and b as necessary. This form differs slightly from the various standardor canonical forms used in textbooks, in that we handle arbitrary simplebound and constraint types, so as to make explicit the distinction betweenthe bounds and constraints of the original problem and those introduced bythe Dantzig-Wolfe reformulation in chapter 3.

The set P = {x ∈ Rn3 : Ax ./ b} is the feasible region, a polyhedron. TheLP (1.2) is feasible if P is non-empty, and infeasible otherwise. A feasibleLP is bounded if z∗ is finite, and unbounded otherwise. A point x ∈ Rn isfeasible for (1.2) if x ∈ P .

1So dual problems, introduced in chapter 3, will be minimisation problems in general.

1.3. DANTZIG-WOLFE DECOMPOSITION 3

1.3 Dantzig-Wolfe Decomposition

LP optimisation software use two main classes of methods. The simplexmethod is a gradient descent method that moves along the edge of the fea-sible region [Chv83, Dan63]. Interior point methods (IPM) move throughthe interior of the feasible region [Wri97]. We do not dwell on these wellknown methods, but take the solution of an LP problem by either of thesetwo methods as granted, provided that practical considerations allow it.

Dantzig-Wolfe decomposition was developed in the late 1950s, a decadeafter the simplex method, and many years before interior point methodswere applied to LPs [Dan63, Dan83]. It immediately aroused widespreadinterest, and many attempts were made to implement it as a computationalmethod. Practical experiences however were mixed, with some claims ofsuccess but no lasting achievements, when measured by the methods usedto solve practical problems. There has been no evaluation into and devel-opment of the different options and strategies in computational implemen-tations, where as there has been continued research and development forover fifty years with the simplex method, and for over fifteen years withIPM. Perhaps the greatest challenge of Dantzig-Wolfe decomposition hasbeen that, when viewed simply as an alternative LP optimisation method,any successes have been rapidly overtaken by improvements in simplex andIPM implementations. Yet this continued improvement in LP optimisationtechnology could be used to improve implementations of Dantzig-Wolfe de-composition. We survey practical experiences obtained with Dantzig-Wolfedecomposition in chapter 4.

The principle of decomposition as a solution technique is to break aproblem down into a set of smaller problems, and, by solving the smallerproblems, obtain a solution to the original problem. Dantzig-Wolfe decom-position is a solution method for the class of LP problems in which theconstraint matrix, A, exhibits the primal block angular structure defined inchapter 2. To apply it, there are two major steps. The formulation step,which is carried out implicitly, is to reformulate the original problem intoa form amenable to decomposing: the reformulated problem is called themaster problem, and in many respects is actually larger than the originalproblem. The algorithmic step is to solve the master problem using a spe-cialised algorithm, which treats a much smaller, restricted master problemand a set of small pricing problems to obtain a solution to the full masterproblem, and thus the original problem. The pricing problems are basedon the block structure present in the original problem. The method is fullydescribed in chapter 3.

A major ingredient of Dantzig-Wolfe decomposition is the use of LPoptimisation within the algorithm to solve the restricted master problem andthe set of pricing problems—each of these normally solved once per iterationof the Dantzig-Wolfe algorithm. By using state of the art LP optimisation


software as a component within a Dantzig-Wolfe implementation, which canbe updated and replaced over time, we are able to exploit the advances inmainstream LP optimisation within the Dantzig-Wolfe algorithm—advancesthat include presolve techniques, matrix scaling, pricing, and sparse matrixhandling [Bea96, Bix92, Bix94]. This is a prerequisite for a state of the artDantzig-Wolfe decomposition implementation.

One particularly attractive feature of decomposition, and in contrast tothe simplex method, is the relatively straightforward potential for a paral-lel computational version. The pricing phase of the algorithm consists ofsolving a set of mutually independent LP problems, which can be easily ac-complished in parallel. The parallelism is coarse grained, uses distributedmemory, and is ideally suited to networks of serial computers that are incommon everyday use. Furthermore, provided that the work to solve thepricing problems is not trivial, and, in particular, it is not dominated by thework to solve the master problem, the suggested parallelism is likely to beefficient.

With the availability of sophisticated LP optimisation software compo-nents, widely available parallel computer platforms in the shape of high per-formance parallel computers and networks of serial computers, easy-to-useparallel tools which allow shared and distributed memory parallel softwareto be developed for a variety of common platforms, and the ingenious yetunexploited ideas of researchers into decomposition over the past fifty years,it is time for a re-appraisal of Dantzig-Wolfe decomposition.

1.4 Structure of Thesis

Following this introduction, the thesis is divided into six chapters. Thepurpose of chapter 2 is to show how block structure can be identified in realworld LP problems, a subject overlooked in work on decomposition. Westart by defining and illustrating the classes of block structures in which weare interested. By means of an example, we show how such block structurescan be inferred from the algebraic formulation of an LP problem, and derivesome general principles for detecting block structure in LP problems. Wealso discuss other methods that have been proposed for detecting blockstructure.

In chapter 3 we provide the theoretical basis of Dantzig-Wolfe decompo-sition required for a computational implementation. Our two guiding prin-ciples have been to provide a mathematically sound development, withoutunnecessary simplifications. This allows us to apply it directly to a com-putational implementation, and does not place unnecessary limits on theimplementation. In our experience, simplifications made in theoretical workare later un-done in computational work, where they are loosely justified ascomputational extensions.

1.5. CONTRIBUTION TO KNOWLEDGE 5

We first provide a brief revision of the terminology and basic theory oflinear algebra and linear programming, followed by an overview of Dantzig-Wolfe decomposition. The two main sections then cover the formulation ofthe master problem and its solution by the column generation algorithm.Along the way we consider the degree of equivalence between the solutionobtained by the decomposition method, and a solution to the original prob-lem.

Chapter 4 surveys past experience with Dantzig-Wolfe decomposition.Our aim is to identify the state of the art, so that we can both apply bestpractice and make further advances. We describe the techniques used in andthe performance of previous computational implementations.

Our primary objective in this work is a definitive evaluation of Dantzig-Wolfe decomposition as a practical optimisation method for a primal blockangular LP problems, and chapter 5 describes the major practical work un-dertaken to meet this goal. We give details of our computational implemen-tation, DecOpt, drawing on the theoretical basis and computational surveyundertaken in the previous two chapters. We then proceed to describe ourevaluation methodology: the suite of test problems; the computational envi-ronments for testing; the experiments and analysis undertaken; and finishingwith the results, discussion, conclusions and possible further work.

Chapter 6 returns to the issue of applying decomposition in practice.Prompted by our own experiences of applying decomposition to real worldproblems, we argue that modelling software support for decomposition is asequally important as is optimisation software itself. We review the benefitsof modelling support in mathematical programming in general and the addi-tional features required for decomposition, survey existing modelling supportand software for decomposition, and finally present our own contribution,DecMod.

The final chapter summarises the main achievements of our work.

1.5 Contribution to Knowledge

• Little has been written on identifying block structure in practical LPproblems. We show how block structure can be inferred from thealgebraic model description of the problem, by recognising that un-derlying structure in the problem is made apparent through index setsin the model. Our algebraic approach differs from previous compu-tational approaches, but is more readily applicable by practitionersto their problems. To support this approach, we have designed andimplemented new modelling software to automate the extraction ofblock structure from LP problems, a necessary precursor to the appli-cation of optimisation by decomposition, and part of our wider aim ofincreasing accessibility to and application of decomposition in general.


• A new theoretical development of the Dantzig-Wolfe decompositionalgorithm. We emphasise the crucial distinction between the trans-formation from the original problem to the master problem, and thealgorithm used to solve the master problem. We make the mappingfrom the original problem to the master problem explicit, allowing usto state how all of the various solutions (viz. bounded optimal point,unbounded direction, certificate of infeasibility) obtained from the al-gorithm relate to the original problem. We give an explicit exampleof the structure of the master problem, and explain what implicationsthe structure of the master problem has on the solution algorithm.We present a flexible column generation algorithm, showing how the‘computational strategies’ of previous work fit neatly into the frame-work of the algorithm as resolutions for open choices evident ratherthan ad hoc later additions.

• A critical review of previous practical work, drawing out issues for anew implementation and opportunities for further advances.

• The design and implementation of integrated serial and parallel Dantzig-Wolfe decomposition software, applying best practice from previouscomputational work together with our own new ideas. These novelideas include: an automatic means of varying the proposal generationfrequency as the algorithm progresses; specialised scaling routines usedin proposal generation; and a parallel implementation, fully integratedwith the serial implementation, suitable for use on both dedicated par-allel machines and networks of serial computers. There has been lim-ited work with parallel Dantzig-Wolfe decomposition before, on certainspecialised parallel computers, but not using a state of the art decom-position implementation, and not designed for such a wide variety ofarchitectures.

The software uses state of the art software library components forLP optimisation and parallel control and communication, which canbe updated and replaced easily as new advances are made in thesetechnologies. Using Dantzig-Wolfe decomposition implemented in oursoftware, we were able to solve problems of up to 52 blocks, 83,500rows, 83,700 columns, and 622,000 non-zero elements in 63% of thetime taken by a commercial LP optimiser2 to solve the problem usingits simplex method. If we were content with a solution guaranteed tobe within 0.1% of optimality, then the solution time can be cut bya further 32%. Using parallel hardware we can cut the solution timefurther: by 59% on a four processor PC (equivalent to a speed-up of2.42) or 81% on seven processors of a Silicon Graphics workstation (aspeed-up of 5.22).

2The same LP optimisation software used within our decomposition software.

1.5. CONTRIBUTION TO KNOWLEDGE 7

• A definitive evaluation of Dantzig-Wolfe decomposition as a practi-cal optimisation method. This is the first work to compare differ-ent optimisation strategies for Dantzig-Wolfe decomposition using arecognised statistical methodology, and the first work to evaluate theperformance of Dantzig-Wolfe decomposition on a variety of commonparallel architectures, using a single consistent and flexible softwareharness.


Chapter 2

Block Structures in LinearProgramming Problems

Decomposition methods for linear programming problems are designed toexploit block structure in the constraint matrix of the problem. In this chap-ter we describe some commonly occurring block structures that are amenableto decomposition methods, and suggest ways in which they might arise inpractice. We discuss methods of identifying block structure in real worldproblems, an area often overlooked in the literature despite its fundamentalimportance to the application of decomposition methods, and focus in par-ticular on detecting block structure from natural structure in the algebraicmodel. Our objective is to show how block structure may be identified inreal world LP problems, in preparation for the application of Dantzig-Wolfedecomposition.

Little has been written on identifying block structure in practical LPproblems. We show how block structure can be inferred from the algebraicmodel description of the problem, by recognising that underlying structurein the problem is made apparent through index sets in the model. Our alge-braic approach differs from previous computational approaches, but is morereadily applicable by practitioners to their problems. Chapter 6 describesnew modelling software to support this approach.

2.1 Sparsity

Linear programming problems encountered in practice invariably have manyzero elements in their constraint matrices [Bea71, HM00, OH68]. This oc-curs because most activities, or variables, only feature in a few constraints,typically between five and ten—a consequence of the way LPs are formu-lated. Larger LP problems thus have a greater proportion of zero elements,because there are more rows but the same number of non-zero elements ineach row.

9

10 CHAPTER 2. BLOCK STRUCTURES

This phenomenon can be measured by two closely related indices: thedensity is the proportion of non-zero elements in the constraint matrix; andthe sparsity is the proportion of zero elements in the constraint matrix. Letm be the number of constraints and n be the number variables in the LPproblems,1 then

Density = (number of non-zero elements)/(mn);Sparsity = (number of zero elements)/(mn).

Both indices are often expressed as percentages. An LP problem with a veryhigh sparsity is called sparse. Assuming that the average number of non-zeroelements per column, d, is constant, the density is equal to (dn)/(mn) =d/m, which decreases to zero as the number of rows increases.

One consequence of sparsity is that only the non-zero elements of theconstraint matrix are stored when specifying instances of LP problems, i.e.,in memory arrays or a matrix file. A further consequence is that with alarge proportion of the constraint matrix consisting solely of zero elements,it is often possible to recognise patterns in the remaining non-zero elements.In the next section we identify some common patterns that can be exploitedby decomposition methods.

2.2 Block Structures in a Matrix

Given an arbitrary matrix A, consider a pair of partitions of the rows andcolumns: suppose the rows are divided into M + 1 sets {I0, I1, . . . , IM}and the columns into N + 1 sets {J0, J1, . . . , JN}. The rows (respectivelycolumns) in each set Ik (resp. J`) need not be adjacent in the matrix, al-though we shall always illustrate each pair of partitions as if they are. Weshall have more to say on the order of rows and columns when we discussthe problem of finding block structures.

The partitions of the rows and columns impose a partition on the matrixelements as shown in figure 2.1. We say that the elements are partitionedinto blocks Ak` := {aij : i ∈ Ik and j ∈ J`}, and we refer to the parti-tion of the matrix elements as a block structure. A block structure is thusdefined by a pair of partitions on the rows {I0, I1, . . . , IM} and columns{J0, J1, . . . , JN}.

Given the constraint matrix A from a large sparse LP problem, it isoften possible to choose a pair of partitions so that the non-zero elementsof A are confined to relatively few of the blocks in the block structure (andnormally these blocks will themselves be sparse). Indeed, frequently therewill be many ways to obtain block structures with this property.

Some block structures are more useful than others. In the sections andaccompanying figures that follow we illustrate some common block struc-

1The number of variables includes slack and artificial variables.

2.2. BLOCK STRUCTURES IN A MATRIX 11

Figure 2.1: Block structure in a matrix.

Figure 2.2: Primal block angular structure.

tures that are amenable to decomposition methods [FGSV00, Las70]. In thefigures the shaded areas represent blocks containing non-zero elements.

2.2.1 Primal Block Angular Structure

The matrix has a set of rows I0 that connect with all sets of columns, and setsof rows Ik, k = 1, 2, . . . ,K, that each connect with a single set of columnsJk. In practical applications there is usually an additional set of columnsJ0 that interacts solely with the row set I0 as pictured (figure 2.2). Primalblock angular structure is one of the most widely known and recognisedstructures in decomposition, as it is the basis for the original decompositionmethod developed by Dantzig and Wolfe [DW60, DW61], the subject of thiswork.

The structure may typically arise where the system being modelled splitsnaturally into a set of subsystems, e.g., a set of facilities or time periods,independent apart from a number of global constraints. The variables andconstraints referring to a single subsystem k correspond to the rows andcolumns in sets Ik and Jk. The constraints that link the subsystems, cor-responding to rows I0, may express limits of system-wide scarce resources


Figure 2.3: Dual block angular structure.

or ensure that materials balance correctly between the subsystems (e.g., fa-cilities or time periods), and are referred to as global, common or linkingconstraints.

2.2.2 Dual Block Angular Structure

The matrix has a set of columns J0 that connect with all sets of rows, andsets of columns Jk, k = 1, 2, . . . ,K, that each connect with a single set ofrows Ik. There is usually an additional set of rows I0 that interacts solelywith the column set J0 (figure 2.3).

Observe that this structure is just the transpose of the primal blockangular structure. If a problem, with constraint matrix A say, exhibitsa primal block angular structure, then the dual problem, with constraintmatrix AT , will exhibit a dual block angular structure. Dual block angularstructure forms the basis for Benders decomposition [Ben62], well known tobe the dual equivalent of Dantzig-Wolfe decomposition [Las70, § 7.3.2].

Dual block angular structure may typically arise where the system be-ing modelled comprises a number of semi-independent subsystems, linkedby a set of global variables. For instance, such problems occur as the de-terministic equivalent LP problems derived from stochastic programmingproblems [Tho97].

The structures described below also have corresponding dual (transpose)structures which we do not picture.

2.2.3 Block Staircase Structure

The block staircase structure is best explained as a dynamic (time stage)structure. Let column set Jk comprise the columns of variables relateddirectly to time period k. Row set Ik comprises the rows for constraintslinking the decisions made in period k with those made in the previousperiod, k − 1 (figure 2.4). We say this staircase structure has a time lag of

2.2. BLOCK STRUCTURES IN A MATRIX 13

Figure 2.4: Block staircase structure (with time lag 1).

Figure 2.5: Lower block triangular structure.

one, as activities in period k are related directly to those in period k − 1,but not to those from earlier periods.

In general, the staircase structure may have longer time lags. For in-stance, when the row set Ik links columns from the current period k withthe two previous periods, k − 1 and k − 2, we say that the time lag is two.This dual of this type of structure can be exploited by nested Dantzig-Wolfedecomposition [DW60, Ent89, Fou82].

2.2.4 Lower Block Triangular Structure

A block staircase structure with the maximum time lag K has non-zeroblocks in the entire lower triangular portion of the matrix, and is known aslower block triangular structure. In this case decisions made at period k arelinked directly to all previous periods 1, 2, . . . , k − 1 by constraints corre-sponding to rows in Ik (see figure 2.5). The dual form, upper block triangularstructure, can also be exploited by nested Dantzig-Wolfe decomposition.


buym,f,t↓

mstockm,f,t −→ mstockm,f,t+1 −→ mstockm,f,t+2y×REQp,m

makep,f,t↓

pstockp,f,t −→ pstockp,f,t+1 −→ pstockp,f,t+2

↓sellp,f,t

Figure 2.6: Activities and material flows in the production planning model.

2.3 An Example of Block Structure in an LP Prob-lem

In order to apply decomposition methods that exploit block structure, itis first necessary to identify the block structure in the target problem. Insection 2.5 we discuss some methods that have been proposed for identifyingblock structure automatically. It is our belief however that the task is bestperformed by the user, for instance, the person formulating the model of theproblem, and in section 2.4 we describe our algebraic approach. The bestintroduction to our approach is by example, and that is the content of thissection. The model used is based on the Coco problem [Das99b].

Consider a simple production planning model over a set of months t ∈ Tand a set of facilities f ∈ F . In each month at each facility a number ofraw materials m ∈ M are purchased, and manufactured into end productsp ∈ P which are sold. Storage of both materials and products is possible atall facilities. The activities and material flows in the model are pictured infigure 2.6, and table 2.1 presents an algebraic model formulation. Decisionvariables are denoted in lower case; data in upper case; and the namesof constraints in a mixture of upper and lower case. A subscript on theconstraint name indicates that the constraint applies for all values of thesubscript.

Aside from the mass balance equations (PBal and MBal), there are limitson the total sales of each product in each period (MxSell), the total stock ofproducts at each facility in each period (MxPStock), the total production ateach facility in each period (MxMake), the total stock of materials at eachfacility in each period (MxMStock), and the total purchases of each materialin each period (MxBuy). Sales of products generate revenue, whilst all otheractivities incur costs. The objective is to maximise profit.

The model is highly structured, with four sets of objects in the underlying

2.3. BLOCK STRUCTURE IN AN LP PROBLEM 15

Maximise∑pft REV pt sellpft −

∑pft CPSTOCK pstockpft

−∑

pft CMAKE pf makepft −∑

mft CMSTOCK mstockmft−∑

mft CBUY mt buymft

Subject to

PBalpft : pstockpf,t+1 = pstockpft + makepft − sellpftMBalmft : mstockmf,t+1 = mstockmft + buymft

−∑

p REQpm makepftMxSellpt :

∑f sellpft ≤ MXSELLpt

MxPStockft :∑

p pstockpft ≤ MXPSTOCKMxMakeft :

∑p makepft ≤ MXMAKE f

MxMStockft :∑

m mstockmft ≤ MXMSTOCKMxBuymt :

∑f buymft ≤ MXBUY mt

All variables are non-negative.

Table 2.1: An algebraic model of the production planning problem.

Global Set Set p ∈ PConstraints MBal∗∗∗ MxPStock∗∗ PBalp∗∗ MxSellp∗

MxMake∗∗ MxMStock∗∗MxBuy∗∗

Variables mstock∗∗∗ buy∗∗∗ sellp∗∗ pstockp∗∗ makep∗∗

Table 2.2: A partition of the variables and constraints generated by the setof products in the production planning problem.

system: materials M , products P , facilities F and time periods T . Theseform the index sets in the model. Given an index set, some variables andconstraints (collectively entities) refer to just one object from it; others referto all objects or do not apply to any objects at all. Let us consider the indexset P to show what we mean.

Each variable or constraint in the model with a particular index, say p,relates to the single product p in the underlying system. For instance, thevariables make pft (for all f and t) represent the amount of product p madeat the different facilities in the different time periods. Constraints MxSell pt(for all t) limit the amount of product p sold in the different time periods.

Other entities do not apply to a single product p. The constraintsMxMakeft (for all f and t) limit the total production of all products atfacilities f in periods t. The MxMStockft constraints do not refer directlyto any product p.


We can group together the variables in sets according to which product pthey relate to, with an additional set for variables that do not relate to anyone product in particular. We can do the same with the constraints. In thisway we partition the entities according to the structure in the underlyingsystem induced by the set of products. The sets of entities in the partitionsare listed in table 2.2.

If we examine the block structure induced by this partition, we see thatthe constraints in set p include only variables in the same block p. Con-straints in the global set refer to all variables. It is a primal block angularstructure.

Let us perform the same process using the set of time periods T . All ofthe variables and constraints are indexed by time period but the underlyingsystem clearly does not decompose totally by time period due to the materialand product stocks carried over from one period to the next. We couldchoose to regard the mass balance equations, PBal and MBal , as linkingthe time periods, since they include stock variables with indices t and t+ 1.As an alternative, let us choose the stock variables themselves as the linkingentities in the model. If the stocks at the end of each time period werefixed then the system would decompose into independent subsystems, onefor each time period. (A stock variable with index t represents the finalstock of period t− 1 and the initial stock of period t.)

The fact that the system decomposes by time period when the stocksare fixed is reflected in the model, which spits into separate sub-modelswhen the stock variables pstock and mstock are fixed. We can partition theremaining entities according to the time period to which they refer, with thestock variables forming a linking set of variables. The block structure is ofthe dual block angular form.

2.4 Identifying Block Structures in LP Problems

The example in the previous section showed how block structure can be in-ferred from the index sets of the algebraic model. When the same equationor set of equations is repeated with different index values, each time with anappropriate set of variables, this induces a structure in the model [FGSV00].We have seen how the system, and thus the LP model based on the system,splits into a number of loosely coupled components or subsystems corre-sponding to the chosen set of objects. Some activities and constraints relateto single objects, and thus are internal to one of the subsystems; others relateto all, several or none of the objects, and so are external to the individualsubsystems.

This approach for identifying block structure from the underlying setsin the system may be generalised. The entities are partitioned into sets ac-cording to which object or index they relate; entities that relate to no single

2.4. IDENTIFYING BLOCK STRUCTURES IN LP PROBLEMS 17

index in particular are placed in an auxiliary set. The pair of partitions—one for the variables and one for the constraints—induces a block structurein the constraint matrix as we saw earlier. The blocks corresponding tosingle-index entities will be independent of each other, while the blocks cor-responding to the auxiliary sets will link the model together. We refer tothe single-index blocks as subsystem blocks, as they result from subsystemswithin the system being modelled. The blocks corresponding to the auxiliarysets are the global or linking blocks.

Generally, the smaller the number of entities in the global blocks in aparticular instance of an LP problem, the more easily exploited by decompo-sition methods the problem instance will be, and the better the performancewe can expect from decomposition methods to solve the problem. The word‘instance’ is important in this statement—up until now our discussion ofblock structure in LP problems has referred only to the algebraic model.Indeed, when we have talked about non-zero elements in the constraint ma-trix, we have meant that the variable is included in the constraint in thealgebraic model, although in a specific data instance the coefficient mayin fact be zero. The size of the problem, and the actual numbers of vari-ables and constraints in each set, depends on the model parameters and thesize of the index sets in the model, which in general is an attribute of thedata instance. Later we see an example of a problem where different blockstructures resulted in better performance by Dantzig-Wolfe decompositiondepending on the size of the data instance (see section 5.9).

The procedure given above to identify block structure is far from beingwell-defined or automatable. One has to identify a set of subsystems onwhich to base the pair of partitions. We chose the set of products, and thenthe set of time periods above, but we might equally well have tried usingthe set of facilities or the set of materials. (See appendix D.2.3 for figuresillustrating a block structure based on the set of facilities.) Other commonsubsystems to look for are those based on a set of locations or a set ofdistinct processes. If there is some choice, as there was in the example, onecan try the different sets of subsystems in turn, perhaps revealing many blockstructures, which could be tested and evaluated by applying decompositionmethods to solve the problem using each of the block structures. Generally,the less interconnected the blocks are and the smaller the size of the globalblocks, the better the likely performance of decomposition methods. Thus,we look in preference for a set of subsystems that have few variables orconstraints connecting them. (Remember that it is the number of actualvariables and constraints in the data instance(s) that will matter, while thenumber of separate algebraic entities may be misleading.)

Given a block structure, one can derive further block structures by ag-gregating the blocks into fewer, larger blocks. This technique can be partic-ularly useful where it gives the opportunity to reduce the size of the globalblocks. Consider a dynamic model where each period is linked to the pre-


vious period, and suppose that by identifying linking constraints we haveobtained a primal block angular structure. If the sets representing periodswere aggregated into sets representing N consecutive periods, then N − 1of every N global constraints refer solely to variables within one aggregatedblock, and so can be transferred into the subsystem block.

One also has to decide which entities are to be classified as global orlinking. In the product based structure derived above, this was simply acase of identifying all entities that did not relate to any single product—there was no choice involved. However the time period based structureillustrated a choice of classifying the stock variables or the mass balanceequations as linking: we chose the variables, which resulted in a dual blockangular structure. If we had chosen the constraints, we would have obtaineda primal block angular structure. It was not necessary to remove both thevariables and constraints from the subsystem blocks: in the absence of eitherone, the model decomposed completely by time period into independent sub-models. There was in fact a third choice: we could have included both thestock variables and the balance equations in the single-period sets. As theeffect of the stock was to link each period with the previous period, thiswould have resulted in a block staircase structure with a time lag of one.

In general, there is a wide choice in assigning variables or constraints tothe blocks in a partition, and how that is done will determine the type ofblock structure. If the only global entities are variables, then a dual blockangular structure is obtained; if the only global entities are constraints, thena primal block angular structure is obtained; and where the global entitiescan be interpreted as linking only with previous sets, they can be included inthe subsystem blocks resulting in a block staircase or triangular structure.

Large, complex LP problems inevitably have some macro structure aris-ing from the natural divisions of the system being modelled into subsystems.Based on a set of subsystems, we can choose partitions of the variables and ofthe constraints and thus realise a block structure. How the set of subsystemsis chosen, and how we deal with entities that link different subsystems, is amatter of choice and judgement for which there are no hard and fast rules.It is impossible to say how well a decomposition method may perform usingone block structure in a problem compared with using another. Crucially,the performance of a decomposition method depends on the interaction be-tween the block structure and the data instance. Where the choice is open,it will usually be worth picking out several block structures with which toexperiment. Finally we should emphasise that despite the lack of concreteprocedures, it is not difficult to pick out block structures from the algebraicdescription.

Modelling support tools can help to make experimentation with differentblock structures and data sets in a model easier and more automatic. Wediscuss the development and use of one such tool, DecMod, in chapter 6.

2.5. OTHER METHODS OF IDENTIFYING BLOCK STRUCTURE 19

2.5 Other Methods of Identifying Block Structure

In the previous section we abstracted some general principles and guidelinesfor identifying block structure in LP problems by considering the algebraicformulation. The block structures obtained were all derived from a set ofsubsystems in the real world system, represented in the model by an indexset. In this section we discuss two alternative methods of identifying blockstructure and compare them to our algebraic approach.

One superficially attractive and simple method is to study a plot ofthe non-zero elements in the constraint matrix of a problem instance. Thefigures that accompany section 2.2 illustrate the different block structureswith stylised plots that show clearly the particular block structure, but donot correspond to plots of practical problem instances. It might be supposedthat it would be easy to identify a block structure from a constraint matrixof an arbitrary problem instance, but this is not the case for two reasons.

The rows and columns almost invariably do need permuting in orderto realise a block structure. Natural formulations of LP problems groupthe entities by function rather than by index, i.e., all variables associatedwith production for all products occur together, rather than all variables(production, storage, sale) associated with a particular product. Softwaremodelling tools replicate this order when generating matrix files [FGSV00].

Even with a relatively small problem instance it is impossible to pickout a block structure visually from a plot of the non-zero elements, whenthe rows and columns are not arranged to expose it, as the figures in [FH98,FGS01, Lus89] illustrate. The visual method is impractical for anythingother than the most trivial problem instance.

A second approach is to pose the problem of identifying block structureas an optimisation problem in itself, and use optimisation software to iden-tify a structure. Various studies have been conducted into this area butwith limited success [BFM97, BFM98, FH98, Gri80, KL70, WK71]. In gen-eral, the optimisation problems that have been formulated to represent theblock structure identification problem are among the most difficult class ofoptimisation problems.2

One argument against such an approach is that any potential benefitobtained by using a decomposition method to solve the original problemis more than outweighed by the extra effort required to identify the blockstructure in the first place. This argument loses some of its weight if thesame problem is to be solved many times with different data sets, as thestructure obtained can be used for different data sets (provided they are thesame size), but the longer it takes to solve the block structure problem, thegreater the gain needed to compensate in using decomposition methods tosolve the original problem.

2NP -hard in fact.


A further difficulty is that if the structure obtained does not correspondto an underlying structure in the algebraic model, then it will not scale andwill be inapplicable to the same model with different sized data sets. It isour opinion and that of others [Lou97] that the criteria for a good blockstructure are too ill-defined for the result of a block structure optimisationproblem to be useful. Criteria such as minimising the size of the globalblocks or maximising the number of subsystem blocks are used, but theseare poor proxies for the ultimate objective of a block structure that resultsin a good performance of a decomposition method

It is our belief that the best way to obtain a block structure for an LPmodel for use with a decomposition method is to study the algebraic formu-lation and generate block structures from the index sets of the model. Witha little practice it is not difficult, and with good modelling support toolsone can try a number of alternatives to find one that works well. Structuresthat rely on the algebraic formulation, rather than a specific data instance,have the key advantage of being scalable, that is, applicable to any datainstance of the problem—an advantage which cannot be underestimated inindustrial applications.

2.6 Summary

In this chapter we have discussed block structures in practical LP problems.The existence and process of identifying such block structures is a prereq-uisite for decomposition methods to be considered as practical optimisationtechniques. We have described a number of block structures, shown how theyare defined by a partition of the entities, suggested how such structures arisein practical problems, and discussed a number of ways to identify them. Inthe next chapter we describe Dantzig-Wolfe decomposition that allows us toexploit the primal block angular structure.

Identifying a block structure is the first step towards applying decompo-sition methods to an LP problem. It will always be the step which requiresthe most involvement from the user or practitioner, as success at identify-ing block structures is essentially a mixture of practical experience and trialand error. In chapter 6 we describe a software tool developed as part ofthis research to help extract a given block structure from an LP problem,leaving the user free to concentrate on the intellectual work of identifyingblock structures.

Chapter 3

Dantzig-WolfeDecomposition

In this chapter we set out the Dantzig-Wolfe decomposition method whichis the theoretical foundation of our practical work. We provide a construc-tive description of Dantzig-Wolfe decomposition, emphasising its two prin-ciple components: the transformation of the original problem to the masterproblem; and the solution of the master problem by the column generationalgorithm. We develop a complete theory that allows us to make accuratestatements about the algorithm and solutions obtained, but without impos-ing unnecessary simplifications which would restrict the choices available ina computational implementation. We include examples to give added insightinto the transformations described.

The first section of this chapter revises the terminology and basic theoryof linear algebra and linear programming. The second section contains ageneral overview of Dantzig-Wolfe decomposition to act as motivation forthe theory that follows. Then we present the first major aspect of Dantzig-Wolfe decomposition: the transformation of the original problem into themaster problem. We prove Minkowski’s Representation Theorem on whichthe transformation rests and show how the result can be applied to a primalblock angular structured problem in order to unlock the subsystem blocksof the structure. By formulating the transformation as a mapping, we areable to show how the master problem is equivalent to the original problem,but also how its polyhedral structure differs.

Section 3.4 considers the second major aspect of Dantzig-Wolfe decom-position: the column generation algorithm. Following an overview of thealgorithm, we study its three components in depth: the restricted masterproblem, the pricing problems, and the dual bound. We give a formal de-scription of the algorithm, which encapsulates the algorithm encoded in ourcomputational implementation, and discuss convergence of the algorithm.

Despite the invention and subsequent work on Dantzig-Wolfe decompo-

21

22 CHAPTER 3. DANTZIG-WOLFE DECOMPOSITION

sition over the past forty years, we have several contributions to make inthis chapter. We emphasise the crucial distinction between the transforma-tion from the original problem to the master problem, and the algorithmused to solve the master problem. We make the mapping from the originalproblem to the master problem explicit, allowing us to state how the var-ious solutions (bounded optimal point, unbounded direction, certificate ofinfeasibility) obtained from the algorithm relate to the original problem. Wegive an explicit example of the structure of the master problem, and explainwhat implications the structure of the master problem has on the solutionalgorithm. We present a flexible column generation algorithm, showing howthe ‘computational strategies’ of previous work fit neatly into the frameworkof the algorithm as resolutions for open choices evident rather than ad hoclater additions.

3.1 Basic Theory

We start by briefly reviewing some basic definitions and theory from linearalgebra and linear programming, in preparation for Minkowski’s Represen-tation Theorem and the formulation of the master problem in section 3.3.For a full treatment including the missing proofs, see, e.g., [NW88, Sch86].

3.1.1 Polyhedra, Extreme Points and Extreme Rays

In LP problems we deal with feasible sets in Rn defined by a set of linearinequalities or equalities. Such sets are called polyhedra.

Definition 1 A polyhedron P ⊆ Rn is the set of points that satisfy a finitenumber (m) of linear constraints; i.e., P = {x ∈ Rn : Ax ./ b}, where A isan m× n matrix called the constraint matrix.

Recall (from chapter 1) that we use the symbol ‘./’ as an arbitrary vectorof constraint symbols ‘≤’, ‘=’ and ‘≥’.

Definition 2 A set S ⊆ Rn is convex if, for any x, y ∈ S and 0 ≤ λ ≤ 1,λx+ (1− λ)y ∈ S.

Proposition 1 A polyhedron is a convex set. �

Definition 3 A point x ∈ Rn is a convex combination of a set S ⊆ Rn ifx can be expressed as x =

∑i λix

i for a finite subset {xi} of S and λ ≥ 0with

∑i λi = 1.

Definition 4 A point x ∈ P is an extreme point of P if there do not existtwo distinct points x1 and x2 ∈ P such that x = 1

2x1 + 1

2x2.

3.1. BASIC THEORY 23

max minxj ≥ 0 uA·j ≥ cjxj ≤ 0 uA·j ≤ cjxj = 0 uA·j freexj free uA·j = cjAi·x ≤ bi ui ≥ 0Ai·x ≥ bi ui ≤ 0Ai·x = bi ui freeAi·x free ui = 0

Table 3.1: The correspondence between primal bounds and dual constraints,and primal constraints and dual bounds. When the primal problem is amaximisation problem and the dual problem is a minimisation problem, xrepresents the primal variables and u the dual variables; and vice versa.

Equivalently, x ∈ P is an extreme point of P if x is not a convex combinationof P , excluding the trivial convex combination of x itself. The extreme pointsof a polyhedron P are in fact the zero dimensional faces—the vertices—ofP [NW88, proposition 4.2].

Definition 5 Given a non-empty polyhedron P = {x ∈ Rn : Ax ./ b}, avector r ∈ Rn , r 6= 0, is a ray of P if Ar ./ 0. Two rays r and s of P aredistinct if there is no constant λ > 0 such that s = λr; if s = λr for someλ > 0 the s is the same ray as r.

Definition 6 A ray r of P is an extreme ray of P if there do not exist twodistinct rays r1 and r2 of P such that r = 1

2r1 + 1

2r2.

The extreme rays of P are closely related to the unbounded one dimensionalfaces of P : r is an extreme ray of P if and only if {x+ λr : λ ≥ 0} is a onedimensional face of P for some extreme point x ∈ P [NW88, proposition4.3].

Proposition 2 A polyhedron has a finite number of extreme points andextreme rays.

Proof. See [NW88, corollary 4.4]. �

3.1.2 Linear Programming Duality

We defined the LP problem in chapter 1:

z∗ = max{cx : Ax ./ b, x ∈ Rn3}. (3.1)


Problem (3.1) is closely related to another LP problem, its dual problem:

w∗ = min{ub : uA ./′ c, u ∈ Rm3′}. (3.2)

The constraints ./′ and bounds 3′ of the dual problem are ‘inverted’ fromthose of the primal problem (3.1) in the sense defined by table 3.1 (p. 23).

Proposition 3 (Weak Duality) If x is primal feasible and u is dual fea-sible, cx ≤ z∗ ≤ w∗ ≤ ub.

Proof. Observe that the correspondence between primal and dual bounds andconstraints given in table 3.1 implies that for any primal feasible x and dual feasibleu, both (c − uA)x ≤ 0 and u(b − Ax) ≥ 0. Hence cx ≤ uAx ≤ ub and the resultfollows. �

Corollary 4 If the primal problem is unbounded then the dual problem isinfeasible. Similarly, if the dual problem is unbounded then the primal prob-lem is infeasible. �

The converse is not true; both the primal and dual problems may be infea-sible.

Theorem 5 (Strong Duality [GKT51]) If either the primal or dual prob-lem is feasible and bounded, the primal and dual problems are both feasibleand bounded with the same optimal value z∗ = w∗.

Corollary 6 Given a primal and dual pair of problems, exactly one of thefollowing four cases must hold:

1. Both the primal and dual problems are feasible and bounded with thesame optimal value.

2. The primal problem is feasible and unbounded, and the dual problemis infeasible.

3. The primal problem is infeasible, and the dual problem is feasible andunbounded.

4. Both the primal and dual problems are infeasible. �

Theorem 7 (Farkas’ Lemma) A polyhedron P = {x ∈ Rn3 : Ax ./ b} iseither non-empty or there exists u ∈ Rm

3′ such that ub < 0 and uA ./′ 0,where Rm

3′ and ./′ are the dual opposites of the primal attributes ./ and Rn3given by table 3.1.

3.2. OVERVIEW OF DANTZIG-WOLFE DECOMPOSITION 25

Proof. Consider the LP problem z = max{0x : Ax ./ b, x ∈ Rn3} and its dualw = min{ub : uA ./′ 0, u ∈ Rm3′}. The dual has a solution u = 0 so it is certainlyfeasible, and thus only of cases 1 and 3 of corollary 6 may occur. Either the primalproblem is feasible and so P is non-empty, or the dual problem is unbounded whichimplies the existence of u. �

Definition 7 If an LP problem is infeasible, i.e., its feasible region P isempty, we say a vector u satisfying the hypothesis of Farkas’ Lemma is acertificate of infeasibility for the LP problem.

3.2 Overview of Dantzig-Wolfe Decomposition

Dantzig-Wolfe decomposition is a method for solving LP problems witha pre-determined primal block angular structure (see chapter 2) [DW60,DW61]. The original problem is split up into a set of pricing problems—one for each of the subsystem blocks in the block structure—and a masterproblem, which co-ordinates the pricing problems and ensures that the globalconstraints are satisfied. The master problem is equivalent to the originalproblem in some respects, and by solving the master problem we obtain asolution to the original problem.

The master problem has a large number of variables—many more thanin the original problem. They are defined implicitly by the set of basicfeasible solutions and unbounded directions of the pricing problems. Ratherthan enumerating all of the variables for the master problem ab initio, aspecialised variant of the simplex algorithm is employed which operates witha small subset of the variables at any one time. The restricted masterproblem is optimised over the current active set of variables; new variableswhich price out suitably to enter the basis are added; and this procedurecontinues until no new variables which price out to enter the basis can befound. As in the standard simplex algorithm, this signifies that an optimalsolution has been found. In general an optimal solution will be found withouthaving to generate explicitly all of the variables of the master problem.

How are the variables which price out to enter the current basis of the re-stricted master problem generated? By defining a suitable objective functionfor each of the pricing problems, based on the dual values of the restrictedmaster problem, we can contrive for the objective value at each extremepoint to be the reduced cost—the price—of the master problem variableassociated with the extreme point. Solving the modified pricing problemsgenerates basic feasible solutions of maximum reduced cost (for maximisa-tion problems). If the objective value at an extreme point is positive, thenthe associated master problem variable is added to the master problem; ifthe minimum objective value over all extreme points is negative, then no


master problem variable exists to improve the current master problem solu-tion.

The computational advantage of Dantzig-Wolfe decomposition arises fromperforming a significant amount of the computational work in the pricingproblems. Based on the subsystem blocks of the original coefficient matrix,they are an order of magnitude smaller than the original problem, and thusin general easier to solve. The pricing problems are also independent of oneanother and thus may be solved concurrently on suitable hardware. Againstthis must be set the potentially huge size of the master problem, with manymore variables—though fewer constraints—than the original problem. Aswe shall show, the feasible region of the master problem is more complexthan that of the original problem and the solution path may be longer. Nu-merical problems may result from dynamically generating the variables ofthe master problem. Thus there are many factors which may influence theperformance of the algorithm and a true evaluation can only be based on athorough practical investigation, the subject of chapter 5.

3.3 The Master Problem

The solution algorithm in Dantzig-Wolfe decomposition solves a reformula-tion of the original problem: the master problem. The master problem isobtained from the original problem by replacing linear constraint represen-tation of the feasible region by one using convex combinations of its extremepoints and rays. In this section we prove Minkowski’s Representation Theo-rem on which the new representation relies, show how it may be applied toa primal block angular LP problem, and examine how close the associationis between the original problem and the master problem.

3.3.1 Minkowski’s Representation Theorem

This result shows that a polyhedron P may be described in terms of itsextreme points and rays alone. It allows us to represent P by an alternativepolyhedron PMk coupled with a mapping from PMk to P .

First we introduce a general hypothesis of this chapter.

Definition 8 The rank of an m × n matrix A is the maximum number oflinearly independent columns of A when viewed as vectors in Rm.

The rank of a matrix can be equivalently defined as the maximum numberof linearly independent rows of A when viewed as vectors in Rn . Whenrank(A) = n, i.e., all columns of A are linearly independent, we say that Ais of full column rank. Intuitively, it means that the polyhedron {x ∈ Rn :Ax ./ 0} contains no vector subspaces, no ‘full lines’.

3.3. THE MASTER PROBLEM 27

The results of this section 3.3 assume that various subsystem blocks ofthe constraint matrix are of full column rank. When checking this hypothesisfor an LP problem, we implicitly include the bounds in the constraint matrix,although we continue to treat them separately in our notation. In practice,provided each variable is bounded above or below, the hypothesis will besatisfied. Where there are free variables, a simple transformation due toGeoffrion can be used to remove them [Geo70, p. 663].

The next result shows that, when looking for optimal solutions to anLP problem, we need only consider extreme points and rays of its feasibleregion.

Theorem 8 Consider a feasible LP problem max{cx : x ∈ P} with rank(A) =n. If it is bounded, it has an optimal solution at an extreme point x∗ of P ;if it is unbounded, it is unbounded along an extreme ray r∗ of P (and socr∗ > 0).

Proof. See [NW88, theorems 4.5 and 4.7]. �

Theorem 9 (Minkowski’s Representation Theorem) If P = {x ∈ Rn :Ax ./ b} is non-empty and rank(A) = n, then P = P ′, where

P ′ :=

x ∈ Rn : x =∑i∈Π

λixi +∑j∈R

µjrj ;∑i∈Π

λi = 1;λ, µ ≥ 0

and {xi}i∈Π and {rj}j∈R are the sets of extreme points and extreme rays ofP respectively.

Proof. [NW88, theorem 4.8.] Since xi ∈ P for i ∈ Π and P is convex, x′ =∑i∈Π λix

i ∈ P for all λ satisfying the conditions. Since the rj are rays of P ,x′′ = x′ +

∑j∈R µjr

j ∈ P for all µ satisfying the conditions. Hence P ′ ⊆ P .Now suppose P ′ P , i.e., there exists a y ∈ P but y 6∈ P ′. So there do not

exist λ, µ ≥ 0 such that ∑λix

i +∑µjr

j = y−∑λi = −1.

By Farkas’ Lemma there exists (π, π0) ∈ Rn+1 such that

πy − π0 < 0πxi − π0 ≥ 0πrj ≥ 0

(3.3)

Consider the LP problem min{πx : x ∈ P}, feasible by hypothesis. If it werebounded, by theorem 8 the optimal solution would be obtained at an extreme pointxi∗

of P for some i∗ ∈ Π—but y ∈ P and πy < π0 ≤ πxi for all extreme points xi

of P . If on the other hand it were unbounded, the same theorem shows that there


Figure 3.1: A simple bounded polyhedron.

would exist an extreme ray rj∗

with πrj∗< 0—but πrj ≥ 0 for all extreme rays rj

of P . Therefore the original assumption was false and P = P ′. �

Henceforth, to reduce notational clutter, we shall use xi to representboth extreme points and extreme rays, where i ranges over E := Π∪R. Weshall use λi for the coefficient of xi in Minkowski representations, and writethe convexity constraint as

∑i δiλi = 1, where the constant δi is one if xi is

an extreme point, and zero if xi is an extreme ray.The Minkowski Representation Theorem allows us to represent a given

polyhedron P by a second polyhedron

PMk =

{λ ∈ R|E| :

∑i∈E

δiλi = 1;λ ≥ 0

},

and the projection

πMk : R|E| → Rn;λ 7→ x =

∑i

λixi.

Given a point λ ∈ PMk , we can obtain the point x ∈ P that it repre-sents by applying the projection x = πMk (λ). We call the polyhedron PMk

the Minkowski Representation of P , and the mapping πMk the MinkowskiProjection of P . In general the projection is non-invertible, meaning thatseveral points λ may represent the same point x; we illustrate this below.Using this new machinery, Minkowski’s Representation Theorem states thatπMk (PMk ) = P .

3.3.2 Examples

To illustrate Minkowski’s Representation Theorem, we present two specimenexamples. The first consists of the two dimensional bounded polyhedron P1


depicted in figure 3.1. It can be described conventionally using two non-negative variables and four constraints as:

P1 =

(xy

)≥ 0 :

−2x + y ≤ 2 (A)−8x + 11y ≤ 36 (B)

4x + 5y ≤ 108 (C)2x − y ≤ 26 (D)

.

Minkowski’s Representation Theorem tells us that the polyhedron can beexpressed as the set of convex combinations of its extreme points and rays.In this case, since P1 is bounded, it has no extreme rays. Its MinkowskiRepresentation consists of a variable λi for each of its six extreme points,

PMk1 =

{λ ∈ R6 :

6∑i=1

λi = 1;λ ≥ 0

},

with Minkowski Projection

πMk1 : R6 → R

2;

(xy

)= λ1

(00

)+ λ2

(02

)+ λ3

(14

)+ λ4

(1212

)+ λ5

(178

)+ λ6

(130

).

For instance, the point (14, 6) can be represented by

λ =(0, 0, 0, 8

28 ,928 ,

1128

)since (

146

)=

828

(1212

)+

928

(178

)+

1128

(130

).

In fact, the point (14, 6) can also be represented by

λ′ =(

213 , 0, 0, 0, 3

4 ,552

).

This demonstrates that the Minkowski Representation is degenerate. Weshall return to the issue of degeneracy and its consequences in section 3.3.5.

Our second example consists of an unbounded polyhedron, and thusillustrates the use of extreme rays. Consider the polyhedron P2 shown infigure 3.2; described by the two variables and three constraints as:

P2 =

(xy

)≥ 0 :

−2x + y ≤ 3 (A)x − 2y ≤ 2 (B)x − y ≤ 4 (C)

.


Figure 3.2: A simple unbounded polyhedron.

The polyhedron has four extreme points and two extreme rays, and has theMinkowski Representation

PMk2 =

{λ ∈ R6 :

4∑i=1

λi = 1;λ ≥ 0

}

πMk2 : R6 → R

2;

(xy

)= λ1

(00

)+ λ2

(03

)+ λ3

(20

)+ λ4

(51

)+ λ5

(12

)+ λ6

(11

).

For instance, the point (6, 5) can be represented by

λ =(0, 0, 2

5 ,35 ,

115 , 0

).

3.3.3 Application to Primal Block Angular LP Problems

Given an LP problem, we may replace the conventional description of itsfeasible region by its Minkowski Representation, solve the reformulated LP,and project the λ∗ solution obtained onto an x∗ solution using the Min-kowski Projection. This may seem slightly perverse—in the two examplespresented above, we replaced two variables x and y by a six variable vectorλ, and introduced additional degeneracy. However, in doing so we replacedall the constraints by a single constraint: the convexity constraint on λ. InDantzig-Wolfe decomposition we exploit this property to remove the con-straints and variables associated with the subsystem blocks in primal blockangular structured LP problems. Later, in section 3.4, we present a spe-cialised algorithm for solving the resulting master LP problem that obviatesthe need to handle all of the additional variables.


It is by reformulating the problem that we unlock the subsystem blocksfrom the global blocks and are thus able to exploit the block structure ina solution algorithm by optimising over each subsystem block separately.Rather than simply taking the Minkowski Representation of the completeproblem, we treat the global constraints and the subsystem constraints sep-arately, and take the Minkowski Representation of the polyhedra based oneach of the subsystem blocks of the problem. In some respects the end re-sult is the same; we end up with a second polyhedron PDW and a projectionπMk such that πMk (PDW ) = P , which allows us to solve the LP over PDW

and project the result onto P . But there is an important difference; theglobal constraints are not eliminated but lifted up, along with the objectivefunction, to the master problem, of which PDW is the feasible region. Wenow describe the formulation of the master problem.

We introduced the primal block angular structure in chapter 2. Consideran LP problem—the original problem—

max{cx : Ax ./ b, x ∈ Rn3}. (3.4)

for which the coefficient matrix A has a primal block angular structure.Let P be its feasible region—a polyhedron. Ordering the variables andconstraints so that the blocks are unified, we can display the block structure

P =

x0 ∈ Rn0

30

x1 ∈ Rn131

...xK ∈ RnK3K

:

A0 A1 · · · AK

B1

. . .BK

x0

x1...xK

./0

./1...

./K

b0b1...bK

.

We decompose the polyhedron into polyhedra of smaller dimension basedon each subsystem block Bk: define Pk := {xk ∈ Rnk3k : Bkxk ./k bk}, k =1, . . . ,K. Also, to hold the global constraints of the problem, define P0 :={x ∈ Rn3 :

(A0 A1 · · · AK

)x ./0 b0}. Then

P = P0

⋂(Rn0 × P1 × · · · × PK) ,

illustrating the fact that, were it not for the global constraints, the feasibleregion would decompose into a number of smaller independent polyhedra.

Taking the Minkowski Representation of each of the polyhedra Pk, k =1, . . . ,K, assuming rank(Bk) = nk,1 we get

PMkk =

λk ∈ R|Ek|+ :∑ik∈Ek

(δk)ik(λk)ik = 1

,

1Recall we implicitly include the simple bounds xk ∈ Rnk3k in the constraints whenchecking this assumption.


with associated projection

πMkk : R|Ek| → R

nk ;λk 7→ xk =∑ik∈Ek

(λk)ikxikk .

By Minkowski’s Representation Theorem, πMkk (PMk

k ) = Pk.Let us write

P := Rn0 × P1 × · · · × PK

and, combining the Minkowski Representations PMkk ,

PMk := Rn0 × PMk

1 × · · · × PMkK .

In tandem, we define a combined Minkowski Projection,

πMk : Rn0 × R|E1|+···+|EK | → Rn; (x0, λ1, . . . , λK) 7→ (x0, x1, . . . , xK),

given by xk = πMkk (λk). We have πMk (PMk ) = P .

Next we lift up the global constraints into this new space PMk ; let P0 bethe pre-image of P0 under πMk :

P0 :={

(x0, λ) ∈ Rn030× R|E1|+···+|EK | : πMk (x0, λ) ∈ P0

}.

and define

PDW := P0

⋂PMk .

PDW is the feasible region of the Dantzig-Wolfe master problem, which,through πMk , projects onto P , the feasible region of the original problem.

Theorem 10 (Dantzig-Wolfe Representation) Let P be defined as above,with rank(Bk) = nk for each k = 1, . . . ,K. Then

πMk (PDW ) = P,

where PDW and πMk are defined above. �

To complete the formulation of the master problem, we just need tolift up the objective function so that it is invariant under the projection.The objective coefficient of a variable (λk)ik is simply ckx

ikk , where xikk is the

extreme point or ray associated with (λk)ik and ck is its objective coefficient.Let us define (ck)ik := ckx

ikk ; also define (Ak)ik := Akx

ikk .

Thus the master problem may be expressed in full as

max c0x0 +∑

i1(c1)i1(λ1)i1 + · · · +

∑iK

(cK)iK (λK)iKs.t. A0x0 +

∑i1

(A1)i1(λ1)i1 + · · · +∑

iK(AK)iK (λK)iK ./0 b0∑

i1(δ1)i1(λ1)i1 = 1

. . ....

...∑iK

(δK)iK (λK)iK = 1x0 ∈ Rn0

30;λ1 ∈ R|E1|

+ , . . . , λK ∈ R|EK |+

;


or more simply

max c0x0 + c1λ1 + · · ·+ cKλKs.t. A0x0 + A1λ1 + · · ·+ AKλK ./0 b0

δ1λ1 = 1. . .

......

δKλK = 1x0 ∈ Rn0

30;λ1 ∈ R|E1|

+ , . . . , λK ∈ R|EK |+

;

or even max c0x0 + cλs.t. A0x0 + Aλ ./0 b0

∆λ = 1x0 ∈ Rn0

30;λ ∈ R|E1|+···+|EK |

+

, (3.5)

where ∆ := diag(δ1, . . . , δK).

3.3.4 The Correspondence between Solutions of the MasterProblem and Solutions of the Original Problem

In this section we examine the correspondence between the master prob-lem (3.5), with feasible region PDW , and the original problem (3.4), withfeasible region P . We demonstrate that

in the master problem in the original problema primal feasible point ⇔ a primal feasible point

an optimal primal point ⇔ an optimal primal pointan unbounded ray ⇔ an unbounded ray

a dual feasible point ⇔ a dual feasible pointan optimal dual point ⇔ an optimal dual point

a certificate of infeasibility ⇔ a certificate of infeasibility

under the Minkowski mapping. Thus the master problem and original prob-lem are equivalent, in the sense that a solution to one implies an equivalentsolution to the other, where solution is taken in its broadest sense to encom-pass a primal or dual point or ray.

Throughout this section, x is a point in Rn (where n = n0+n1+· · ·+nK),(x0, λ) is a point in Rn0×R|E1|+···+|EK |, (π, ν) is a point in Rm0×Rm1+···+mK ,and (π, µ) is a point in Rm0×RK . We shall write πMk (x0, λ) for πMk ((x0, λ)).

First, we consider primal solutions.

Proposition 11 If (x0, λ) is primal feasible for the master problem, thenx = πMk (x0, λ) is primal feasible for the original problem and has the sameobjective value. Conversely, if x is primal feasible for the original problem,then there exists (x0, λ) ∈ (πMk )−1(x) that is primal feasible for the masterproblem and has the same objective value.


Proof. A restatement of theorem 10, P = πMk (PDW ). The correspondence ofobjective values is immediate from the invariance of the objective function under theprojection πMk : if πMk (x0, λ) = x then c0x0 + cλ = c0x0 +

∑k ck

∑ik

(λk)ikxikk =

c0x0 +∑k ckxk = cx. �

Corollary 12 If (x0, λ) is primal feasible and optimal for the master prob-lem, then x = πMk (x0, λ) is primal feasible and optimal for the originalproblem. Conversely, if x is primal feasible and optimal for the originalproblem, then there exists (x0, λ) ∈ (πMk )−1(x) that is primal feasible andoptimal for the master problem. �

Proposition 13 If (x0, λ) is a ray of the master problem, then x = πMk (x0, λ)is a ray of the original problem. Conversely, if x is a ray of the original prob-lem, then there exists a ray of the master problem (x0, λ) ∈ (πMk )−1(x). Theobjective function coefficients of the rays have the same sign.

Proof. Suppose first that (x0, λ) is a ray of the master problem—i.e.,

A0x0 + Aλ = 0 (3.6)∆λ = 0 (3.7)

and (x0, λ) 6= 0. Let x = πMk (x0, λ). Expanding Aλ in (3.6) gives A0x0 +A1x1 + ... + AKxK = 0. From (3.7) we deduce that all non-zero elements of λcorrespond to extreme rays xikk of Pk, k = 1, . . . ,K, with Bkx

ikk = 0; and so

Bkxk =∑ik

(λk)ikBkxikk = 0. Then, once we show x 6= 0, we will have shown that

x is an extreme ray of the original problem.Since (x0, λ) 6= 0, either x0 6= 0 or ∃(λk)i 6= 0. In the former case x is certainly

non-zero. In the latter case suppose xk =∑ik

(λk)ikxikk = 0, where all non-zero

terms of the sum are extreme rays of Pk. Given one term of the sum is non-zero,there must be at least two non-zero terms, i.e.,

(λk)ikxikk + x′k = 0,

where xikk is an extreme ray of Pk and x′k is linear sum of extreme rays of Pk, andthus is itself a ray of Pk. Thus −xikk = x′k/λ is a ray of Pk, and so the linearsubspace {µxikk : µ ∈ R} ⊆ Pk, contradicting the hypothesis rank(Bk) = nk. Thusxk 6= 0 and therefore x 6= 0.

Conversely, suppose x is a ray of the original problem. Then A0x0 + · · · +AKxK = 0; Bkxk = 0 for each k = 1, . . . ,K; and x 6= 0. In particular, for eachk = 1, . . . ,K, if xk 6= 0, xk is a ray of Pk, and by Minkowski’s RepresentationTheorem, xk is a linear combination of extreme rays of Pk, xk =

∑ik

(λk)ikxikk for

some λk ≥ 0 with δkλk = 0. If xk = 0, let λk = 0.Consider the point (x0, λ) ∈ Rn0

30×R|E1|+···+|EK |. By construction πMk (x0, λ) =

x; clearly (x0, λ) 6= 0 (if it did, x = πMk (0) = 0 but x 6= 0); and it is easy to showthat (3.6) and (3.7) hold. Thus (x0, λ) is a ray of the master problem.

The correspondence of signs of the objective values is immediate from the invari-ance of the objective function under the Minkowski projection, as in proposition 11;


we cannot however equate the objective values since all (positive) multiples of a rayare identified. �

We now turn to dual solutions. The dual of the original problem is

min πb0 + ν1b1 + · · ·+ νkbks.t. πA0 ./′0 c0

πA1 + ν1B1 ./′1 c1...

. . ....

...πAK + νKBK ./′K cK

πRm0

3′0; νk ∈ Rmk3′k

, k = 1, . . . ,K

. (3.8)

The dual of the master problem ismin πb0 + µ1s.t. πA0 ./′0 c0

πA + µ∆ ≥ cπ ∈ Rm0

3′0;µ ∈ RK

. (3.9)

Note that the second set of constraints is equivalent to{(ck − πAk)xikk ≤ µk ∀ik ∈ Πk

(ck − πAk)xikk ≤ 0 ∀ik ∈ Rk

}k = 1, . . . ,K. (3.10)

Given a dual solution (π, ν) to the original problem, an obvious corre-sponding dual solution to the master problem is (π, µ), where µk := νkbk.Going the other way is less obvious, but we can obtain a valid ν given (π, µ)such that (π, ν) is a dual solution to the original problem by means of theLP problems

z∗k = max{(ck − πAk)xk : Bkxk ./k bk, xk ∈ Rnk3k}, (3.11)

k = 1, . . . ,K. Observe that the feasible region of problem k is Pk, andthat provided Pk is non-empty, problem k is feasible with extreme pointsand rays {xikk }ik∈Ek . It is clear that solving problem k will either provethat conditions (3.10) hold, or identify an extreme point or ray that violatesthem. Thus it is not surprising that their dual solutions provide the νk we areseeking. These problems are called pricing problems and play an importantrole in the column generation algorithm described below (section 3.4).

Proposition 14 If (π, ν) is dual feasible for the original problem, then(π, µ) is dual feasible for the master problem, where µk := νkbk. Conversely,if (π, µ) is dual feasible for the master problem, then (π, ν) is dual feasible forthe original problem, where νk is a dual solution to pricing problem k (3.11).


Proof. The proof is simply a case of checking the dual feasibility conditions.Suppose first that we are given (π, ν) and we wish to obtain a dual solution (π, µ)to the master problem. We define µk := νkbk and check that it satisfies masterproblem dual feasibility (3.9). Using the property noted in the proof of weak duality(proposition 3, p. 24), we have, for all primal feasible points xk, (i) (ck − (πAk +νkBk))xk ≤ 0; and (ii) νk(bk −Bkxk) ≥ 0. Thus

µk := νkbk ≥ νkBkxk ≥ (ck − πAk)xk

for all extreme points {xikk }ik∈Πk of Pk. Similarly, for all primal rays xk, (i) holdsand in addition Bkxk = 0. Thus

0 = νkBkxk ≥ (ck − πAk)xk

for all extreme rays {xikk }ik∈Rk of Pk. This demonstrates that (3.10) holds; theremaining master problem dual feasibility conditions carry over from the originalproblem.

Conversely, suppose that we are given (π, µ) and we wish to obtain a dualsolution (π, ν) to the original problem. Consider the set of pricing problems (3.11).If problem k is feasible, then by (3.10) it is bounded with objective value ≤ µk, andby strong duality there exists a dual solution νk ∈ Rmk3′k

. If problem k is infeasible,then by Farkas’ Lemma there exists a certificate of infeasibility νk ∈ Rmk3′k

. Inboth cases νkBk ./′k ck − πAk. The remaining original problem dual feasibilityconditions carry over directly from the master problem, and (π, ν) is dual feasiblefor the original problem. �

Observe that, given a master problem dual solution (π, µ), the originalproblem dual solution (π, ν) obtained in the proposition above does notdepend on µ.

We can make further deductions about the dual objective value of thedual solutions obtained. Given (π, ν), a master problem dual solution (π, µ)is obtained that has the same dual objective value in the master problemas (π, ν) had in the original problem. Given (π, µ) however, at best wecan obtain an original problem dual solution (π, ν) that has a dual objectivevalue in the master problem no worse than (π, ν) had in the original problem.This follows from the second part of the proof above: if problem k is feasible,νkbk = z∗k ≤ µk; while if problem k is infeasible, νkbk < 0, and we can scaleνk to ensure that νkbk ≤ µk.

Corollary 15 If (π, µ) is dual feasible and optimal for the master problemthen (π, ν) obtained in proposition 14 is dual feasible and optimal for theoriginal problem. Conversely, if (π, ν) is dual feasible and optimal for theoriginal problem then (π, µ) obtained in proposition 14 is dual feasible andoptimal for the master problem.

Proof. This follows from fact that the dual objective value does not degrade whenapplying either procedure of proposition 14. �


Finally we consider certificates of infeasibility, which, as Farkas’ Lemmashows, are dual unbounded rays. Thus the definition of certificates of infea-sibility for the original and master problems follows the definitions of dualfeasible solutions for the original and master problems (see (3.8), (3.9) and(3.10)) but with all dual RHS vectors c replaced by zero. So it is not surpris-ing that we have an analogous result to proposition 14, which uses modifiedpricing problems (cf. equation 3.11) in which ck is replaced by zero:

zk = max{(−πAk)xk : Bkxk ./k bk, xk ∈ Rnk3k}. (3.12)

Proposition 16 If (π, ν) is a certificate of infeasibility for the original prob-lem, then (π, µ) is a certificate of infeasibility for the master problem, whereµk := νkbk. Conversely, if (π, µ) is a certificate of infeasibility for the masterproblem, then (π, ν) is a certificate of infeasibility for the original problem,where νk is a dual solution to the modified pricing problem k (3.12).

Proof. The proof proceeds as in proposition 14. The fact that the dual objectivevalue of the dual ray constructed is strictly less than zero follows from observationabove that the value of the dual solution constructed is no worse than the givendual solution, which satisfies a similar strict inequality by hypothesis. �

We have now demonstrated the equivalence of all types of solution be-tween the original and master problems as summarised in the introductionto this section. In particular, we have the corollary:

Corollary 17 The master problem (3.5) and the original problem (3.4) areequivalent in the following sense:

1. The master problem is primal feasible if and only if the original prob-lem is primal feasible.

2. The master problem is dual feasible if and only if the original problemis dual feasible.

3. The master problem is bounded and if and only if the original prob-lem is bounded; if the problems are bounded then the optimal objectivevalues co-incide. �

Furthermore, we have shown how a master problem primal solution(point or ray) can be converted into an equivalent original problem primalsolution by the Minkowski Projection, and a master problem dual solutioncan be converted into an equivalent original problem dual solution by solv-ing the pricing problems. Given an original problem primal solution, wecan show the existence of an equivalent master problem primal solutionbut constructing one is not easy. An original problem dual solution can beconverted into an equivalent master problem dual solution by simple pro-jection. There is no guarantee that the primal and dual solutions for the


Figure 3.3: The feasible region P of the original problem.

original problem, obtained from primal and dual master problem solutions,will be compatible in the sense of mutually satisfying the complementaryslackness conditions, even if the master problem solutions satisfy their owncomplementary slackness conditions.

The results of this section show that the master problem and orig-inal problem are theoretically equivalent for the purposes of a solutionalgorithm—in particular, we can solve the master problem as a means toobtaining a solution for the original problem. However, there are many fac-tors that might aid or inhibit this method in practice. Given that we cannoteasily construct a master problem primal solution from the equivalent orig-inal problem solution, there is no general way to force a solution algorithmapplied to the master problem to follow the same path as a solution algo-rithm applied to the original problem. Structural properties of the masterproblem may render it more difficult or more easy to solve than the originalproblem, and in the next section, we investigate the polyhedral structure ofthe master problem.

3.3.5 The Polyhedral Structure of the Master Problem

The mapping from the original problem to the master problem is many-to-one, a lifting, and introduces additional degeneracy. In this section weillustrate how this additional degeneracy may account for additional extremepoints, rendering the polyhedral structure of the master problem more com-plex than that of the original problem. We present a small geometric ex-ample of the transformation from an original problem to the correspondingmaster problem, which illustrates how additional extreme points are intro-duced and gives an insight into the polyhedral structure of Dantzig-Wolfemaster problems in general.

Consider the polyhedron P depicted in figure 3.3 which represents the


Figure 3.4: The feasible region P1 of the pricing problem.

feasible region of our original problem. The six constraints (faces) havebeen labelled A–D, P and Q, where constraint C is redundant. We formthe master problem by classifying constraints P and Q as global constraints.The remaining constraints A–D are assigned to a single subsystem block,and figure 3.4 depicts the feasible region of the resulting pricing problem.The four extreme points (vertices) are labelled 1–4, while the significanceof E and F will become clear in due course.

Our first step in forming the master problem is to obtain the Min-kowski Representation PMk

1 of P1. The four vertices 1–4 are mapped tofour non-negative variables λ1, . . . , λ4, constrained by a single convexityconstraint. PMk

1 is thus the upper quadrant of four dimensional space(λ ≥ 0) intersected with a three dimensional hyperplane (

∑i λi = 1), and

can be visualised as a three dimensional tetrahedron (see figure 3.5). Thefour vertices of P1 are represented by four vertices of PMk

1 , (λ1, . . . , λ4) =(1, 0, 0, 0), (0, 1, 0, 0), (0, 0, 1, 0) and (0, 0, 0, 1). We can also readily identifythe images of the four faces A–D of P1: for example, since A is defined bypoints 1 and 2 in P1, we can label its image A in PMk

1 —see figure 3.5.In this way we can associate four of the six edges of PMk

1 with faces ofP1. What are we to make of the remaining two edges E and F of PMk

1 ?Edge E joins points 1 and 3 of PMk

1 ; projecting it back onto P1 reveals thatit represents an internal face of P1 (see figure 3.4). Similarly, edge F linkspoints 2 and 4 and represents another internal face of P1.

On reflection it is clear that the Minkowski Representation QMk of anarbitrary polyhedron Q contains an edge for every pair of vertices of Q—sometimes corresponding to real faces of Q, but more often not. Further-more, QMk contains a face for every triple of vertices of Q. The effect ofthe Minkowski transformation is to inflate the polyhedron, pushing out thevertices into independent dimensions, and making all inter-vertex distancesuniform. Any original polyhedron with a given number of extreme points


Figure 3.5: The feasible region PMk1 of the Minkowski Representation of the

pricing problem.

results in the same Minkowski Representation.To complete the Dantzig-Wolfe transformation, we must lift up the global

constraints and superimpose them on PMk1 . Their position can be ascer-

tained by comparing PMk1 (figure 3.5) to the original problem P (figure 3.3):

constraint P cuts off points 3 and 4, intersects constraints B and D, andrenders constraint C redundant; constraint Q cuts off point 2 and intersectsconstraints A and B. Figure 3.6 shows the feasible region PDW obtained.It can be seen also how P intersects the internal face F , and Q intersectsboth internal faces E and F .

Comparing the feasible regions of the original problem and the masterproblem, we see that the master problem has eight vertices of which fivecorrespond to vertices of the original problem. The other three correspondto points of the original problem which are not vertices. (In fact they areinternal vertices that involve at least one internal face.)

In summary, the general effects of the Minkowski transformation are:

• to introduce additional extreme points and edges;

• to destroy non-adjacency—all pairs of the extreme points of the orig-inal problem are adjacent in the master problem;

• to destroy scale—all pairs of extreme points of the original problemare equidistant in the master problem.

The polyhedral structure of the master problem is evidently very differentfrom that of the original problem.

3.4. THE COLUMN GENERATION ALGORITHM 41

Figure 3.6: The feasible region PDW of the master problem.

The peculiar structure of the master problem has implications on theapplication of a solution algorithm to the master problem. The additionalextreme points mean that there are potentially more bases that a simplexbased procedure may visit in the process of finding an optimal solution.Conversely, the additional edges mean that a simplex based procedure couldtake a shorter path to an optimal solution. How important these two effectsare in practice is difficult to determine, and would require detailed analysisof the paths taken by specific solution methods, and on problem instances ofdimension too great to visualise. It is perhaps one explanation for differingperformance obtained by solving the master problem and solving the originalproblem.

3.4 The Column Generation Algorithm

We have now constructed the master problem and seen how a solution ofthe master problem—whether an optimal solution, an unbounded ray, or acertificate of infeasibility—leads easily to a solution of the original problem.We are in a position to consider the second major aspect of Dantzig-Wolfedecomposition: solving the master problem.

The master problem is an LP problem and so, in principle at least, canbe solved by any standard LP algorithm; e.g., simplex or interior pointmethods. This neglects the key property of the master problem: its hugenumber of variables, one for every extreme point and ray of the polyhedraPk. This is an obstacle, because to solve the master problem using standardmethods would entail enumerating all of its variables ab initio, which for anyreasonably sized practical problem would be both computationally expensive


and result in a huge master problem that would be difficult to solve inpractice; but also an opportunity, because an alternative solution methodpresents itself, tailored specifically to the master problem and which exploitsthe decomposition of the subsystem blocks.

In this final theoretical section we present the column generation algo-rithm which was first invented by Dantzig and Wolfe specifically to solvethe Dantzig-Wolfe master problem, although now has a wider applicabil-ity [BJN+94, Dan63, DW60, DW61]. We first give an overview of the al-gorithm to put the subsequent development into context. We study thethree components of the algorithm in greater detail—the restricted masterproblem, the pricing problems, and the dual bound. We then present thealgorithm itself, which encapsulates the operations of our computationalimplementation (chapter 5), and discuss convergence of the algorithm.

3.4.1 Overview

The column generation algorithm is a specialised simplex method in whichonly a small number of the non-basic variables are handled at any one time—the remaining variables are effectively fixed at zero. Whenever an optimalsolution is found over the current set of available variables, the set is up-dated: variables that are unlikely to enter the basis may be removed; whilenew variables with profitable reduced costs at the current basis are added.The master problem is then re-optimised and the process iterated. By ensur-ing that if there are any variables with profitable reduced cost, at least oneof them is added to the set of available variables, and taking any necessaryprecautions against cycling, the column generation algorithm is guaranteedto reach an optimal solution if one exists. The proof of optimality is justthat of the simplex algorithm: when no more variables with profitable re-duced cost can be found, all non-basic variables have unprofitable reducedcosts and the current solution cannot be improved.

As the variables of the master problem correspond to extreme points andrays of the polyhedra Pk, they can be generated by solving the K pricingproblems with feasible regions Pk, k = 1, . . . ,K. Using the dual variables ofthe master problem to define the objective function of the pricing problems,we can arrange for the objective value at an extreme point to equal thereduced cost of the corresponding master problem variable. Thus by solvingthe pricing problems, we effectively price out the master problem variables,identifying those with the more profitable reduced costs.

There is no need to enumerate the variables of the master problem fromthe start—we can just add variables and calculate their constraint coef-ficients when they are identified as having profitable reduced costs. Wetherefore work with a restricted master problem which has a dynamic set ofvariables altered at each iteration of the column generation algorithm. Thepricing problems not only identify which variables should be added to the


restricted master problem, but also generate the vector of master problemconstraint coefficients—the column—needed to include the variable in theconstraint matrix of the restricted master problem.

3.4.2 The Restricted Master Problem

The restricted master problem is purely an artifice of the column generationalgorithm. It is defined by subsets E′k of the index sets Ek of master problemvariables. Given E′k ⊆ Ek, k = 1, . . . ,K, let λ′, c′, A′ and ∆′ be λ, c, A and∆ of the master problem (3.5) restricted to E′k. Then the restricted masterproblem is

z =

max c0x0 + c′λ′

s.t. A0x0 + A′λ′ ./0 b0∆′λ′ = 1

x0 ∈ Rn030

;λ′ ∈ R|E′1|+···+|E′K |

+

. (3.13)

On one level, the restricted master problem is simply the master problemwith many of its variables removed. The missing variables can be regardedas fixed at zero, prevented from entering the basis, and so restricting theproblem. However, it is important to realise that the restricted masterproblem is dynamic: the sets of variables E′k change during the iteration ofthe column generation algorithm. New variables are added—variables fromthe master problem that are missing in the restricted master problem—thusfreeing them up to enter the basis and take non-zero values. Variables whichleave the basis may be deleted, joining the missing variables fixed at zero.

In theory, there is no difference between fixing the missing variables tozero, or removing them altogether from the problem. However, given thelarge number of variables in the master problem, significant computationaladvantages are gained by removing the condemned variables. The restrictedmaster problem is orders of magnitude smaller than the master problem,and so fits into a computer memory whereas the master problem may not.The basic linear algebra operations of an LP optimiser are performed morequickly on the smaller problem. Finally, it turns out that it is much easierto generate the constraint and objective coefficients of the new variables asrequired, rather than to generate the coefficients of all variables initially,and then identify which ones are to be to freed up during the progress ofthe column generation algorithm. These computational aspects are goodreasons to opt for the concept of adding and deleting variables rather thanthat of freeing and fixing variables at this theoretical stage.

How does a solution to the restricted master problem relate to the masterproblem? Clearly, a primal feasible solution to the restricted master problemis also a primal feasible solution to the master problem, and so gives a primal(lower) bound on the optimum value of the master problem. A dual feasiblesolution of the restricted master problem will not in general be a dual feasible


solution of the master problem, although in section 3.4.4 below we show howit can be used together with the dual solutions of the pricing problems toobtain a dual feasible solution of the original problem. If the restrictedmaster problem is unbounded, then so too is the master problem, and anyunbounded ray of the restricted master problem is an unbounded ray of themaster problem. Conversely, if the restricted master problem is infeasible,then no deduction can be made about the feasibility or not of the masterproblem—to prove infeasibility of the master problem, we need additionalinformation. The column generation algorithm uses the pricing problems toguarantee that no variable can be brought into the basis to reduce the sumof infeasibilities.

Observe that a valid basis for the restricted master problem is also avalid basis for the master problem, and the primal and dual solutions of therestricted master problem at a particular basis are identical to the primaland dual solutions of the master problem at the same basis. So the reducedcosts of variables in the master problem at a specific basis may be calculatedusing the dual values obtained from the restricted master problem at thatbasis.

3.4.3 The Pricing Problems

The K pricing problems are defined so that the possible solutions of problemk correspond to the vector of master problem variables λk. By definingpricing problem k to have Pk as its feasible region, the set of basic feasiblesolutions and extreme rays of the pricing problem are precisely the set ofextreme points Πk and extreme rays Rk (collectively Ek) of Pk. Solvingthe pricing problem with a given objective function will identify an elementxk = xikk , ik ∈ Ek, being either an extreme point ik ∈ Πk, or an extreme rayik ∈ Rk. To price out the variables of the master problem, we must define anobjective function that relates the objective value at these solutions to thereduced costs of the corresponding master problem variables (λk)ik , ik ∈ Ek,at a particular basis of the master problem.

The reduced cost (dk)ik of a variable (λk)ik can be derived by referenceto the master problem (3.5) (p. 33). Let (π, µ) ∈ Rm0

3′0× RK be the dual

values of the master problem.2 Then

(dk)ik = (ck)ik − [π(Ak)ik + µ(δk)ik ]

= (ck − πAk)xikk −

{µ if xikk is an extreme point;0 if xikk is an extreme ray.

The objective function is thus defined to be zk = (ck − πAk)xk, dependentupon the dual values (π, µ) of the master problem at a particular basis.

2In section 3.4.2 above we observed that these dual values may in fact be obtainedfrom the restricted master problem.


Pricing problem k is therefore

z∗k =

max (ck − πAk)xks.t. Bkxk ./k bkxk ∈ Rnk3k

, (3.14)

which we have already used in section 3.3.4.Given dual values (π, µ) of the master problem, a basic feasible solution

xk = xikk , ik ∈ Πk, corresponds to a master problem variable (λk)ik withreduced cost (dk)ik = zk − µk; whilst an extreme ray xk = xikk , ik ∈ Rk, cor-responds to a master problem variable (λk)ik with reduced cost (dk)ik = zk.By solving the pricing problem, we obtain an extreme point with maximalobjective value z∗k, or an unbounded extreme ray with z∗k > 0. If the opti-mum value is bounded and z∗k > µk, then the corresponding master problemvariable is a candidate to enter the basis of the master problem, and indeedany basic feasible solution of the pricing problem with zk > µk correspondsto a master problem variable that is a candidate to enter the basis. If thepricing problem is found to be unbounded, then an extreme ray with zk > 0also corresponds to a master problem variable that is a candidate to enterthe basis. If the pricing problem is bounded and z∗k ≤ µk, then no masterproblem variable derived from Pk is a candidate to enter the basis.

By selecting the optimum solution when the pricing problem is bounded,we are effectively using the Dantzig criterion to price out variables in themaster problem—choosing the variable with the maximum reduced cost.Other pricing schemes, such as Devex pricing and steepest edge pricing, areused in LP optimisation to prevent some problems of degeneracy and reducethe number of iterations required overall [Har73, GR77]. These schemesrequire set-up time and more work per iteration, and because the pricingproblems’ objective functions are altered each cycle, are less efficient andimpractical within Dantzig-Wolfe decomposition. We do not investigatealternative pricing schemes at this level (we do investigate them at the levelof solving the individual pricing problems and restricted master problemsthat occur on each cycle).

If we only have an infeasible solution to the restricted master problem,then we can use a certificate of infeasibility of the master problem (π, µ)to set the objective functions of the pricing problems. This results in thepricing problem associating a phase I reduced cost value with the points andrays representing master problem variables. Thus the pricing problems canbe used to identify variables that will reduce the infeasibility of the masterproblem, or prove that the infeasibility cannot be reduced.

Let us consider the consequences of a pricing problem being infeasible.It is clear that this would imply that the original problem too is infeasible,for the pricing problem can be viewed as a Lagrangean relaxation of theoriginal problem. Indeed a certificate of infeasibility for pricing problem k,


νk, can trivially be extended into a certificate of infeasibility for the originalproblem by padding with zeros. The master problem is clearly infeasibletoo: since Ek is empty, there are no variables (λk)ik and so the kth convexityconstraint

∑ik∈Ek(δk)ik(λk)ik = 1 cannot be satisfied. A trivial certificate

of infeasibility for the master problem is (π∗, µ∗) := (0, ek), where ek is thekth unit vector. Finally, note that the feasibility of the pricing problemsdepends only on the polyhedra Pk, not on the choice of objective functionwhich varies with (π, µ), and so may be ascertained at the beginning of thealgorithm.

Observe that a non-extreme solution to the pricing problems can be ex-pressed as a convex combination of extreme solutions (definitions 4 and 6,section 3.1). If we generate a variable for the master problem from the solu-tion, the variable would be equivalent to a fixed convex combination of theconventional master problem variables arising from the extreme solutions,and solutions obtained involving it would be valid solutions to the masterproblem. In practice, such solutions may be generated by using interiorpoint methods to solve the pricing problems (see section 4.4) and may forma useful additional to the computational technique. The finite convergencediscussed in section 3.4.6 may be destroyed, but this is not especially relevantin practical work anyway.

3.4.4 A Dual Bound

A dual feasible solution to the restricted master problem (for given E′k, k =1, . . . ,K) is not a dual feasible solution to the complete master problem (seesection 3.4.2). Another way of viewing this is that fixing the primal variablesto zero implies that the corresponding dual constraints are relaxed. However,in proposition 14 (p. 35), we showed how, given π satisfying πA0 ./

′0 c0 only

(and that the pricing problems are feasible and bounded), we can complete adual feasible solution (π, ν) to the original problem using the dual solutionsof the pricing problems, and thence into a dual solution (π, µ) to the masterproblem.

Let νk ∈ Rmk3′kbe a dual feasible solution to pricing problem k, where the

objective function is defined using π as usual. Then (π, ν) is a dual feasiblesolution to the original problem, and the corresponding dual objective valuez = πb0 + ν1b1 + · · · + νKbK is a dual (upper) bound on the optimal valueof the original problem, and thus also of the master problem. A set of dualoptimal solutions to the pricing problems {ν∗k}Kk=1 gives a smallest upperbound. We can express the bound in terms of the optimal objective values


z of the master problem and z∗k of the pricing problems:

z = πb0 + ν∗1b1 + · · ·+ ν∗KbK

= (z − µ · 1) + z∗1 + · · ·+ z∗K

= z +K∑k=1

(z∗k − µk),

assuming that all objective values are finite.At each iteration of the column generation algorithm, once the restricted

master problem is feasible, a new dual solution of the restricted master prob-lem is available. Provided we obtain a bounded feasible optimal solution toeach of the pricing problems we can calculate a new dual bound. The dif-ference between the primal value of the restricted master problem—a lowerbound on the value of the master problem—and the least dual bound ob-tained during the course of the algorithm—an upper bound—is the opti-mality gap. It provides a guarantee on the quality of the current solutionto the restricted master problem. In practice, a near optimal solution maybe acceptable and so the column generation algorithm may be halted earlyonce the optimality gap falls to within an acceptable limit.

3.4.5 The Algorithm

In this section we present the column generation algorithm. Earlier treat-ments have imposed simplifications which limit flexibility when implement-ing the algorithm. For the first time, we give a straightforward theoreticalpresentation of the algorithm without unnecessary simplifications, whichcorresponds to that which has previously been employed in computationalimplementations (e.g., [HL81]).

Initialisation. Choose E′k ⊆ Ek, k = 1, . . . ,K. Initialise the best knowndual value of the master problem z∗ =∞ (a least upper bound).

Solve the restricted master problem. Solve

z =

max c0x0 + c′λ′

s.t. A0x0 + A′λ′ ./0 b0∆′λ′ = 1

x0 ∈ Rn030

;λ′ ∈ R|E′1|+···+|E′K |

+

.

Let (π∗, µ∗) be an optimal dual solution found if the problem is feasibleand bounded, or a certificate of infeasibility if the problem is infeasible.

If the restricted master problem is unbounded, then the complete mas-ter problem is unbounded, and any unbounded ray identified is anunbounded ray of the complete master problem. Stop.


Solve the pricing problems. Solve some of the k = 1, . . .K pricing prob-lems

z∗k =

max (ck − π∗Ak)xks.t. Bkxk ./k bkxk ∈ Rnk3k

.

until at least one basic feasible solution is found with zk > µ∗k, or anunbounded extreme ray is found with zk > 0, or having found that nosuch solutions exist to any of the pricing problems. Any such solutionsfound correspond to λk variables of the master problem with positivereduced cost, and thus candidates to enter the basis of the currentrestricted master problem.

If pricing problem k is infeasible, then the complete master problemis infeasible. Stop.

Optimality test. If z∗k ≤ µ∗k for each k = 1, . . . ,K, then no master prob-lem variable prices out positive to enter the basis of the restrictedmaster problem. The current restricted master problem solution is anoptimal solution to the complete master problem, or, if the restrictedmaster problem is currently infeasible, then the complete master prob-lem is infeasible and (π∗, µ∗) is a certificate of infeasibility. Stop.

Near-optimality test. If the restricted master problem is feasible, andthe pricing problems are all feasible and bounded, calculate a newdual value (an upper bound) of the master problem

z = z +K∑k=1

(z∗k − µ∗k).

If z < z∗, update the best known dual value z∗ := z.

Calculate the absolute optimality gap dA = z∗ − z and the relativeoptimality gap dR = dA/(1 + |z∗|). If they are within user specifiedoptimality tolerances, then the current restricted master problem so-lution solves the master problem to the requested tolerance. Stop.

Update the restricted master problem. As the optimality test failed,at least one master problem variable (λk)ik was identified when solv-ing the pricing problems that is a candidate to enter the basis of therestricted master problem. Choose at least one of these to add tothe current restricted master problem, i.e., update E′k by adding thechosen ik.

Any current non-basic variable (λk)ik of the restricted master prob-lem with negative reduced costs may be removed, i.e., update E′k byremoving the chosen ik.

Return to solve the restricted master problem with the updated E′k.


There are various decisions in the algorithm as stated that have been leftopen, all related to the handling of the candidate variables, which are calledproposals. They are: the choice of the initial proposals E′k (proposal initiali-sation); the choice of the proposals to add to the restricted master problem,which encompasses the selection of pricing problems to solve on a given cy-cle and to what degree they should be solved (proposal generation); and thechoice of the redundant proposals to remove from the restricted master prob-lem (proposal management). From a theoretical point of view, convergencecan be guaranteed with simple choices: the initial E′k may be empty; atleast one proposal pricing out negative must be added to the restricted mas-ter problem; and no proposals need to be deleted (see section 3.4.6 below).However, a solution can be obtained more quickly in practice if differentchoices are made. In particular, there is no need to solve all of the pricingproblems to optimality—we could stop as soon as we have one good pro-posal to add to the restricted master problem, although this would preventthe calculation of a dual bound. We explore the various options and manyother computational issues in chapters 4 and 5.

As we shall use this algorithm as the basis for a computational proce-dure, it is worth noting changes necessary to minimise the master problem.Both the restricted master problem and the pricing problems are minimisedrather than maximised, and the optimal value of a feasible restricted masterproblem gives an upper bound on the optimal value of the master problem.Conversely, the dual values calculated provide lower bounds on the optimalvalue of the master problem, and the best dual bound must be initialised to−∞. The master problem is solved when no variables have negative reducedcosts; hence we look for basic feasible solutions of the pricing problems withzk < µ∗k and unbounded extreme rays with zk < 0; the optimality test ispassed when all z∗k ≥ µ∗k. And at the final step, we may delete non-basicvariables with positive reduced costs from the restricted master problem.

3.4.6 Finite Convergence

We have already remarked that the column generation algorithm is in somerespects merely a restricted variant of the simplex algorithm. The restrictedmaster problem contains a working basis for the master problem; the setof non-basic variables permitted to enter the basis is restricted. When theavailable set has been exhausted and the restricted master problem is solved,new variables are made available, identified by the pricing problems.

To show that the column generation algorithm has finite convergence, wejust need to show that the parallels with the simplex algorithm are indeedaccurate. This means showing that the optimality test is equivalent to thestandard reduced cost of the simplex algorithm, and then finite convergencefollows easily. In this section we assume that the near-optimality test is sus-pended; its validity derives from the dual bound established in section 3.4.4,


and of course its consequence is to terminate the algorithm in advance ofthe other stopping criteria.

First, we confirm that if the optimality test is passed, then the restrictedmaster problem solution is indeed optimal for the master problem.

Proposition 18 Given an optimal solution (x∗0, λ∗) of the restricted master

problem, and that all of the pricing problems are feasible and bounded withoptimal values z∗k ≤ µ∗k, the solution (x∗0, λ

∗) to the master problem obtainedby padding λ′ with zeros is optimal.

Proof. The construction of the pricing problems in section 3.4.3, and the obser-vation in section 3.4.2 that the dual values of the restricted master problem at aparticular basis are identical to the dual values of the master problem at the samebasis, show that the reduced costs of all master problem variables not included inthe restricted master problem are negative. The result follows from the validity ofthe simplex algorithm [Dan63]. �

Corollary 19 Assume that no variables are removed from the restrictedmaster problem and that the near-optimality test is suspended. Then thecolumn generation algorithm of section 3.4.5 halts at an infeasible, boundedoptimal, or unbounded solution in a finite number of iterations.

Proof. At an optimal bounded solution to the restricted master problem, allvariables included in the restricted master problem have zero or negative reducedcost. The construction of the pricing problems guarantees that these variables willnot be chosen to be added to the restricted master problem. So at each iterationeither the algorithm terminates, or at least one new master problem variable isadded to the restricted master problem. Since the master problem has a finitenumber of variables (proposition 2 and theorem 9), in the worst case all variablesare added and then the algorithm must terminate. �

In general the path taken to the optimal solution will not pass throughevery vertex of the master problem. When only a small proportion of thevertices are visited before reaching the optimal solution, many master prob-lem variables will never be explicitly generated, and the performance of thealgorithm will be much better than the worst case.

The result of corollary 19 depends on the implicit assumption in thecolumn generation algorithm of being able to obtain an optimal solution tothe restricted master problem. Any problems of degeneracy or cycling havebeen delegated to the task of solving the restricted master problem. Whenwe permit variables to be removed from the restricted master problem, theseissues are promoted to the level of the column generation algorithm. Thereis the possibility, familiar in the simplex algorithm, of cycling between asequence of bases of the master problem.

3.5. SUMMARY 51

If the problem is not degenerate, then the strict improvement in theobjective function at each iteration prevents the same basis occurring morethan once, and the finite number of bases of the master problem guaranteesfinite convergence.

If the master problem is degenerate, then a serious weakness of the col-umn generation algorithm becomes apparent. The usual methods to preventcycling cannot be applied, as the selection of candidate variables to enter thebasis is limited to the variables chosen by the pricing problems. Our onlyoption is to fall back on not removing variables from the restricted masterproblem, which delegates the problem back down to the LP optimiser, butthis is often not possible in practice. It also seems that the master prob-lem is likely to be degenerate in practice [Ho84, Naz84] (see section 3.3.5).This issue is part of the wider computational aspects of the Dantzig-Wolfedecomposition which we return to in section 5.8.

3.5 Summary

This has been a long but important chapter, in which we have laid the frame-work for our computational work. We have seen how the original problemis first transformed into the master problem, solved using the column gener-ation algorithm, and how the solution is transformed back into one for theoriginal problem. We have touched upon various computation aspects whichare the focus of the following two chapters: in the next chapter we use ourunderstanding of the method to review previous computational work, whilstin chapter 5 we implement and evaluate the method.


Chapter 4

Survey of ComputationalExperience

In this chapter we survey reported computational experience with Dantzig-Wolfe decomposition. Our intention is threefold—to identify the compu-tational techniques and strategies that have been used to implement thealgorithm, so that our implementation can draw on the best practice ofprevious work; to describe the practical results obtained so far with decom-position on real applications; and to identify opportunities for us to makefurther progress. Our contribution here is a critical review of previous prac-tical work, drawing out issues for a new implementation and opportunitiesfor further advances.

Our survey is in the main chronological, but we have identified five dis-tinct phases of work into which the chapter is structured. The first andlargest section covers the period up until the late 1970s of the first prac-tical experiences with decomposition. This period is characterised by awealth of ideas, software implementations and problems tackled, but littleevolutionary improvement in the computational development. The secondsection considers the major contribution made by Ho, Loute et al over aperiod of ten years. By drawing on their own experiences and other work,they developed a series of second generation software that removed many ofthe doubts about decomposition, but finally showed that it cannot competewith mainstream LP methods on most problems. Their work had just twoweaknesses—a lack of a final systematic evaluation of the algorithm, anda tight integration of their software to certain LP software that preventedre-use and improvement of their code once the LP code had been superseded.

The third section reports on the limited experiences with parallel im-plementations of the decomposition method: there is much scope here forfurther work. During the 1990s, the innovation of interior point methodsreached decomposition with some positive effect, with a significant amountof work done at Logilab in Geneva, and this is reported in section four.

53

54 CHAPTER 4. SURVEY OF COMPUTATIONAL EXPERIENCE

Finally we mention the most recent topic of interest: that of incorporatingdecomposition into a branch and bound framework in order to solve integerprogramming problems.

We end the chapter by drawing together and discussing issues for acomputational implementation, and noting opportunities for further work.

4.1 First Experiences

Beale, Hughes and Small reported the first practical application of Dantzig-Wolfe decomposition in 1965, applying decomposition to an oilfield opera-tions problem [BHS65]. In many ways their study remains the most useful ofall those implementing Dantzig-Wolfe decomposition, for it contained inno-vative ideas for implementing the algorithm together with positive practicalresults; the authors were always able to come up with new ideas to circum-vent poor performance.

The algorithm was implemented as an internal extension to the com-mercial lp/90/94 mathematical programming software. The software hada theoretical capacity of 100 blocks, 100,000 constraints in total, 1000 con-straints in any block, and an unlimited number of variables.

The major innovation was to retain the coefficients of the global con-straints (Ak) in the pricing problems—as non-binding constraints with zeroRHS. This had three apparent advantages:

1. The modified objective function of each pricing problem, c′k = ck −πAk, could be calculated directly in the underlying LP optimiser as alinear combination of these constraints. (A simple modification to theinternal simplex routine FORMC allowed an easy implementation.)

2. The proposal coefficients for extreme point proposals could be readdirectly from the slack values of the global constraints; or, in the caseof extreme ray proposals, directly from the matrix coefficients of theseconstraints, as represented in the current basis.

3. The reconstruction of the full primal solution could be implementedeasily—setting the RHS of the global constraints in each of the pric-ing problems to be the computed optimal allocation and solving thepricing problems with the constraints enforced.

Exploiting the underlying simplex optimiser to perform this additional workrequired by the decomposition algorithm greatly simplified the additionalprogramming. However, the authors’ claim that the resulting implementa-tion was ‘more efficient’ cannot be left unchallenged; whilst the amount ofwork required in the decomposition-specific routines was certainly reduced,it was not shown that this outweighed the extra work required by the simplexroutines to carry the non-binding constraints.

4.1. FIRST EXPERIENCES 55

It was noted that optimisation of the master problem and pricing prob-lems could be resumed from the advanced bases obtained at the last cycle.

The paper also addressed the three open algorithmic issues of proposalinitialisation, proposal generation and proposal management. At the start,a representative range of proposals was computed from each block by ex-ploiting certain problem specific knowledge. (Each proposal had a naturalinterpretation as an activity vector for one of the oilfields.) Multiple pro-posal generation was used—that is, many proposals generated from onepricing problem per cycle. Proposals were generated at a certain frequencyof pricing problem pivots, using a dynamic frequency—high at the start ofthe algorithm, to try to quickly build up the representation of each block inthe master problem, then low, to prevent the master growing too large, andthen high again towards the end of the algorithm. The actual frequency wascontrolled manually during computation from the on-line card reader basedon many experimental runs. Whether a similar strategy could be appliedwithout such detailed experiments appears unlikely. At the beginning of thealgorithm, optimal proposals were intentionally suppressed, as it was foundthey actually slowed overall progress by misleading the master problem.

Dantzig described three strategies for deciding how long proposals shouldpersist in the master problem—retain all proposals; delete all non-basic pro-posals immediately (apart from those just generated); or keep proposals untila certain proposal capacity is reached, and then purge those proposals withthe worse reduced costs [Dan63]. Beale, Hughes and Small chose the firstoption: they felt that the benefit of retaining coverage of the feasible regionin the master problem outweighed the extra work implied by accumulatingproposals.

The best reported result, on a problem of 450 constraints, found thedecomposition method performed marginally better than the rival simplexmethod. The authors claimed that this comparison was slightly unfair tothe simplex method, as no tuning had been carried out for simplex, but thisoverlooked the fact that the simplex method in general had already been sub-ject to much refinement, whereas this was one of the first real applications ofdecomposition. Nevertheless, some of the more problem specific refinementsto the decomposition method did prove crucial; notably the generation ofinitial proposals and the dynamic proposal generation frequency.

The work of Beale, Hughes and Small was important because it estab-lished a benchmark implementation for Dantzig-Wolfe decomposition. Thecomputational techniques they used were not matched or improved on untilrediscovered by Ho and Loute over a decade later.

Orchard-Hays described an implementation of decomposition as an in-ternal addition to the optimal mathematical programming software for theCDC 6000 [OH68]. He suggested an alternative criterion for generating pro-posals: generating one from the first solution to pass the reduced cost test,followed by the next five for which the objective value (i.e., the reduced


cost of the corresponding proposals adjusted by a constant) improved by astated percentage. At unbounded solutions, generate both an extreme pointproposal and an extreme ray proposal.

One application, to a logistics problem, was reported. Convergence wasslow, with a long tail of near optimal solutions but for which the final op-timality gap could not be closed—asymptotic convergence. Proposals werecompletely dense, which had the effect of reducing the simplex iterationspeed when solving the master [OH72]. Decomposition was not competitivewith alternative methods.

Summarising his practical experience up to 1973, Orchard-Hays said:

[ . . . ] in two decades, while LP capabilities have flourished inscope, power, generalisation and application, decomposition re-mains more a principle than a practical method. [ . . . ] I knowof not a single instance of the use of general decomposition ina routine, production environment such has been commonplacewith general LP for over a decade. [OH73]

Kutcher reported an implementation of decomposition using ICL’s xdla

LP software, and the application of it to an economic planning model foragriculture in Mexico [Kut73]. Like Beale et al, he found that the pricesfluctuate wildly in the early stages of the algorithm, and, like Orchard-Hays, he observed asymptotic convergence. He experimented by varyingthe degree of decomposition, and found that a finer grain decomposition ledto fewer cycles (we return to this issue in section 5.9, p. 112).

A breakthrough in software came in 1974, when the first implementationsof decomposition using IBM’s mpsx mathematical programming softwareappeared. mpsx allowed algorithms to be written that called into mpsx’sown routines, by using mpsx’s pl/i based Extended Control Language.

Williams and Redwood reported an mpsx implementation decomp fora food blending application of 1800 constraints and 3200 variables in to-tal [WR74].1 The same problem was solved repeatedly in a productionenvironment with different data instances. Two block structures were ex-ploited, one based on food products and one on time periods. The productdecomposition gave rise to proposals with a natural interpretation as recipes,which allowed recipes generated on earlier runs to be used as initial propos-als on later runs. Other features of the implementation worth noting arethat proposals were generated once the reduced cost fell below an appar-ently arbitrary value of −0.2, and that the algorithm was halted once theimprovement in the overall objective function from one cycle to the next fellbelow 0.5%. Observe that either feature may have concealed numerical in-stability at the expense of potentially early, non-optimal termination of the

1In the same year Ho and Manne reported an implementation of nested Dantzig-Wolfedecomposition using mpsx and pl/i [HM74].

4.1. FIRST EXPERIENCES 57

algorithm, perhaps far from optimal, and although the dual bound wouldhave provided a measure of the quality of the solution it was not reported.

In comparison to solving the problem with the mpsx simplex methoddirectly, the time based decomposition took half the time to get a solutionwithin 0.28% of optimal, while the product based decomposition had littleif any advantage over the simplex method. The authors noted that whilethe decomposition method was accelerated by the strategies they used, thecrucial feature of the simplex method was the ability to save and restorebases from one run to the next, a feature that could not really be matchedby decomposition despite re-using recipes.

A number of practical experiences were summarised in the oft cited bookof Dirickx and Jennergren [DJ79]. They reported that in general the greaternumber of proposals are used to initialise the algorithm, the faster the so-lution can be found. It is clear that if work can be done outside the al-gorithm, then the algorithm will run more quickly. But if a representativeset of proposals can be generated cheaply or proposals generated on otherdata instances can be re-used, then the benefit is real. On the managementof proposals in the master, they quoted Schiefer, who suggested deletingproposals once they had been non-basic for a certain number of iterations,such as 15.

Their general assessment was that the decomposition applications re-ported had ‘not been very positive.’ The method was ‘time consumingand cumbersome [and] converges only slowly towards an optimal solution.’Echoing Orchard-Hays’ 1973 comment, they saw that no general Dantzig-Wolfe software was available; each study started by building its own fromscratch, and thus there was little scope for evolutionary improvement. Theirconclusion was that though in some cases decomposition may be the onlyalgorithmic choice where the problem is too large to be solved in one piece,if a standard LP code was available that can handle the given problem, itshould be used in preference to a decomposition code.

Certainly, the opinion of the state of the art in decomposition towardsthe end of the 1970s was one of disillusionment. A lot of research had beencarried out, often focused on solving a particular application. There waslittle re-use or evolutionary improvement of software, and no systematicstudy of the algorithm with its various strategies on a variety of problems.Dantzig summarised the deficiencies of the research:

The problem has been that [ . . . ] the only final criterion can besystematic experimentation with representative models. Clearly,for such experimentation to give meaningful and reliable results,the implementation must be sophisticated and the test problemslarge enough to give a guide to real problem behaviour. [Dan77]


4.2 The Advanced Implementations of Ho andLoute

During the 1970s and 1980s, Ho, Loute, and others published a series ofpapers which described a second generation implementation and evaluationof Dantzig-Wolfe decomposition. Ho’s interest sprang from one in nestedDantzig-Wolfe decomposition, an idea originally proposed (though oftenoverlooked) in one of Dantzig’s and Wolfe’s original papers, and first investi-gated in depth by Glassey and Ho and Manne [DW60, Gla73, Ho74, HM74].Nested Danzig-Wolfe decomposition is a recursive application of Dantzig-Wolfe decomposition, and some of the ideas and experience we quote be-low, though applicable to ordinary (two-level) decomposition, were origi-nally made with reference to nested decomposition. Loute meanwhile hadoriginally investigated the exploitation of block structure in the simplexalgorithm itself, without the need for reformulation, by the use of blockfactorisation techniques [HL80].

Already by 1978, Ho, in the context of nested decomposition, had de-fined a set of four parameters to control multiple proposal generation, whichformalised and unified the various strategies described in the previous sec-tion [Ho78]. The first proposal is generated at the first feasible solution thatmeets the reduced cost criterion. Proposals are then generated every FREQsimplex iterations, or whenever the reduced cost is improved by PERC%, untilTERM proposals have been generated or the problem is optimal, and only thefinal MAX proposals are submitted to the master problem. Suggested rangesand default values are MAX: 1–10, default 5; FREQ: 5–15, default 5; PERC:0.1–1.0%, default 0.1% [Ame81]. Ho said that one should aim for rapidsub-optimal proposal generation in the early cycles, and complete optimalproposal generation in the later cycles. If the pricing problem becomesunbounded so that an extreme ray proposal is generated, further propos-als can be generated (up to MAX) by fixing the unbounded pricing problemvariable and re-optimising. Experience showed that there were often manyunbounded extreme rays in close vicinity, and it may otherwise have takenone cycle to eliminate each ray. Ho also identified data structures, the LPsoftware, and the proposal management as three major areas on which theefficiency of the implementation depends.

Ho’s and Loute’s first study of two-level decomposition was a prelimi-nary investigation into solving large multi-national multi-period energy mod-els [HLSvdV79]. They used a decomposition code decomp, originally writ-ten by Carlos Winkler at Stanford’s System Optimisation Laboratory (SOL)and based on John Tomlin’s revised simplex code lpm1 [HS89, Tom75b].This enabled them to exploit low-level efficiencies, such as directly modifyingthe internal simplex routine FORMC to automatically calculate the modifiedpricing problem objective function (following [BHS65]).

4.2. THE ADVANCED IMPLEMENTATIONS OF HO AND LOUTE 59

The model under investigation had up to 52,000 rows and 65,000 columnsin its largest form, although only small instances of it were solved for thereport. They argued that decomposition would win on large structuredproblems, based on the observed workload of decomposition, which increasedlinearly with problem size, as compared to the simplex method, where theworkload increased cubically.

Ho’s and Loute’s main contribution was the decompsx implementationand associated experience [HL81, HL83]. decompsx was implemented usingthe Extended Control Language and LP routines of mpsx/370, at thattime state of the art and the only sophisticated LP software to providea programming interface [BGHR77]. In fact, various mpsx routines weremodified, to implement a similar low-level interface as had been used withthe earlier decomp code.

decompsx incorporated many techniques developed piecemeal in thepast—for example, the data handling scheme of [BHS65], and Ho’s unifiedcontrols for proposal generation. They also introduced some new ideas,such as partial cycles: they noted that provided at least one proposal wasgenerated each cycle, there was no need to solve all pricing problems oneach cycle, thus saving on problem setup time. In practice they skipped aproblem if it had failed to generate a profitable proposal on the last cycle.They developed a simple input format: a single MPS file in which the masterproblem and pricing problems were listed consecutively. The matrix scalingstrategy of mpsx had to be augmented for decomposition, as experienceshowed that proposals tended to contain elements which differed significantlyin magnitude from the unity element in the convexity row, for what the mpsx

scaling was insufficient to handle. Instead, each pricing problem was givena problem scaling factor, used to scale all proposals generated from thatpricing problem [ALR81].

The initial experience comprised a test set of 12 problems from a varietyof applications such as forestry management and multi-national dynamicenergy planning, with up to 6000 rows, 12,000 columns, 9 pricing problems,and with density 0.04–1.2%. All tests were conducted with the defaultstrategies, and the problems were solved to within 0.01% of optimality. Theproblems took on average 27 cycles to solve, the largest up to 128 cycles, butall but two under 30 cycles. The largest problem took 72 minutes to solve,but the average time was 18 minutes. Of the total CPU time, 15–36% wastaken up by the high level decomposition routines, and the rest by the lowlevel mpsx LP routines. The authors thus proclaimed decompsx a robustand efficient decomposition solver, meaning that it could successfully solvea variety of problems, and that the time spent in the higher level routinesas regards the mpsx LP routines was relatively small. However, comparedto the simplex method implemented in mpsx, decompsx always came outworse, in many cases taking twice as long as mpsx.

All problems were solved to a relatively tight tolerance in a relatively


small number of cycles, in contrast to the accepted wisdom that decompo-sition yields slow or asymptotic convergence. Ho noted that for problemsthat don’t converge quickly, one of two behaviours could be observed—theyeither become asymptotic, or exhibit symptoms of ill-conditioning [Ho84].Asymptotic convergence may be attributed to numerical inaccuracy prevent-ing the duality gap from closing. The error can be standardised by invertingbefore generating each proposal, but this can be expensive, especially if mul-tiple proposal generation is used. Ho suggested (but did not implement) atest to calculate the error, so that an invert can be carried out when theerror grows large. However, in decomposition there is really no comparablefeature to the invert in the revised simplex method which periodically resetsthe computation with the original data [Ho87].

In an assessment of their work, Ho and Loute made three major points.Firstly, echoing Dirickx and Jennergren, they claimed that decompositioncould not compete with mainstream LP techniques, but could only extendthe capability of mainstream LP techniques by being applied solve largerproblems than would otherwise be possible to solve. Secondly, echoingDantzig, they claimed that for a definitive evaluation of decomposition, it isnecessary to implement decomposition at the level of state of the art LP soft-ware. Thirdly, to promote understanding of decomposition, they said thatdecomposition must be made available as simple to use software, ideally aspart of mainstream commercial LP codes.

Ho and Loute’s work was the first to really build on previous work, andat the same time to highlight its weaknesses: they incorporated variousdata handling techniques and computational strategies devised by previousstudies and developed a simple set of parameters for controlling them. Theyrealised the importance of using sophisticated LP software, and exploitedthe LP subroutine libraries that had become available. Unfortunately theydid not consider the relative merits of a tight integration with the underlyingLP software as compared to a clear interface; by opting for the former, withsource code modifications to the LP software, they prevented easy portingof their decomposition routines to other LP software, with the result thattheir code is no longer in use.

Ho and Loute evaluated their implementations on a sufficient variety ofproblems to validate decomposition as a capable and robust problem solver.Their results shifted the focus of decomposition from competing with main-stream LP methods to standing as a method in its own right, offering asolution method for a small but significant number of problems. However,despite their original intention, they did not perform a systematic study ofthe algorithm, and its different computational strategies, on a comprehen-sive set of problems, and in this respect fell short of a definitive study ofdecomposition.

4.3. PARALLEL IMPLEMENTATIONS 61

4.3 Parallel Implementations

The decomposition method naturally lends itself to a coarse grained dis-tributed memory parallelism based on assigning the pricing problems to dif-ferent processors and solving them simultaneously.2 At the simplest level themaster problem and pricing problems reside on separate processors; whenthere are more problems than processors, each processor may hold severalproblems which may be solved locally in sequence. Mirroring the algorithm,control is maintained by the process handling the master problem: it sendsout prices to start up the slave processes; the slave processes solve the pricingproblems and send the generated proposals back to the master processes.

Two measures are widely used to assess the performance of a parallelapplication in comparison to a serial application performing the same task.Let Tp be the time taken to perform the application when p processors areavailable. The speed-up, Sp, is the performance improvement ratio

Sp = T1/Tp

The efficiency, Ep, is the ratio of the speed-up to the number of processors

Ep = Sp/p

In general, we would hope for near-linear speed-up, or, equivalently, anefficiency close to one.

The first parallel implementation of the decomposition method was theprototype decompar by Ho et al [HLS88, HS89], based on the serial de-composition software decomp used by Ho and Loute. decomp, and itsunderlying LP software lpm1, were by now over fifteen years old, and muchinferior to other LP software available. The authors no doubt chose it dueto the availability of complete source code and their familiarity with it, butit could not be used for practical computational work. Indeed, the maxi-mum problem size was very restricted, with at most 10 blocks, each with400 rows, 1000 columns and 10,000 non-zeros; and up to 99 global rows.

When implemented in parallel, a new area of flexibility is opened up inthe algorithm: the order in which the problems are solved on each cycle,and whether all problems are solved on every cycle [HLS88]. Five strategiesmay be distinguished by the action taken in a typical cycle:

Basic strategy Solve all pricing problems, then solve the master problem.

First pricing problem strategy Start solving all pricing problems, thenas soon as one finishes, solve the master problem.

2Such an implementation is described as MIMD—multiple instructions (meaning theprocesses follow different instructions or code) multiple data (meaning the processes op-erate on different data).


First proposal strategy Start solving all pricing problems, then as soonas one proposal is generated, solve the master problem.

Instant feedback strategy All new information is acted on immediately—all pricing problems and master constantly being solved, send out pro-posal from each pricing problem after every pivot, and send out pricesfrom master after each pivot.

Accelerated feedback strategy Whenever a pricing problem finishes, sendout proposals generated, check for new prices, and start solving againas soon as new prices found. Whenever the master problem finishes,send out new prices, check for new proposals, and start solving againas soon as any found.

The latter strategies aimed for less idle time than the basic strategy, with theinstant feedback strategy taking this aim to its logical conclusion; however,they result in computation being based on sub-optimal information, and sosmaller steps taken by each process, with slower overall convergence. Theaccelerated feedback strategy was developed to strike a compromise betweenthese two factors, as a result of experience with the instant feedback strategy.

Some results were reported on tiny test problems—up to 700 rows and1250 columns in total—and so can only be taken as validating the implemen-tation rather than providing any real evidence as to the efficiency of it andwith the different computational strategies. The number of processors usedvaried (one per pricing problem) from 5–11, giving efficiencies in the range0.5–1.0, and 0.71 on average, although a potentially major complicating fac-tor here was the lack of disk I/O required by decompar (each problemheld in the memory of a dedicated processor) as compared to decomp (eachproblem written to disk between cycles), which was not investigated. Theresults indicated that the basic strategy was faster than the first problemstrategy, which in turn was faster than the first proposal strategy, but theaccelerated feedback strategy was on average 1.3 times faster than the ba-sic strategy. However, doubts must remain about the applicability of theresults.

Ho and Gnanendran reported a more sophisticated parallel implementa-tion decube in 1993 [GH93]. Equally importantly, more extensive testingis reported with larger problems, although many of the problems were firstsolved ten years previously. decube was an MIMD implementation for theiPSC/2 hypercube, and, as with decompar before, each problem was as-signed to a dedicated processor.

A set of parameters (K, k, p) were introduced for describing the variousstrategies for sequencing the problems solved on a given cycle—the masterproblem was solved as soon as K proposals have been submitted from allpricing problems in total, or k proposals received from each pricing problem,or p pricing problems terminated.

4.4. THE USE OF INTERIOR POINT METHODS 63

decube was applied to the complete set of problems used earlier by Hoand Loute, together with much larger instances (up to 30,000 rows) of theMRP problems from the same set [HL83]. Making a somewhat question-able comparison with mpsx/370 (much better would have been to comparedecube with its own internal LP software, lpm1, to minimise discrepan-cies attributable to software differences), decube achieved parity at about12,000 rows, and beat mpsx by factors of four and six on the two largestproblems. No account was given of other factors that may have influencedthe comparison. An improvement of 20% was attributed to the load balanc-ing ideas, but little raw data was provided to assess this claim.

Entriken made an excellent study of nested Dantzig-Wolfe decomposi-tion, including a sophisticated parallel implementation and comprehensivepractical evaluation [Ent89]. Of particular relevance to us is his descriptionof the implementation, which we draw on in chapter 5, and his methodologyfor testing. His tests, on a particularly wide range of problems culled fromother studies, had four explicit aims: to evaluate parallel nested decom-position in comparison to standard LP methods; to investigate algorithmperformance under different parameter settings; to extrapolate the perfor-mance results beyond the test set; and to identify limitations of the codeand areas for improvement. It was the most comprehensive evaluation of adecomposition method made in forty years. The results obtained were alsopositive, showing both that nested decomposition itself provided a speed-upover the simplex method for larger problems—up to ten times in cases—andnaturally parallel nested decomposition provided further speed-up.

By contrast, the practical work for plain two-level decomposition in par-allel has been nowhere near as comprehensive. There have been some verygood ideas, and recently some further work using interior point methods(see below), but no state of the art implementation.

4.4 The Use of Interior Point Methods

The idea of exploiting structure with interior point methods has seen muchinterest, and in 1991 Kim and Nazareth published an excellent paper whichproposed the idea of using interior point methods (IPM) within the contextof the decomposition method [KN91]. The idea might at first seem slightlyodd, since decomposition is a specialised variant of the simplex algorithm,and makes use of the extreme point and ray solutions generated by thesimplex method when solving the pricing problems. But as we mentioned insection 3.4.3, proposals can be generated from non-extreme points or raysof the pricing problems.

Kim and Nazareth used IPM to solve the pricing problems, retaining thesimplex method for the master problem. They argued that when the pricingproblems are degenerate, there may be many extreme points in the vicinity


of the extreme point required by the master problem to represent the opti-mal, or indeed an intermediate, solution. Thus many cycles are expandedgenerating similar proposals to take tiny steps towards the optimal solutionof the master. Further, numerical problems may redouble the problem bymisrepresenting extreme points of the pricing problem. Conversely, interiorpoints would be less susceptible to the exact combinatorial structure of thepricing problem (see [KN91] for a numerical example). A limited amountof practical work was also conducted (using random problems, which is of-ten not the best test for practical optimisation codes; see [CDM78]) whichshowed that the IPM variant always used fewer cycles than the standardsimplex variant. However, much greater work and time was required eachcycle by the IPM variant to solve the pricing problems, as each pricing prob-lem had to be solved from scratch. In the simplex variant the optimal basisfor each pricing problem from the previous cycle can be exploited to give anadvanced start.

Other authors considered using IPM to solve just the master problemas part of the decomposition method. Here, the argument in favour is thatthe dual solution (the prices) obtained by the IPM will give better guid-ance to the pricing problems than an extreme dual solution, although itis also argued that IPM are best suited to solving the large scale prob-lems arising in decomposition [GSV97]. Goffin et al noted that the masterproblem need only be solved to within a certain tolerance of the optimalsolution at each cycle, thus saving work in the IPM [GHVZ93]. An approx-imate warm start for the IPM method was applied. Brief experience withmulti-commodity network flow problems suggested that convergence of thedecomposition method was always faster.

Picking up the work of Goffin et al, Gondzio, Sarkissian and Vial useda new implementation of the analytic centre cutting plane method (AC-CPM) [GSV97]. They tested the method on mostly random instances ofnonlinear multi-commodity network flow problems, with up to 1000 blocks,2000 constraints and 80,000 variables in each block, and 4000 global con-straints. No comparisons were given with other methods, but the authorsthemselves commented on the long solution times, attributed to having tosolve the master problem from scratch on each cycle, with some instancestaking 24 cycles to converge. Comparisons with a classical simplex baseddecomposition have been carried out by Chardiare and Lisser, with the sim-plex variant coming out a clear winner [CL96]. They questioned whether thebelief in the superiority of interior points over extreme points in the contextof decomposition is warranted.

Gondzio and Sarkissian used an implementation hopdm of an alternativeIPM, the primal dual logarithmic barrier method (PDCGM) [GS96]. Fol-lowing the earlier idea of solving the master problem to a certain tolerance,they suggested gradually increasing the tolerance so that less work is donein the early stages of the algorithm, but greater precision is obtained closer

4.4. THE USE OF INTERIOR POINT METHODS 65

to the optimal solution. Other advanced features of the implementationincluded a form of warm start and a removal of redundant proposals fromthe master problem—‘essential’ if the Cholesky factorisation is to remainsparse. Tests showed the purge strategy reduces CPU time by 50%. Usingthe same test set as [GSV97], they found that their hopdm implementationwas on average 25% faster than the accpm implementation on all but thesmallest problems, but that classical simplex based decomposition was fasteron all but the largest problems, where the simplex variant fails completely.Their implementation of the simplex variant was missing many advancedfeatures—such as the purge—that gave improved performance in previouswork.

Martinson and Tind used IPM to solve both the master problem and thepricing problems within the decomposition method [MT99]. By using inte-rior points on the central path between the analytic centre and the optimalpoint, they viewed classical simplex based decomposition and the analyticcentre IPM variant of Goffin et al as special cases. They claimed fewer cyclesare required in general than either of these two cases, but based this claimon a test set of random small dense problems. The classical simplex baseddecomposition did win on several instances, and the authors’ implementa-tion of simplex based decomposition was poor, with no multiple proposalgeneration for instance, which raises more doubts about the conclusions. In-deed the authors themselves admitted that little attention was paid to thecomputational efficiency of any of the implementations, as they just wanteda unified platform for comparative testing.

Gondzio and Vial suggested and evaluated three enhancements to IPMbased decomposition [GV99]. They used a trust region around the dualvariables (prices) of the master problem, a form of warm start that speededup solution of the restricted master and prevented severe fluctuations in theprices, apparently with no negative effects. They halted the pricing prob-lems early with guaranteed ε-optimality, which saved IPM iterations whensolving the pricing problems but may also have lead to an increased numberof algorithm cycles. This technique, also suggested in [MT99], can be seen asa form of suboptimal proposal generation as implemented by, e.g., [BHS65],but with a guarantee on the optimality gap. They implemented an approxi-mate warm start, by recording a weaker ε0-optimality solution (ε0 > ε) to beused on the next cycle to restart the pricing problem. The looser tolerancewas necessary so that the solution is a valid IPM solution with the updatedprices (and recall that the change in the prices is bounded by the trust re-gion). The resulting implementation, developed from the accpm softwarefor the master problem and hopdm for the pricing problems, can properlybe classified as the first second generation IPM based decomposition code.

The three enhancements were tested using four problems from real ap-plications, with between two and 16 pricing problems, and up to 42,000constraints and 34,000 variables in total. The trust region resulted in sav-


ings of between 17–66%; the suboptimal proposal generation a saving ofabout one fifth, and the warm start a saving of about one third. But onlythe numbers of iterations were reported, not CPU time, which makes itdifficult to assess their work.

Gondzio, Sarkissian and Vial used the software of [GV99] as a ba-sis for a straightforward parallel implementation of IPM based decompo-sition [GSV01]. They used a set of 16 Linux Pentium Pro PCs linked by100Mb/s ethernet and an implementation (mpich) of the MPI message pass-ing standard. Using the same test set as [GSV01], they obtained reasonablespeed-ups of 1.54–1.94 with two processors; 3 with four processors; and 5with eight processors.

The chief advantage of IPM based decomposition would appear to be im-proved numerical stability for degenerate problems, and given the reportednumerical problems with decomposition, this may prove important. Themain drawback is still the extra computational effort required to solve themaster problem and pricing problems repeatedly using IPM, despite theadvances made with warm start and trust regions.

4.5 IP Decomposition: Branch and Price

The popular topic of recent research has been the application of decomposi-tion to integer programming (IP) problems. Decomposition can be embed-ded within a branch and bound framework in a technique called variouslybranch and price or column generation. The difficulty is ensuring that pric-ing problems do not simply regenerate parts of the feasible region of themaster problem that have been eliminated by branching; the solution is tochoose branching rules that can easily enforced in the pricing problems aswell as in the master problem. This topic is beyond the scope of the presentwork, see Barnhart et al [BJN+94] for a survey. Our interest in this sectionlies in how recent work impinges on field of LP decomposition.

Vanderbeck and Wolsey applied IP decomposition to various cuttingstock problems [VW96]. They noted that most of the work is accountedfor by solving the IP pricing problems, investigated ways to reduce thetime taken to solve the pricing problems, and developed an early termina-tion rule by exploiting the characteristics of the surrounding branch andbound algorithm. Savelsbergh suggested using the familiar LP decomposi-tion techniques of suboptimal and multiple proposal generation to achievethis end [Sav97]. He also made the novel suggestion of solving the pricingproblems and generating suitable columns quickly using approximate meth-ods. If the approximate method failed, then the more time consuming butreliable optimal methods could be used to ensure that a valid column isgenerated.

Barahona and Jensen tackled plant location IP problems by solving the

4.6. DISCUSSION 67

LP relaxation at the top node with decomposition, then used standardbranch and bound to solve the resulting IP with no further columns added(what might be termed price and branch) [BJ98]. Of note is the use ofsub-gradient optimisation to ‘improve’ the prices obtained from the masterproblem before applying them in the pricing problems. In fact the sub-gradient optimisation requires the solution of pricing problems very similarto those used in the decomposition, with the objective being to optimise thedual bound. A side effect, and quite possibly the major factor in the posi-tive results obtained, was that they were able to use the solutions from thesub-gradient pricing problems to create additional proposals for the masterproblem, which had the effect of improving the primal value much morerapidly than without the additional proposals.

In the context of branch and price, du Merle et al used a dynami-cally perturbed master problem, which they claimed offered improved nu-merical stability and thus fewer iterations to locate degenerate optimalpoints [dMVDH99].

IP decomposition is still a relatively immature field, and parallels can bedrawn with the first experiences of LP decomposition reported in section 4.1where each study concentrates on a particular application. This trait is evenmore marked in IP decomposition where the scope for exploiting application-specific knowledge is much larger. Many of the studies use such knowledgeto generate initial columns for the master problem, and to provide specialrules for terminating the solution of the pricing problems or master problemearly, in the context of the surrounding branch and bound algorithm. Oneunifying feature has been the use of a common software, minto, in manyof the studies; its continued use and development should allow practicalexperience to be accumulated and exploited in contrast to the experienceswith early LP decomposition.

4.6 Discussion and Opportunities for Further Work

In this concluding section we draw together elements of the survey thatimpinge on a new computational implementation, and note opportunities forfurther work. In 1973 Orchard-Hays noted three major obstacles that hadinhibited successful practical experience with decomposition [OH73], andlet us first consider these points to see how relevant they are in prevailingconditions.

The computer code required to implement decomposition is amagnitude more complicated than that required for the simplexalgorithm. Even if decomposition could be added to an exist-ing simplex code, the resulting code would be more difficult todevelop and maintain.


At the time of writing commercial LP software was not sufficiently advancedto allow the simplex method to be called as a subroutine. Following the leadof mpsx, all major commercial LP software is now able to be employed as asubroutine library, essentially providing a black box LP optimiser. This al-lows a decomposition algorithm to be implemented as a high level algorithm,with a clear interface to the LP optimiser. The writer of a decompositioncode need not be concerned with the implementation specifics of the simplexmethod, and there is no inherent reason for the decomposition code to becomplex.

One of the lasting consequences of the state observed by Orchard-Haysis that past implementations of decomposition were built with no clear in-terface between the LP optimiser and the decomposition algorithm. AsLP optimiser software advanced, the decomposition software remained tiedin with legacy technology, unable to take advantage of the subsequent ad-vances, and falling out of use. decomp and derivatives, reliant on the 25year old lpm1 optimiser, is still usable but hardly state of the art; whiledecompsx, built using the first optimiser subroutine library mpsx, actuallyrelied on certain bespoke modifications to internal mpsx routines and is thusno longer available.

The major advantage of using an optimiser subroutine library withindecomposition software with a clear interface, is that the component canbe updated or replaced over time. This allows the decomposition softwareto exploit further advances in LP optimisation technology, and at the sametime prevents the decomposition software being discarded if the particularLP optimiser is discontinued. A well-defined distinction between the LPsoftware and the decomposition software also aids code development andmaintenance.

The potential disadvantage of using an LP optimiser library is that ad-vantage cannot be taken of being able to modify its internal routines andshare its data structures to give low level efficiencies. Many of the imple-mentations described above employed modifications of various sorts, suchas adjusting the FORMC routine to calculate the objective function of thepricing problems automatically, and specialised scaling routines. This pointis valid to a certain extent, although most implementation details describedabove, and even specialised scaling routines, can be implemented throughthe interface provided an optimiser subroutine library.

We contend that the disadvantages with the decomposition softwarecompletely wedded to the LP software completely outweigh the advantages.Furthermore, state of the art LP software is proprietary and so does notpermit an interface at the level of the source code. (It would of course be adifferent matter if there was serious commitment to develop and maintainboth the decomposition algorithm and the underlying LP optimiser by anowner of high quality LP software.) There has been no published implemen-tation of decomposition which uses a modern optimiser subroutine library,

4.6. DISCUSSION 69

so here is an opportunity to establish a new more lasting implementation.One or other of these approaches is essential for an efficient implementa-

tion. A third approach, of using a stand-alone optimiser, perhaps controlledthrough modelling software, is commonly used for classroom implementa-tions. The problem is that the master problem and each pricing problemmust be regenerated on each cycle, giving a great overhead. Kutcher foresawthis point in 1973, and as a result of using LP optimisation software witha high overhead, tuned his decomposition code to use a minimum numberof cycles [Kut73]. It is worth noting that some of the conclusions drawnby previous works, e.g., regarding computational strategies, may have beenmade under experimental conditions which no longer apply.

The scope of standard LP software has been continually increas-ing, so that any problem solvable only by decomposition todaymay well become amenable to standard methods within a yearor so.

There is certainly a lack of incentive to develop, sell and maintain decom-position as a practical technique. Despite all the claimed successes of thedecomposition, there has been no lasting implementation or application.But perhaps the more pertinent question is: does decomposition offer asignificantly superior performance over, or wider applicability than, stan-dard methods? If there is a significant margin of benefit for decomposition,then the method would have a buffer against increases in raw computingpower and resource that could give it a significant, albeit moving, market ofapplications. We return to this issue later on, in section 5.10.

There is a smaller class of problems that are available to de-composition than to LP in general, and the application of de-composition requires the additional effort of identifying a blockstructure, so there is less motivation to develop, apply and im-prove decomposition methods and software.

The lack of experience and practice is self-reinforcing. And it manifests itselfin other ways—no common interface for problem input; no standardisedtest set of problems. Software implementations, where they have existed,were immature and needed expert control, and fell into misuse once thedevelopers had moved on to other areas. All of these problems would havedisappeared if the use of decomposition had reached a critical mass. Thisis a big incentive to produce and promote good software tools, to which wehope this work contributes.

A related problem has been an unwillingness or ignorance to re-use anddevelop existing code. With the notable exception of the work by Ho andLoute, past experience has not been exploited when building new implemen-tations, even in the most recent work.


The experience we have surveyed does debunk one of the myths of de-composition: that it always suffers from slow convergence. Whilst therehave been reports of slow convergence, there have equally been reports inwhich slow convergence was not observed. Ho and Loute evaluated their im-plementations on a sufficient variety of problems to validate decompositionas a capable and robust problem solver.

The most significant gap in previous work has been that no systematicstudy of the algorithm with its different computational strategies on a varietyof problems has been carried out. This is essential for a definitive study ofdecomposition.

The experience gained in parallel implementations of decomposition ispatchy. The range of hardware, quality of the LP optimiser technology, andthe scope of problems solved have all fallen short of a definitive investigation.This is in contrast to the excellent work undertaken by Entriken to imple-ment and evaluate nested Dantzig-Wolfe decomposition. Different parallelcomputational strategies have been proposed, but the range of applicabilityof the conclusions is narrow. Even so, it does seem likely that the basicstrategy, which is the most simple to implement is among the most efficient.Whichever strategy is used, it can only alter the order in which proposalsare added to the master problem on a micro level.

Amdahl’s Law of parallel computing states that the potential speed-upand efficiency is limited by the amount of work that must be done in serial.In the case of decomposition, the minimum time is governed by the masterphase together with the bottleneck pricing problem, no matter how manyprocessors are available. If the time to solve the master problem dominatesthe decomposition algorithm for a specific problem instance, then it is fair tosay that the problem instance is just not suited to parallel decomposition. Hoand Gnanendran suggested performing certain non-critical tasks during idletime—the LP re-factorisation, periodic update of the primal solution, theaccuracy check, and housekeeping associated with incorporating proposalsinto the master problem [GH93]. But implementing these efficiencies wouldentail breaking into the LP software and destroying the clean interface,which, following our discussion above, could only be justified if the savingwas significant.

To summarise the work with IPM based decomposition, the work byGondzio, Sarkissian and Vial in particular has enabled rapid progress tobe made. They have developed advanced software, capable of solving someof the largest problems currently amenable to decomposition, and makinga number of theoretical and computational advances on the way. In par-ticular, their developments of warm start have enabled the method to beapplied efficiently. However, from our perspective there are still many areasopen to further investigation. Testing on problems derived from real worldapplications has been limited to a few specific classes and instances. Wherecomparisons have been made with simplex based decomposition, the imple-

4.6. DISCUSSION 71

mentations of simplex based decomposition used have been poor, not usingmany of the advanced features or tuned computational strategies describedin the earlier sections. The authors’ interest has lain in applying IPM ratherthan trying to make both IPM and simplex variants produce the best possi-ble results, and the conclusions are in any case inconclusive. The work withparallel implementations is still in its early stages.

The implementations of decomposition used in these comparisons—par-ticularly the simplex based variants—missed many of the computational fea-tures that are reported to offer improved performance. An implementationof decomposition incorporating these features, which used LP optimisationas a library, would allow a proper comparison of different LP optimisationtechnologies in the same environment. Our implementation would enablethis, although we do not consider the use of IPM methods in our evaluation.

The use of decomposition in IP is an exciting area, and it is very likelythat it could benefit more from work done with LP decomposition.

It should be clear from this survey that practical work in decompositionhas been characterised by a lot of ideas and inventiveness, a willingness totry things out and see what works. At various times, it seems that decompo-sition has been comparable to mainstream LP methods. However, it shouldbe equally clear that there have been two serious deficiencies: no full eval-uation of the different computational strategies; and a lack of evolutionaryimprovement in implementations, drawing on and improving previous work,and using the best available components for the LP optimisation. In ad-dition, the potential for parallel exploitation of the algorithm has not beenfully explored, particularly with regard to common parallel architecturessuch as multi-processor PCs and networks of PCs. We aim to rectify thesedeficiencies, whilst drawing on the wealth of techniques reviewed here, withthe implementation and evaluation described in the next chapter.


Chapter 5

A Computational Evaluationof Dantzig-WolfeDecomposition

Is Dantzig-Wolfe decomposition a useful practical tool for solving primalblock angular structured LP problems? A Dantzig-Wolfe decompositionoptimisation code was written and a series of experiments carried out ona suite of problems to answer this question. The code enabled differentoptimisation strategies, and permitted both serial and parallel operation.Four phases of experiments were carried out:

• to validate that the code actually solved problems correctly;

• to compare different optimisation strategies and identify a good overallstrategy;

• to evaluate performance on various parallel architectures; and

• to compare performance to that of a standard LP optimisation soft-ware.

This chapter contains the central contribution of our work: a thoroughevaluation of the practical merits of Dantzig-Wolfe decomposition. This isthe first work to compare different optimisation strategies for Dantzig-Wolfedecomposition using a sound statistical methodology. This is the first workto evaluate the performance of Dantzig-Wolfe decomposition on a varietyof common parallel architectures. All experiments were carried out usinga single consistent and flexible software harness, designed and implementedfollowing the theoretical framework laid down in chapter 3, and building onthe best practice reviewed in chapter 4.

Novel features of our software include: an automatic means of varyingthe proposal generation frequency as the algorithm progresses; specialised

73

74 CHAPTER 5. COMPUTATIONAL EVALUATION

scaling routines used in proposal generation; and a parallel implementation,fully integrated with the serial implementation, suitable for use on bothdedicated parallel machines and networks of serial computers. The softwareuses state of the art software library components for LP optimisation andparallel control and communication, which can be updated and replacedeasily as new advances are made in these technologies. Using Dantzig-Wolfedecomposition implemented in our software, we were able to solve problemsof up to 52 blocks, 83,500 rows, 83,700 columns, 622,000 non-zero elementsin 63% of the time taken by a commercial LP optimiser to solve the problemusing its advanced dual simplex method. If we were content with a solutionguaranteed to be within 0.1% of optimality, then the solution time can becut by a further 32%. Using parallel hardware we can cut the solution timefurther: by 59% on a four processor PC (equivalent to a speed-up of 2.42)or 81% on seven processors of a Silicon Graphics workstation (a speed-up of5.22).

Our presentation of computational results follows the recommendationsof [CDM78] and draws on studies of a similar nature such as [Cor99, Ent89,HL81]. The statistical methodology of section 5.5 is taken from [Mon01].The first section describes our practical implementation of Dantzig-Wolfedecomposition with particular emphasis on our own novel ideas and exten-sions. Section 5.2 gives details of the problem suite used for testing, andsection 5.3 describes the computational environments used for testing. Sec-tions 5.4 to 5.7 present the results of the four phases of experimentationintroduced above.

Our decomposition software failed to solve some of the problems in-cluded in our test set under certain conditions, and we discuss these andthe wider issue of numerical robustness in section 5.8. Section 5.9 considersthe results obtained on our most important set of problems, those obtainedfrom PowerGen, in particular comparing the results obtained with differentblock structures on the same underlying LP problem. Finally, drawing to-gether all of the experimental work, we discuss the merits of Dantzig-Wolfedecomposition as a practical tool for optimisation.

5.1 Implementation

5.1.1 Overview

Our implementation of Dantzig-Wolfe decomposition, DecOpt, is written in7600 lines, 129 functions and 37 files of Standard C. It uses two subroutinelibraries in addition to the Standard C library: the xpress-mp OptimiserSubroutine Library (xosl), which provides the LP solver; and High Perfor-mance Parallel Virtual Machine (hppvm), which provides the parallel har-ness. There are no inherent limits on problem size in general, with systemresources being the only constraint, although xpress-mp imposes a limit

5.1. IMPLEMENTATION 75

of 62 pricing problems. It has been designed both as practical software forsolving problems, and as a flexible tool for evaluating Dantzig-Wolfe decom-position. What this means is that DecOpt can be used on one level as a blackbox optimiser, requiring very little knowledge of decomposition or the differ-ent computational strategies from the user, whilst the options and analysisavailable on demand enable the more sophisticated user to experiment withdifferent strategies, serial or parallel operation, using one consistent code.DecOpt has been used successfully, to a high degree of sophistication, byboth PowerGen and BASF as part of the MEMIPS project.

The main modules of DecOpt initialise the control variables; read inthe problems from a set of matrix files and allocate memory; perform theDantzig-Wolfe algorithm; and calculate a summary of performance. Thealgorithm itself is divided into three phases: (i) find an initial feasible solu-tion to each of the pricing problems; (ii) solve the master problem to proveninfeasibility, optimality, or unboundedness; and (iii) compute and output anoptimal primal solution to the original problem.

DecOpt implements the column generation algorithm for Dantzig-Wolfedecomposition as presented in section 3.4.5 and we do not repeat the detailshere. Rather, we concentrate on the open aspects of the theoretical algo-rithm, and computational details of the implementation. Following previoususage, we use the term computational strategies to refer to possible resolu-tions of open aspects of the algorithm, although heuristics would be a moreaccurate term.

For a detailed user guide and reference see the DecOpt User Manual,which is included in this work as appendix C.

5.1.2 Proposal Generation

The most open aspect of the column generation algorithm, and thus thearea with greatest scope for computational strategies, is that of proposalgeneration. In chapter 3, we noted that every basic feasible solution of eachpricing problem corresponds to a valid proposal, whilst the convergence ofthe algorithm is guaranteed if we simply generate one profitable proposaleach cycle.

From the first implementations of Dantzig-Wolfe decomposition, it hasbeen recognised that the number of cycles can be reduced by generating mul-tiple proposals from each pricing problem on each cycle (see our review of[BHS65] in section 4.1). This can be achieved by generating proposals frombasic feasible points encountered by the simplex algorithm before the opti-mal solution of the pricing problem is reached. Proposals generated beforean optimal solution is obtained are intermediate proposals; those generatedat an optimal or unbounded solution are terminal proposals. However, gen-erating too many intermediate proposals can cause its own problems—lotsof similar proposals in the master problem offers little benefit in reducing


the number of cycles, and may hinder progress by increasing the amount ofwork required to solve the master problem and creating numerical instabilityin the master problem.

We use a set of heuristics called a proposal generation strategy to gen-erate multiple proposals whilst controlling the number and similarity of theproposals to limit ill effects. DecOpt supports several mechanisms for con-trolling multiple proposal generation: limiting the number of proposals gen-erated from each pricing problem; preventing two proposals being generatedwithin a minimum number of simplex pivots; preventing a proposal beinggenerated without a minimum percentage improvement in the reduced costover the previous proposal, both in order to avoid similar proposals; limit-ing the number of proposals generated from all pricing problems during onecycle; and limiting the number of pricing problems solved during one cycle.These mechanisms are controlled by a set of parameters, which allow theuser to control each mechanism. A proposal generation strategy is specifiedby values for the control variables, which we indicate here in typewriterstyle.

A check is made to generate a proposal every propfreq pivots during theoptimisation of a pricing problem, and also when the final optimal solutionis attained. (A value of zero for propfreq disables the generation of inter-mediate proposals.) A proposal is generated if the solution is feasible, andthe reduced cost of the solution, when viewed as a prospective variable inthe master problem, is below the reduced cost criterion propcostmax. Eachtime a proposal is generated, propcostmax is increased by propcost%. (Avalue of zero for propcost disables the increase.) Proposal generation con-tinues until optimality is obtained, or propgcol proposals, of total length(number of non-zero coefficients) propgelt, have been generated. The pric-ing problem is always solved to optimality (unless unbounded), so that thefinal objective value can be used in calculating a new dual bound if possible.

The proposal generation frequency, propfreq, is initialised with a com-mon value and then varies independently for each pricing problem using ourown innovative scheme. We define the observed proposal frequency to bethe number of simplex pivots taken to re-optimise the pricing problem afterthe first proposal is generated, divided by propgcol − 1, where propgcolis the number of proposals allowed from a single pricing problem. It is thevalue of propfreq that would have resulted in propgcol proposals beinggenerated evenly during re-optimisation of the pricing problem. At the endof each cycle propfreq is updated using the most recent observed proposalfrequency. A decay factor weights the influence of the most recent frequencywith past frequencies. The new value of propfreq is:

(1− propfreqdcay)× propfreq

+ propfreqdcay× (observed proposal frequency).

A value of zero for propfreqdcay means that propfreq will remain constant;


whilst a value of one means that propfreq will be updated each cycle with nomemory of earlier cycles. Intermediate values give combinations of these twoextreme policies. An additional control, propfreqmin, allows a minimumvalue for propfreq to be specified.

If a pricing problem is found to be unbounded, a different mechanism isemployed to generate multiple proposals. There are often many unboundedrays emanating from the same basic solution, or within close proximity, andgenerating a single unbounded ray in this case means it can take severalcycles to fathom the unboundedness [Ho78]. Hence, once an extreme rayproposal has been generated, we fix the vector causing the unboundedness tozero, and resume optimisation and proposal generation. Several unboundedvectors may be found and fixed to zero in this way. At the end of the cycle,the fixed vectors are freed up again.

Each pricing problem is considered in turn, until cycsubmax pricing prob-lems have been solved, or the total number or length of proposals generatedduring the cycle exceeds propicol and propielt respectively. (A value ofzero for cycsubmax means all pricing problems are solved on each cycle.) Inparallel DecOpt, these limits apply independently to the set of pricing prob-lems held on each slave process (see section 5.1.9 below). On the next cycle,the pricing problem phase resumes at the next pricing problem in sequence,so that the pricing problems get equal treatment on average.

The use of a proposal frequency was first described in [BHS65]; theyobtained their best results by varying the frequency during the algorithm aswe described earlier in section 4.1. Their approach of manually varying thefrequency is not practicable in general, and requires detailed experimentalwork with a problem to obtain a good strategy. Our proposal frequencydecay factor scheme offers a novel means to automate a dynamic proposalfrequency. Orchard-Hays describes the use of a reduced cost criterion andthe percentage improvement factor [OH68]. Ho integrated and parame-terised these two mechanisms and also suggested the strategy when thepricing problem becomes unbounded [Ho78]. The cycle limits were used byHo and Loute [HL81]. Other mechanisms for controlling proposal generationhave been proposed: Loute has suggested comparing prospective proposalsby their angular difference [Lou97]; the computation could be accomplishedusing a scalar product. This may be a more effective but possibly costlymeans of finding dissimilar proposals than either the proposal frequency orthe reduced cost schemes. At the time of writing this scheme had not beeninvestigated.

5.1.3 Proposal Management

As the column generation algorithm proceeds, proposals are added to therestricted master problem in order to build up a representation of the fullmaster problem on a path to its optimal solution. The algorithm leaves open


the question of whether proposals are deleted from the restricted masterproblem. If proposals are deleted, there is a chance that the same proposalwill have to be regenerated in a later cycle, thus delaying progress. Con-versely, progress can also be delayed by the presence of many proposals inthe master problem, as they can be expensive to price out. (Proposals aretypically dense, much more dense than columns in practical LPs [OH72].Some of the trickier problems tackled later in this chapter had average pro-posal densities of up to 42%, and even the problems that perform well hadproposal densities averaging 10–20%.)

As with proposal generation, DecOpt provides a set of parameters to de-fine a proposal management strategy, a strategy for managing the persistenceof proposals in the master problem. A check is made every purgfreq cyclesto delete (or purge) proposals from the master problem, or whenever thenumber and length of proposals in the master problem, including those justgenerated, exceeds propmcol and propmelt respectively. Although there isno guarantee that a proposal will not be required again, we offer two meth-ods to measure redundancy: the number of cycles that a proposal has beennon-basic, and the reduced cost of a proposal. A proposal is condemned ifit has been out of the basis for purgnbas or more cycles, and its reducedcost is above a criterion level, purgcostmax. (Either test may be disabled.)Proposals that are basic or generated during the previous pricing problemphase are not considered for deletion. The purge is repeated with relaxedcriteria until the number and length of proposals remaining are within thespecified limits.

Dantzig described the use of a purge, and the reduced cost as a measureof redundancy [Dan63]. Schiefer suggested deleting proposals once they hadbeen non-basic for a certain number of iterations [DJ79].

5.1.4 Proposal Initialisation

One minor unresolved issue is the set of proposals {E′k}Kk=1 used to initialisethe algorithm. The master problem may be started without any propos-als, just the logical (artificial and slack) variables on the global and con-vexity constraints. However, previous studies have indicated that startingthe algorithm with a pre-computed set of proposals does accelerate con-vergence [DJ79, WR74], and indeed for [BHS65] it was one of the mostimportant factors. We should be a little bit wary of such claims, as anywork done outside of the algorithm will obviously give the effect of reducingthe work done within the algorithm, although the overall workload may beunchanged. But, if the proposals can be generated cheaply, perhaps by someheuristic, or perhaps a range of proposals is immediately suggested by theapplication, then the benefits will be real.

In previous studies, proposals were generated in advance by exploitingproblem-specific knowledge. In [BHS65], the proposals represented the ac-


tivities of different oil fields, and the pricing problems could be fathomedby computing proposals representing various extreme cases of activities.In [WR74], the proposals represented recipes, and recipes generated on pre-vious data sets could be re-used to warm-start algorithm for the currentdata. Where the application area is not known, as is the case for a gen-eral solver such as DecOpt, problem-specific methods cannot be used. Onecould try to generate a range of proposals by solving the pricing problems inadvance, perhaps by varying the dual values to give several representativeproposals, but this is unlikely to be better than generating proposals guidedby the dual values of the master problem, within the column generationalgorithm.

The resolution adopted in DecOpt is to generate one proposal for eachpricing problem before entering the column generation algorithm proper, bysolving the pricing problem with its original objective function, i.e., effec-tively setting the dual values of the master problem to zero. This has usefulside-effects: the feasibility of each pricing problem is checked at the begin-ning, and the proposals generated can be used as part of a simple proposalcrash, by making each basic on its respective convexity row, in place of theartificial variable on the convexity row.

An extension which could give some of the benefits of proposal pre-computation but without requiring any problem-specific components, wouldbe to allow pre-computed proposals to be input from file. These proposalscould be included in the master problem from the start, but then treatedjust as other proposals, with redundant ones being removed at later cycles.Indeed, one simple mechanism would be to include initial proposals in theoriginal master problem matrix file as ordinary matrix columns, with a con-trol parameter specifying the index of the first column in the matrix to betreated as a proposal. This would be especially useful where a class of modelproposals could be specified at the formulation stage in the model.1

5.1.5 Termination

The algorithm terminates when one of the following events occurs:

1. No proposals are generated on a cycle, which proves that the currentfeasible solution for the master problem is optimal, or, if infeasible,that no feasible solution exists.

2. The master problem is unbounded.1The idea of model proposals is related to that of model cuts implemented in xpress-

mp [Das00]. There, constraints (cuts) may be specified when formulating a MIP problemthat are not required to guarantee a feasible solution but improve the quality of theLP relaxation. The cuts are often numerous, so only those that are violated in an LPrelaxation are included to the matrix during MIP optimisation. The term model cuts isused because the cuts are present in the original model rather than being generated frompartial solutions.


3. A pricing problem is infeasible.

4. A feasible solution is found which is within optabs or optrel% of thedual bound.

5. The decomposition cycle (maxcyc) or time (maxtim) limits are reached.

6. The algorithm is interrupted by the user pressing control-C.

5.1.6 Proposals Computation

Proposals are generated direct from the pricing problem matrix using thescheme of [BHS65]; see our earlier discussion in section 4.1. The portion ofthe global rows pertaining to a pricing problem is included in that pricingproblem’s matrix as non-binding constraints with zero RHS. To obtain thecoefficients of a proposal generated from an extreme point, the slack valuesof the global constraints are read and negated, which, since the RHS iszero, gives us the desired activity values of the global constraints; while thecoefficients of a proposal generated from an extreme ray are the entries of thecoefficient column in the current basis of the unbounded vector, which arecalculated by obtaining the original coefficient column, and pre-multiplyingit by the current basis inverse. One or both of two scaling strategies maybe applied: proposal columns may be scaled by their infinity norm; and theconvexity row values of all proposals from a given pricing problem may bescaled by the inverse of the infinity norm of the first proposal generated.Scale factors are rounded to the nearest power of two to maintain numericalaccuracy [Tom75a].

The proposal generation buffer, or g-buffer, stores the proposals gener-ated from one pricing problem in a cycle until work finishes on that pricingproblem. Proposals are then transferred to the intermediate proposal buffer,or i-buffer, which holds them until the pricing problem phase of a cycle isfinished. Proposals are then loaded into the master problem matrix, wherethe space designated for that purpose is known as the master proposal buffer,or m-buffer. All buffers hold the proposals in sparse column-wise format.In parallel DecOpt, the proposals collected in the local i-buffer on a slaveprocess are sent to the i-buffer on the master process at the end of a cycle,before being loaded into the master problem matrix.

All data arrays required by DecOpt, including working arrays, are allo-cated during the initialisation of DecOpt. No resizing takes place during thealgorithm itself.

5.1.7 Price Computation

In section 3.4.3 we showed that the objective function of each pricing prob-lem is defined in terms of the dual variables of the master problem (see


equation (3.14), p. 45),

zk = c′kxk := (ck − πAk)xk,

so that the objective value of a feasible point corresponds to the reducedcost of the corresponding proposal. When solving the phase I master prob-lem, the contribution of the original objective, c, is excluded. In DecOpt,the updated objective function coefficients c′k of each pricing problem arecomputed as a scalar product

c′k = −(−δ : π)(ck : Ak),

where δ is zero if the master problem is infeasible and one if it is feasible.In fact we store the objective coefficients ck and the global constraint coef-ficients Ak together, in one set of sparse row-wise arrays, and extend the πvector to include the δ indicator. The objective coefficients c′k are updatedat each cycle using the (−δ : π) vector obtained from the master problem.

This method originated in [BHS65] (see also [OH68, p. 68]). It is possibleto implement the calculation more directly than has been done in DecOpt,by modifying the internal FORMC routine of the LP solver to perform thecalculation; alternatively, the objective function can be declared to be alinear combination of the global constraints [BHS65, HS89]. In DecOpt this isnot possible, because the LP solver did not allow this access or functionality,and the calculation is performed within DecOpt’s intrinsic code.

5.1.8 LP Optimisation

The choice of the LP software to optimise the master problem LP and pric-ing problem LPs that occur throughout the column generation algorithm iscrucial to the overall efficiency of the decomposition software. We use thexpress-mp Optimiser Subroutine Library (xosl), which is one of the lead-ing LP optimiser codes available. All of its advanced features are exploitedor available in DecOpt, including its presolve, matrix scaling, alternativeLP algorithms, primal simplex crash, pricing strategies, and re-optimisationwithout requiring a basis re-factorisation. We have already discussed themerits of using an LP optimiser subroutine library (section 4.6); in particu-lar, new releases can be plugged in when they become available, and indeedit would be reasonably easy to replace the library by another which offeredsimilar functionality.

The optimiser subroutine library also performs various house-keepingroles: it is used to input the matrices and output the solution in widelyaccepted formats; its own multiple matrix facility is used to manage thematrices of the master and pricing problems, with all associated data struc-tures necessary for LP optimisation; its matrix scaling routine is used tore-scale the master problem whenever a purge occurs; and the routine to


check whether a proposal should be generated from the current solution isimplemented using an entry point within the simplex algorithm routine.

When comparing an implementation of decomposition to standard LPmethods, it is important to use the same LP software that is embeddedwithin the decomposition software. This removes a potentially large sourceof variability from the experiment.

5.1.9 Parallel Implementation

The parallel implementation of DecOpt uses the coarse grained distributedparallelism offered by the independence of the pricing problems in the pricingproblem phase. The master process, the DecOpt program initiated by theuser, itself initiates a number (nslave) of slave processes, which run a cutdown version of DecOpt called DecSlave. The master process allocatesthe pricing problems evenly among the slaves and itself, with a slight biastowards the slave processes. The master process retains the master problem,and co-ordinates the slave processes by broadcasting the prices (π) obtainedby solving the master problem, and receiving the proposals generated onthe slave processes in return. Thus the parallelism springs directly from thealgorithm itself, and echoes the decentralised planning methods proposedfor the Dantzig-Wolfe method when it was first devised [DW60].

We can predict the performance of DecOpt in parallel using the resultsof the serial performance. Let tm be the total time taken to solve the masterproblem, and let ts be the average total time to solve one pricing problem,with K pricing problems in total. The total time to solve the problem inserial is then

T1 = tm +Kts,

and the approximate time Tp required to solve the same problem on p pro-cessors is

Tp = tm + dK/pets,

where d·e means to round up to the next integer, and allows for the granu-larity of the decomposition. This estimate ignores the overhead of paralleli-sation and the possibility that the column generation algorithm may take adifferent solution path. But it does take into account the serial execution ofthe master problem phase.

The speed-up is

Sp =T1

Tp=

1 +Kts/tm1 + dK/pets/tm

.

The efficiency is

Ep =Spp

=1 +Kts/tm

p+ pdK/pets/tm.


We can see how the enforced serial optimisation of the master problemrestricts the scope for good speed-up and efficiency in parallel DecOpt, fol-lowing Amdahl’s Law. Let R = Kts/tm, the ratio of the time taken to solvethe pricing problems to the time taken to solve the master problem. Ignor-ing the granularity of the parallelisation, the maximum possible efficiency isapproximately

Ep ≈1 +R

p+R.

We see that if R is much greater than zero, Ep ≈ 1 and the speed-up willbe near-linear; whereas if R is close to zero, Ep ≈ 1/p and the speed-up willbe non-existent.

Parallel DecOpt is implemented using an implementation of the ParallelVirtual Machine (PVM) protocol [GBD+]. The parallel virtual machine maybe a single parallel computer or a set of computers linked by a network—bothtypes of platform are treated in the same way under PVM, with independentprocesses running on the different processors using independent memory. Alldata transfer and operational co-ordination is handled by explicit messagessent between the processes. We use an industrial strength implementationof PVM, known as High Performance PVM (hppvm), in the form of a sub-routine library and a set of daemon programs which govern the spawningof slave processes and the communications between the processes [Mel98].hppvm is known for its reliability and speed, is used in commercial softwaresuch as xpress-mp, and is available for many common platforms, includ-ing networks of PCs, workstations, and dedicated parallel computers suchas multi-processor PCs and SGI machines, giving DecOpt a wide range ofpotential platforms.

There are minimal algorithmic differences between the parallel and serialversions of DecOpt, and indeed 82% of the code is common between thetwo. This removes one level of variability between the serial and parallelversions, giving more credence to comparisons between them. We study theperformance of parallel DecOpt in section 5.6 below.

5.1.10 User Interface

DecOpt has been designed both as a black box optimiser, to allow a usersimply to input the problem and solve it, and as an experimental code, toallow the sophisticated user great scope to test the performance of DecOptwith the wide range of strategies described above.

The important algorithmic and heuristic controls have been mentionedabove; the DecOpt User Manual contains a complete list (see appendix C,section C.10). The default values have been selected following the experi-mental work reported later in this chapter. The user can alter the defaultsettings using command line arguments, useful during experimentation to


quickly try a number of different settings, or in an initialisation file, usefulfor preserving preferred control values for a particular class of problems.

Of particular note are the rules used to set the maximum numbers andlengths of proposal buffers. The g-buffer can handle up to propgcol propos-als of total length propgelt; the i-buffer can hold up to propicol proposalsof total length propielt; and the m-buffer can hold up to propmcol propos-als of total length propmelt. All sizing controls are set automatically usinga set of built-in rules: given a value for propgcol, propicol is set to allowa full quota of proposals from each pricing problem (or just cycsubmax pric-ing problems per cycle if this control has been set), propmcol is set to allowa full set of proposals to be added to the master problem for purgcycmincycles before a purge is required, and all proposal length controls are setto allow proposals of density propdens%, or, if preferred, of propnelt ele-ments. Any of the default settings can be overridden by the user, in whichcase the remaining controls are deduced from the user’s settings using thesesame rules.

The output of DecOpt is similarly designed for the practitioner and ex-pert alike: a comprehensive set of messages are graded into levels, and thelevel of messages output to the console and log file can be varied indepen-dently to suit the user’s needs. The messages allow the user to analyse theoperation of DecOpt, especially those areas like proposal generation wheredifferent strategies can be applied. In addition, a set of performance statis-tics are output on demand.

Other features include the following. If optimisation is halted, the stateof DecOpt can be saved to file, and optimisation resumed from the file in alater session. The master and pricing problem matrices can be assembledinto one matrix representing the full original problem, which can be outputand used in a conventional LP optimiser to verify the solution obtained withDecOpt. A full primal solution to the original problem can be computed byDecOpt using the scheme of [BHS65].

5.2 Experimental Problem Suite

Our suite of problems for evaluating DecOpt has been drawn from the sixsources described below. Tables 5.1 to 5.6 give the dimensions of the prob-lems in each set: the number of subsystem blocks in the block structure(i.e., the number of pricing problems); the number of rows, columns andmatrix non-zeros (Elts) in the complete problem, excluding slack columns;the number of global rows (GRows), i.e., the number of rows in the masterproblem; the density of the complete problem; and the optimal objectivevalue where published and verified using the xpress-mp Optimiser.

Ho & Loute (Table 5.1) This set of sixteen diverse problems was collectedby Ho and Loute to evaluate their implementations of Dantzig-Wolfe

5.2. EXPERIMENTAL PROBLEM SUITE 85

Problem Blocks Rows GRows Cols Elts Density Objectivedregem 10 2580 67 1533 15,937 0.174% unboundeddyna 5 2331 596 5520 16,563 0.103% unboundedegd034 10 582 8 272 1454 0.410% 39,728.00221egd085 10 1443 8 680 3635 0.166% 99,320.00552fixmar 4 325 18 452 2601 1.159% 18,416.75903forest 6 402 11 603 3794 1.039% 61,542.19551meat12 6 381 46 311 2992 1.279% 150,907.6186meat31 8 384 11 577 4284 1.265% 517,665.0249meat43 9 648 16 605 4720 0.661% 42,223.13685mege06 6 1011 17 687 6066 0.412% 351.2506712mege08 8 1343 17 908 8200 0.316% 552.6390475mege10 10 1675 17 1172 10,610 0.258% 744.0053656mrp3 3 301 31 522 2010 0.933% 30,009.97201mrp5 5 961 61 1740 8450 0.363% 109,469.8188regem 9 1263 9 2300 31,652 0.731% 139,381.9013scorpion 6 389 53 358 1708 0.722% 1878.124823

Table 5.1: Dimensions of the Ho & Loute problem set.

Problem Blocks Rows GRows Cols Elts Density Objectivescfxm2 2 661 6 914 5229 0.566% 36,660.26157scfxm3 3 991 11 1371 7846 0.378% 54,901.25455sctap1 12 301 61 480 2052 1.001% 1412.250000sctap2 47 1091 151 1880 8124 0.284% 1724.807143sctap3 62 1481 241 2480 10,734 0.208% 1424.000000

Table 5.2: Dimensions of the Netlib problem set.

Problem Blocks Rows GRows Cols Elts Density Objectivesmall 16 1330 18 24,625 290,966 0.847% 199.4868165med1 16 2738 18 51,377 614,166 0.416% 199.4673396med2 16 1970 18 36,785 437,654 0.576% 199.4781955large 16 3634 18 68,401 820,502 0.315% 199.4545125

Table 5.3: Dimensions of the Gondzio problem set.


Problem Blocks Rows GRows Cols Elts Density Objective7-01-s 23 1659 182 1839 10,954 0.217% 9941.8241437-01-t 7 1659 271 1839 10,954 0.217% 9941.8241437-02-s 23 1577 182 1757 10,332 0.227% 7053.5802157-02-t 7 1577 271 1757 10,332 0.227% 7053.5802157-03-s 23 1489 182 1669 9647 0.237% 4739.5872287-03-t 7 1489 271 1669 9647 0.237% 4739.5872287-04-s 23 1595 182 1775 10,546 0.226% 6773.4102067-04-t 7 1595 271 1775 10,546 0.226% 6773.4102067-05-s 23 1531 182 1711 10,112 0.235% 5251.2819937-05-t 7 1531 271 1711 10,112 0.235% 5251.2819937-06-s 23 1671 182 1851 11,147 0.218% 5239.4293687-06-t 7 1671 271 1851 11,147 0.218% 5239.4293687-07-s 23 1769 182 1949 11,885 0.208% 6603.9289877-07-t 7 1769 271 1949 11,885 0.208% 6603.9289877-08-s 23 1839 182 2019 12,413 0.201% 6037.2962247-08-t 7 1839 271 2019 12,413 0.201% 6037.2962247-09-s 23 1839 182 2019 12,413 0.201% 7382.3573897-09-t 7 1839 271 2019 12,413 0.201% 7382.3573897-10-s 23 1809 182 1989 12,196 0.204% 7374.6657117-10-t 7 1809 271 1989 12,196 0.204% 7374.6657117-11-s 23 1839 182 2019 12,413 0.201% 8279.0166117-11-t 7 1839 271 2019 12,413 0.201% 8279.0166117-12-s 23 1769 182 1949 11,906 0.208% 7897.6450447-12-t 7 1769 271 1949 11,906 0.208% 7897.64504484-s 23 19,259 804 19,433 132,154 0.020% 84,758.6990784-t2 42 19,259 1159 19,433 132,154 0.020% 84,758.6990784-t4 21 19,259 676 19,433 132,154 0.020% 84,758.6990784-t7 12 19,259 469 19,433 132,154 0.020% 84,758.6990784-t14 6 19,259 331 19,433 132,154 0.020% 84,758.69907364-t7 52 83,531 1669 83,705 622,257 0.005% 368,652.9263364t14 26 83,531 1071 83,705 622,257 0.005% 368,652.9263364-t28 13 83,531 772 83,705 622,257 0.005% 368,652.9263

Table 5.4: Dimensions of the PowerGen problem set.

5.2. EXPERIMENTAL PROBLEM SUITE 87

Problem Blocks Rows GRows Cols Elts Density Objectivecinca2 4 314 166 612 2004 0.797% 0.000000000minsil7 4 715 319 1260 4178 0.346% 4477218.830minsil19 4 4885 2145 8564 28,434 0.051% 249364667.3navia90 4 268 120 468 1552 0.923% 1057644.000tajo6 4 217 117 392 1416 1.236% 694620.1629

Table 5.5: Dimensions of the Hydrothermal problem set.

Problem Blocks Rows GRows Cols Elts Density Objectivepds001 11 1474 88 3729 10,472 0.156% 29,083,930,523pds003 11 4594 304 12,287 34,969 0.051% 28,597,374,145pds005 11 8100 554 23,639 67,749 0.030% 28,054,052,607pds009 11 14,832 1049 42,680 122,419 0.016% 26,974,586,241pds012 11 20,043 1431 60,808 174,933 0.012% 26,103,493,922pds020 11 33,875 2448 105,728 304,153 0.007% 23,821,658,640pds050 11 83,061 5720 270,095 764,161 0.003% 16,603,525,724

Table 5.6: Dimensions of the PDS problem set.

decomposition [GH93, HLS88, HL81, HL83]. Applications include pro-duction scheduling, multi-product ingredient mix optimisation, multi-national dynamic energy planning, forest management, and materialand capacity planning; see [GH93, HL83] for the sources of the indi-vidual problems. By today’s standards the problems are small: thelargest contains 2580 rows and 5520 columns, the minimum density is0.01%, and they have at most ten blocks. The problems were obtainedin the form of MPS matrix files with the block structure pre-defined.

Netlib (Table 5.2) The Netlib library of LP problems is widely used toreport testing of LP software [Gay97, Lus89]. Unfortunately only thematrix files of the problems are available, not the algebraic models,which makes it difficult to identify and exploit block structure withinthe problems. Nevertheless, five problems were found to have a suffi-ciently obvious block structure, which allowed us to apply decompo-sition. (Some other Netlib problems are included in the Ho & Louteproblem set.) These problems are also small, although the number ofblocks ranges up to 62.

Gondzio (Table 5.3) This set comprises four problems which arise from aMarkov decision problem in manufacturing [GSV01, GV99]. As withthe Ho & Loute set, we obtained the problems in the form of matrixfiles, with a pre-defined block structure. Each problem comprises six-


teen blocks, with up to 68,500 rows, 821,000 columns, and a minimumdensity of 0.31%.

PowerGen (Table 5.4) This set of unit commitment problems is used todetermine how electricity generating units are scheduled (committed)to meet predicted demands [Pow98]. In the largest data instance, a 1%change in the objective value represents £520 million. These problemswere particularly important to us because of their known importance,and because the model and a wide range of data sets were available.The problems are linear relaxations of MIP problems, although the LPsolutions are themselves useful for heuristic methods. The probleminstances all use the same underlying model, with twelve data setsrepresenting planning periods of one week, one data set representinga period of 84 days, and one data set representing a period of 364days—this last data set had 83,500 rows and 83,700 columns. Thenumber of days in each instance is indicated by the first component ofthe name given to the instance (see table 5.4).

As the model was available to us, we were able to consider differentblock structures based on index sets in the model. Two were used: onebased on generating unit (set); the other based on time period. Fur-thermore, by grouping time periods together, we were able to obtainvariants of the time period decomposition for the two larger data sets.The types of data sets are indicated by the second component of theproblem name in table 5.4: ‘s’ for the set decomposition; and ‘t’ for thetime period decomposition, with the final component (‘t7’, ‘t14’, etc.)giving the number of periods grouped into a block. The decomposedmatrix files were obtained using the DecMod decomposition modellingsoftware, which is described in chapter 6.

The different block structures enabled us to obtain several decomposi-tion problem instances from one LP problem instance (see chapter 2).The different block structures result in different master problem andpricing problem formulations.

Hydrothermal (Table 5.5) This set of five multi-commodity network flowproblems concerns hydrothermal energy generation [CN96, Fra00]. Oneblock structure was identified and used, with four blocks in each prob-lem. The largest problem has 4885 rows and 8564 columns. The de-composed matrices were converted directly from the ‘PPRN’ data fileformat [Fra97] in which the instances were obtained using a bespokesoftware utility.

PDS (Table 5.6) This set of multi-commodity network flow problems rep-resent different scenarios from one underlying data set, and arise fromthe area of military transportation planning [Fra00]. One block struc-

5.3. EXPERIMENTAL PLATFORMS AND SOFTWARE 89

ture was identified and used with eleven blocks. The problem size in-creases up to 83,000 rows and 270,000 columns for the largest instance.The decomposed matrices were converted directly from the ‘mnetgen’4-file data format [JL92] in which the instances were obtained using abespoke software utility.

Ideally, we would choose our evaluation problems randomly from some myth-ical population of all primal block angular problems. This would allow usto use the results from our sample of problems to make statistical inferencesabout the entire population. However, we face two interrelated obstacles:defining the population of all problems, and the practical process of choosingproblems from this population.

There are essentially two methods of selecting test problems [CDM78].One is to generate random instances, given certain parameters such as prob-lem dimension and number of blocks. For this method the process of select-ing problem instances from the population is relatively easy; however, thepopulation from which we are choosing them is not the one that interestsus. This is because randomly generated problems are not representative ofproblems encountered in the real world—the probability distribution of realworld problems in the space of all problems is far from uniform. One canrefine the method somewhat by imposing a known problem structure, typi-cal of one occurring in certain practical problems; however, this still resultsin a skewed sample of problems. All we have done is move the difficultyof random selection with an accurate distribution to the choice of problemgenerator, and in any case the random problems are still far from ideal. Forfurther discussion of the use of random problem generators, see [RL82].

The alternative method is to select problems from available sets of realworld problems. This at least allows us to draw useful conclusions aboutthe performance of our software on the sets of real world problems used.And the more diverse range of applications covered by the problem sets, themore certain we can be about the general applicability of our conclusions.However, we face the inevitable difficulty that, since the population fromwhich these problems are drawn is not known, it is impossible to make firmstatistical inferences from the results obtained and generalise the conclu-sions. Accepting these limitations, it is this second method we have optedfor, because it gives us the best chance of evaluating the method for practicalproblems.

5.3 Experimental Platforms and Software

Four computer platforms were used to perform the experiments.


Platform 1: Windows PCComputer Dell Inspiron 5000 PC: Intel Pentium III 650 MHz

processor; 128 MB RAM; 256 kB L2 cacheOperating System Windows 2000 (version 5.00.2195)

Platform 2: Windows multi-processor PCComputer PC Server: 4 × Pentium Pro 200 MHz processors;

1536 MB RAM; 512 kB L2 cacheOperating System Windows NT 4.0 Server (build 1381 service pack 6)

Platform 3: Four Windows PCs linked by ethernetComputers 4 Compaq Deskpro PCs, each: Pentium Pro 200 MHz

processor; 128 MB RAM; 256 kB L2 cacheOperating System Windows NT 4.0 Workstation (build 1381 service

pack 6)Network 10 Mbit/s ethernet

Platform 4: IRIX multi-processor workstationComputer SGI Workstation: 64 × IP27 300 MHz processors;

65536 MB main memoryOperating System 64 bit irix 6.5.10f

Windows SoftwareLanguage ANSI 1989 Standard CCompiler Watcom C32 Optimising Compiler version 10.6Compiler Options Pentium stack calling conventions; level ‘x’ control

optimisation; Windows NT target operating systemExecutable Size Serial DecOpt: 98,816 B

Parallel DecOpt: 117,760 B; DecSlave: 116,736 BDecOpt Options As given in the DecOpt User Manual (appendix C,

section C.10) except where stated belowLP Library xosl version 12.13LP Options Default options were used (see [Das99a, Das00]), ex-

cept where specified below, and: the primal simplexLP optimiser was used; slack variables on equalityrows were not allowed to leave the basis once ba-sic; access to the solution file was disabled; and non-binding rows were not removed from the matrix

Parallel Library hppvm version 2.9Parallel Options See appendix A.1

5.4. VALIDATION 91

IRIX SoftwareLanguage ANSI 1989 Standard CCompiler MIPSpro C compiler version 7.3.1.1mCompiler Options Global (‘O2’) control optimisationExecutable Size Parallel DecOpt: 2,752,344 B; DecSlave: 2,748,084 BDecOpt Options As given in the DecOpt User Manual (appendix C,

section C.10) except where stated belowLP Library xosl version 11.14LP Options Default options were used (see [Das99a]), except

where specified below, and: the primal simplex LPoptimiser was used; slack variables on equality rowswere not allowed to leave the basis once basic; ac-cess to the solution file was disabled; and non-bindingrows were not removed from in the matrix

Parallel Library hppvm version 2.10Parallel Options See appendix A.1

We used two statistics to measure the performance of DecOpt. The totalelapsed time of the DecOpt session was the absolute performance measure;this includes program initialisation, problem input, optimisation, and closedown. However, when comparing the performance of DecOpt with differentparameter settings on a given problem, we used the elapsed optimisationtime, which excludes the fixed set-up and close down times. In our results,all times are quoted in seconds. The timings were calculated by DecOptand output directly to results files, which was then imported into MicrosoftExcel for analysis. In all experiments, the value of the objective functionwas used to check that the correct solution had been found.

5.4 Validation

The purpose of our first experiment was to validate that DecOpt could in-deed solve the problems under consideration. The settings for the DecOptcontrols given in the DecOpt User Manual (see appendix C, section C.10)were applied apart from those listed in table 5.7 (p. 92), and all problemswere solved using serial DecOpt on the Windows PC (platform 1, p. 90).

We were able to solve to optimality all problems in our test set apart fromthree. The Ho & Loute, Netlib, Gondzio, and all but one of the Hydrother-mal problems solved within 30 seconds. The 7-day PowerGen problemssolved within a minute, and the larger PowerGen problems (with the excep-tion of 84-s) within an hour. The tables of results given in appendix B.1(p. 143) include the total elapsed time taken by DecOpt and full tables ofother performance statistics which we quote in this section. Where prob-lems were not solved to optimality, we give the statistics to the point at


Control ValueProposal generation frequency propfreq 0Proposal generation frequency decay propfreqdcay 50%Proposal generation cost factor propcost 0.0Proposal generation column limit propgcol 2Purge frequency purgfreq 4Purge non-basic criterion purgnbas 1Purge cost criterion purgcostmax 0.0xpress-mp presolve xp ifpres 0xpress-mp pricing xp iphsf −1xpress-mp re-invert xp fastopt 0

Table 5.7: Initial control settings.

which optimisation halted, the gap remaining, and the reason for the earlytermination.

The remainder of our discussion focuses on those problems which weredifficult to solve.

The PowerGen problem 84-s took eight hours to solve, in contrast tothe time based decomposition instances derived from the same 84-day dataset. The much greater number of cycles and master problem iterationstaken indicate that the decomposition was less suitable—it seems that moreinformation needed to be collected in the master problem before the solutioncould be found. This behaviour is in contrast to that observed for the 7-day data sets, where the set based decomposition performed better than thetime based decomposition. We return to this issue in section 5.9.

The hydrothermal problem minsil19 was halted after 24 hours with agap of 24% remaining, having completed 3051 cycles. The slow progresscan be illustrated by the fact that, on average, the master problem took 28seconds per cycle to solve an average of just 500 simplex iterations. This isdue to the formulation of the master problem: proposals have an averagedensity of 42.40%, and with 2145 global rows, each proposal column hason average 910 non-zero elements. We conclude that the block structureidentified was simply not suitable for Dantzig-Wolfe decomposition.

The set of PDS problems exhibited the classical asymptotic convergenceof Dantzig-Wolfe decomposition, with progress slowing considerably as theoptimality narrowed. We halted optimisation when the relative optimalitygap fell below 0.00001%, which eliminated some of the tail without adverselyaffecting the quality of solution. The time taken grew from 1.11 seconds tosolve pds001, to 2 hours 20 minutes to solve pds012. pds020 took 34 hoursbefore it was halted with an optimality gap of 0.00195%, whilst pds050 had agap of 84% remaining after 12 hours. Here a combination of two factors helpsexplain the large amount of time required. Taking pds020 as an example, the

5.4. VALIDATION 93

majority of the time, 57%, is spent solving the master problem: an average of7.41 seconds and 143 simplex pivots per cycle, whereas the pricing problemstake an average of 0.51 seconds and 1049 simplex pivots each per cycle. Onceagain, the difficulty of the master problem may be at least partly explainedby its density: 8.41%, with an average of 206 non-zero elements per proposalcolumn. However, another factor is the sheer amount of work required tosolve the pricing problems: 109 million simplex iterations altogether, anaverage of 11,540 per cycle.

Looking at the PDS problem set as a whole, we observe that the growthin the number of global rows, although not as large as the growth in theoverall problem size, is large enough to make the master problem increasinglydifficult to solve. The density of the proposals does fall, but not enough tooutweigh the growth in the number of global rows, so that the number ofnon-zero elements increases until it becomes unmanageable. The secondfactor, the effort required to solve the pricing problems, is not nearly asimportant—it is a natural result of greater problem size, and on its owncould be overcome using parallel decomposition.

The extreme case of minsil19 illustrates one potential difficultly of Dantzig-Wolfe decomposition. The proposal columns of the master problem are oftenfar more dense than columns occurring in typical LP problems. Whereascolumns in a real world LP problem represent activities of a particular com-ponent of a real world system, and thus rarely contain more than ten non-zero elements, proposal columns often have no direct meaning in the realworld, and being aggregated solutions to complex systems, can be far moredense, as we have seen. Perhaps the most important measure of a goodblock structure on a problem instance for Dantzig-Wolfe decomposition isnot the number of global rows or number of blocks, but the average numberof non-zero elements in the proposal columns of the master problem.

During preliminary testing of DecOpt, we tried solving problems from aseventh test set, a set of production planning problems obtained from BASF.Unfortunately we were unable to make any progress solving the problemswith DecOpt. Many tests failed with the problem being declared infeasible;with other settings the xpress-mp optimiser displayed repeated warningsof numerical instability with no advance being made in the master prob-lem. One cause appears to be the large range in magnitude of the proposalcoefficients—in one case, they ranged from 10−3 to 6 × 108, whilst the re-duced costs of the proposals were of the order of 109. (We return to this issuein section 5.8, below.) Another cause may have been the lack of well-formedpricing problems, with two of the three problems unbounded on almost allof the cycles. As a result, we did not include the problem in our final testset. Since conducting the experiments, we have been able to obtain solu-tions within 0.2% of optimal, but numerical warnings are still displayed andprogress on the master problem eventually stops.


Factor/Control Low HighA Prop. gen. frequency propfreq 10 100B Prop. gen. frequency decay propfreqdcay 50% 90%C Prop. gen. cost factor propcost 0.0% 1.0%D Prop. gen. column limit propgcol 2 6E Purge frequency purgfreq 4 12F Purge non-basic criterion purgnbas 3 9G Purge cost criterion purgcostmax 0.0 1.0H xpress-mp presolve xp ifpres 0 1J xpress-mp pricing xp iphsf −1 0K xpress-mp re-invert xp fastopt 0 1

Table 5.8: Control settings evaluated.

5.5 Evaluation of Computational Strategies

What is the effect of the different computational strategies encoded withinDecOpt? Which factors cause a significant effect? Can we identify a goodoverall strategy? To answer these questions we carried out a fractional facto-rial experiment on a subset of our problem suite. Fractional factorial designsare known to be the most efficient type of experiment for studying the effectsof several factors, and are widely used in industry and agriculture [Mon01].They are ideal for evaluating different control settings in computer software,and it is surprising that they are not more widely used. For those unfamiliarwith factorial designs a short introduction is given in appendix B.2 (p. 147).Details specific to our experimental design are given in this section withfurther reference to appendix B.3.

We investigated the effect of ten controls, or factors, each at two fixedlevels, given in table 5.8. These factors represent the primary strategicchoices available in DecOpt, and the levels were chosen following experiencegained in preliminary testing to cover the range of normal operating condi-tions. Note that the low and high labels are somewhat arbitrary, and shouldnot be equated with off and on—the low settings of propfreq, purgfreqand purgnbas are more proactive than the high settings.

We adopt a common statistical shorthand to describe the experimentaldesign: the letters A–K (excluding I) represent the ten factors; the lowand high levels of a factor are represented by ‘−’ and ‘+’; and treatmentcombinations are represented using strings of lower case letters for the factorsat high levels, so treatment ‘bdf ’ means factors B, D and F were appliedat their high level, and all other factors at their low level. In contrast, thecombined effect of applying factors B, D, and F is labelled in uppercase,‘BDF ’. The effect BDF is not the same as the result of applying treatmentbdf ; indeed, the factorial design allows the results of all treatments to be

5.5. EVALUATION OF COMPUTATIONAL STRATEGIES 95

Set ProblemHo & Loute meat12Ho & Loute mrp5Netlib sctap2Gondzio largePowerGen 7-07-sPowerGen 7-07-tPowerGen 84-t7PowerGen 84-t14Hydrothermal minsil7PDS pds003

Table 5.9: Problems used to evaluate computational strategies.

used to calculate each effect. (See appendix B.2.)In a fractional factorial design, we sacrifice the ability to measure all

possible effects for the sake of fewer experimental runs. The effects are par-titioned into equivalence classes, called alias chains, and effects in the samealias chain are aliases of each other. We are able to measure the combinedeffect of each alias chain, but cannot distinguish between the effects withinan alias chain. Thus when selecting a fraction, we must ensure that no twopotentially interesting effects are members of the same alias chain. We as-sume that with many variables the process is driven primarily by some ofthe main effects and low-order interactions—an assumption known as thesparsity of effects principle. Certainly we require that all main effects andtwo factor interactions are distinguishable from each other, which can beachieved by a standard experimental design known as resolution V . Weare also interested in some higher order effects: those involving the fourproposal generation controls (factors A–D), and those involving the threepurge controls (factors E–G). We therefore require that none of the inter-actions ABC, ABD, ACD, BCD, ABCD, and EFG are aliased with eachother or with lower order effects.

We selected a 210−3V design, with 128 runs. The design generators were

H = ABEF , J = ACEG and K = ADFG (adapted from [Mon01]). Thecomplete defining relation, design matrix and the alias structure are givenin appendix B.3. The alias structure shows which effects are in the samealias chain, and we used it when assigning the factors to ensure that thespecial requirements on factors A–D and E–G were satisfied.

The large number of runs made it impractical to perform this experi-ment on our complete test suite. We selected one or two problems fromeach problem set, and four from the important PowerGen set. We tried tochose problems that took neither too little time—for which computationalstrategies would not be important—nor too great a time—for which decom-


position was unlikely to be a useful method anyway. The ten problems usedare listed in table 5.9. The complete experiment comprised 1280 runs.

The different problems constituted a known and controllable nuisancefactor. The variation due to the different problems was separated fromthe variation due to the effects of the factors using a randomised completeblock design. Each treatment combination from the fractional factorial wasapplied to each problem, and the complete set of runs was conducted in arandom order.

The response variable, the elapsed optimisation time t, is a rate. Wetherefore considered a multiplicative model, or equivalently an additivemodel after applying the transformation y = log t. Let xi be coded vari-ables representing the level of ten factors i, with xi = −1 when factor i isat the low level, and xi = 1 when factor i is at the high level.2 Our effectsmodel is

y = log t

=∑i1

βi1xi1 +∑i1<i2

βi1i2xi1xi2 +∑

i1<i2<i3

βi1i2i3xi1xi2xi3 (5.1)

+ · · ·+ βi1...i10xi1 · · ·xi10 + κp + ε,

where the β are unknown coefficients representing the main and interactioneffects of the factors; κp is the effect of block (problem) p, and ε is the error.The error includes any interaction effect between blocks and treatments—itcannot be distinguished from other sources of error. The variables y, t, andε are of course functions of xA, xB, . . . , xK and p; we have omitted this fromthe notation in model (5.1) for clarity.

The 1280 runs were run in random order on the Windows PC (platform 1,p. 90). The raw response data, the elapsed optimisation times t, are given inappendix B.4.1 (p. 157). Thirteen of the 1280 runs failed due to numericalproblems, which are discussed in section 5.8. Our analysis requires a full setof results, and there are well established procedures to remedy the problemwhich do not invalidate the analysis. Following [Mon01, § 4.1.3], we estimatethe missing response data by minimising the error sum of squares, and reducethe degrees of freedom for the error by thirteen. It is worth mentioning thatthe treatments that failed were very different from the strategy adoptedbelow for general use in DecOpt.

An analysis of variance was performed on the transformed responsedata y, and this is summarised in appendix B.4.2 (p. 159).3 The analysis isalso shown graphically in figure B.1 (p. 161), which presents a normal prob-ability plot of the effect estimates. The axes of the plot are less importantthan the shape—any normal distribution is represented by a straight line, so

2Throughout this section, we use indices i to range over the ten factors, A–H, J–K.3We used Microsoft Excel to collect the results and Minitab for the analysis of variance,

for which [GG98] proved helpful.


Effect Sig. EstimateH 0.0% 0.7354D 0.0% 0.1279B 0.0% 0.0626EH–ABF 0.0% −0.0463FH–ABE 0.3% −0.0393BH–AEF 0.6% 0.0359GH 2.2% −0.0303AE–BFH–CGJ 2.2% 0.0303HJ 2.6% 0.0294DE 2.8% 0.0289K 3.8% −0.0273J 7.3% 0.0236EGH–BDK 7.6% 0.0234EF–ABH 7.7% 0.0233CD 8.5% −0.0227

Table 5.10: Significant Effects in the evaluation of computational strategies.

effects that are far from the fitted line are significant. The model accountsfor almost all of the observed variability in the response, with R2 = 98.40%and R2

Adj = 98.21%, and the model is significant at a level of 1%.Our analysis indicates that fifteen effects, or rather, alias chains of ef-

fects, are significant at a level of 10%. Table 5.10 lists the fifteen chains,disregarding interaction effects of order greater than three, together withthe level of significance and the estimate of the value of the effect. (Refer totable 5.8 p. 94 for the meaning of the ten factors.)

At this point, a subjective analysis is necessary to distinguish betweenthe different aliases where more than one is indicated.4 We chose the lowestorder effect in general, noting that three of these choices, EH, FH, and BH,include effects H or B that are highly significant in their own right. For thealias chain AE–BFH–CGJ , we picked BFH because B and H are bothsignificant in their own right, in contrast to A and E. The likely significanceof BH and FH lends weight to this choice. There are two difficult decisions,both of marginal (5–10%) significance. In the case of EGH–BDK, we optedfor EGH, given the high significance of H, EH and GH, and the fact thatnone of the interactions BD, BK or DK appear remotely significant. In thecase of EF–ABH, we selected EF , firstly because it is of lower order thanABH, and secondly, because factor A appears to have no other significanteffect.5

4Another resolution would have been to run further experiments to increase the reso-lution of the design, i.e., its ability to distinguish between effects.

5It turns out that the strategy derived below is unaffected by either of these choices,


We fitted a reduced model using the 15 significant effects identified above,together with six additional effects, C, E, F , G, BF and EG, in order forour model to be hierarchical. The regression analysis is summarised inappendix B.4.3. The reduced model is:

y = log t= 0.0313xB − 0.0022xC + 0.0640xD + 0.0036xE + 0.0098xF− 0.0062xG + 0.3677xH + 0.0118xJ − 0.0137xK − 0.0007xBxF+ 0.0179xBxH − 0.0113xCxD + 0.0144xDxE + 0.0116xExF+ 0.0072xExG − 0.0232xExH − 0.0196xFxH − 0.0151xGxH+ 0.0147xHxJ + 0.0117xExGxH + 0.0151xBxFxH + κp. (5.2)

The standard error of each regression coefficient is 0.00681, and the reducedmodel has R2 = 98.35% and R2

Adj = 98.31%.Expressing the model in terms of the original response, t, we have

t = ey

= 1.032xB × 0.998xC × 1.066xD × 1.004xE × 1.010xF

× 0.994xG × 1.444xH × 1.012xJ × 0.986xK × 0.999xBxF

× 1.018xBxH × 0.989xCxD × 1.015xDxE × 1.012xExF

× 1.007xExG × 0.977xExH × 0.981xF xH × 0.985xGxH

× 1.015xHxJ × 1.015xBxF xH × 1.012xExGxH × eκp . (5.3)

Appendix B.4.4 gives details of the model adequacy checking. The as-sumptions of the analysis are a normal error distribution with constant vari-ance across blocks, and there is some departure from this, including someobvious outliers. However, in a balanced fixed effects model such as weare using, the F -test is very robust under abnormal conditions or if thehomogeneity of variances is violated. Indeed the F -test may regarded asan approximation to the generally applicable randomisation test [Mon01,§3.3.2]. We therefore feel justified in accepting the analysis and model.

The model (5.3) implies that the best strategy in general, one that min-imises y and thus t = ey, is given by xC = xG = xK = 1 and xB = xD =xE = xF = xH = xJ = −1. No setting is prescribed for factor A. Thevalues of the corresponding DecOpt controls are given in table 5.11. Wediscuss the assumptions and limitations of this model and our conclusionsand recommendations below.

The most striking factor is the use of the xpress-mp presolve (factorH)—it more than doubles solution times when considered in isolation tothe other factors, and has several significant effects when combined withother factors. The main effect implies that extra effort required to repeat-edly presolve and post-solve each matrix in each cycle is not paid off, that

other than to suggest A should be at its low setting rather than indeterminate.


Factor/Control ValueA Prop. gen. frequency propfreq n/aB Prop. gen. frequency decay propfreqdcay 50%C Prop. gen. cost factor propcost 1D Prop. gen. column limit propgcol 2E Purge frequency purgfreq 4F Purge non-basic criterion purgnbas 1G Purge cost criterion purgcostmax 1H xpress-mp presolve xp ifpres 0J xpress-mp pricing xp iphsf −1K xpress-mp re-invert xp fastopt 1

Table 5.11: Control settings chosen following the evaluation.

the resulting matrices are not significantly quicker to solve than if presolvehad not been applied. We must remember that significantly larger prob-lems than the ones used in this experiment are likely to be encountered inpractice (indeed our complete test set contains many larger problems), andit might be expected that presolve would have a more beneficial effect onlarger problems. However, it is also worth noting that the LP problemsencountered within the column generation algorithm rarely take many iter-ations or much time to solve, as an advanced basis is always available, sothe scope for presolve to reduce the number of iterations or amount of time,even on much larger problems, will always be limited.

The adverse effect of presolve is mitigated to a very limited extent whenapplied in combination with any of the following factors: a large purgefrequency (E), high non-basic and reduced cost purge criteria (F and G),a low proposal frequency decay factor (B), and the use of partial pricingonly in xpress-mp, rather than a choice between partial pricing and devexpricing (J). The three purge factor settings suggested all have the effectof increasing the average size of the master problem, with fewer propos-als being eliminated less often. This suggests that presolve has the mostbeneficial effect on the large master problems that arise in the algorithm,and any measure that reduces the size of the master problems limits theimprovements presolve is able to make without any corresponding saving inthe time taken to perform the presolve. A low proposal frequency decayfactor means that the proposal frequency changes less rapidly in responseto the number of iterations taken to solve a pricing problem; it is difficultto hypothesise what interaction this factor has with presolve. In any case,even applying all of these mitigating factors, and ignoring the main effectsand other interactive effects of these factors, applying presolve still resultsin solution times 74% greater than without applying presolve.

The number of proposals generated per pricing problem per cycle (D)


is highly significant. Solution times are 13.6% greater when generating sixproposals compared to two proposals. Generating a proposal takes time, itinterrupts the optimisation of the pricing problem, and adds to the numberor proposals (and proposal non-zeros) in the master problem. To offset this,an improvement must come from being able to obtain a better solution to themaster problem, and if the additional proposals are not sufficiently distinctthis will be unlikely, as appears to be the case. When coupled with a largepurge frequency (E), meaning that proposals build up in the master problemfor longer before being deleted, the solution times worsen by a further 2.9%.On the other hand, if a larger number of proposals are generated, a slightimprovement appears to result from applying a high proposal generation costfactor (C), which has the effect of preventing the generation of proposalswhose reduced costs do not differ by more than 1%. These two interactioneffects are not surprising.

Our dynamic proposal frequency has a highly significant effect. Solutiontimes are 6.5% shorter when the proposal frequency decay factor (B) is 50%rather than 90%. A lower percentage means that the proposal generationfrequency is more responsive to the observed frequency with which propos-als are generated. When a pricing problem takes few iterations to solve,the frequency will be reduced; when it takes many iterations to solve, thefrequency will be increased (see section 5.1.2). This probably explains theinsignificance of the initial proposal generation frequency (A), which appearsto have no effect whatsoever. With a low proposal frequency decay factor,its value is only of importance during the first few cycles of the algorithm,and even with a higher decay factor its influence diminishes each cycle.

The xpress-mp re-factorisation control (K) determines whether a newrepresentation of the basis is computed when a matrix is resolved, or whetherthe existing representation is re-used (if possible). Re-using an existingrepresentation appears to improve solution times by 2.8%.

Turning to the effects of questionable significance, further evidence issupplied that partial pricing only in xpress-mp (J) should be used. Theinteraction between the purge frequency and non-basic purge criterion (EF )suggests that low settings should be used, resulting in smaller master prob-lems, that presumably are quicker to solve without lowering the quality ofthe solutions produced.

Many of the factors tested had no significant effect; C, E, F and Ghave an effect only in combination with other factors, and we have alreadyremarked on the insignificance of A. In general, it appears that the columngeneration algorithm is relatively unaffected by these factors. The columngeneration algorithm must always generate the proposals that give a path tothe optimal solution, and all of these factors—the purge controls, the initialproposal generation frequency, the proposal cost criterion—can be seen asrelatively minor adjustments that have little impact on the overall progress.

At this point it is worth discussing some of the assumptions and limi-

5.6. EVALUATION OF PARALLEL PERFORMANCE 101

tations of this model. The experiment used only a limited number of ouralready limited set of problem instances. This was a practical necessity, asdiscussed above, but what impact does it have on our conclusions? Certainlyit prevents us saying with any certainty that the ‘best’ strategy derived aboveis indeed the best strategy in general for all problems to which DecOpt mightbe applied. We cannot say that the strategy is the best for all problems onwhich decomposition is a worthwhile solution method. It is not even certainthat such a ‘best’ strategy exists. With that said, most of the significanteffects have a reasonable explanation. Given the confirmatory results below,in section 5.7, which showed an average improvement of 23% compared tothe original settings used in section 5.4, the chosen settings do appear to begood in general, and in particular on the PowerGen problems. If our workhelps to improve the application of decomposition, and results in a widerrange of test problems becoming available, further experimental work can beundertaken to extend the validity of our work and modify our conclusions.Our analysis provides a first attempt to systematically evaluate the effectsof the different strategies within the decomposition algorithm, provides ademonstration of how such an analysis can be performed, and opens thedoor to further work when further problem instances become available.

The experiment considered only the values for the controls given in ta-ble 5.8 (p. 94). The factorial design ensures that all combinations of thesevalues are taken into account, but to interpolate between these values orextrapolate beyond them requires an assumption of linearity in the effect ofthe variables, and this assumption has not been tested.6 Three of the fac-tors, the xpress-mp controls H, J , and K, are simple off/on switches andno other values are relevant. The other factors all take values within a rangeand we have to be very careful making judgements on values not included inthe design. Further experimentation, for instance using a response surfacemethodology, would be required to find a fully optimised strategy. However,given the limited significance and effect of most factors, and the fact thatthe most important factor H takes just the two tested values anyway, webelieve that the experimental work undertaken provides a reasonable basisto state that the strategy given in table 5.8 is a good overall strategy. It isthe strategy we employ for the remainder of our experimental work.

5.6 Evaluation of Parallel Performance

How does DecOpt perform on parallel computer platforms? Does the per-formance increase in proportion to the increase in computing power? Wetested DecOpt on three diverse parallel architectures to find out.

We selected the longer running problems from our test suite for which wepredicted an efficiency greater than 50%, using the analysis of section 5.1.9

6To do so would require the addition of centre points to the factorial design.


Problem K tm ts R E2 E4

sctap3 62 5.01 0.30 232.7 83% 60%7-10-s 23 0.87 0.11 69.0 78% 56%7-10-t 7 10.60 2.63 12.2 69% 46%84-t2 42 1446.47 32.81 40.0 66% 39%84-t4 21 390.68 53.74 60.7 77% 53%84-t7 12 9.75 12.65 186.8 94% 85%84-t14 6 5.23 53.23 366.4 98% 73%364-t7 52 301.87 14.84 133.0 78% 54%364t14 26 105.10 49.40 317.7 93% 77%364-t28 13 40.75 183.42 760.7 92% 78%pds003 11 1.85 1.16 75.9 83% 68%

Table 5.12: Predicted parallel efficiency Ep using p processors. K is thenumber of blocks; tm the total time taken to solve the master problem; tsthe total average time taken to solve one pricing problem; and R the parallelefficiency ratio—see section 5.1.9 (p. 82) for an explanation of the notationand analysis.

combined with the results of the validation experiment in section 5.4. Thisexcluded some of the problems that took the longest time to solve, suchas the larger PDS problems, mnsil19, and 84-s, for which the maximumefficiency was predicted to be low. However, most of the large problemsfrom our most important test set, PowerGen, were included. Two of thesmaller PowerGen problems were also included, for comparison. The elevenproblem instances are listed in table 5.12, together with estimated efficiencieson two and four processors.

The asynchronous message passing system in parallel DecOpt inevitablycauses variation in the performance between otherwise identical runs. Theorder that the messages arrive from the slave processes at the master pro-cess determines the order in which the proposals are added to the masterproblem. When the variables in the master problem are ordered in differentways, the result can be different basic solutions with different dual vari-ables, and hence pricing problems with different objective functions. If thisoccurs, the next set of proposals generated will be completely different, andthe algorithm will take a different path to the optimal solution.

We used a very simple experimental design. Each problem listed intable 5.12 was solved with parallel DecOpt using zero, one, two and threeslaves (corresponding to one, two, three and four processors) on the multi-processor Windows PC (platform 2, p. 90) and the set of four Windows PCslinked by ethernet (platform 3). To allow for the variation in the results,five repeated runs were performed on platform 2 and three repeated runs onplatform 3. The order in which the problems were solved was completely


Platform 2 Platform 3 Platform 4Problem E2 E3 E4 E2 E3 E4 E2 E4 E∞sctap3 64% 51% 41% 69% 59% 50% 46% 16% 3%7-10-s 57% 45% 33% 60% 42% 35%7-10-t 64% 48% 37% 65% 48% 40%84-t2 74% 52% 43% 73% 47% 36%84-t4 56% 49% 35% 56% 40% 37%84-t7 79% 68% 59% 76% 74% 62% 80% 54% 23%84-t14 91% 85% 71% 95% 78% 73% 100% 71% 53%364-t7 64% 48% 38% 63% 46% 38%364t14 88% 71% 60% 89% 79% 66% 125% 97% 58%364-t28 88% 72% 67% 92% 82% 77% 85% 75% 45%pds003 65% 47% 44% 63% 49% 39% 47% 23% 8%

Table 5.13: Observed parallel efficiency Ep using p processors. E∞ showsthe efficiency when enough processors were used to treat the master problemand each pricing problem on a dedicated processor, apart from problemsctap3 for which 16 processors were used and problem 364t14 for which 14processors were used.

randomised.We subsequently tested DecOpt on a high performance dedicated parallel

computer, a 64-processor SGI machine (platform 4, p. 90). We solved thesix problems that performed well under the parallel tests performed on plat-forms 2 and 3. Each problem was solved three times using various numbersof processors up to a maximum of 16.

The elapsed times are reproduced in appendix B.5. A few results aremissing due to numerical problems which we discuss later in section 5.8. Thetables in appendix B.5 also give the mean time, the sample standard devia-tion, the speed-up and the parallel efficiency. The running times, scaled toallow all problems to be displayed together, have been plotted in figures 5.1,5.3 and 5.5 (pp. 104–106). Figures 5.2, 5.4 and 5.6 (pp. 104–106) show theparallel speed-ups, and the efficiencies are summarised in table 5.13. Thespeed-ups and efficiencies are heavily dependent on the times taken on oneprocessor, and so are liable to distortion if those times are unreliable. Oneshould always consider the actual improvements in the running time as wellas the speed-up and efficiency.

The tables and plots of results show some very good results. On plat-form 2, problems 84-t7, 84-t14, 364t14 and 364-t28 produced excellent effi-ciencies on two processors (80–90%) and creditable efficiencies on four pro-cessors (60–70%), where the speed-ups were in the range 2.37–2.68. Forinstance, on platform 2, the mean time of 364t14 was cut from 2842 sec-onds on one processor to 1176 seconds on four processors. The results on


Figure 5.1: Parallel DecOpt: Running times on platform 2.

Figure 5.2: Parallel DecOpt: Speed-up on platform 2.








Problem E2 E2 E2:E2 E4 E4 E4:E4

sctap3 83% 64% 77% 60% 41% 68%7-10-s 78% 57% 73% 56% 33% 59%7-10-t 69% 64% 93% 46% 37% 81%84-t2 66% 74% 112% 39% 43% 110%84-t4 77% 56% 72% 53% 35% 66%84-t7 94% 79% 84% 85% 59% 70%84-t14 98% 91% 92% 73% 71% 98%364-t7 78% 64% 82% 54% 38% 70%364t14 93% 88% 95% 77% 60% 78%364-t28 92% 88% 96% 78% 67% 86%pds003 83% 65% 78% 68% 44% 64%Average 83% 72% 87% 63% 48% 77%

Table 5.14: Comparison of the predicted efficiency Ep, and the efficiencyobserved on platform 2, Ep.

platform 3 were very similar, and in most cases slightly better.On platform 4, problems 84-t7, 84-t14, 364t14 and 364-t28 produced

even better performance on two and four processors. (Recall that on plat-form 4 we only solved problems that performed well on platforms 2 or 3.)In some instances the efficiency was greater than 100%, which in each casewas explained by the algorithm taking a different solution path. The bestinstance again was problem 364t14, where the time of 2188 seconds on oneprocessor was cut to 873 seconds on two processors and 563 seconds on fourprocessors. However, further speed-up on more processors was limited asthe plots and E∞ column in table 5.13 make clear.

Other problems did not perform so well. The smaller problems had poorefficiencies, due to the overhead of setting up the parallel application andtransmitting the pricing problem matrices from the master process, wherethey were read in, to the slave processes, where they were solved. Therewas little gained in performance by solving problems 84-t2, 84-t4 and 364-t7 with more than two processors, and both 84-t2, 84-t4 experienced somenumerical problems. Problems sctap3 and pds003, which performed wellenough on platforms 2 and 3, did not perform at all well on platform 4. Thetotal time remained constant at ten seconds no matter how many processorswere used, suggesting that the performance was dominated by overhead.

The observed performance was some way below the predicted perfor-mance, as shown by table 5.14,7 indicating that the overhead incurred bythe parallel application is not as insignificant as we might have hoped. Onaverage the observed efficiency was 87% of the predicted efficiency for two

7Based on the results obtained on platform 2.


processors, and 77% of the predicted efficiency for four processors. Tak-ing this factor into account, the performance was broadly in line with thatpredicted, with most of the ratios Ep:Ep near to the average ratio. Somenotable exceptions are 84-t2, which performed better than expected, and84-t4, which performed much worse than expected.

The performance discussed so far has been based on the average runtimes obtained. We can investigate the robustness of the run times byconsidering the standard deviation results (see appendix B.5, p. 166). Ingeneral, the standard deviation rises with the number of processors as onewould expect. Platform 3 exhibited higher variation than platform 2, whichagain is not surprising given that parallel virtual machine relies on an eth-ernet network connection. Variation is much higher on platform 4, both inabsolute terms and relative to the average run times, and this was almostcertainly due to the fact that multi-user access could not suspended.

There are some unusually high variations. The two problems that expe-rienced difficulties, 84-t2 and 84-t4, had large deviations in their times onthree and four processors, as did 364-t7 and to a certain extent 364t14 and364-t28. In all cases the variation was due to a different solution path beingtaken, with the number of cycles varying within a range of 20%.

To conclude we have shown that parallel DecOpt is capable of solvingcertain large problem instances using parallel decomposition, with excellentspeed-up on two processors and creditable speed-up on four processors. Theperformance is reasonably robust and similar across a diverse range of par-allel platforms. Subject to a degradation factor, the performance is in linewith the performance predicted using the simple analysis in section 5.1.9,allowing problems to be assessed in advance to see if they would gain fromparallel optimisation. It would be interesting to test the performance ofparallel DecOpt on a larger set of problem instances.

5.7 Comparison with Standard LP Software

How does the Dantzig-Wolfe decomposition method compare to mainstreamLP optimisation techniques? We compared the performance of the Dantzig-Wolfe decomposition method in our serial DecOpt implementation to thesimplex method in the xpress-mp Optimiser.

For this final experiment we solved our complete problem suite usingDecOpt and xpress-mp on the Windows PC (platform 1, p. 90). We usedthe DecOpt control settings established in the evaluation of computationalstrategies (table 5.11, p. 99) and the default settings in xpress-mp. Athree hour time limit was imposed for each problem. The run order wasrandomised.

The results from the experiment are presented in appendix B.6 (p. 175).The columns headed Time give the elapsed run time using DecOpt and

5.7. COMPARISON WITH STANDARD LP SOFTWARE 109

Problem Set DecOpt xpress-mp

Ho & Loute 0.65 0.12NetLib 1.91 0.21Gondzio 8.99 12.67PowerGen 7-day 5.55 1.00PowerGen 84-day 452.56 140.22PowerGen 364-day 1441.25 2060.43Hydrothermal 6.57 0.16PDS 74.45 2.23Overall 6.39 1.06

Table 5.15: Comparison of Dantzig-Wolfe and simplex methods: mean (ge-ometric) timings for each problem set.

xpress-mp; the columns headed Opt give the elapsed optimisation timeonly. The DecOpt results include the numbers of cycles and proposals; thexpress-mp results include the number of pivots. The Gap is the relativegap between the primal value and dual bound when the column genera-tion algorithm finished. Note that three problems—mnsil19, pds020 andpds050 —could not be solved by DecOpt within the time limit; whilst five ofthe PDS problems were halted with an ε-optimal solution only.

Rates may be compared using geometric means, and in table 5.15 wegive the geometric means of the total run times for DecOpt and xpress-

mp, individually for each problem set and overall. These figures exclude thethree problems that DecOpt could not solve in 3 hours.

The results show that DecOpt wins on the Gondzio problem set and onthe largest examples from the PowerGen problem set. On all other problemsets, DecOpt is out-performed by xpress-mp. However, these results do nottell the whole story. The decomposition method, unlike the simplex method,provides bounds on the optimal solution as it progresses. If a 0.1% solutionwere sufficient, DecOpt would have solved two of the 84-day PowerGen prob-lem instances in less time than xpress-mp; 84-t4 in 99 seconds, and 84-t7in 107 seconds, compared to 140 seconds with xpress-mp and 453 secondsto achieve optimality with DecOpt. Most problems solved in one secondor less using xpress-mp, so a similar comparison could not be performed.We discuss ε-optimality of the larger PowerGen problems in greater detailbelow, in section 5.9.

Clearly decomposition can be an efficient solution technique on certainclasses of problems. The weakness of our test set is exposed somewhat,in that few of our problems are challenging for modern LP optimisationsoftware. Of the two problems sets that are challenging, the PDS problemsdo not appear suited to decomposition, while the PowerGen problems appear


eminently suited to decomposition.This is the first time we have solved all of the problems in our test suite

since deriving a good computational strategy in section 5.5, and it is instruc-tive to compare the times obtained here with those obtained in our initialvalidation experiment from section 5.4. Both experiments were performedon platform 1; the only difference being the control settings—the initial set-tings given in table 5.7 (p. 92) for the validation experiment as comparedto the chosen settings given in table 5.11 (p. 99) for this experiment. Theelapsed optimisation times are compared in appendix B.7 (p. 177), wherethe percentage improvement is also displayed. All but ten of the 67 prob-lems in the test suite8 were solved in shorter time using the chosen controlsettings, with an overall average improvement of 23%. 41% fewer cycles and16% fewer proposals were required.

The ten problems that did not exhibit improved performance include allfour problems from the Gondzio test set, for which the performance degradedby 2.8–10.2%. These problems are unusual in requiring just two cycles, andtwo proposals from each pricing problem, to achieve optimality.

Overall these results confirm our belief that the control settings derivedin section 5.5 are a good general strategy.

5.8 Numerical Robustness

In section 5.4 we remarked upon the failure to find a solution to minsil19,pds020 and pds050 within a reasonable time, and the numerical problemsexperienced with the BASF problems. In the subsequent experiments, weobserved some failures by DecOpt to solve problems under particular controlsettings that under other conditions could be solved. In this section weexamine the cases where DecOpt failed and make some general remarksabout numerical robustness.

Thirteen runs failed in our experiment evaluating computational strate-gies, listed in appendix B.4.1 (p. 157). In eleven cases one of the pricingproblems was mistakenly declared to be infeasible by the xpress-mp opti-miser at one of the intermediate cycles of the decomposition algorithm; intwo cases the xpress-mp optimiser crashed during the optimisation routine.What is interesting is that they all involved problems 84-t7 and 84-t14, andin eleven of the cases, the treatment included the common element hjk:the xpress-mp presolve was enabled, the xpress-mp automatic pricing wasused, and the xpress-mp rapid re-inversion control was set. It appears thatthis combination of settings together with the particular attributes of thetwo problems within the column generation algorithm caused the xpress-

mp optimiser to experience numerical difficulties. Our chosen strategy for

8We excluded the three problems that did not solve in 3 hours.

5.8. NUMERICAL ROBUSTNESS 111

DecOpt has xpress-mp presolve and re-invert controls at their low setting,which should prevent such behaviour occurring.

The evaluation of parallel performance also resulted in some failed runs.On platform 2, four runs failed, involving problems 84-t2 and 84-t4, and,more strangely, all using two slaves (i.e., three processors). In all cases theobjective value returned by DecOpt was erroneous, and in addition, the timetaken to solve the problems was 11

2–2 times longer than a correct run. Anexamination of the logs showed that the xpress-mp optimiser had difficultysolving the master problem, with repeated lapses into infeasibility.

Two reasons have been proposed that might explain the numerical prob-lems experienced by xpress-mp, both tied to the column generation algo-rithm. By the very nature of the algorithm, the constraint coefficients forthe proposal columns in the master problem are generated as the algorithmprogresses. Thus unlike the LP problems normally faced by LP optimisers,the coefficients are not primary data, but are taken from solution values tothe pricing problems, the result of a computation which will have introducednumerical inaccuracy. As [Ho84, Ho87] noted, the great advantage of thesimplex method is its ability to periodically refresh its working data by refer-ence to the original problem data in the re-factorisation routine. There is noequivalent in decomposition: if the columns generated are subject to error,they cannot be recalculated from the original problem data. Each columnwill be subject to a certain level of error, meaning that the restricted masterproblem is really an approximation to the restricted master problem [Naz84].

Not only are the proposal coefficients subject to error, they are also not‘nice’ in general. What do we mean by this? A well formulated LP problemhas coefficients within a few orders of magnitude to unity. If very large orvery small values are used, the fixed tolerances used in an LP optimisercan result in significant non-zero values being equated with zero—this iswhat is meant by numerical instability. Now the user can control the valuesappearing in an LP problem, and an LP optimiser can use scaling when amatrix is input to try to remove any very large or very small values thatoccur. In decomposition, the user has very little direct control over thecoefficient values of proposals in the master problem, and the LP optimiseris unable to scale the problem well when variables are continually beingadded and deleted. Any very large or very small values generated may giverise to difficulties in the LP optimiser.

DecOpt incorporated two types of scaling to try to mitigate this prob-lem as described in section 5.1.6. Unfortunately, the column scaling did notperform well: when a column contained very large values, the scaling some-times reduced the convexity row coefficient from one to a value less than thecoefficient tolerance in xpress-mp. In an extreme case, the Netlib problemsctap3 was incorrectly found to be unbounded as a result, whilst xpress-

mp encountered numerical instabilities when solving some of the PowerGenproblems. Clearly, one could scale a column so that the minimum element


Figure 5.7: Convergence of two PowerGen 7-day problems. The objectivevalue descends from above and the dual bound ascends from below.

was greater than the zero tolerance in xpress-mp; conversely one could alterthe zero tolerance in xpress-mp. At the time of writing neither had beentried, and column scaling was not used in the evaluation.

5.9 The Performance of the PowerGen Problems:A Comparison of Different Block Structures

We were fortunate to have had access to the PowerGen problem set, com-plete with model formulation and a range of data sets, which allowed us toconsider problem instances of different sizes and with different block struc-tures. The experimental work described in this chapter was designed toevaluate the performance of DecOpt, but in this section we take the oppor-tunity to consider the performance of Dantzig-Wolfe decomposition on thePowerGen problems in particular, using the results of previous sections.

The set based decomposition is clearly superior to the time based de-composition for the 7-day instances: the results from section 5.7 show theset based decomposition out-performing the time based decomposition byan average9 elapsed run time of 3.12s compared to 9.87s. Figure 5.7 plotsthe progress of a typical pair of problems, 7-10-s and 7-10-t, showing howthe objective values and bounds converge as the number of cycles increases.The problems are minimisation problems, so the primal (objective) value

9geometric mean

5.9. THE PERFORMANCE OF THE POWERGEN PROBLEMS 113

Problem Set DecOpt xpress-mp

PowerGen 7-day 3.12 0.95PowerGen 84-day 151.13 126.50PowerGen 364-day 1254.02 1991.58PowerGen (overall) 6.33 2.32

Table 5.16: Shortest run times over all block structures obtained for thePowerGen problems.

descends from above whilst the dual bound ascends from below. The plotshows just how much more rapid the convergence is with the set decomposi-tion, and the length of the tail of the time decomposition. The set decompo-sition required one third fewer proposals than the time decomposition (700on average compared to 991), but the most telling statistic is the numberof cycles: 26.8 for the set decomposition; 98.9 for the time decomposition.The time decomposition exhibits the classical asymptotic convergence.

The situation is reversed with the larger data sets. The set decompositioncertainly did not perform well: the 84-day instance took 3159 seconds; the364-day instance has not been considered in the previous sections, but a trialrun was halted after 12 hours with no feasible solution found. In contrast,the best performing time decompositions enabled us to solve the 84-dayinstance in just 21

2 minutes, and the 364-day instance in 21 minutes. Theseare exceptionally good times, close to or better than the times obtained withxpress-mp. Table 5.16 shows the best (geometric mean) times obtained forthe PowerGen problems over all block structures. Using parallel DecOpt withfour processors of similar performance to platform 1, and supposing we couldobtain an efficiency of 67% (as observed in section 5.6 on a multi-processorPC), we would require just 7.80 minutes to solve the 364-day instance.

Figures 5.8 and 5.9 show the convergence of the Dantzig-Wolfe algorithmon the 84-day and 364-day problem instances. They highlight the differentrates of convergence for the different block structures, contrasting the poorperformance of 84-s, 84-t2 and 364-t28 against the good performance of 84-t7, 364-t7 and 364t14. All problem instances exhibit a tail to some extent,but it is noticeably short for 84-t7.

In a practical application, there is often the need to get solutions quickly,within a given time, even if it means compromising the quality of the so-lutions. Both the Dantzig-Wolfe method and the simplex method providefeasible solutions if halted prematurely, and furthermore the quality of thesolutions obtained improves monotonically as the algorithms progress. How-ever the Dantzig-Wolfe algorithm gives the practitioner one crucial benefit:the dual bound allows the quality of the current solution to be assessed, sothat the tradeoff between time and quality can be quantified.

Figures 5.10 and 5.11 show the relative gap between the primal objective


Figure 5.8: Convergence of the PowerGen 84-day problems. The objectivevalue descends from above and the dual bound ascends from below.

Figure 5.9: Convergence of the PowerGen 364-day problems. The objectivevalue descends from above and the dual bound ascends from below.

5.9. THE PERFORMANCE OF THE POWERGEN PROBLEMS 115

Figure 5.10: The relative duality gap observed solving the PowerGen 84-dayproblems.

Figure 5.11: The relative duality gap observed solving the PowerGen 364-day problems.


Problem 10% 1% 0.1% 0.01% Optimal xpress-mp

84-t7 38 78 107 127 151 14084-t14 65 136 192 222 251 14784-t4 29 54 99 241 565 13184-t2 33 125 509 1040 1959 14384-s 156 751 2094 2700 3159 127364-t7 304 583 847 1053 1254 2079364t14 305 645 882 1064 1256 2112364-t28 614 1054 1413 1753 1901 1992

Table 5.17: Elapsed optimisation time taken by DecOpt to solve the Power-Gen problems to guaranteed ε-optimality.

value and dual bound as the algorithm progresses, using a logarithmic scalefor the gap. They provide a different perspective on the convergence of theproblems to the previous figures, and one may observe the points at whichthe algorithm could have been halted early if a solution of a particular qualitywas required. Table 5.17 shows the elapsed optimisation times required tosolve the problems to selected criteria.

Observe in figure 5.10 that the convergence of 84-t7 and 84-t14 is quali-tatively different to that of 84-t2 and 84-s (and, though not shown, 84-t4 ).Both figures 5.10 and 5.11 show how the convergence tends to be logarith-mic up until 0.01%, after which it noticeably improves. These plots supportthe view that Dantzig-Wolfe decomposition is subject to asymptotic conver-gence.

We can speculate why the set and time decompositions perform so dif-ferently with different sized data sets. The PowerGen problems are demanddriven, meaning that power must be generated in each time period to meeta known demand. In the time based decomposition, the demand is handledat the level of the pricing problem, which includes all generating sets for asubset of time periods. In the set based decomposition, the sets are parti-tioned amongst the pricing problems, and to ensure that demand is satisfiedin a given period the demand must be enforced via the master problem.The amount of demand information is proportional to the number of timeperiods, so the master problem sees an increasing amount of work in the setbased decomposition. In contrast, the time based decomposition handlesonly inter-set information in the master problem, and the number of sets isconstant across all data instances.

The time based decompositions are interesting in themselves. The bestperformances, taking into account the parallel and final serial experiments,were exhibited by problems 84-t7 and 364-t14. These would seem to havethe right combination of size and number of pricing problems—in both casesdecompositions with a fewer large pricing problems and decompositions with

5.10. PRACTICAL MERITS 117

more small pricing problems performed less well.A few previous works have considered the optimal degree of decompo-

sition, i.e., whether it is better to have a smaller number of larger pric-ing problems (coarse grained decomposition) or a greater number of smallerpricing problems (fine grained decomposition).10 [Mad73], in the context ofnested Dantzig-Wolfe decomposition, [Kut73], applying Dantzig-Wolfe de-composition, and [HL86], also applying Dantzig-Wolfe decomposition, allagree that the finest grained decomposition is preferable. However, the ba-sis of their conclusions is open to some question. The implementation usedby Kutcher had a high overhead per cycle (see section 4.1), as a consequenceof which he tried to minimise the number of cycles. Ho and Loute basedtheir conclusions solely on the number of cycles, and, furthermore, generatedone proposal only per pricing problem per cycle. It is clear that a smallernumber of pricing problems (blocks) would have allowed fewer proposals tobe generated, and in the extreme case they considered, all the subsystemblocks aggregated into just one pricing problem—only one proposal wouldbe generated per cycle. With multiple proposal generation, their resultsand conclusions may have been different, and of course, the absolute perfor-mance measure must be elapsed time. They do not seem to have consideredif generating one proposal per cycle might not have resulted in much quickercycles. Our conclusions, using an advanced implementation and comparingelapsed time, suggest that the matter is not so clear cut, at least on thePowerGen problems.

5.10 The Practical Merits of Dantzig-Wolfe De-composition

In this chapter we have described and evaluated a computational implemen-tation of Dantzig-Wolfe decomposition. We have shown that it is capable ofsolving a wide variety of LP problems correctly. We have evaluated differentcomputational strategies, some the subject of past research, and used wellestablished statistical techniques to derive good overall settings. We haveshown that good speed-up can be obtained when the software is applied onparallel hardware, both for a small number of processors using commonlyavailable hardware and for a greater number of processors on high perfor-mance parallel hardware. Finally, we have shown that the decompositionmethod, as implemented in DecOpt, is at least comparable to the simplexmethod, as commercially implemented in xpress-mp.

If our limited experiences show anything, it is that if a reasonable blockstructure can be found, the decomposition method is worth trying. We have

10Other works, e.g., [JLFP93], have considered alternative formulations of the originalproblem.


seen that for certain large scale primal block angular problems, Dantzig-Wolfe decomposition was able to obtain the optimal solution in 63% of thetime taken by the simplex method. If we were content with a solutionguaranteed to be within 0.1% of optimality, then the solution time can becut by a further 32%. Using parallel hardware we can cut the solution timefurther: by 59% on a four processor PC (equivalent to a speed-up of 2.42)or 81% on seven processors of a Silicon Graphics workstation (a speed-up of5.22).

Dantzig-Wolfe decomposition will not rival mainstream techniques as anoptimisation method for all LP problems. But our work has shown it doeshave some niche areas of application: certain large scale classes of primalblock angular structured problems, and in particular where the context de-mands rapid results using parallel optimisation or near optimal solutionswith a guaranteed quality. For such problems, decomposition offers thecapability to solve them in much quicker time than would otherwise be pos-sible, providing evidence that, no matter what increases in raw computingpower and resources occur in the future, there will be a class of applica-tions, perhaps of ever increasing size and scope, for which decomposition isthe solution method of choice.

Our work provides the software tools required for further application andexperimentation, and it is our hope that the practical experience and use ofDantzig-Wolfe decomposition will grow as a result.

There are a number of ways in which our work could be extended. Ourexperimental work could be widened to cover more problem instances. Ourevaluation of computational strategies could be developed into a systematicsearch for the optimal computational strategy settings, using a responsesurface methodology. Our evaluation of parallel Dantzig-Wolfe optimisa-tion could be extended to a greater number of processors. Regarding ourimplementation of Dantzig-Wolfe decomposition, there are a few additionaltechniques that could be implemented and evaluated. In particular, therewould seem to be scope to do something much more inventive with presolve:the drawback in our current implementation is that each matrix must becontinually presolved and post-solved. It would seem sensible if each matrixcould be presolved once at the beginning of the algorithm, and post-solvedonce at the end of the algorithm, and all intermediate calculations performedon the presolved matrix. The presolve procedures would have to allow forthe updated cost coefficients in the pricing problems, and the additionalvariables in the master problem. Such procedures could be written at thelevel of the decomposition software, where there is knowledge of the com-plete problem and the updates that will occur, or, with more difficulty, atthe level of the LP optimisation software, where there is knowledge only ofan individual matrix.

Chapter 6

A Decomposition Modeller

An essential component of a mathematical programming system is mod-elling software, which interprets the formulation of the problem supplied bythe user, interfaces with external data repositories, and creates an explicitspecification of the problem instance in a canonical form for the optimiser.For decomposition methods one additional facility is required—the meansto specify a block structure decomposition of the problem instance to thedecomposition optimiser.

In this chapter we describe a new modelling and decomposing softwaretool DecMod. The motivation for DecMod arose from the need to automatethe process of constructing the master problem matrix and pricing problemmatrices required by the decomposition optimiser described in the previouschapter. Whilst we had access to a limited number of problems already posedin a suitably decomposed form, to exploit other problems that were availablein standard model formulations required a lot of work. Our decompositionoptimiser reads in structured problems in the form of separate matrix files forthe master problem and each pricing problem, and it soon became apparentthat constructing these matrix files was not a trivial task, and would provea serious handicap if decomposition were to be applied on anything morethan an ad hoc scale.

The software we describe here permits any block decomposition schemeto be specified and extracted. Thus, in conjunction with a suitable optimiser,it can be used with any form of mathematical programming decomposition,such as nested Dantzig-Wolfe decomposition, Benders decomposition, andcross decomposition, as well as Dantzig-Wolfe decomposition. Throughoutthis chapter, we consider all types for block structure such as those describedin chapter 2, and not just primal block angular structure.

The software we describe simplifies the specification of a block structurefor a problem and automates the process of generating an explicit repre-sentation of the problem instance in a canonical form for a decompositionoptimiser. No attempt has been made to automate the process of identify-

119

120 CHAPTER 6. A DECOMPOSITION MODELLER

ing the block structures in a problem, which, as we discussed in chapter 2,is best performed by the human modeller, e.g., the person formulating themodel. Nevertheless, if automatic techniques were developed, then theycould certainly be incorporated into our software.

The structure of this chapter is as follows. We first review the role ofmodelling software in mathematical programming (MP) systems and theadditional requirements for decomposition. We examine existing modellingsupport and software for decomposition. We introduce our new software,show how it contributes to the development of decomposition as a practicaltechnique, illustrating its use through an example, and finally note somepossible extensions.

Our contribution builds on that made in chapter 2, by providing softwareto support our algebraic approach for identifying block structure. We havedesigned and implemented integrated modelling software for decomposition,which performs all of the functions of standard modelling software for MP,and in addition allows a block structure to be identified and extracted. Thisis a necessary tool for the wider application of decomposition methods. Itproved essential when experimenting with different block structures in thePowerGen problems in preparation for the application of the DecOpt softwareas described in chapter 5.

6.1 The Role of Modelling Software in Mathemat-ical Programming Systems

The principle purpose of modelling software, a modeller, is to produce an ex-plicit description of the problem instance in a canonical form—the matrix—suitable for input to an optimiser. However, modellers are now much morethan the simple matrix generators from which they have evolved [Bea68,BM82, Fou83, FG01, Gre92, GM92, SS93, TW92]. They construct the prob-lem instance by interpreting an understandable and concise description ofthe problem—the model—that is close to the user’s own notation, and gath-ering data from various external data sources. Often, the modeller is alsoused after the solution has been obtained by the optimiser, to make thesolution available in a form understandable to the user, for further analysis,and for export to other software or formats. This mode of use is illustratedin figure 6.1.

Let us briefly review the three major roles of modelling software.Optimisers read in problem instances in specific formats, such as the

open MPS format and a variety of proprietary formats. These formats de-scribe the problem as an explicit canonical LP problem, made up of thecoefficient matrix, RHS, objective function and variable bounds, with justenough structural information to relate each numerical datum to the rele-vant variable and constraint in the problem, but with no macro structure

6.1. THE ROLE OF MODELLING SOFTWARE 121

Figure 6.1: Sequential operation of modeller and optimiser software.

information. Because the bulk of the information is the coefficient matrix,even in a sparse format, the formats used to describe a problem instance toan optimiser are collectively called matrix formats, and the files containingthe problem are called matrix files, or simply matrices. No matrix format iseasy to read, particularly at a structural level, and creating or even editingmatrices by hand or using ad hoc matrix generation programs is prone toerror. A modeller gives the means to generate matrix files automatically,from sources that the user can more easily understand, create and correct.A modeller may also interpret the output files from an optimiser, convertingthe solution from an equally inaccessible output format into one that canbe understood and manipulated in the original context of the problem.

The user must provide a description of the problem to the modeller inorder for it to generate the matrix. Typically, this description is written ina notation that is close to the user’s own notation as befits the applicationbeing modelled. Notations used range from general purpose algebraic baseddeclarative modelling languages, to higher level and more restrictive systemsused for specific applications. They permit the user to specify the problemconcisely in a natural form that is easy to write, understand, and maintain.For large practical problems, it is also desirable to separate the logical spec-ification from the numerical data of the problem, so that the same genericproblem may be solved repeatedly with different instances of data, perhapsas part of a production process, or because the generic problem is embeddedwithin an application that is distributed to many users to use with their owndata. The generic description of the problem is called the model. Softwarefor specific applications may have the model embedded in the software, andthe user simply has to fill data tables to specify a problem instance. Alge-braic modelling languages, the focus of our further discussion, potentiallyallow any model to be specified, by declaring the parameters, data tables,variables, the sources of data, and formulating the constraints in an alge-braic format, without explicit specification of data. Algebraic modellinglanguages also provide means to manipulate the data, allowing them to beinput in their original form without external massaging.


Figure 6.2: The modeller as a front-end to the optimiser.

The third major role of a modeller is to interface with external datasources, to extract data and subsequently return the corresponding solu-tion. Data may be stored in a variety of forms, such as databases, spread-sheets, and text files, and modellers are typically able to access all of these.Statements in the model specify from where the data are to be obtained.

Some modelling software also give interactive access to the optimiser, al-lowing models to be constructed, solved, altered, re-solved, etc., perhaps byincluding primitives to control the optimiser from the modelling language.In this mode of use the modeller performs the additional function of being afront-end for the optimiser, and we illustrate this mode of use in figure 6.2.We do not consider this mode of use in our work, although such an extensioncould be implemented if desired. Of course, some of the apparent benefitsare superficial—solving a sequence of problems may just as easily be accom-plished by running the modeller and optimiser in sequence as a batch job.The main benefit is being able to write meta-algorithms in the model, whichsolve a sequence of problems as part of one larger problem via repeated callsto the optimiser; indeed, many implementations of Dantzig-Wolfe and otherdecomposition algorithms have been written in this way. Such implementa-tions are invariably inefficient, as they involve repeated regeneration of thematrix in between iterations, and operating the modeller and optimiser si-multaneously can incur a substantial memory overhead which is particularlysignificant for our work with large problems.

To support decomposition, a modeller must also enable the user to spec-ify a block structure for a general LP problem, which is then interpreted inthe context of the problem, and used to generate the explicit description ofthe problem instance in a format suitable for the decomposition optimiser.So the standard modeller functionality of reading in a model and generatingthe matrix is extended to read in a decomposition structure and generatea ‘matrix’ in some extended form containing the decomposed problem in-stance.

We have described the three important roles that modelling softwareplays in MP systems. There is no doubt that a modeller greatly facilitates

6.2. MODELLING SUPPORT FOR DECOMPOSITION 123

the application of MP and is one of the major reasons for its frequent andwidespread use [Fou96a, Fou96b, FG00, SR95]. For decomposition optimisa-tion to be at all successful beyond dedicated researchers, modelling supportfor decomposition is essential. The next section reviews some developmentsthat have been made in this area, followed in section 6.3 by our own contri-bution.

6.2 Modelling Support for Decomposition

In this section we describe what modelling support already exists or has beenused for decomposition, and alternative ideas and software to our own thatare currently proposed or being developed. We comment on the advantagesand disadvantages of these other methods (actual or proposed).

Until recently, there has been no modelling software to assist the ap-plication of decomposition methods to general purpose problems—neithermodelling software specifically for decomposition nor decomposition exten-sions to general MP modellers. Two methods to generate the problem in-stances (matrices) required for decomposition were open to practitioners:writing ad hoc decomposition matrix generators for specific problems; orusing available modelling software to do the bulk of the work, and obtaininga structured matrix or decomposed set of matrices by manual manipulationof the model or matrix.

An ad hoc matrix generator may be suitable where a particular model-decomposition pair was to be solved with a large set of data instances.However, it could not be used with other models, it would be difficult toexperiment with different block structures, and it would require significantmaintenance to keep up with developments in the model. This approach isnot widely accessible nor maintainable, as it requires detailed knowledge ofcoding, data extraction, matrix generation, the specific model, and decom-position within one project team, and thus requires a large commitment ofresources. These are much the same reasons that Orchard-Hays identifiedas an obstacle to the general development of decomposition in 1973 [OH73].Much of the effort involved is spent duplicating facilities of other matrixgenerators, and indeed these were all motivating factors for the originaldevelopment of general modelling software [Fou83].

A standard MP modeller may be used to interpret a model, extractdata, and generate matrix files, and then a set of decomposed matrices oran ordered matrix may be obtained either by manipulating the input modelor the output matrix file. The main disadvantage is the likely possibility ofintroducing errors into the final matrix or matrices, and the restriction on thesize of the problem that can be easily manipulated by hand, or even by semi-automated tools. Furthermore, the number of problems that can be solved,and experiments performed with other decompositions and data instances,


is restricted, given the effort it takes for each model/data/decompositioninstance.

In both cases, some or all of the benefits that would be enjoyed by usingmodelling software, as reviewed in section 6.1, are lost or overwhelmed. Thelack of proper modelling support for decomposition is a definite obstacle tothe wider and more successful use of decomposition optimisers.

There have been several efforts recently to address this issue, and twomain approaches are apparent: providing explicit support for decompositionin a general MP modelling language as implemented in modelling software;and using a module attached to but separate from general MP modellingsoftware.

Several proposals have been made to extend the ampl modelling softwareto provide support for stochastic programming [FG97, GI95, GI96], for whichthere are several Benders’ decomposition based optimisers available. Oneidea is to introduce the concept of scenarios, giving each entity in the modela scenario index implemented as an additional suffix in the ampl modellinglanguage, to be communicated to an optimiser that made use of such aconcept. Such an extension has an obvious application to decomposition ingeneral, as the scenario index could be used to indicate the block numberwithin a block structure. However, the proposals have come to nothing.

The second approach has proven more fruitful, with two pieces of soft-ware splitdat and spi produced. (Our software also falls into this category.)splitdat was developed as an extension to gams for use with a Benders’decomposition solver [CF96]. It requires as input a single index set occurringin the model which is used to classify each entity in the model into blocks,based on the index value taken by each entity. Entities which have the firstelement of the selected index set as an index are placed in block 1; those withthe second element of the index set in block 2; and so on. Entities which arenot indexed by the chosen index set are placed in block 0, representing themaster problem. splitdat builds up two block index arrays internally—onefor variables and one for constraints—containing the block index of each en-tity. These arrays are used to split the entire problem matrix into matricesfor each block, which are used by the decomposition optimiser.

For instance, using the production planning model example introducedin chapter 2 (see section 2.3, p. 14), we might specify a dual block angularstructure simply by indicating the product set, t ∈ T . Entities with t = 1form the first pricing problem, entities with t = 2 form the second pricingproblem, and so on, with any remaining entities forming the master problem.

splitdat exploits the dictionary file produced by gams, which lists eachentity in the problem instance giving a name based on the algebraic notationused in the model and the position of the entity in the matrix produced.The algebraic name includes the generic name of the entity and the namesor values of the indices of the entity, e.g., an instance of the variable sellpftmight be represented as

6.2. MODELLING SUPPORT FOR DECOMPOSITION 125

VARIABLE."sell"."NAILS"."Doncaster"."March"

splitdat first uses the dictionary to read the index values from the selectedindex set. It then checks each variable and constraint in the list to see if itincludes an index value from the selected index set, and allocates it to oneof the blocks accordingly.

Specifying a single index is simple enough, but does have some majorweaknesses. It is only suitable when the class of block structure (e.g., primalblock angular) is a given. It becomes more clumsy where one wishes to groupentities with a range of index values, say t = 1–5, t = 6–10, etc., into oneblock, or where the block structure is not described neatly by one index set.The particular implementation in splitdat is also open to problems if theelements of the selected index set occur elsewhere in the model (perhaps inother index sets), or if some entities are indexed twice by the same indexset. The user may specify that some of the initial elements of the chosenindex set are to be allocated to the master problem, but no other flexibilityis allowed.

The experience of splitdat led to the development of a more advancedpartitioning software, Structure Passing Interface (spi) [FGS98, FGS01,FGSV00]. Like splitdat, spi is also an extension to gams and uses thegams dictionary file to relate the entities in the matrix to the algebraicstructure of the model. In principle the software could be interfaced toother modelling software, such as ampl, that also produced a dictionary file(but the user input would have to change—see comment below).

The major advance over splitdat is that spi takes as input an expres-sion for each variable and constraint block that defines the block structure,giving much greater flexibility for describing block structures. Any blockstructure can be described, whereas splitdat was restricted to dual blockangular for use with Benders’ decomposition. The input expression is aregular expression, which allows the user to specify a pattern or a series ofalternatives. The dictionary file, containing the algebraic names and indicesof each entity used in the matrix, is scanned, and each variable or constraintthat matches one of the regular expressions supplied is assigned to the vari-able or constraint set indicated. The output is a pair of arrays, one for thevariables and one for the constraints, giving the set index of each variableand constraint.

The use of regular expressions confers both advantages and disadvan-tages. In principle they allow any block structure to be specified (as a lastresort the regular expression could simply be a list of all entities for the oneset), although the expressions become progressively more complex for blockstructures that are less clean and not reliant on a single index set. Theuser does not have to list each variable and constraint within each block,providing a common index value or range of values is used to define eachblock. It does not use the modelling software itself to detect entities and


indices, which may lead to complications if, for instance, the order of indicesis not the same in all entities. The language of regular expressions is verydifferent from the gams or any other MP modelling language, making itdifficult to learn and use, and relate to the model. The regular expressionsmust specify the entities as they are written in the dictionary file producedby the modelling software, rather than the model supplied by the user, asubtle but important difference, and the syntax of expressions is thereforedependent on the syntax of the dictionary file. There is the potential forambiguity if, for instance, an expression used to define a repeated index alsooccurs accidentally as part of another name. Finally, the block structurespecification is data dependent—it cannot be parameterised using parame-ters in the model and similar blocks must be specified individually. So if onemodel is used with several data sets of differing sizes, different decomposi-tion specifications must be supplied, even if the pattern for defining blocksis the same in each.

spi has been used with the Dantzig-Wolfe decomposition optimisationsoftware accpm and hopdm (see section 4.4 and the references therein),together known as Structure Exploiting Solver (set); and has been usedwith the sp/osl Benders decomposition solver, together known as set-

stoch [CLF98]. It has been applied successfully to problems of up to 1million variables and 2.5 million constraints [FGV00].

6.3 DecMod: A Decomposition Modeller

We have put forward the case for a decomposition modeller and reviewedexisting tools. We now present our new decomposition modeller DecMod.

DecMod takes a model and data, as with a standard MP modelling sys-tem, and in addition takes a block structure or decomposition specifica-tion. The model is written in the xpress-mp modelling language, and thexpress-mp modeller is used in a callable form to parse the model, importthe data, and generate a complete representation of the problem instance.The decomposition specification is written using a language specially de-veloped for the purpose, allowing entities to be specified in the same formthat the user used in his/her model, and using a grammar similar in styleto the xpress-mp modelling language, making it easy to understand anduse. The language has range constructs to eliminate redundancy, and al-lows arithmetic expressions and reference to model parameters and data toremove data dependency. DecMod parses the decomposition specification,building up an internal symbolic representation, extracts the matrix datafor each block from xpress-mp’s representation, and writes them out to aset of matrix files, which can be used to input the problem into DecOpt. Anyarbitrary block structure may be specified, and a variety of output formatsare available, so that DecMod can be and indeed has been used with various

6.4. EXAMPLE 127

decomposition optimisation software.DecMod allows the user to concentrate on identifying a suitable block

structure for the decomposition method being considered, without havingto spend time and effort explicitly extracting the block structure from theproblem. Writing and modifying the block structure specification is an easytask, so users can rapidly experiment with and assess different block struc-tures within one model, given a suitable decomposition optimiser. DecModremoves one of the major obstacles towards applying decomposition as ageneral solution technique. DecMod has been used successfully by both Pow-erGen and BASF as part of the MEMIPS project.

We illustrate the use of DecMod with an example in the next section.The remaining sections describe aspects of the design and implementationin detail and discuss possible extensions.

6.4 Example

This example illustrates how DecMod may be used to extract various primalblock angular structures from a simple LP problem. For a detailed guideand reference see the DecMod User Manual, which is included in this workas appendix D.

6.4.1 A Simple Production Planning Model

The example is based on the production planning model example introducedin chapter 2 (section 2.3, p. 14). Table 6.1 (p. 128) presents the body of anxpress-mp model formulation of the problem, which is a direct translationfrom the algebraic formulation given in table 2.1 (p. 15).

In chapter 2 we observed that the model is highly structured, with foursets of indices present throughout the model: finished products p, raw ma-terials m, facilities f, and time periods t. We will consider block structuresbased on the facility and time period structures, and show how DecMod al-lows alternative block structures to be extracted easily from a single model.

6.4.2 Decomposition by Facility

Inspecting the model we see that most of constraints—PBal, MBal, MxPStock,MxMake and MxMStock—are indexed by facility, and furthermore they onlycontain variables indexed by the same facility index f. Thus these con-straints all partition into blocks based on the facility index. The objectivefunction MaxProfit and constraints MxSell and MxBuy are not indexed byfacility, and apply across the different facilities in the problem. By labellingthese constraints global constraints, we are able to identify a primal blockangular structure.


CONSTRAINTS

MaxProfit:SUM(p=1:NP,f=1:NF,t=1:NT) REV(p,t) * sell(p,f,t) &- SUM(p=1:NP,f=1:NF,t=1:NT) CPSTOCK * pstock(p,f,t) &- SUM(p=1:NP,f=1:NF,t=1:NT) CMAKE(p,f) * make(p,f,t) &- SUM(m=1:NM,f=1:NF,t=1:NT) CMSTOCK * mstock(m,f,t) &- SUM(m=1:NM,f=1:NF,t=1:NT) CBUY(m,t) * buy(m,f,t) $

PBal(p=1:NP,f=1:NF,t=1:NT): pstock(p,f,t+1) &= pstock(p,f,t) + make(p,f,t) - sell(p,f,t)

MBal(m=1:NM,f=1:NF,t=1:NT): mstock(m,f,t+1) &= mstock(m,f,t) + buy(m,f,t) &- SUM(p=1:NP) REQ(p,m) * make(p,f,t)

MxSell(p=1:NP,t=1:NT): SUM(f=1:NF) sell(p,f,t) <= MXSELL(p,t)

MxPStock(f=1:NF,t=1:NT): SUM(p=1:NP) pstock(p,f,t) <= MXPSTOCK

MxMake(f=1:NF,t=1:NT): SUM(p=1:NP) make(p,f,t) <= MXMAKE(f)

MxMStock(f=1:NF,t=1:NT): SUM(m=1:NM) mstock(m,f,t) <= MXMSTOCK

MxBuy(m=1:NM,t=1:NT): SUM(f=1:NF) buy(m,f,t) <= MXBUY(m,t)

Table 6.1: An xpress-mp model of the production planning problem.

To specify the decomposition to DecMod, we supply a decompositionspecification containing lists of the entities in each of the blocks, writtenin DecMod’s own decomposition specification language. Table 6.2 (p. 129)shows how the facility decomposition can be specified; let us consider whatit means.

Spaces, blank lines, and comments can be included in the specificationfile to improve readability and self document the file. Comments are pre-ceded by an exclamation mark and last until the end of the line.

The specification contains two sets of blocks, each introduced by theBLOCKS keyword, and named master and sub. The block names are arbi-trary and have no meaning to DecMod. We exploit the fact that the pricingproblems follow a common pattern to specify them all in one go. The spec-ification is terminated by the END keyword.

The master block has three sets of constraints and no variables. Theconstraints are introduced by the CONSTRAINTS keyword, and written usingexactly the same name that was used in the model file. For an indexedconstraint set such as MxSell, we must supply the correct number of indicesor index ranges—in this case the indices ‘(*,*)’ mean that the entire set of

6.4. EXAMPLE 129

! Production planning problem; facility decomposition

BLOCKS master ! Master problemCONSTRAINTS ! Global constraintsMaxProfitMxSell(*,*)MxBuy(*,*)

BLOCKS sub ! Pricing problemsf=1:NF ! 1 for each facility

CONSTRAINTS ! Global constraintsMaxProfitMxSell(*,*)MxBuy(*,*)PBal(*,f,*) ! Pricing problem constraintsMBal(*,f,*)MxPStock(f,*)MxMake(f,*)MxMStock(f,*)

VARIABLES ! Pricing problem variablessell(*,f,*)pstock(*,f,*)make(*,f,*)mstock(*,f,*)buy(*,f,*)

END

Table 6.2: DecMod specification for the facility decomposition.

MxSell constraints is included in the master block.The BLOCKS sub section defines a set of blocks, one for each value of f in

the range 1–NF. The dummy variable f is local to the decomposition specifi-cation; when DecMod extracts the blocks, it will set f to one and substituteit in the variable and constraint lists in the current block, then set f to twoand repeat, and so on, until f equals NF. Any constant symbol not recog-nised within the decomposition specification, such as NF, will be looked upin the model and its value obtained from the data instance. This makes thedecomposition specification file independent of the data instance—whateverthe size of the data instance used with the model, the decomposition speci-fication will be valid.

Following the range specification, the constraints and variables are listedin CONSTRAINTS and VARIABLES sections. DecOpt requires the pricing prob-lem matrix file to contain the global constraints as well as the pricing prob-lem constraints. (Blocks Ak and Bk in figure 2.2, p. 11.) Observe the dif-ference between the pricing problem constraints and the global constraints:


the pricing problem constraints are indexed by the dummy variable f, andso will be partitioned amongst the pricing problems; the global constraintsall appear in each pricing problem.

DecMod will output a matrix corresponding to each block defined byeach set of BLOCKS. The matrix will contain the entities listed, togetherwith the matrix coefficients occurring in both a constraint and variable inthe block. As the master problem contains as no variables, the constrainttypes and RHS will be included in the master problem matrix, but no matrixcoefficients.

6.4.3 Decomposition by Time Period

! Production planning problem; time period decomposition

BLOCKS master ! Master problemCONSTRAINTS ! Global constraintsMaxProfitPBal(*,*,*)MBal(*,*,*)VARIABLES ! Global variablespstock(*,*,1)mstock(*,*,1)

BLOCKS sub ! Pricing problemst=1:NT ! 1 for each time periodCONSTRAINTS ! Global constraintsMaxProfitPBal(*,*,*)MBal(*,*,*)CONSTRAINTS ! Pricing problem constraintsMxSell(*,t)MxPStock(*,t+1)MxMake(*,t)MxMStock(*,t+1)MxBuy(*,t)VARIABLES ! Pricing problem variablessell(*,*,t)pstock(*,*,t+1)make(*,*,t)mstock(*,*,t+1)buy(*,*,t)

END

Table 6.3: DecMod specification for the 1 time period decomposition.

We can identify another primal block angular structure using the time

6.4. EXAMPLE 131

period index. Table 6.3 (p. 130) shows a DecMod decomposition specificationfor a simple time period block structure.

The time decomposition illustrates that occasionally it is useful to be ableto include entities in a block with different index values to the majority ofentities in the block. In this example, we placed variables pstock(*,*,t+1)and constraints MxPStock(*,*,t+1) in pricing problem t.

In sections 2.4 and 5.9 we discussed the possibility of aggregating blockstogether to produce a variation of a block structure with fewer blocks.DecMod contains a number of features to support this: entities can be spec-ified using a range of index values instead of a single index value, and therange of blocks can be specified using integer arithmetic and additionaldummy variables.

! Production planning problem; 7 time period decomposition

BLOCKS master ! Master problemCONSTRAINTS ! Global constraintsMaxProfitPBal(*,*,*)MBal(*,*,*)

VARIABLES ! Global variablespstock(*,*,1)mstock(*,*,1)

BLOCKS sub ! Pricing problemst=1:NT/7 ! 1 for each set of 7 time periodsl=7*(t-1)+1 ! First time period in block tu=7*t ! Last time period in block t

CONSTRAINTS ! Global constraintsMaxProfitPBal(*,*,*)MBal(*,*,*)

CONSTRAINTS ! Pricing problem constraintsMxSell(*,l:u)MxPStock(*,l+1:u+1)MxMake(*,l:u)MxMStock(*,l+1:u+1)MxBuy(*,l:u)

VARIABLES ! Pricing problem variablessell(*,*,l:u)pstock(*,*,l+1:u+1)make(*,*,l:u)mstock(*,*,l+1:u+1)buy(*,*,l:u)

END

Table 6.4: DecMod specification for the 7 time period decomposition.


Figure 6.3: The design of DecMod: its modules, data structures, inputs andoutputs.

A second time based decomposition, shown in table 6.4 (p. 131), illus-trates these features. The model is partitioned into blocks of 7 days, withNT/7 blocks in total. We define a sub block for each t in the range 1 to NT/7,and define two extra dummy variables, l and u, to be the first and last timeperiods in each block respectively. Each block includes the pricing problementities with time period index values in the range l to u. The specificationis easily understood, concise and remains free of explicit data, even thoughthe block structure itself is more complex.

6.5 The Design and Implementation of DecMod

In this section we describe the inputs, outputs, decomposition language,modules, and data structures that together make up DecMod. The design ofDecMod is summarised in figure 6.3.

6.5.1 Inputs

Model An MP model written in the xpress-mp modelling language.

If users have previously used a different modelling language, it wouldbe necessary to translate their models into the xpress-mp modellinglanguage in order to use DecMod. However, this is not such a hurdleas it might appear—it is normally possible to translate LP modelsbetween different algebraic modelling languages fairly easily.

Data Data may be retrieved from databases, spreadsheets, data files or acombination of these, using the data interfaces available with xpress-

mp. The model specifies where the data is held, and how it is to beused in the model. It is also possible to include data directly in themodel.

6.5. THE DESIGN AND IMPLEMENTATION OF DECMOD 133

Decomposition A specification of the block structure to be extracted fromthe model. Written in a special purpose language with a similar syntaxto the xpress-mp modelling language. The language is described insection 6.5.3, below.

The separation of the decomposition specification from the model, asidefrom being a practical necessity, does confer benefits. The model is freeof any additional elements necessary for the decomposition specification, socan still be used in the normal way to be solved without decomposition;conversely, existing models can be subjected to the decomposition solversimply by writing the decomposition file.

The PowerGen problems used in chapter 5 illustrate how important itis to be able to try out different block structures in one model—not onlyblock structures induced by different index sets within the model, such asthe basic set and time based decompositions employed with the PowerGenproblems, but also variants within one basic block structure, as illustratedby the different time based blocks structures used for the larger PowerGendata sets. With DecMod, different block structures can be extracted from asingle model simply by substituting different decomposition files.

6.5.2 Outputs

DecMod is able to output the decomposed problem using three mechanisms.

Set of matrix files Each block in the decomposition is output to a sepa-rate matrix file.

Single ordered matrix file The whole problem is output to a single ma-trix file, in which the entities are ordered by block. For example, theentities occurring in the first block are listed first, then entities insecond block, etc.

Single matrix file with block index file The whole problem is outputto a single matrix file with no special ordering. An additional blockindex file gives a block index value for each entity occurring in thematrix.

All three of these mechanisms are equally applicable to memory transfer, asmight occur in combined modeller/optimiser software.

The first mechanism, a set of matrix files, is the input format used byDecOpt (see appendix C, section C.6). The other two mechanisms havebeen provided for interfaces to other decomposition optimisers; e.g., [And98]uses the block index file mechanism with his IPM optimiser, which exploitsthe block structure in the Cholesky factorisation but otherwise treats theproblem as a whole. It would be a relatively simple task to support otheroutput formats if required.


The matrices may be output in the standard MPS format, a high preci-sion variant of the MPS format, or the xpress-mp binary matrix format.

6.5.3 Decomposition Language

An algebraic modelling language provides the most general purpose and flex-ible modelling component, suitable for all classes of problems. The use ofan algebraic modelling language to specify the basic model makes it easy todevise a mechanism for specifying the block structure (which we refer to asthe decomposition). The user has already provided a name for each entity inthe model, and, having written the model, is familiar with notations of themodelling language. The existing conventions and structures of the mod-elling language can be exploited to allow the block structure to be specifiedin a way that is concise and powerful, yet intuitive and clear to the user.

The decomposition language has been designed to allow an arbitraryblock structure to be specified using a notation that is concise, exploits pat-terns to eliminate redundancy, allows reference to parameters in the modelto remove data dependency, and remains close in grammar to the xpress-mp

modelling language.The basic unit of the specification is a set of blocks. Each set may

have a range quantifier, giving the set of values that define the individualblocks in the set. This allows a set of blocks sharing a common pattern,such as the subsystem blocks in primal block angular decomposition, to bespecified together. With no range specification, the set of blocks consistsof a single block, such as may form the master problem in Dantzig-Wolfedecomposition.

Each set of blocks possess a list of entities, and the intersection of thevariables and constraints in the problem matrix defines the block matrixextracted for each block. The entities are described using exactly the samenames, indices and general notation as used in the model.

An index of an entity may be a single value, a range of indices, or thecomplete range of values indicated using the wild card range ‘*’. The rangequantifier and other dummy variables may be used as part of the index, topartition the entity amongst the blocks in the set. See the specifications intables 6.2, 6.3 and 6.4 for examples.

A decomposition specification comprises a number of sets of blocks, eachwith an optional range, and a list of variables and constraints. This allowsfor any conceivable block structure.

6.5.4 Modules

DecMod comprises 3492 lines, 36 functions and 20 files of Standard C.

Model parser The xpress-mp modeller is used to parse the model, inter-face to data sources and extract relevant data, and generate a repre-

6.5. THE DESIGN AND IMPLEMENTATION OF DECMOD 135

sentation of the complete problem instance within xpress-mp. It isused in the form of the xpress-emosl subroutine library.

Without doubt the objectives of providing modelling software for de-composition—promoting wider use of decomposition optimiser soft-ware—could best be achieved by providing decomposition supportwithin a general MP modelling language. However, to implement de-composition extensions within existing modelling software or indeed tobuild a new general modelling software incorporating decompositionwould require a significant investment of resources, which would prob-ably not be justified by the expected returns. Without backing froman existing modelling software vendor, and their expert knowledge ofand access to the software, such a task would be very difficult, and thealternative approach of developing an add-on module for decomposi-tion support to a proprietary modelling system is probably the mostpractical way of achieving the goal. This view is supported by the factthat while three extensions to modelling software have been developedand applied, efforts to incorporate decomposition support into exist-ing modelling software have been confined to ideas and proposals (seesection 6.2).

Using an established MP modeller within DecMod automatically pro-vides the facilities reviewed in section 6.1. It also allows existing mod-els written in the particular modelling language to be used with thedecomposition software.

The use of xpress-mp within DecMod would not have been possiblewithout the availability of the xpress-emosl subroutine library. Itprovides the means to interrogate the problem instance in memoryusing the algebraic entities defined in the model to retrieve parame-ters and data [DT01]. This feature is crucial to implement a blockstructure specification module that recognises and uses entities fromthe algebraic model.

Block structure parser The decomposition parser is an interpreter thatembodies the DecMod decomposition language. It parses the decompo-sition file and builds up a symbolic representation of the block struc-ture.

Matrix extractor The matrix extractor evaluates symbolic representationof each set of blocks, and obtains the data necessary to describe thematrix for each block of the decomposition. It loops over the rangequantifier (if present) included in the set of blocks, producing a differ-ent block for each. It evaluates integer arithmetic expressions, recog-nises model data objects and obtains their values from xpress-mp.It identifies the entities to be included in each block, and obtains thedata to construct the block matrix from xpress-mp.


Matrix exporter The matrix exporter exports the matrix for each blockto a matrix file.

If an alternative output mechanism has been selected, the matrix ex-tractor and exporter behave accordingly. The matrix extractor canbuild up an alternative complete matrix for the problem, using theorder given by the decomposition, so that the resulting matrix is or-dered by block. The matrix exporter then simply exports the completeordered matrix. Alternatively, for each entity within each block, thematrix extractor may just obtain the sequence numbers—the row andcolumn numbering—from the original matrix, so that it can build uparrays to accompany the original matrix giving the block index of eachentity.

6.5.5 Data Structures

Problem instance This data structure is internal to xpress-mp. It con-tains the entire problem instance, combining the model notation withthe explicit data of the matrix. It is constructed by and accessedthrough the xpress-emosl subroutine library, and used as the sourceof all enquiries about the original problem.

Block structure A symbolic representation of the block structure is con-structed by the decomposition parser and evaluated by the matrixextractor. Its form directly corresponds to the structure of sets ofblocks and lists of entities imposed by the decomposition language.

Matrix block data The matrices built up by the matrix extractor arestored in a standard set of sparse matrix arrays.

6.6 Extensions

Combining our ideas for a decomposition modeller and optimiser we have thebasic design for a decomposition modelling and optimisation system similarto that pictured earlier in figure 6.1 (p. 121). However the third stage, thatof reading the solution found by the optimiser and making it available in aconvenient form, is missing. In the case of decomposition, the third stagewould need to take the solutions to the decomposed problem and recombinethem, to make available a solution to the whole problem, so the user coulduse the solution in the same way as if conventional MP methods had beenemployed to find it.

As it stands, our decomposition modeller only does half the job—it takesa standard, complete model, and decomposes it, in order for the problemto be solved with a decomposition optimiser, but it doesn’t take the de-composed solutions and recombine them so that the solution to the whole

6.6. EXTENSIONS 137

problem can be easily interpreted and analysed. The user must work withseparate solutions for the master problem and each of the pricing problems,which is not very satisfactory and still imposes an obstacle to the practicaluse of the decomposition. A very natural extension for DecMod would be toimplement a solution combination phase, and indeed a full solution listingcould be implemented relatively easily. However, for real practical use itwould be desirable to exploit the embedded xpress-mp software to furthermanipulate and export the solution to external databases, as would be pos-sible if the solution had been obtained entirely with conventional means.This task was not necessary for our research, and would require a lot of ef-fort to implement in a similar way to DecMod. It would however be relativelyeasy to implement in association with the developers of xpress-mp if theywished to do so.


Chapter 7

Conclusion

LP decomposition techniques, and Dantzig-Wolfe decomposition in partic-ular, have been the subject of a lot of research over the past forty or moreyears. What has our contribution added to the body of knowledge that hasbeen built up; what more can we say about Dantzig-Wolfe decomposition?

Our contribution has been in the main a practical one: we have takenexisting theory and ideas, together with some new techniques of our own,and applied them in a computational implementation. We have not beenthe first to implement Dantzig-Wolfe decomposition, but we have been thefirst to do so in a way that imposes no artificial limits on the size of problemthat may be solved, that allows consistent evaluation of serial and parallelimplementations, and that is straightforward to use by the practitioner yetallows systematic experimentation and analysis of a wide range of compu-tational strategies by the expert user. And no less important for the futureapplication of decomposition, our implementation has a clean and well de-fined interface with the LP optimisation library, that will allow it to bemaintained and exploit future advances in LP optimisation technology.

We have been able to evaluate, in combination, a number of differenttechniques that have been proposed for use with Dantzig-Wolfe decomposi-tion, and recommended combined settings that would appear to give goodoverall performance in practice. We have demonstrated some importantpractical instances where decomposition obtains the solution in a shortertime than the simplex method. If a solution within 1% or 0.1% of optimal-ity is sufficient in the context of the problem, solution times may be cut byup to 54% or 32% respectively. We have been able to show that despite, orperhaps in association with, all the advances in LP optimisation technology,decomposition is a useful practical technique.

Perhaps our most important contribution has been the development andapplication of the parallel variant of Dantzig-Wolfe decomposition. Onceagain, we are not the first to have attempted this, but as we showed inchapter 4, previous work in this area has been very limited. We have im-

139

140 CHAPTER 7. CONCLUSION

plemented Dantzig-Wolfe decomposition in parallel, using all the techniquesand computational strategies (and much common code) of the serial imple-mentation. We have obtained creditable results with the parallel application(allowing for the fact that we were able to predict and eliminate problemswhich were not suited to parallel optimisation), and although the efficien-cies were not as great as originally hoped, we did achieve efficiencies in therange of 59–75% for four processors using the larger more successful prob-lems. This translates into cutting solution times by 59% on a four processorPC (equivalent to a speed-up of 2.42) or 81% on seven processors of a Sil-icon Graphics workstation (a speed-up of 5.22). We have shown that theperformance is similar across a variety of parallel hardware, ranging fromthe sophisticated parallel computers usually associated with parallel appli-cations, to the more widespread multi-processor PCs and networks of PCsthat are available in most organisations.

We have formalised an ‘algebraic’ approach to identifying block structurein LP problems, and developed modelling software to support preparationof problems for the application of decomposition techniques.

In summary, we have demonstrated that Dantzig-Wolfe decompositionis an important practical technique and produced software tools to enablefurther experimentation and promote its wider application.

Appendix A

Experimental Platforms andSoftware

A.1 HPPVM Options

We reproduce the hppvm config file used which describes the options andthe chosen settings.

# Relative speed of machine for config commandspeed = 1000

# Number of processorsprocs = 1

# error level no | env | fatal | warning | allerror_level = all

# use sequence numbers and checksums to verify data integrityverify = no

# Allowed to use shared memory?shmem = yesshmem_total = 2000000shmem_link_size = 30000

# Choose port automatically or generate within fixed rangeauto_socket_port = yes

# Buffering of pvm output into tmp directory log files. "no",# "line" or "full" bufferingtmp_file_buffering = full

# List of trusted machines that can use me as a slave machine# The inet addresses must be able to be looked up by this

141

142 APPENDIX A. EXPERIMENTAL PLATFORMS AND SOFTWARE

# machine from these names. Use name "all" to allow any# machine to use us.machine = all

# Should slave processes have pvmlib debug windows displayed.# They will only be created when the first error or warning# is displayed# no - not at all# yes - yes, but they should be created minimised# popup - yes, and they should popup into viewslave_window = yes

# output pvm calls in tmp filesdebug_pvm_commands = no

# output dump of data packed and unpacked in tmp filesdebug_pvm_data = no

Appendix B

Experimental Designs andResults

B.1 Validation

This section gives the results of first experiment, described in section 5.4(p. 91), which was to validate that DecOpt could indeed solve the problemsunder consideration. All problems were solved using serial DecOpt on aWindows PC (platform 1 p. 90). DecOpt was applied using the settingsgiven in the DecOpt User Manual (appendix C, section C.10) apart fromthose listed in table 5.7 (p. 92).

Table B.1 reports the elapsed time (Time), the elapsed optimisation time(Opt), the number of cycles taken by the column generation decompositionalgorithm (Cyc), the final objective value reported (Obj ), and the final rela-tive duality gap reported (Gap). Table B.2 reports the number of proposalsgenerated (Props), the number of iterations taken by the simplex algorithmto solve the master problem (MPiv), the average number of iterations takenby the simplex algorithm to solve one pricing problem (PPiv), the timetaken handling the master problem (MTime), and the average time takenhandling one pricing problem (PTime).

Table B.1: Validation results (part 1).

Problem Time Opt Cyc Obj Gapdregem 2.22 1.86 25 unboundeddyna 0.50 0.14 1 unboundedegd034 0.43 0.17 7 39728.00221 0egd085 0.56 0.28 8 99320.00552 0fixmar 0.91 0.69 43 18416.75903 0forest 0.84 0.59 21 61542.19551 0meat12 1.32 1.06 56 150907.6186 0meat31 0.60 0.31 11 517665.0249 0meat43 0.60 0.31 11 42223.13685 0

143

144 APPENDIX B. EXPERIMENTAL DESIGNS AND RESULTS

Table B.1: Validation results (part 1, continued).

Problem Time Opt Cyc Obj Gapmege06 0.46 0.21 9 351.2520143 1.07E−07mege08 0.59 0.30 11 552.6427855 1.11E−08mege10 0.67 0.36 10 744.0105035 3.13E−07mrp3 0.42 0.20 15 30009.97201 0mrp5 1.76 1.49 46 109469.8187 0regem 0.68 0.30 3 139381.8848 0scorpion 0.42 0.17 9 1878.124823 0ken07 0.88 0.28 1 -679520443.4 0scfxm2 0.76 0.53 18 36660.26157 0scfxm3 0.93 0.68 15 54901.25455 0sctap1 1.59 1.31 51 1412.250000 0sctap2 7.85 7.26 77 1724.807143 0sctap3 24.57 23.84 192 1424.000000 0small 4.16 2.04 2 199.4868165 1.57E−06med1 9.81 5.58 2 199.4673396 2.26E−07med2 9.42 6.38 2 199.4781938 5.51E−07large 12.75 7.34 2 199.4545125 1.83E−067-01-s 4.22 3.80 57 9941.824143 07-01-t 6.03 5.72 95 9941.824143 07-02-s 3.41 3.00 44 7053.580215 07-02-t 5.01 4.70 82 7053.580215 07-03-s 4.05 3.64 59 4739.587228 07-03-t 9.99 9.68 216 4739.587228 07-04-s 4.42 3.99 59 6773.410206 07-04-t 14.35 14.04 279 6773.410206 07-05-s 4.11 3.71 58 5251.281993 07-05-t 11.84 11.60 238 5251.281996 3.05E−097-06-s 4.61 4.27 65 5239.429368 07-06-t 28.82 28.57 556 5239.429368 07-07-s 4.67 4.35 63 6603.928987 07-07-t 60.20 59.95 1023 6603.929002 7.34E−097-08-s 4.73 4.39 60 6037.296224 07-08-t 44.01 43.75 789 6037.296225 7.84E−097-09-s 3.93 3.52 47 7382.357389 07-09-t 48.66 48.40 816 7382.357389 7.15E−097-10-s 3.83 3.47 47 7374.665711 07-10-t 29.26 28.99 514 7374.665711 07-11-s 4.64 4.29 58 8279.016611 07-11-t 12.05 11.79 181 8279.016611 1.19E−097-12-s 2.65 2.31 34 7897.645044 07-12-t 26.22 25.95 488 7897.645097 6.82E−0984-t2 2826.02 2824.59 2859 84758.69912 3.66E−0984-t4 1520.48 1519.33 2649 84758.69911 2.42E−0984-t7 162.84 161.51 131 84758.69908 084-t14 325.58 324.54 189 84758.69908 084-s 27165.64 27164.09 17172 84758.70000 0364-t7 1078.88 1073.43 170 368652.9264 1.27E−09

B.1. VALIDATION 145


Problem Time Opt Cyc Obj Gap364t14 1393.87 1389.20 182 368652.9263 0364-t28 2429.51 2424.93 175 368652.9263 0cinca2 0.33 0.16 13 0.000000000 0minsil7 14.03 13.83 167 4477218.834 0mnsil19 86425.01 86423.66 3051 252327000.0 0.2401

navia90 0.69 0.52 36 1057644.000 0tajo6 0.43 0.26 21 694620.0777 0pds001 1.08 0.81 18 29083930523 0pds003 15.20 14.60 114 28597374167 7.64E−082

pds005 111.59 110.69 351 28054052643 8.50E−082

pds009 1421.34 1419.59 1287 26974586263 6.62E−082

pds012 8401.92 8399.92 3800 26103500000 02

pds020 122397.35 122392.77 9432 23821700000 2.10E−051

pds050 43257.77 43210.52 398 17003200000 0.8411

Table B.2: Validation results (part 2).

Problem Props MPiv PPiv MTime PTimedregem 417 411 4017 0.18 1.69dyna 5 0 556 0.02 0.12egd034 52 43 613 0.03 0.12egd085 69 50 1510 0.05 0.24fixmar 138 168 2307 0.21 0.48forest 89 85 2124 0.18 0.41meat12 254 459 2796 0.32 0.73meat31 66 62 1240 0.08 0.21meat43 55 43 916 0.05 0.25mege06 35 11 329 0.03 0.16mege08 51 16 442 0.06 0.22mege10 63 18 599 0.04 0.32mrp3 38 30 537 0.10 0.08mrp5 204 206 7389 0.33 1.15regem 13 10 1265 0.03 0.28scorpion 65 63 208 0.05 0.12ken07 49 24 2113 0.03 0.26scfxm2 35 39 1858 0.08 0.45scfxm3 43 43 2784 0.06 0.61sctap1 833 1233 4478 0.46 0.84sctap2 3989 5604 19016 1.71 5.54sctap3 8456 11752 41051 5.01 18.80small 32 26 2480 0.02 2.13med1 32 29 6176 0.01 5.85med2 32 27 12660 0.03 6.53

1mnsil19 was stopped after 24 hours; pds020 after 34 hours; pds050 after 12 hours.2ε-optimal.



Problem Props MPiv PPiv MTime PTimelarge 32 27 3500 0.02 7.707-01-s 875 1735 8599 1.05 2.747-01-t 598 3461 12062 2.10 3.607-02-s 729 1977 6905 0.91 2.077-02-t 487 2665 11927 1.57 3.127-03-s 796 1909 7381 0.96 2.687-03-t 1120 3698 10917 3.72 5.977-04-s 970 2058 9732 1.04 2.967-04-t 1326 5130 19069 5.20 8.847-05-s 888 1951 9375 0.91 2.817-05-t 1378 4327 17448 4.16 7.437-06-s 1010 2348 11401 0.98 3.297-06-t 3282 8390 38969 9.75 18.827-07-s 1009 2351 12692 1.04 3.317-07-t 6181 16020 96079 21.59 38.347-08-s 1076 2521 12426 1.09 3.317-08-t 4250 13864 54083 15.08 28.677-09-s 891 2335 11346 0.85 2.677-09-t 4926 16552 53169 18.34 30.057-10-s 922 2184 11789 0.87 2.617-10-t 2981 10112 32650 10.60 18.407-11-s 1008 2907 11311 1.16 3.127-11-t 1173 6971 23911 4.17 7.637-12-s 585 1603 6034 0.57 1.757-12-t 2878 10626 25171 9.58 16.3584-t2 81493 325880 1434288 1446.47 1377.9784-t4 41605 114989 1155610 390.68 1128.5584-t7 1489 12557 373305 9.75 151.8084-t14 1090 5237 504633 5.23 319.3584-s 371823 697200 19144341 20204.85 302.54364-t7 6446 88233 1194940 301.87 771.81364t14 3932 39865 1640961 105.10 1284.35364-t28 2130 18002 2191060 40.75 2384.47cinca2 33 46 319 0.02 0.14minsil7 668 5729 43441 9.91 3.92mnsil19 12204 1525693 6805060 85559.51 216.43navia90 144 388 1941 0.24 0.27tajo6 84 189 1267 0.07 0.19pds001 185 282 2092 0.08 0.74pds003 880 2638 40015 1.85 12.76pds005 2439 9130 253761 23.99 86.74pds009 9171 58842 2843449 458.36 961.30pds012 25596 167716 10717289 3490.77 446.30pds020 76685 1350365 108844983 69875.96 4774.42pds050 4313 729150 2161583 20657.12 2050.48

B.2. INTRODUCTION TO FRACTIONAL FACTORIAL DESIGNS 147

B.2 Introduction to Fractional Factorial Designs

We give an short introduction to fractional factorial designs, to explain whatthey are and why they are used, and to introduce the basic concepts andnotation. The 210−3

V that we use in chapter 5 acts as a motivating example.Our explanation is derived from [Mon01], to which we refer the interestedreader for a complete treatment.

Suppose we have ten factors, labelled A–H, J–K, each of which maybe applied at two fixed levels, labelled ‘−’ and ‘+’. Each of the 210 − 1combinations of factors may have an effect, which is defined to be the changein the response produced by a change in the level of the given combination offactors. For instance, the effect BDF is the change in the response producedby an equal change in the three factors B, D, and F . Effects of a singlefactor are called main effects; effects due to combinations of two or morefactors are called interaction effects.

We can portray these effects in a regression model:

y =∑i1

βi1xi1 +∑i1<i2

βi1i2xi1xi2 +∑

i1<i2<i3

βi1i2i3xi1xi2xi3

+ · · ·+ βi1...i10xi1 · · ·xi10 + ε,

where y is the response, the indices i range over the ten factors A–H, J–K, the β’s are parameters whose values are to be determined, and xi is avariable that represents factor i. The variables xi are defined on a codedscale from −1 to +1. The single factor terms βixi represent the main effectof factor of factor i; the product terms represent the interaction effects, so,e.g., βBDFxBxDxF represents the interaction effect BDF .

Suppose we applied each of the 210 combinations of factors and measuredthe response. Each combination, known as a treatment, comprises some ofthe factors at their high levels (+) and the other factors at their low levels(−), and is written using lower case letters for the factors applied at theirhigh levels. For instance, treatment cejk means that factors C, E, J andK were applied at their high levels, with the other six factors at their lowlevels. The treatment with all factors applied at their low level is labelled(1). The complete experiment is called a (full) factorial design.

Given the results of a full factorial design experiment, we can estimatethe main effect of a factor to be the difference between the average responseat the high level of the factor and the average response at the low level ofthe factor. For instance, in a two factor factorial design (22) experiment,four treatments are applied: (1), a, b and ab. The effect of A is estimated as

A =ya + yab

2−y(1) + yb

2.

Observe that the estimate uses the responses of all 22 treatments. The effect


of B can be estimated similarly

B =yb + yab

2−y(1) + ya

2.

Given the estimate of the effect of A, the regression coefficient βA is equalto one half of the value, because the change from low to high of xA in theregression model takes two units on the coded scale, from −1 to +1. Ingeneral, the main effects and regression coefficients of all factors in a 2k

factorial design can be calculated in a similar way, each calculation usingthe responses of all 2k treatments.

A two way interaction effect such as AB can be estimated as the averagedifference between the effect of A at the high level of B and the effect of Aat the low level of B as

AB =(yab − yb)− (ya − y(1))

2.

Note that the RHS can be written

AB =yab − yb − ya + y(1)

2,

showing that the effect is symmetric in A and B, and could equally havebeen derived by considering the effect of B at the two levels of A. Once againthe effect is calculated using all 22 responses, and once again the regressioncoefficient βAB is equal to one half of the value of the effect.

You may notice that the sign of each term in the expression for the effectof AB is the product of the signs for the same terms in the expressions forthe effects of A and B. For instance, the sign of ya in the expression ofA is +1, in the expression for B it is −1, and in the expression for AB itis −1 = (+1) × (−1). This relation holds true in general, and allows usto calculate estimates for the effects and regression coefficients of all higherorder interactions in a general 2k factorial design. Each estimate uses all 2k

treatment responses.Because the estimate for each effect is calculated using all treatment

responses, factorial designs are the most efficient experiments for estimatingthe effects of a number of factors in the minimum number of treatments.Factorial designs also allow us to estimate the effect of all 2k − 1 effects andtest their significance independently, although we do not illustrate it here.

Despite the efficiency of the factorial design, when the number of factorsis large the number of treatments is often more than can be handled in anexperiment. For this reason use is made of fractional factorial designs, inwhich a fraction of the 2k treatments are applied.

Consider the case of three factors A,B and C each at two fixed levels. Afull 23 factorial design would require eight treatments, but suppose we areable to afford only four runs. By running the four treatments a, b, c and

B.2. INTRODUCTION TO FRACTIONAL FACTORIAL DESIGNS 149

abc we are still able to estimate the effects to a certain degree. For instance,the effect of A can be estimated as

A =12

(yabc − yb − yc + ya).

However, the same expression also estimates the effect of BC. We say thetwo effects A and BC are aliases and it is impossible to differentiate betweenthe two effects given the four runs performed. Similarly we can estimate allof the main effects, and we find that B and AC are aliases, and C and ABare aliases. In general there may be many effects that are mutual aliases ofone another, and we call a set of mutual aliases an alias chain.

We call this experimental design the one half fraction of the 23 design,written as 23−1. It allows us to estimate the main effects provided thatwe assume the interaction effects are insignificant. The four treatmentsapplied are those which have a coefficient of +1 in the estimate for the effectABC. Because the average response is always calculated by summing upthe responses of all treatments, the effect ABC is indistinguishable from theaverage response. Representing the average response by I, we say I = ABC,the defining relation for the design. In general, the defining relation for afractional factorial design includes all of the effects that are indistinguishablefrom the average response, and from it we can derive all the alias chains forthe design.

A one half fraction of the 2k design may be constructed by writing downa full factorial design for the 2k−1 design, and adding the kth factor bysetting its low and high levels to be the ‘product’ of the levels for all of thek − 1 factors. For instance, the 23−1 design can be constructed from a full22 design by setting ‘C = AB’ as follows:

Run A B C = AB Treatment1 − − + c2 + − − a3 − + − b4 + + + abc

This table tells us which treatment to apply on each run and is called thedesign matrix. The expression C = AB is the design generator. Note howthe design generator is related to the defining relation; in general, if P = Qis a design generator, then I = PQ will be one of the defining relations.

When the number of factors is larger, we may consider larger fractions.For instance, a 210−3 fractional factorial design requires just one eighth ofthe runs of the full 210 factorial design. The design may be constructedfrom a full 27 factorial design and three design generators which define theremaining three factors H, J and K in terms of the seven factors in the 27

design. For instance, the three design generators H = ABEF, J = ACEGand K = ADFG give rise to the design matrix shown in section B.3.3.


The important attribute of this particular 210−3 fractional factorial de-sign over other 210−3 designs (i.e., given by different generators) is that thisdesign has the highest possible resolution, meaning that the maximum num-ber of lower order effects are not aliased with each other. This particulardesign is of resolution V (roman 5) which means main effects are aliasedwith four-factor effects (and higher), and two-factor effects are aliased withthree-factor effects (and higher), but no main or two-factor effect is aliasedwith any other main or two-factor effect. Thus if we assumed all three-factorand higher effects were negligible, we would estimate all main and two-factoreffects. In practice we must use our discretion to distinguish between twoand three-factor effects that are aliases of each other. Each effect is an aliasof seven other effects, and we can write down the alias chains of eight effectsthat are mutual aliases (see section B.3.4).

B.3. THE 210−3V FRACTIONAL FACTORIAL DESIGN 151

B.3 The 210−3V Fractional Factorial Design

B.3.1 Design Generators

H = ABEF

J = ACEG

K = ADFG

B.3.2 Defining Relation

I = ABEFH = ACEGJ = ADFGK = BCFGHJ = BDEGHK =CDEFJK = ABCDHJK.

B.3.3 Design Matrix

Table B.3: Design matrix.

A B C D E F G H J K Treatment1 − − − − − − − + + + hjk2 + − − − − − − − − − a3 − + − − − − − − + + bjk4 + + − − − − − + − − abh5 − − + − − − − + − + chk6 + − + − − − − − + − acj7 − + + − − − − − − + bck8 + + + − − − − + + − abchj9 − − − + − − − + + − dhj

10 + − − + − − − − − + adk11 − + − + − − − − + − bdj12 + + − + − − − + − + abdhk13 − − + + − − − + − − cdh14 + − + + − − − − + + acdjk15 − + + + − − − − − − bcd16 + + + + − − − + + + abcdhjk17 − − − − + − − − − + ek18 + − − − + − − + + − aehj19 − + − − + − − + − + behk20 + + − − + − − − + − abej21 − − + − + − − − + + cejk22 + − + − + − − + − − aceh23 − + + − + − − + + + bcehjk24 + + + − + − − − − − abce25 − − − + + − − − − − de26 + − − + + − − + + + adehjk27 − + − + + − − + − − bdeh28 + + − + + − − − + + abdejk29 − − + + + − − − + − cdej


Table B.3: Design matrix (continued).

A B C D E F G H J K Treatment30 + − + + + − − + − + acdehk31 − + + + + − − + + − bcdehj32 + + + + + − − − − + abcdek33 − − − − − + − − + − fj34 + − − − − + − + − + afhk35 − + − − − + − + + − bfhj36 + + − − − + − − − + abfk37 − − + − − + − − − − cf38 + − + − − + − + + + acfhjk39 − + + − − + − + − − bcfh40 + + + − − + − − + + abcfjk41 − − − + − + − − + + dfjk42 + − − + − + − + − − adfh43 − + − + − + − + + + bdfhjk44 + + − + − + − − − − abdf45 − − + + − + − − − + cdfk46 + − + + − + − + + − acdfhj47 − + + + − + − + − + bcdfhk48 + + + + − + − − + − abcdfj49 − − − − + + − + − − efh50 + − − − + + − − + + aefjk51 − + − − + + − − − − bef52 + + − − + + − + + + abefhjk53 − − + − + + − + + − cefhj54 + − + − + + − − − + acefk55 − + + − + + − − + − bcefj56 + + + − + + − + − + abcefhk57 − − − + + + − + − + defhk58 + − − + + + − − + − adefj59 − + − + + + − − − + bdefk60 + + − + + + − + + − abdefhj61 − − + + + + − + + + cdefhjk62 + − + + + + − − − − acdef63 − + + + + + − − + + bcdefjk64 + + + + + + − + − − abcdefh65 − − − − − − + + − − gh66 + − − − − − + − + + agjk67 − + − − − − + − − − bg68 + + − − − − + + + + abghjk69 − − + − − − + + + − cghj70 + − + − − − + − − + acgk71 − + + − − − + − + − bcgj72 + + + − − − + + − + abcghk73 − − − + − − + + − + dghk74 + − − + − − + − + − adgj75 − + − + − − + − − + bdgk76 + + − + − − + + + − abdghj



A B C D E F G H J K Treatment77 − − + + − − + + + + cdghjk78 + − + + − − + − − − acdg79 − + + + − − + − + + bcdgjk80 + + + + − − + + − − abcdgh81 − − − − + − + − + − egj82 + − − − + − + + − + aeghk83 − + − − + − + + + − beghj84 + + − − + − + − − + abegk85 − − + − + − + − − − ceg86 + − + − + − + + + + aceghjk87 − + + − + − + + − − bcegh88 + + + − + − + − + + abcegjk89 − − − + + − + − + + degjk90 + − − + + − + + − − adegh91 − + − + + − + + + + bdeghjk92 + + − + + − + − − − abdeg93 − − + + + − + − − + cdegk94 + − + + + − + + + − acdeghj95 − + + + + − + + − + bcdeghk96 + + + + + − + − + − abcdegj97 − − − − − + + − − + fgk98 + − − − − + + + + − afghj99 − + − − − + + + − + bfghk

100 + + − − − + + − + − abfgj101 − − + − − + + − + + cfgjk102 + − + − − + + + − − acfgh103 − + + − − + + + + + bcfghjk104 + + + − − + + − − − abcfg105 − − − + − + + − − − dfg106 + − − + − + + + + + adfghjk107 − + − + − + + + − − bdfgh108 + + − + − + + − + + abdfgjk109 − − + + − + + − + − cdfgj110 + − + + − + + + − + acdfghk111 − + + + − + + + + − bcdfghj112 + + + + − + + − − + abcdfgk113 − − − − + + + + + + efghjk114 + − − − + + + − − − aefg115 − + − − + + + − + + befgjk116 + + − − + + + + − − abefgh117 − − + − + + + + − + cefghk118 + − + − + + + − + − acefgj119 − + + − + + + − − + bcefgk120 + + + − + + + + + − abcefghj121 − − − + + + + + + − defghj122 + − − + + + + − − + adefgk123 − + − + + + + − + − bdefgj



A B C D E F G H J K Treatment124 + + − + + + + + − + abdefghk125 − − + + + + + + − − cdefgh126 + − + + + + + − + + acdefgjk127 − + + + + + + − − − bcdefg128 + + + + + + + + + + abcdefghjk

B.3.4 Alias Structure

I = ABEFH = ACEGJ = ADFGK = BCFGHJ = BDEGHK = CDEFJK = ABCDHJKA = BEFH = CEGJ = DFGK = ABCFGHJ = ABDEGHK = ACDEFJK = BCDHJKB = AEFH = ABCEGJ = ABDFGK = CFGHJ = DEGHK = BCDEFJK = ACDHJKAB = EFH = BCEGJ = BDFGK = ACFGHJ = ADEGHK = ABCDEFJK = CDHJKC = ABCEFH = AEGJ = ACDFGK = BFGHJ = BCDEGHK = DEFJK = ABDHJKAC = BCEFH = EGJ = CDFGK = ABFGHJ = ABCDEGHK = ADEFJK = BDHJKBC = ACEFH = ABEGJ = ABCDFGK = FGHJ = CDEGHK = BDEFJK = ADHJKABC = CEFH = BEGJ = BCDFGK = AFGHJ = ACDEGHK = ABDEFJK = DHJKD = ABDEFH = ACDEGJ = AFGK = BCDFGHJ = BEGHK = CEFJK = ABCHJKAD = BDEFH = CDEGJ = FGK = ABCDFGHJ = ABEGHK = ACEFJK = BCHJKBD = ADEFH = ABCDEGJ = ABFGK = CDFGHJ = EGHK = BCEFJK = ACHJKABD = DEFH = BCDEGJ = BFGK = ACDFGHJ = AEGHK = ABCEFJK = CHJKCD = ABCDEFH = ADEGJ = ACFGK = BDFGHJ = BCEGHK = EFJK = ABHJKACD = BCDEFH = DEGJ = CFGK = ABDFGHJ = ABCEGHK = AEFJK = BHJKBCD = ACDEFH = ABDEGJ = ABCFGK = DFGHJ = CEGHK = BEFJK = AHJKABCD = CDEFH = BDEGJ = BCFGK = ADFGHJ = ACEGHK = ABEFJK = HJKE = ABFH = ACGJ = ADEFGK = BCEFGHJ = BDGHK = CDFJK = ABCDEHJKAE = BFH = CGJ = DEFGK = ABCEFGHJ = ABDGHK = ACDFJK = BCDEHJKBE = AFH = ABCGJ = ABDEFGK = CEFGHJ = DGHK = BCDFJK = ACDEHJKABE = FH = BCGJ = BDEFGK = ACEFGHJ = ADGHK = ABCDFJK = CDEHJKCE = ABCFH = AGJ = ACDEFGK = BEFGHJ = BCDGHK = DFJK = ABDEHJKACE = BCFH = GJ = CDEFGK = ABEFGHJ = ABCDGHK = ADFJK = BDEHJKBCE = ACFH = ABGJ = ABCDEFGK = EFGHJ = CDGHK = BDFJK = ADEHJKABCE = CFH = BGJ = BCDEFGK = AEFGHJ = ACDGHK = ABDFJK = DEHJKDE = ABDFH = ACDGJ = AEFGK = BCDEFGHJ = BGHK = CFJK = ABCEHJKADE = BDFH = CDGJ = EFGK = ABCDEFGHJ = ABGHK = ACFJK = BCEHJKBDE = ADFH = ABCDGJ = ABEFGK = CDEFGHJ = GHK = BCFJK = ACEHJKABDE = DFH = BCDGJ = BEFGK = ACDEFGHJ = AGHK = ABCFJK = CEHJKCDE = ABCDFH = ADGJ = ACEFGK = BDEFGHJ = BCGHK = FJK = ABEHJKACDE = BCDFH = DGJ = CEFGK = ABDEFGHJ = ABCGHK = AFJK = BEHJKBCDE = ACDFH = ABDGJ = ABCEFGK = DEFGHJ = CGHK = BFJK = AEHJKABCDE = CDFH = BDGJ = BCEFGK = ADEFGHJ = ACGHK = ABFJK = EHJKF = ABEH = ACEFGJ = ADGK = BCGHJ = BDEFGHK = CDEJK = ABCDFHJKAF = BEH = CEFGJ = DGK = ABCGHJ = ABDEFGHK = ACDEJK = BCDFHJKBF = AEH = ABCEFGJ = ABDGK = CGHJ = DEFGHK = BCDEJK = ACDFHJKABF = EH = BCEFGJ = BDGK = ACGHJ = ADEFGHK = ABCDEJK = CDFHJKCF = ABCEH = AEFGJ = ACDGK = BGHJ = BCDEFGHK = DEJK = ABDFHJKACF = BCEH = EFGJ = CDGK = ABGHJ = ABCDEFGHK = ADEJK = BDFHJKBCF = ACEH = ABEFGJ = ABCDGK = GHJ = CDEFGHK = BDEJK = ADFHJKABCF = CEH = BEFGJ = BCDGK = AGHJ = ACDEFGHK = ABDEJK = DFHJKDF = ABDEH = ACDEFGJ = AGK = BCDGHJ = BEFGHK = CEJK = ABCFHJKADF = BDEH = CDEFGJ = GK = ABCDGHJ = ABEFGHK = ACEJK = BCFHJKBDF = ADEH = ABCDEFGJ = ABGK = CDGHJ = EFGHK = BCEJK = ACFHJKABDF = DEH = BCDEFGJ = BGK = ACDGHJ = AEFGHK = ABCEJK = CFHJKCDF = ABCDEH = ADEFGJ = ACGK = BDGHJ = BCEFGHK = EJK = ABFHJKACDF = BCDEH = DEFGJ = CGK = ABDGHJ = ABCEFGHK = AEJK = BFHJKBCDF = ACDEH = ABDEFGJ = ABCGK = DGHJ = CEFGHK = BEJK = AFHJKABCDF = CDEH = BDEFGJ = BCGK = ADGHJ = ACEFGHK = ABEJK = FHJKEF = ABH = ACFGJ = ADEGK = BCEGHJ = BDFGHK = CDJK = ABCDEFHJK


AEF = BH = CFGJ = DEGK = ABCEGHJ = ABDFGHK = ACDJK = BCDEFHJKBEF = AH = ABCFGJ = ABDEGK = CEGHJ = DFGHK = BCDJK = ACDEFHJKABEF = H = BCFGJ = BDEGK = ACEGHJ = ADFGHK = ABCDJK = CDEFHJKCEF = ABCH = AFGJ = ACDEGK = BEGHJ = BCDFGHK = DJK = ABDEFHJKACEF = BCH = FGJ = CDEGK = ABEGHJ = ABCDFGHK = ADJK = BDEFHJKBCEF = ACH = ABFGJ = ABCDEGK = EGHJ = CDFGHK = BDJK = ADEFHJKABCEF = CH = BFGJ = BCDEGK = AEGHJ = ACDFGHK = ABDJK = DEFHJKDEF = ABDH = ACDFGJ = AEGK = BCDEGHJ = BFGHK = CJK = ABCEFHJKADEF = BDH = CDFGJ = EGK = ABCDEGHJ = ABFGHK = ACJK = BCEFHJKBDEF = ADH = ABCDFGJ = ABEGK = CDEGHJ = FGHK = BCJK = ACEFHJKABDEF = DH = BCDFGJ = BEGK = ACDEGHJ = AFGHK = ABCJK = CEFHJKCDEF = ABCDH = ADFGJ = ACEGK = BDEGHJ = BCFGHK = JK = ABEFHJKACDEF = BCDH = DFGJ = CEGK = ABDEGHJ = ABCFGHK = AJK = BEFHJKBCDEF = ACDH = ABDFGJ = ABCEGK = DEGHJ = CFGHK = BJK = AEFHJKABCDEF = CDH = BDFGJ = BCEGK = ADEGHJ = ACFGHK = ABJK = EFHJKG = ABEFGH = ACEJ = ADFK = BCFHJ = BDEHK = CDEFGJK = ABCDGHJKAG = BEFGH = CEJ = DFK = ABCFHJ = ABDEHK = ACDEFGJK = BCDGHJKBG = AEFGH = ABCEJ = ABDFK = CFHJ = DEHK = BCDEFGJK = ACDGHJKABG = EFGH = BCEJ = BDFK = ACFHJ = ADEHK = ABCDEFGJK = CDGHJKCG = ABCEFGH = AEJ = ACDFK = BFHJ = BCDEHK = DEFGJK = ABDGHJKACG = BCEFGH = EJ = CDFK = ABFHJ = ABCDEHK = ADEFGJK = BDGHJKBCG = ACEFGH = ABEJ = ABCDFK = FHJ = CDEHK = BDEFGJK = ADGHJKABCG = CEFGH = BEJ = BCDFK = AFHJ = ACDEHK = ABDEFGJK = DGHJKDG = ABDEFGH = ACDEJ = AFK = BCDFHJ = BEHK = CEFGJK = ABCGHJKADG = BDEFGH = CDEJ = FK = ABCDFHJ = ABEHK = ACEFGJK = BCGHJKBDG = ADEFGH = ABCDEJ = ABFK = CDFHJ = EHK = BCEFGJK = ACGHJKABDG = DEFGH = BCDEJ = BFK = ACDFHJ = AEHK = ABCEFGJK = CGHJKCDG = ABCDEFGH = ADEJ = ACFK = BDFHJ = BCEHK = EFGJK = ABGHJKACDG = BCDEFGH = DEJ = CFK = ABDFHJ = ABCEHK = AEFGJK = BGHJKBCDG = ACDEFGH = ABDEJ = ABCFK = DFHJ = CEHK = BEFGJK = AGHJKABCDG = CDEFGH = BDEJ = BCFK = ADFHJ = ACEHK = ABEFGJK = GHJKEG = ABFGH = ACJ = ADEFK = BCEFHJ = BDHK = CDFGJK = ABCDEGHJKAEG = BFGH = CJ = DEFK = ABCEFHJ = ABDHK = ACDFGJK = BCDEGHJKBEG = AFGH = ABCJ = ABDEFK = CEFHJ = DHK = BCDFGJK = ACDEGHJKABEG = FGH = BCJ = BDEFK = ACEFHJ = ADHK = ABCDFGJK = CDEGHJKCEG = ABCFGH = AJ = ACDEFK = BEFHJ = BCDHK = DFGJK = ABDEGHJKACEG = BCFGH = J = CDEFK = ABEFHJ = ABCDHK = ADFGJK = BDEGHJKBCEG = ACFGH = ABJ = ABCDEFK = EFHJ = CDHK = BDFGJK = ADEGHJKABCEG = CFGH = BJ = BCDEFK = AEFHJ = ACDHK = ABDFGJK = DEGHJKDEG = ABDFGH = ACDJ = AEFK = BCDEFHJ = BHK = CFGJK = ABCEGHJKADEG = BDFGH = CDJ = EFK = ABCDEFHJ = ABHK = ACFGJK = BCEGHJKBDEG = ADFGH = ABCDJ = ABEFK = CDEFHJ = HK = BCFGJK = ACEGHJKABDEG = DFGH = BCDJ = BEFK = ACDEFHJ = AHK = ABCFGJK = CEGHJKCDEG = ABCDFGH = ADJ = ACEFK = BDEFHJ = BCHK = FGJK = ABEGHJKACDEG = BCDFGH = DJ = CEFK = ABDEFHJ = ABCHK = AFGJK = BEGHJKBCDEG = ACDFGH = ABDJ = ABCEFK = DEFHJ = CHK = BFGJK = AEGHJKABCDEG = CDFGH = BDJ = BCEFK = ADEFHJ = ACHK = ABFGJK = EGHJKFG = ABEGH = ACEFJ = ADK = BCHJ = BDEFHK = CDEGJK = ABCDFGHJKAFG = BEGH = CEFJ = DK = ABCHJ = ABDEFHK = ACDEGJK = BCDFGHJKBFG = AEGH = ABCEFJ = ABDK = CHJ = DEFHK = BCDEGJK = ACDFGHJKABFG = EGH = BCEFJ = BDK = ACHJ = ADEFHK = ABCDEGJK = CDFGHJKCFG = ABCEGH = AEFJ = ACDK = BHJ = BCDEFHK = DEGJK = ABDFGHJKACFG = BCEGH = EFJ = CDK = ABHJ = ABCDEFHK = ADEGJK = BDFGHJKBCFG = ACEGH = ABEFJ = ABCDK = HJ = CDEFHK = BDEGJK = ADFGHJKABCFG = CEGH = BEFJ = BCDK = AHJ = ACDEFHK = ABDEGJK = DFGHJKDFG = ABDEGH = ACDEFJ = AK = BCDHJ = BEFHK = CEGJK = ABCFGHJKADFG = BDEGH = CDEFJ = K = ABCDHJ = ABEFHK = ACEGJK = BCFGHJKBDFG = ADEGH = ABCDEFJ = ABK = CDHJ = EFHK = BCEGJK = ACFGHJKABDFG = DEGH = BCDEFJ = BK = ACDHJ = AEFHK = ABCEGJK = CFGHJKCDFG = ABCDEGH = ADEFJ = ACK = BDHJ = BCEFHK = EGJK = ABFGHJKACDFG = BCDEGH = DEFJ = CK = ABDHJ = ABCEFHK = AEGJK = BFGHJKBCDFG = ACDEGH = ABDEFJ = ABCK = DHJ = CEFHK = BEGJK = AFGHJK


ABCDFG = CDEGH = BDEFJ = BCK = ADHJ = ACEFHK = ABEGJK = FGHJKEFG = ABGH = ACFJ = ADEK = BCEHJ = BDFHK = CDGJK = ABCDEFGHJKAEFG = BGH = CFJ = DEK = ABCEHJ = ABDFHK = ACDGJK = BCDEFGHJKBEFG = AGH = ABCFJ = ABDEK = CEHJ = DFHK = BCDGJK = ACDEFGHJKABEFG = GH = BCFJ = BDEK = ACEHJ = ADFHK = ABCDGJK = CDEFGHJKCEFG = ABCGH = AFJ = ACDEK = BEHJ = BCDFHK = DGJK = ABDEFGHJKACEFG = BCGH = FJ = CDEK = ABEHJ = ABCDFHK = ADGJK = BDEFGHJKBCEFG = ACGH = ABFJ = ABCDEK = EHJ = CDFHK = BDGJK = ADEFGHJKABCEFG = CGH = BFJ = BCDEK = AEHJ = ACDFHK = ABDGJK = DEFGHJKDEFG = ABDGH = ACDFJ = AEK = BCDEHJ = BFHK = CGJK = ABCEFGHJKADEFG = BDGH = CDFJ = EK = ABCDEHJ = ABFHK = ACGJK = BCEFGHJKBDEFG = ADGH = ABCDFJ = ABEK = CDEHJ = FHK = BCGJK = ACEFGHJKABDEFG = DGH = BCDFJ = BEK = ACDEHJ = AFHK = ABCGJK = CEFGHJKCDEFG = ABCDGH = ADFJ = ACEK = BDEHJ = BCFHK = GJK = ABEFGHJKACDEFG = BCDGH = DFJ = CEK = ABDEHJ = ABCFHK = AGJK = BEFGHJKBCDEFG = ACDGH = ABDFJ = ABCEK = DEHJ = CFHK = BGJK = AEFGHJKABCDEFG = CDGH = BDFJ = BCEK = ADEHJ = ACFHK = ABGJK = EFGHJK

B.4. EVALUATION OF COMPUTATIONAL STRATEGIES 157

B.4 Evaluation of Computational Strategies

This section gives the results of second experiment, described in section 5.5(p. 94), which was to estimate the effect of ten key controls (or factors) inDecOpt. 1280 different treatments were applied to each of ten problems,solved using serial DecOpt on a Windows PC (platform 1 p. 90).

B.4.1 Experimental Results

Table B.4: Evaluation of computational strategies. The table shows the elapsed optimisation timeto solve each problem under each treatment.

Treatment meat12 mrp5 sctap2 large 7-10-s 7-10-t 84-t7 84-t14 minsil7 pds003 Totalhjk 1.12 2.15 10.40 35.22 4.47 37.21 524.02 412.04 19.53 17.66 1063.82a 0.69 1.39 5.85 7.40 2.50 10.67 134.60 244.83 8.65 6.30 422.88bjk 0.83 1.12 5.67 8.56 2.87 17.56 145.25 233.35 9.92 6.42 431.55abh 1.09 2.44 11.85 18.67 4.83 46.11 669.50 1501.21 22.26 16.25 2294.21chk 1.07 2.15 11.79 20.11 3.71 38.99 669.11 1223.65 14.97 16.87 2002.42acj 0.74 1.31 5.16 7.39 2.79 14.21 130.62 234.21 10.66 7.17 414.26bck 0.78 1.22 5.82 7.95 2.62 16.40 140.99 232.51 8.45 8.65 425.39abchj 1.15 2.19 13.85 34.47 4.59 40.03 761.25 1445.69 20.46 16.21 2339.89dhj 0.95 2.81 18.01 33.40 4.91 33.10 725.33 1526.75 25.36 14.23 2384.85adk 0.77 1.42 5.98 7.30 3.23 11.48 164.59 236.46 11.22 6.96 449.41bdj 0.81 1.56 5.11 8.56 3.28 18.49 154.89 274.73 10.27 10.18 487.88abdhk 0.95 2.62 27.09 31.97 5.03 38.78 665.78 625.06 24.36 15.00 1436.64cdh 1.04 2.77 12.70 18.17 4.72 38.77 781.31 1411.58 23.60 15.74 2310.40acdjk 0.73 1.40 6.18 7.30 2.85 12.41 152.86 283.74 10.96 7.61 486.04bcd 0.87 1.43 6.18 7.98 2.99 18.15 159.88 261.92 9.48 7.77 476.65abcdhjk 1.06 2.69 24.00 33.86 4.38 26.84 557.45 416.67 29.27 13.85 1110.07ek 0.61 1.24 5.75 8.58 2.55 11.19 127.56 211.25 8.96 6.51 384.20aehj 0.96 2.62 12.56 35.83 4.19 22.29 316.92 532.98 24.37 13.62 966.34behk 1.33 3.11 10.92 18.66 5.12 30.54 360.24 651.88 28.30 14.18 1124.28abej 0.73 1.61 6.49 7.33 2.88 13.60 149.51 256.49 10.30 7.78 456.72cejk 0.70 1.28 5.50 7.96 2.75 13.47 135.40 234.95 9.02 6.96 417.99aceh 1.14 2.72 11.21 18.71 4.45 20.49 296.61 473.76 22.89 13.78 865.76bcehjk 1.31 2.32 9.71 35.73 5.32 22.60 313.04 786.66 28.46 15.70 1220.85abce 0.68 1.55 6.57 7.37 2.74 13.50 140.58 248.25 9.79 6.98 438.01de 0.74 1.57 7.33 8.58 3.44 13.42 163.32 240.76 12.37 9.64 461.17adehjk 1.09 3.27 23.24 36.00 5.37 21.83 269.52 384.22 28.97 12.10 785.61bdeh 1.42 3.24 23.45 19.96 7.61 22.39 369.26 688.96 23.34 14.93 1174.56abdejk 0.71 1.77 6.21 7.39 3.40 13.78 175.08 237.76 14.13 7.17 467.40cdej 0.77 1.45 7.22 8.07 3.11 14.14 168.70 270.05 13.47 8.73 495.71acdehk 1.08 3.15 22.27 18.39 5.09 23.72 416.88 494.04 25.97 13.72 1024.31bcdehj 1.39 3.24 22.33 34.45 5.39 28.05 329.63 628.37 26.77 13.86 1093.48abcdek 0.71 1.50 9.31 7.30 2.84 14.72 188.88 293.28 12.35 6.49 537.38fj 0.69 1.37 4.89 8.57 3.31 12.16 143.87 225.97 9.51 6.30 416.64afhk 1.00 3.06 10.43 18.56 4.70 26.57 290.09 436.67 19.16 17.64 827.88bfhj 1.57 2.44 10.46 33.80 5.55 31.03 341.35 700.03 25.20 14.96 1166.39abfk 0.74 1.35 5.69 7.41 2.74 13.17 148.83 217.54 10.95 6.62 415.04cf 0.70 1.34 4.91 7.95 2.70 11.29 141.86 225.68 10.03 6.94 413.40acfhjk 1.04 2.69 13.25 33.85 4.42 30.35 277.51 461.25 26.89 14.27 865.52bcfh 1.29 2.75 14.07 18.14 5.41 34.27 401.00 853.74 32.73 14.78 1378.18abcfjk 0.75 1.41 5.96 7.29 2.92 14.08 143.45 223.66 12.10 6.19 417.81dfjk 0.86 1.45 6.70 8.67 3.36 13.04 176.07 248.40 14.19 7.61 480.35adfh 1.11 3.89 16.40 19.55 6.28 24.89 318.87 469.73 23.55 12.40 896.67bdfhjk 1.36 3.00 24.28 35.04 7.89 30.54 378.13 396.94 23.23 12.36 912.77abdf 0.79 1.85 6.43 7.50 3.97 14.10 182.05 285.88 11.49 8.98 523.04cdfk 0.72 1.51 5.71 7.93 3.59 14.44 185.88 228.65 11.41 7.03 466.87acdfhj 1.08 3.53 19.71 34.47 5.14 25.08 275.75 457.95 32.04 14.05 868.80


Table B.4: Evaluation of computational strategies (continued).

Treatment meat12 mrp5 sctap2 large 7-10-s 7-10-t 84-t7 84-t14 minsil7 pds003 Totalbcdfhk 1.18 3.36 24.96 20.06 7.21 30.03 387.31 493.34 26.40 13.05 1006.90abcdfj 0.77 1.79 5.16 7.31 3.61 15.53 195.51 248.61 14.76 6.66 499.71efh 1.06 2.71 10.56 18.64 4.85 25.71 303.27 362.06 16.58 12.15 757.59aefjk 0.67 1.48 5.57 7.31 3.22 13.04 163.35 224.38 11.36 5.61 435.99bef 0.78 1.35 5.35 8.58 3.30 14.73 156.35 221.60 11.03 6.83 429.90abefhjk 1.17 3.02 11.38 36.41 4.60 25.77 323.59 329.04 23.76 13.91 772.65cefhj 1.14 2.62 13.08 34.44 4.33 19.78 309.74 353.31 25.75 13.90 778.09acefk 0.60 1.39 52.60 7.42 3.08 13.24 148.63 216.51 12.29 5.89 461.65bcefj 0.74 1.42 5.17 7.96 3.39 13.94 149.98 266.77 10.75 7.29 467.41abcefhk 1.19 3.02 11.95 18.62 4.07 21.87 305.96 586.74 29.29 14.22 996.93defhk 1.07 3.56 24.09 19.94 6.53 25.24 350.00 405.31 29.69 12.64 878.07adefj 0.84 2.24 12.88 7.40 3.44 18.47 195.01 226.69 19.04 6.65 492.66bdefk 0.79 1.76 7.08 8.56 3.44 16.88 207.37 259.73 10.59 9.39 525.59abdefhj 1.50 4.27 30.98 35.64 6.57 26.71 372.59 616.66 32.19 11.75 1138.86cdefhjk 1.13 3.17 23.64 36.17 5.61 22.76 309.12 168.17 30.07 14.22 614.06acdef 0.65 2.16 25.33 7.29 3.45 14.22 194.89 247.59 13.68 8.46 517.72bcdefjk 0.72 1.64 7.40 7.93 4.27 16.61 198.06 253.48 11.99 8.30 510.40abcdefh 1.32 3.92 29.25 18.74 6.02 29.26 342.85 620.00 27.86 15.76 1094.98gh 1.03 2.34 12.18 18.33 4.11 23.54 410.04 591.04 20.24 15.74 1098.59agjk 0.71 1.27 4.96 7.38 2.95 13.55 128.16 228.82 9.16 7.06 404.02bg 0.81 1.20 5.26 8.65 2.95 12.65 148.81 265.92 11.59 7.92 465.76abghjk 1.39 2.42 14.68 36.45 4.49 28.70 418.91 502.16 25.28 17.75 1052.23cghj 0.99 2.63 9.44 34.83 4.34 24.28 402.49 521.58 26.95 16.54 1044.07acgk 0.63 1.35 5.17 7.38 2.80 11.03 152.11 217.99 10.35 6.70 415.51bcgj 0.89 1.28 5.66 7.97 3.05 15.28 151.33 266.44 11.11 8.39 471.40abcghk 1.33 2.40 14.75 18.63 4.04 30.93 406.51 424.63 22.17 16.79 942.18dghk 0.86 3.43 18.87 19.75 5.27 23.87 383.93 541.94 25.63 13.73 1037.28adgj 0.88 1.68 5.53 7.30 3.31 12.37 156.49 242.11 11.94 8.14 449.75bdgk 0.72 1.33 5.92 8.59 3.21 15.41 162.52 233.11 10.10 9.77 450.68abdghj 1.13 3.28 29.72 35.66 5.51 28.80 273.52 365.72 35.22 13.60 792.16cdghjk 1.13 2.73 19.42 35.90 5.11 20.24 269.11 383.81 28.63 15.42 781.50acdg 0.63 1.66 4.94 7.41 3.68 14.03 146.89 280.03 11.08 7.67 478.02bcdgjk 0.75 1.35 5.44 8.07 3.64 13.27 185.40 243.48 11.22 8.66 481.28abcdgh 1.11 2.76 28.43 18.49 4.85 33.59 288.16 355.84 25.33 18.38 776.94egj 0.72 1.25 5.35 8.56 2.57 13.03 145.25 221.15 10.73 7.26 415.87aeghk 1.01 2.83 10.42 18.61 4.71 20.33 225.66 345.76 23.05 12.39 664.77beghj 1.48 2.71 13.15 33.80 4.14 22.87 259.04 355.93 30.24 14.48 737.84abegk 0.68 1.48 5.89 7.39 2.77 13.01 144.17 242.13 12.16 6.55 436.23ceg 0.76 1.21 5.66 7.95 2.78 13.05 131.69 239.90 9.83 6.94 419.77aceghjk 1.13 2.73 10.98 34.26 4.94 20.04 240.27 378.87 24.56 14.37 732.15bcegh 1.42 2.37 24.39 18.25 5.50 23.32 274.03 365.65 25.92 16.49 757.34abcegjk 0.85 1.36 6.29 7.38 2.80 15.11 158.94 247.08 10.79 6.57 457.17degjk 0.82 1.35 6.91 8.56 3.18 17.59 167.23 364.82 13.78 7.40 591.64adegh 1.12 3.62 22.94 19.81 7.15 32.01 329.69 351.71 27.87 13.44 809.36bdeghjk 1.43 3.10 26.95 34.56 6.70 30.91 320.01 381.21 27.59 12.76 845.22abdeg 0.82 1.68 13.27 7.31 4.19 16.88 171.44 250.31 13.03 9.64 488.57cdegk 0.72 1.67 5.71 7.92 3.48 12.76 195.07 255.00 12.05 8.16 502.54acdeghj 1.46 3.89 22.27 34.05 5.36 26.16 259.73 315.83 31.56 13.08 713.39bcdeghk 1.10 3.21 26.17 20.35 6.13 31.67 324.02 341.87 27.32 13.36 795.20abcdegj 0.82 1.72 8.31 7.30 3.46 15.53 170.44 280.53 13.96 6.76 508.83fgk 0.61 1.34 5.21 8.58 3.13 13.80 155.68 230.82 9.97 6.95 436.09afghj 1.17 2.96 11.02 36.35 5.65 20.75 261.49 356.12 25.44 13.44 734.39bfghk 1.50 2.95 11.39 18.47 6.07 25.93 279.03 348.82 27.14 14.91 736.21abfgj 0.75 1.59 4.82 7.30 3.53 15.64 157.78 243.47 10.49 6.75 452.12cfgjk 0.73 1.34 5.04 7.90 3.05 14.06 136.51 218.10 9.38 5.79 401.90acfgh 1.00 2.83 14.98 18.72 4.65 21.34 257.12 349.23 24.13 12.30 706.30bcfghjk 1.48 2.74 11.71 35.72 6.09 41.65 277.75 354.78 24.87 17.12 773.91abcfg 0.73 1.51 6.03 7.30 3.49 14.41 163.48 253.15 11.40 6.41 467.91dfg 0.74 1.65 6.04 8.55 3.70 16.99 212.65 260.92 13.19 9.27 533.70adfghjk 1.24 3.26 23.68 35.54 7.42 25.04 309.47 86.36 36.26 13.98 542.25


Table B.4: Evaluation of computational strategies (continued).

Treatment meat12 mrp5 sctap2 large 7-10-s 7-10-t 84-t7 84-t14 minsil7 pds003 Totalbdfgh 1.39 4.35 28.33 19.97 7.37 33.02 271.23 389.67 32.03 15.38 802.74abdfgjk 0.78 1.91 6.05 7.30 4.12 16.71 213.42 251.80 15.62 7.01 524.72cdfgj 0.77 1.65 5.82 7.99 3.42 16.74 178.38 246.51 15.05 7.85 484.18acdfghk 1.06 3.69 24.56 18.38 5.74 22.07 346.49 317.10 28.93 11.73 779.75bcdfghj 1.27 4.22 25.58 34.91 7.61 34.77 291.52 342.08 30.87 14.76 787.59abcdfgk 0.77 1.94 5.96 7.31 3.23 16.45 190.66 243.96 25.46 6.16 501.90efghjk 1.09 2.59 11.46 35.64 5.80 24.77 292.65 332.81 22.43 14.38 743.62aefg 0.71 1.72 24.11 7.59 3.07 13.87 166.05 216.72 11.57 6.22 451.63befgjk 0.83 1.57 5.10 8.57 3.16 15.33 153.78 238.02 10.76 6.01 443.13abefgh 1.16 3.09 15.94 18.87 4.23 24.15 299.30 324.87 30.10 15.66 737.37cefghk 1.11 2.31 10.10 20.04 4.96 23.29 291.37 304.96 27.49 14.49 700.12acefgj 0.76 1.57 5.95 7.30 3.03 11.91 152.69 203.66 11.76 5.78 404.41bcefgk 0.79 1.27 6.49 7.96 2.99 14.71 156.16 238.89 11.29 7.00 447.55abcefghj 1.20 3.52 13.82 34.13 6.43 33.30 314.67 347.80 32.42 15.82 803.11defghj 1.23 3.74 18.07 33.84 7.18 32.69 316.07 342.66 34.59 14.15 804.22adefgk 0.74 1.92 14.78 7.31 3.75 16.67 189.81 265.45 14.03 6.45 520.91bdefgj 0.83 1.89 6.40 8.67 4.01 19.08 198.60 262.46 12.63 8.79 523.36abdefghk 1.33 4.18 32.97 19.85 6.33 36.87 375.27 322.62 29.19 13.09 841.70cdefgh 1.15 4.01 24.54 18.11 5.42 28.92 327.43 300.91 27.47 14.94 752.90acdefgjk 0.82 1.69 8.18 7.48 3.56 16.72 205.00 369.29 17.04 6.55 636.33bcdefg 0.78 1.78 7.08 8.09 3.64 17.41 199.69 254.74 14.24 6.87 514.32abcdefghjk 1.07 4.19 31.36 33.88 5.59 32.20 403.80 353.29 37.15 13.43 915.96Total 123.96 293.09 1658.63 2240.24 553.58 2734.80 33,822.92 48,504.55 2473.27 1397.18 93,802.21

Thirteen runs failed: five treatments for problem 84-t7 (bcehjk, adehjk,acfhjk, cdghjk, bdfgh), and eight treatments for problem 84-t14 (hjk,abcdhjk, adehjk, bdfhjk, cdghjk, aceghjk, degjk, acdefgjk). The responsehas been estimated by minimising the error sum of squares, and this isincluded in table B.4.

B.4.2 Analysis of Variance

Table B.5 summarises the analysis of variance. It includes effects that aresignificant at the 10% level together with those added to make the modelhierarchical—e.g., the insignificant effect BF is included as a componenteffect of BFH, which is significant. Other effects have been grouped intothe lack of fit error term. The residual is the sum of pure error and lack offit error.

The analysis is also shown graphically in figure B.1, which presents anormal probability plot of the effect estimates. The axes of the plot areless important than the shape—any normal distribution is represented by astraight line, so effects that are far from the fitted line are significant. Theplot identifies the effects H, D, B, BH, HJ , AE, DE, EF , CD, K, GH,FH, and EH as significant at the 10% level, confirming the results shownin table B.5.


Source Sum Sq DF Mean Sq F Value Prob > FB 1.253 1 1.253 22.400 0.000C 0.006 1 0.006 0.114 0.734D 5.237 1 5.237 93.614 0.000E 0.016 1 0.016 0.291 0.588F 0.122 1 0.122 2.185 0.138G 0.049 1 0.049 0.879 0.346H 173.083 1 173.083 3093.965 0.000J 0.178 1 0.178 3.177 0.073K 0.239 1 0.239 4.268 0.038BF 0.001 1 0.001 0.013 0.909BH 0.412 1 0.412 7.367 0.006CD 0.165 1 0.165 2.947 0.085DE 0.267 1 0.267 4.777 0.028EF 0.174 1 0.174 3.102 0.077EG 0.067 1 0.067 1.198 0.271EH 0.687 1 0.687 12.288 0.000FH 0.493 1 0.493 8.813 0.003GH 0.293 1 0.293 5.243 0.022HJ 0.277 1 0.277 4.947 0.026BFH 0.293 1 0.293 5.241 0.022EGH 0.175 1 0.175 3.123 0.076Blocks 3949.865 9Model 4133.353 30 137.778 2462.869 0.000Lack of Fit 2.054 106 0.019 0.326Pure Error 67.091 1130 0.059Residual 69.145 1236 0.056Total 4202.498 1266

Table B.5: Analysis of variance.


Figure B.1: Normal probability plot of the effects.

Variable Estimate exp(Est) β exp(β) Sum SqB 0.0626 1.065 0.0313 1.032 1.253C −0.0045 0.996 −0.0022 0.998 0.006D 0.1279 1.136 0.0640 1.066 5.237E 0.0071 1.007 0.0036 1.004 0.016F 0.0195 1.020 0.0098 1.010 0.122G −0.0124 0.988 −0.0062 0.994 0.049H 0.7354 2.086 0.3677 1.444 173.083J 0.0236 1.024 0.0118 1.012 0.178K −0.0273 0.973 −0.0137 0.986 0.239BF −0.0015 0.999 −0.0007 0.999 0.001BH 0.0359 1.037 0.0179 1.018 0.412CD −0.0227 0.978 −0.0113 0.989 0.165DE 0.0289 1.029 0.0144 1.015 0.267EF 0.0233 1.024 0.0116 1.012 0.174EG 0.0145 1.015 0.0072 1.007 0.067EH −0.0463 0.955 −0.0232 0.977 0.687FH −0.0393 0.962 −0.0196 0.981 0.493GH −0.0303 0.970 −0.0151 0.985 0.293HJ 0.0294 1.030 0.0147 1.015 0.277BFH 0.0303 1.031 0.0151 1.015 0.293EGH 0.0234 1.024 0.0117 1.012 0.175

Table B.6: Effect estimates.


Problem κ exp(κ)meat12 -0.065 0.937mrp5 0.755 2.128sctap2 2.365 10.643large 2.656 14.2377-10-s 1.418 4.1307-10-t 2.992 19.93584-t7 5.482 240.21284-t14 5.815 335.149minsil7 2.865 17.543pds003 2.325 10.228

Table B.7: Problem mean estimates.

B.4.3 Regression Model

Table B.6 shows the estimates of the effects in the model (5.1) (p. 96). Theestimate is the predicted effect of applying the variable on the transformedresponse, y = log t. The regression coefficient, β, is the estimated regressioncoefficient in the additive model; the fitted values appear in (5.2) (p. 98).Note that β = 1

2Estimate. The standard error of each effect estimate is0.01362; the standard error of each regression coefficient is 0.00681.

Applying the inverse transformation exp(·), we obtain the estimated ef-fects and regression coefficients in terms of the original response, t. Thevalue exp(Est) gives the multiplicative effect of the variable, so a value of 2means that applying the variable results in double the solution time. (Infact it is not quite so simple, as other interaction variables will also have aneffect.) The values exp(β) appear in (5.3) (p. 98).

Table B.7 shows the mean response of each problem in the model (5.1).The κ value for problem P is mean response κP of problem P in the trans-formed model, y = log t; the fitted values appear in (5.2). Once again, byapplying the inverse transformation exp(·), we obtain the mean in terms ofthe original response, so exp(κ) is the estimated mean elapsed optimisationtime for each problem, and features in (5.3).

B.4.4 Model Adequacy Checking

Figure B.2 presents a normal probability plot of the residuals. It shows thatthere is some departure from the assumption of normality, with the distri-bution being skewed to the right and some noticeable outliers. Figure B.3(p. 164) is a plot of the residuals against the fitted values. The bunching hereis due to the problems (blocks). The variance appears fairly constant, with aslight tendency to increase for larger values. Figures B.4–B.7 (pp. 164–165)present dot diagrams of the residuals by problem (block). They indicate


Figure B.2: Normal probability plot of the residuals.

that the variance is relatively constant across the blocks in general, but islarger for problems 84-t14, large and sctap2, and problems 84-t14 and sctap2both have significant outliers. The normality assumption is questionable forproblems large and sctap2 for which the residual distributions are bimodal.

Moderate departures from normality are of little concern in the balancedfixed effects model that we are using, whilst the F -test is only slightly af-fected if homogeneity of variances is violated. Indeed the F -test is veryrobust under abnormal conditions, and may regarded as an approximationto the generally applicable randomisation test [Mon01, §3.3.2]. We thereforeaccept the analysis and model.


Figure B.3: Residuals against the fitted values.

Figure B.4: Residuals by problem: 84-t14, 7-10-t, 7-10-s.


Figure B.5: Residuals by problem: meat12, large, 84-t7.

Figure B.6: Residuals by problem: pds003, mrp5, minsil7.

Figure B.7: Residuals by problem: sctap2.


B.5 Evaluation of Parallel Performance

This section gives the results of the experiment to evaluate the performanceof DecOpt on a range of parallel computer platforms, described in section 5.6(p. 101).

We used eleven problems from our test set and the DecOpt control set-tings established in the evaluation of computational strategies (table 5.11,p. 99). Five runs were performed on a multi-processor Windows PC (plat-form 2, p. 90), using 1–4 processors. Table B.8 shows the elapsed times, withfailed runs marked by an asterisk. Tables B.9–B.12 (pp. 168–169) summarisethe results, giving the mean time, the standard deviation, the speed-up andthe efficiency.

Table B.8: Parallel DecOpt performance on platform 2.

Processors 1 2 3 4sctap3 33.80 26.32 21.94 20.02

33.79 26.79 21.34 21.2633.78 26.53 20.96 19.9233.72 26.33 25.18 18.9933.94 26.28 21.63 21.75

7-10-s 8.64 7.55 6.10 6.618.59 7.51 6.58 6.728.87 7.91 6.62 6.848.60 7.51 6.77 6.098.97 7.76 6.56 6.53

7-10-t 35.00 27.53 23.52 23.6135.05 27.37 26.43 23.8935.44 27.55 26.51 24.6535.08 27.35 22.53 23.5035.08 27.38 22.65 22.41

84-t2 4584.05 3090.89 * 2742.064588.32 3085.07 * 2549.114600.32 3090.16 3000.38 2471.644588.88 3087.87 2882.97 2859.934599.70 3087.89 * *

84-t4 1396.22 1250.39 948.13 970.121397.10 1250.76 961.06 869.531397.03 1251.03 858.62 1042.671396.86 1250.49 * 1116.971402.75 1253.20 1036.64 1055.08

84-t7 358.23 226.58 174.22 149.91357.69 226.89 174.60 156.23357.62 227.35 174.45 158.21

B.5. EVALUATION OF PARALLEL PERFORMANCE 167

Table B.8: Parallel DecOpt performance on platform 2 (continued).

Processors 1 2 3 4357.83 226.42 174.94 141.78357.75 227.53 174.75 148.07

84-t14 569.29 312.14 223.07 210.78570.12 312.28 225.35 198.11569.03 312.73 219.17 189.03568.67 313.03 225.00 207.36569.51 313.44 223.14 195.23

364-t7 2953.10 2291.11 2029.22 2034.072950.72 2292.23 2032.17 2014.022948.27 2294.75 2086.05 1838.922955.01 2293.06 1951.76 1914.912951.78 2292.03 2063.28 1798.22

364t14 2844.37 1609.96 1338.32 1205.492841.30 1612.97 1336.50 1157.392841.65 1613.28 1335.67 1159.342843.11 1615.31 1337.22 1161.662840.54 1610.25 1335.64 1195.43

364-t28 3895.36 2214.50 1702.47 1447.113901.30 2214.95 1674.89 1453.893899.64 2216.01 1869.72 1457.303901.39 2216.11 1879.62 1461.193896.02 2213.80 1941.11 1449.58

pds003 23.41 18.19 16.28 12.8223.44 18.03 17.44 15.0523.84 18.28 15.44 13.0623.45 17.99 17.30 11.0323.70 18.22 16.49 14.61


Processors 1 2 3 4sctap3 33.81 26.45 22.21 20.397-10-s 8.73 7.65 6.53 6.567-10-t 35.13 27.44 24.33 23.6184-t2 4592.25 3088.38 2941.68 2655.6984-t4 1397.99 1251.17 951.11 1010.8784-t7 357.82 226.95 174.59 150.8484-t14 569.32 312.72 223.15 200.10364-t7 2951.78 2292.64 2032.50 1920.03364t14 2842.19 1612.35 1336.67 1175.86364-t28 3898.74 2215.07 1813.56 1453.81pds003 23.57 18.14 16.59 13.31

Table B.9: Performance statistics on platform 2: mean time.


Table B.10: Performance statistics on platform 2: standard deviation.



Table B.11: Performance statistics on platform 2: speed-up.

Processors 1 2 3 4sctap3 100% 64% 51% 41%7-10-s 100% 57% 45% 33%7-10-t 100% 64% 48% 37%84-t2 100% 74% 52% 43%84-t4 100% 56% 49% 35%84-t7 100% 79% 68% 59%84-t14 100% 91% 85% 71%364-t7 100% 64% 48% 38%364t14 100% 88% 71% 60%364-t28 100% 88% 72% 67%pds003 100% 65% 47% 44%

Table B.12: Performance statistics on platform 2: efficiency.


Three runs were performed on a set of four Windows PCs linked byethernet (platform 3, p. 90), using 1–4 processors. Table B.13 shows theelapsed times, with failed runs marked with an asterisk. Tables B.14–B.17(pp. 171–172) summarise the results, giving the mean time, the standarddeviation, the speed-up and the efficiency.


Processors 1 2 3 4sctap3 43.64 31.47 25.63 20.02

43.60 31.40 24.63 20.7643.74 31.68 24.12 24.74

7-10-s 10.33 7.55 8.04 6.6110.37 8.83 8.39 7.429.78 9.13 7.99 7.99

7-10-t 42.59 27.53 32.52 23.6142.48 33.52 23.52 25.4641.94 36.16 32.59 31.09

84-t2 6320.83 4490.89 4590.12 2742.066322.26 4272.60 4418.87 5387.936322.53 4275.58 4412.82 4888.47

84-t4 1834.48 1650.39 1495.49 1170.121834.64 1652.82 1576.65 1296.581833.63 1640.55 1495.72 1230.71

84-t7 458.23 305.29 222.19 174.22440.76 297.62 199.00 188.10468.83 298.24 197.30 186.66

84-t14 769.29 412.14 339.20 280.78798.30 400.07 339.23 271.37813.62 437.53 339.59 258.98

364-t7 3998.38 3191.11 2829.22 2634.073990.17 3115.84 2891.70 2598.263994.17 3128.67 2906.33 2623.53

364t14 4041.62 2272.87 1721.54 1555.494038.80 2285.45 1638.32 1509.264046.43 2284.05 1787.77 1532.74

364-t28 5692.70 3063.40 2360.00 1867.115790.68 3164.30 2302.47 1851.665791.03 3160.51 2335.56 1861.64

pds003 26.81 21.43 16.28 17.5826.77 21.27 19.42 19.3927.12 21.46 19.02 15.05









Processors 1 2 3 4sctap3 100% 69% 59% 50%7-10-s 100% 60% 42% 35%7-10-t 100% 65% 48% 40%84-t2 100% 73% 47% 36%84-t4 100% 56% 40% 37%84-t7 100% 76% 74% 62%84-t14 100% 95% 78% 73%364-t7 100% 63% 46% 38%364t14 100% 89% 79% 66%364-t28 100% 92% 82% 77%pds003 100% 63% 49% 39%



Three runs were performed on a 64-processor SGI workstation (plat-form 4, p. 90), using up to 16 processors. Table B.18 shows the elapsedtimes. Tables B.19–B.22 (pp. 173–174) summarise the results, giving themean time, the standard deviation, the speed-up and the efficiency.


Processors 1 2 4 6 7 8 12 13 14 16sctap3 12 14 28 10 20

10 12 15 15 3210 9 6 7 25

84-t7 132 86 71 53 46129 75 50 39 37129 83 59 47 45

84-t14 257 135 96 61260 125 95 71269 133 84 81

364t14 2188 890 542 450 2732253 886 598 457 2592124 844 549 351 277

364-t28 1987 1104 631 477 3231885 1069 682 372 2711856 1183 592 370 308

pds003 9 9 13 6 810 11 7 8 89 10 10 6 12

Processors 1 2 4 6 7 8 12 13 14 16sctap3 10.67 11.67 16.33 10.67 25.6784-t7 130.00 81.33 60.00 46.33 42.6784-t14 262.00 131.00 91.67 71.00364t14 2188.33 873.33 563.00 419.33 269.67364-t28 1909.33 1118.67 635.00 406.33 300.67pds003 9.33 10.00 10.00 6.67 9.33







Processors 1 2 4 6 7 8 12 13 14 16sctap3 100% 46% 16% 13% 3%84-t7 100% 80% 54% 40% 23%84-t14 100% 100% 71% 53%364t14 100% 125% 97% 75% 58%364-t28 100% 85% 75% 67% 45%pds003 100% 47% 23% 23% 8%


B.6. COMPARISON WITH STANDARD LP SOFTWARE 175

B.6 Comparison with Standard LP Software

This section gives the results of final experiment, described in section 5.7(p. 108), which was to compare the performance of the Dantzig-Wolfe decom-position method in our serial DecOpt implementation to the simplex methodin the xpress-mp Optimiser. We solved our complete problem suite usingDecOpt and xpress-mp on a Windows PC (platform 1, p. 90). We usedthe DecOpt control settings established in the evaluation of computationalstrategies (table 5.11, p. 99) and the default settings in xpress-mp. A threehour time limit was imposed for each problem.

The results are given in table B.23. The columns headed Time give theelapsed run time using DecOpt and xpress-mp; the columns headed Opt givethe elapsed optimisation time only. The DecOpt results include the numbersof cycles (Cyc) and proposals (Props); the xpress-mp results include thenumber of pivots (Piv). The Gap is the relative gap between the primalvalue and dual bound when the column generation algorithm finished. Notethat three problems—mnsil19, pds020 and pds050 —could not be solved byDecOpt within the time limit; whilst five of the PDS problems were haltedwith an ε-optimal solution.

Table B.23: Comparison of Dantzig-Wolfe and simplex methods.

DecOpt xpress-mp

Problem Time Opt Cyc Props Gap Time Opt Pivdregem 3.34 2.97 42 711 0.27 0.17 579dyna 0.48 0.13 1 5 0.14 0.03 0egd034 0.38 0.13 6 57 0 0.05 0.03 232egd085 0.58 0.32 9 104 1.47E−16 0.12 0.07 560fixmar 0.78 0.55 31 152 0 0.08 0.05 293forest 0.69 0.45 14 102 −6.74E−15 0.30 0.28 802meat12 1.03 0.78 35 258 0 0.06 0.03 174meat31 0.52 0.26 10 86 1.12E−16 0.09 0.06 277meat43 0.56 0.26 8 69 0 0.08 0.04 245mege06 0.44 0.18 8 38 1.07E−07 0.19 0.03 238mege08 0.51 0.23 8 52 0 0.11 0.05 326mege10 0.60 0.30 8 66 0 0.14 0.07 441mrp3 0.38 0.15 10 32 0 0.06 0.03 332mrp5 1.55 1.28 34 245 0 0.29 0.23 1388regem 0.64 0.27 3 13 0 0.26 0.09 271scorpion 0.37 0.15 9 62 1.21E−16 0.04 0.02 61scfxm2 0.73 0.49 14 39 0 0.18 0.15 683scfxm3 1.05 0.80 17 69 0 0.46 0.29 1083sctap1 1.11 0.83 28 582 −4.83E−16 0.06 0.03 184sctap2 5.52 4.88 34 2110 1.32E−16 0.19 0.13 324sctap3 11.76 10.94 65 3374 0 0.32 0.26 485small 4.28 2.23 2 32 1.57E−06 6.59 5.00 290med1 10.23 6.09 2 32 2.03E−07 16.84 12.52 330med2 9.57 6.56 2 32 1.49E−07 10.66 8.09 307large 15.61 8.09 2 32 1.83E−06 21.77 16.73 3287-01-s 2.76 2.34 23 643 0 0.73 0.68 1279


Table B.23: Comparison of Dantzig-Wolfe and simplex methods (continued).

DecOpt xpress-mp

Problem Time Opt Cyc Props Gap Time Opt Piv7-01-t 5.47 5.15 59 605 0 0.68 0.61 12087-02-s 2.69 2.27 24 597 0 0.73 0.66 11637-02-t 5.73 5.41 67 635 2.63E−10 0.72 0.65 12077-03-s 2.76 2.35 27 630 0 0.94 0.87 14267-03-t 5.52 5.21 72 657 1.92E−16 1.12 1.05 15847-04-s 3.07 2.63 27 669 0 1.09 1.04 15237-04-t 8.83 8.53 89 901 2.69E−16 0.98 0.91 14397-05-s 2.81 2.40 28 654 6.93E−16 1.07 1.00 14047-05-t 7.52 7.20 83 796 −1.73E−16 1.14 0.92 13207-06-s 3.36 2.95 29 777 0 1.18 1.12 15837-06-t 11.11 10.80 113 1021 1.74E−16 1.14 1.07 15197-07-s 4.05 3.59 34 858 0 1.11 1.03 14687-07-t 21.58 21.26 158 1593 3.28E−08 1.03 0.97 14877-08-s 3.97 3.52 30 837 0 1.03 0.94 13307-08-t 18.30 17.90 141 1406 1.33E−08 1.36 1.28 17157-09-s 3.30 2.87 26 758 0 1.22 1.14 16297-09-t 12.27 11.93 97 1063 7.15E−09 1.48 1.40 18067-10-s 3.22 2.76 24 690 6.57E−11 1.19 1.11 16717-10-t 12.13 11.81 115 1161 −3.70E−16 0.99 0.91 14507-11-s 3.50 3.06 27 758 9.37E−11 0.93 0.85 13587-11-t 11.44 11.10 88 1030 −2.20E−16 0.93 0.85 13657-12-s 2.44 2.02 22 526 2.30E−16 0.80 0.77 13277-12-t 9.93 9.62 105 1027 −1.15E−16 0.80 0.72 136584-t2 1958.67 1957.12 587 21790 1.26E−09 143.10 140.86 1819684-t4 565.48 564.22 368 8577 2.41E−09 131.00 129.32 1791684-t7 151.13 149.99 95 1663 1.67E−09 139.93 139.27 1811584-t14 250.60 249.47 110 1041 2.41E−10 147.39 145.59 1859084-s 3158.92 3157.63 752 23733 1.55E−15 126.50 124.51 17839364-t7 1254.02 1246.66 94 6208 1.10E−09 2079.24 2075.74 81052364t14 1255.86 1249.21 106 3794 9.75E−10 2112.37 2108.81 82786364-t28 1900.94 1893.33 100 2191 2.17E−10 1991.58 1987.89 81343cinca2 0.41 0.18 9 42 0 0.06 0.05 230minsil7 9.23 8.95 63 468 −6.49E−14 0.20 0.15 856mnsil19 10807.01 10805.94 436 3420 0.7714650113 3.83 3.58 5875navia90 0.61 0.39 18 122 −5.72E−15 0.05 0.03 245tajo6 0.49 0.26 13 92 −2.01E−14 0.05 0.03 196pds001 1.12 0.77 12 198 7.21E−084 0.18 0.10 172pds003 7.79 7.23 36 547 8.80E−084 1.24 1.00 1000pds005 54.75 53.86 88 1235 6.30E−094 3.09 1.96 2295pds009 1295.78 1294.35 267 3564 5.90E−084 7.28 6.40 4386pds012 3695.25 3693.22 514 6689 4.05E−084 10.97 9.68 5766pds020 10812.92 10806.82 421 5625 0.0135029783 35.93 33.35 11193pds050 10813.67 10800.53 121 1564 0.8170760273 318.46 309.55 52013

33 hour time limit.4ε-optimal.

B.7. COMPARISON OF INITIAL AND CHOSEN STRATEGIES. 177

B.7 Comparison of Initial and Chosen Strategies.

Table B.24 compares the times obtained in the final experiment, using thechosen control settings (section 5.7, p. 108), with those obtained in thevalidation experiment, using the initial control settings (section 5.4, p. 91).The results are taken from sections B.6 and B.1. Both experiments wereperformed on the same Windows PC (platform 1, p. 90); the only differencebeing the control settings—the initial settings given in table 5.7 (p. 92)compared to the chosen settings given in table 5.11 (p. 99).

The table shows the elapsed optimisation times (Opt), the number ofcycles taken by the column generation decomposition algorithm (Cyc), thenumber of proposals generated (Props) under the two control settings (cho-sen and initial). The ratio of the responses for the chosen settings over theinitial settings for the three statistics is given in the final three columns.The final line gives the average (arithmetic mean) ratios.

Table B.24: Comparison of initial and chosen strategies (sections 5.4 and 5.7).

Chosen Strategy Initial Strategy Ratio Chosen:InitialProblem Opt Cyc Props Opt Cyc Props Opt Cyc Propsdregem 2.97 42 711 1.86 25 417 159.7% 168.0% 170.5%dyna 0.13 1 5 0.14 1 5 92.9% 100.0% 100.0%egd034 0.13 6 57 0.17 7 52 76.5% 85.7% 109.6%egd085 0.32 9 104 0.28 8 69 114.3% 112.5% 150.7%fixmar 0.55 31 152 0.69 43 138 79.7% 72.1% 110.1%forest 0.45 14 102 0.59 21 89 76.3% 66.7% 114.6%meat12 0.78 35 258 1.06 56 254 73.6% 62.5% 101.6%meat31 0.26 10 86 0.31 11 66 83.9% 90.9% 130.3%meat43 0.26 8 69 0.31 11 55 83.9% 72.7% 125.5%mege06 0.18 8 38 0.21 9 35 85.7% 88.9% 108.6%mege08 0.23 8 52 0.30 11 51 76.7% 72.7% 102.0%mege10 0.30 8 66 0.36 10 63 83.3% 80.0% 104.8%mrp3 0.15 10 32 0.20 15 38 75.0% 66.7% 84.2%mrp5 1.28 34 245 1.49 46 204 85.9% 73.9% 120.1%regem 0.27 3 13 0.30 3 13 90.0% 100.0% 100.0%scorpion 0.15 9 62 0.17 9 65 88.2% 100.0% 95.4%scfxm2 0.49 14 39 0.53 18 35 92.5% 77.8% 111.4%scfxm3 0.80 17 69 0.68 15 43 117.6% 113.3% 160.5%sctap1 0.83 28 582 1.31 51 833 63.4% 54.9% 69.9%sctap2 4.88 34 2110 7.26 77 3989 67.2% 44.2% 52.9%sctap3 10.94 65 3374 23.84 192 8456 45.9% 33.9% 39.9%small 2.23 2 32 2.04 2 32 109.3% 100.0% 100.0%med1 6.09 2 32 5.58 2 32 109.1% 100.0% 100.0%med2 6.56 2 32 6.38 2 32 102.8% 100.0% 100.0%large 8.09 2 32 7.34 2 32 110.2% 100.0% 100.0%7-01-s 2.34 23 643 3.80 57 875 61.6% 40.4% 73.5%7-01-t 5.15 59 605 5.72 95 598 90.0% 62.1% 101.2%7-02-s 2.27 24 597 3.00 44 729 75.7% 54.5% 81.9%


Table B.24: Comparison of initial and chosen strategies (continued).

Chosen Strategy Initial Strategy Ratio Chosen:InitialProblem Opt Cyc Props Opt Cyc Props Opt Cyc Props7-02-t 5.41 67 635 4.70 82 487 115.1% 81.7% 130.4%7-03-s 2.35 27 630 3.64 59 796 64.6% 45.8% 79.1%7-03-t 5.21 72 657 9.68 216 1120 53.8% 33.3% 58.7%7-04-s 2.63 27 669 3.99 59 970 65.9% 45.8% 69.0%7-04-t 8.53 89 901 14.04 279 1326 60.8% 31.9% 67.9%7-05-s 2.40 28 654 3.71 58 888 64.7% 48.3% 73.6%7-05-t 7.20 83 796 11.60 238 1378 62.1% 34.9% 57.8%7-06-s 2.95 29 777 4.27 65 1010 69.1% 44.6% 76.9%7-06-t 10.80 113 1021 28.57 556 3282 37.8% 20.3% 31.1%7-07-s 3.59 34 858 4.35 63 1009 82.5% 54.0% 85.0%7-07-t 21.26 158 1593 59.95 1023 6181 35.5% 15.4% 25.8%7-08-s 3.52 30 837 4.39 60 1076 80.2% 50.0% 77.8%7-08-t 17.90 141 1406 43.75 789 4250 40.9% 17.9% 33.1%7-09-s 2.87 26 758 3.52 47 891 81.5% 55.3% 85.1%7-09-t 11.93 97 1063 48.40 816 4926 24.6% 11.9% 21.6%7-10-s 2.76 24 690 3.47 47 922 79.5% 51.1% 74.8%7-10-t 11.81 115 1161 28.99 514 2981 40.7% 22.4% 38.9%7-11-s 3.06 27 758 4.29 58 1008 71.3% 46.6% 75.2%7-11-t 11.10 88 1030 11.79 181 1173 94.1% 48.6% 87.8%7-12-s 2.02 22 526 2.31 34 585 87.4% 64.7% 89.9%7-12-t 9.62 105 1027 25.95 488 2878 37.1% 21.5% 35.7%84-t2 1957.12 587 21790 2824.59 2859 81493 69.3% 20.5% 26.7%84-t4 564.22 368 8577 1519.33 2649 41605 37.1% 13.9% 20.6%84-t7 149.99 95 1663 161.51 131 1489 92.9% 72.5% 111.7%84-t14 249.47 110 1041 324.54 189 1090 76.9% 58.2% 95.5%84-s 3157.63 752 23733 27164.09 17172 371823 11.6% 4.4% 6.4%364-t7 1246.66 94 6208 1073.43 170 6446 116.1% 55.3% 96.3%364t14 1249.21 106 3794 1389.20 182 3932 89.9% 58.2% 96.5%364-t28 1893.33 100 2191 2424.93 175 2130 78.1% 57.1% 102.9%cinca2 0.18 9 42 0.16 13 33 112.5% 69.2% 127.3%minsil7 8.95 63 468 13.83 167 668 64.7% 37.7% 70.1%navia90 0.39 18 122 0.52 36 144 75.0% 50.0% 84.7%tajo6 0.26 13 92 0.26 21 84 100.0% 61.9% 109.5%pds001 0.77 12 198 0.81 18 185 95.1% 66.7% 107.0%pds003 7.23 36 547 14.60 114 880 49.5% 31.6% 62.2%pds005 53.86 88 1235 110.69 351 2439 48.7% 25.1% 50.6%pds009 1294.35 267 3564 1419.59 1287 9171 91.2% 20.7% 38.9%pds012 3693.22 514 6689 8399.92 3800 25596 44.0% 13.5% 26.1%Average 77.0% 59.1% 84.0%

Appendix C

DecOpt User Manual

For completeness, this manual repeats some material from the main body of thethesis.

C.1 What is DecOpt?

DecOpt is an implementation of the Dantzig-Wolfe decomposition algorithm forsolving a class of linear programming (LP) problems, known as primal block angularLP problems. The algorithm exploits block angular structure in the coefficientmatrix of the problem by partitioning it into a set of pricing problems, linked onlythough a co-ordinating master problem. By solving the master problem, a solutionto the original problem is obtained.

DecOpt uses the xpress-mp Optimisation Subroutine Library (xosl) to solvethe pricing problems and master problem arising in the algorithm. DecOpt alsomakes it possible to solve the pricing problems in parallel on a variety of parallel ar-chitectures including multi-processor computers and a set of serial computers linkedby a network, and for this High Performance Parallel Virtual Machine (hppvm) isrequired.

To use DecOpt you need to supply an initialisation file containing problem pa-rameters and algorithm controls, and separate matrix files (MPS or LP format)containing the master problem and each pricing problem. The decomposition al-gorithm is applied to optimise the problem, and any solution found may be outputto any of the standard xpress-mp solution files. xpress-mp imposes a limit of 62on the number of subsystem blocks (pricing problems) in the decomposition, butotherwise there are no inherent limits on problem size, and system resources arethe only constraint.

DecOpt has been designed both as a practical tool for solving problems, and asa flexible framework for evaluating Dantzig-Wolfe decomposition. What this meansis that you can use DecOpt on one level as a black box optimiser, requiring verylittle knowledge of decomposition or explicit specification of the different strategies,whilst the comprehensive optimisation strategies and statistical reports availableenable you to experiment with different strategies, and serial or parallel operation,using one consistent code.

The main modules of DecOpt initialise the control variables; read in the prob-lems from a set of matrix files and allocate memory; perform the Dantzig-Wolfe

179

180 APPENDIX C. DECOPT USER MANUAL

buym,f,t↓


makep,f,t↓


↓sellp,f,t

Figure C.1: Activities and material flows in the production planning model.

algorithm; and calculate a statistical summary of the program’s performance. Thealgorithm itself is divided into three phases: (i) find an initial feasible solution toeach of the pricing problems; (ii) solve the master problem to proven infeasibil-ity, optimality, or unboundedness; and (iii) compute and output an optimal primalsolution to the original problem.

DecOpt does not identify suitable block structures or extract the various de-composed sub-matrices from a complete matrix; these tasks must be performed byyou in advance. Some guidance is offered below on how to identify block structure,and a separate software tool DecMod is available to extract the individual matrixfiles required from a complete problem formulation (see the DecMod User Manualin appendix D). Neither is the solution available in the form of a basis for theoriginal problem; but of course the output files together list optimal values for allthe variables and constraints of the original problem and these values correspondto a primal solution for the original problem.

DecOpt is a research code. The author would be pleased to hear of your expe-riences with it, and details of any particular successes or failures. You can contactthe author by email at [email protected].

C.2 Tutorial

This introductory tutorial illustrates the whole process of solving a problem withDecOpt, from identifying different block structures in the model, creating the ma-trices and initialisation files for input to DecOpt, operating DecOpt, through tointerpreting the solution in the context of the original problem. Do not worry ifyou are unsure of some of the details at this stage—they are explained fully in thesubsequent sections of this manual.

C.2.1 A Production Planning Problem

Consider a simple production planning model over a set of months t ∈ T and aset of facilities f ∈ F , based on the Coco problem in the Xpress-MP User Guide.In each month at each facility a number of raw materials m ∈ M are purchased,

C.2. TUTORIAL 181



−∑



mft CBUY mt buymft

Subject to


−∑



MxPStockft :∑



MxMStockft :∑




Table C.1: An algebraic model of the production planning problem.

and manufactured into end products p ∈ P which are sold. Storage of both ma-terials and products is possible at all facilities. The activities and material flowsin the model are pictured in figure C.1, and table C.1 presents an algebraic modelformulation. Decision variables are denoted in lower case; data in upper case; andthe names of constraints in mixture of upper and lower case. A subscript on theconstraint name indicates that the constraint applies for all values of the subscript.

Aside from the mass balance equations (PBal and MBal), there are limits onthe total sales of each product in each period (MxSell), the total stock of productsat each facility in each period (MxPStock), the total production at each facility ineach period (MxMake), the total stock of materials at each facility in each period(MxMStock), and the total purchases of each material in each period (MxBuy).Sales of products generate revenue, whilst all other activities incur costs. Theobjective is to maximise profit.

An xpress-mp model of the problem is included in the DecOpt release (filepplan.mod). The equations are contained within the CONSTRAINTS section of themodel, which is reproduced in table C.2. We look for block structure based on oneof the index sets in the model: p, m, f, t.

C.2.2 Identifying a Block Structure

Let us first consider decomposing the problem by the facility index. Later on wewill consider other decompositions based on product and time period, and thiswill allow us to compare the application and performance of the algorithm on thedifferent decompositions and to assess the quality of the decompositions.

Inspecting the model we note that the objective function MaxProfit, constraintsMxSell and constraints MxBuy are not indexed by facilities (f=1–NF). All otherconstraints and all variables are indexed by one facility f. Furthermore, those


CONSTRAINTS









Table C.2: An xpress-mp model of the production planning problem.

Master Problem Pricing Problem fConstraints MaxProfit PBal(*,f,*)

MxSell(*,*) MBal(*,f,*)MxBuy(*,*) MxPStock(f,*)

MxMake(f,*)MxMStock(f,*)

Variables sell(*,f,*)pstock(*,f,*)make(*,f,*)mstock(*,f,*)buy(*,f,*)

Table C.3: A partition of the variables and constraints generated by the setof facilities in the production planning problem.

C.2. TUTORIAL 183






END

Table C.4: DecMod specification for the facility decomposition.

constraints that are indexed by facility f feature only variables indexed by thesame single facility f. We can therefore specify a primal block angular structurebased on the facility index as shown in table C.3.

Note that there are no variables in the initial master problem. This is sometimesthe case in Dantzig-Wolfe decomposition, which uses a column generation methodto add variables to the master problem as it proceeds.

C.2.3 Generating the Matrices using DecMod

DecMod is a tool to extract a given block structure from a problem and create theproblem matrices required by DecOpt. See the DecMod User Manual for details ofhow to use it.

DecMod requires the block structure to be specified in a decomposition file (.dec)which identifies the variables and constraints in each block. The decompositionfile is very similar to the description of the block structure given C.3, the onlysignificant difference being that the global constraints, i.e., the constraints in themaster problem, must be included in each pricing problem. It is important thatthey are listed first in the pricing problems, and in the same order that they appear


for the master problem. A decomposition file for the facility decomposition is shownin table C.4 (p. 183).

C.2.4 The Problem Initialisation File

The initialisation file pplanf.ini contains the following text lines:

nprob=3maxisol=2

This is the minimum amount of information that must be included. It states thatthere are 3 separate problems that make up the complete problem—the masterproblem and two pricing problems; that the problem is to be maximised; and thatwe wish DecOpt to generate xpress-mp print solution files (.prt) for each sub-matrix.

C.2.5 Applying DecOpt

Now we have the three matrices and the initialisation file. To optimise the problemusing DecOpt, simply type

decopt pplanf

The output of DecOpt will scroll rapidly up the screen but near the end of theoutput you should read

Optimal solution obtainedProblem: pplanfMinimisation performed

5 cycles0.08 seconds

Primal value: 162015

The output is also sent to the file pplanf.log so you can browse it as you wish.The important thing to note first of all is that the full optimal solution has beenobtained, with objective (primal) value 162015. You may like to use xpress-mp

to solve the original problem contained in pplan.mod. You should find that theobjective values match.

The next thing to look at is the solution that DecOpt found. This is containedin the solution print files pplanf0.prt, pplanf1.prt and pplanf2.prt. There istoo much information in these files to discuss fully, so let us just consider the deci-sion variables make. These variables were split between the two pricing problems,so looking in the solution print files pplanf1.prt and pplanf2.prt we find theoptimum values shown in table C.5.

C.2.6 Different Block Structures

It turns out that the block structure we have used, based on the facility index, isby far the best for Dantzig-Wolfe decomposition. This is because only two setsof constraints applied across facilities and thus became global constraints. The

C.3. REQUIREMENTS AND INSTALLATION 185

Product 1 Product 2make(1,1,1) 0 make(1,2,1) 0make(2,1,1) 125 make(2,2,1) 375make(1,1,2) 0 make(1,2,2) 0make(2,1,2) 0 make(2,2,2) 150make(1,1,3) 0 make(1,2,3) 0make(2,1,3) 0 make(2,2,3) 150make(1,1,4) 0 make(1,2,4) 0make(2,1,4) 0 make(2,2,4) 0

Table C.5: Optimal solution found by DecOpt.

Master Problem Pricing Problem pConstraints MaxProfit PBal(p,*,*)

MBal(*,*,*) MxSell(*,*)MxPStock(*,*)MxMake(*,*)MxMStock(*,*)MxBuy(*,*)

Variables mstock(*,*,*) sell(p,*,*)buy(*,*,*) pstock(p,*,*)

make(p,*,*)

Table C.6: A partition of the variables and constraints generated by the setof products in the production planning problem.

resulting pricing problems were well-formed: they had constraints, variables andcosts that one might find in standard problems.

By way of comparison, consider two other decompositions specified in tables C.6and C.7. They are based on the product index and time period index respectively.You can immediately see that the master problem is far more complicated, makingthe pricing problems more artificial.

Included in the DecOpt tutorial directory are decomposition files pplant.decand pplanp.dec to generate the required matrices for DecOpt, and the matrices,initialisation files, DecOpt log files and solution print files. You may like to studythese to gain further understanding about using DecOpt.

C.3 Requirements and Installation

DecOpt is available on a variety of platforms including 32 bit Windows (e.g., 95, 98,NT, 2000) and Solaris 2. Potentially it can be ported to any platform supportedby xpress-mp and, for the parallel version, hppvm. You will need to obtain thesoftware and a licence of xosl Release 12, and, if required, the software and alicence of hppvm Release 2.9. Subsequent releases may also be supported.


Master Problem Pricing Problem tConstraints MaxProfit MxSell(*,t)

PBal(*,*,*) MxPStock(*,t+1)MBal(*,*,*) MxMake(*,t)

MxMStock(*,t+1)MxBuy(*,t)

Variables pstock(*,*,1) sell(*,*,t)mstock(*,*,1) pstock(*,*,t+1)

make(*,*,t)mstock(*,*,t+1)buy(*,*,t)

Table C.7: A partition of the variables and constraints generated by the setof time periods in the production planning problem.

Install DecOpt into a new (‘decopt’) directory. In addition you will probablywant a separate (‘data’) directory for your problem matrices and initialisation files.

To run DecOpt, you need the following xosl files:

Windows Unixxpress.pwd xpress.pwdmp-opt.msg mp-opt.msgxpopt32.dllxpressnh.ini

Keep them together in the same directory—it is best to copy them into your decoptdirectory. In Windows, add the directory containing xpopt32.dll to the Windowspath. In Unix, set the environment variable XPRESS to point to the directorycontaining the message and password files.

To run Parallel DecOpt, you also need the following hppvm files:

Windows Unixrshd.exedaemon.exe daemonpvmdll.dllconfig configmc_list mc_list

Keep them together in the same directory—once again it is best to copy them intothe decopt directory. List all of the machines on which you wish to run ParallelDecOpt in the mc_list file on the lead computer. In Windows, add the directorycontaining pvmdll.dll, daemon.exe and the DecOpt slave executable, decslave.exe, to the Windows path and start the Remote Shell Daemon (rshd.exe) oneach machine. In Unix, add the directory containing daemon and the DecOpt slaveexecutable, decslave, to the path on each machine. (See the hppvm documentationfor further details.)

C.4. OVERVIEW OF DANTZIG-WOLFE DECOMPOSITION 187

C.4 Overview of Dantzig-Wolfe Decomposition

Dantzig-Wolfe decomposition is a method for solving LP problems with a pre-determined primal block angular structure (see section C.5 below). The originalproblem is split up into a set of pricing problems—one for each of the subsystemblocks in the block structure—and a master problem—which co-ordinates the pric-ing problems and ensures that the global constraints are satisfied. The masterproblem is equivalent to the original problem in some respects, and by solving themaster problem we obtain a solution to the original problem.

The master problem has a large number of variables—many more than in theoriginal problem. They are defined implicitly by the set of basic feasible solutionsand unbounded directions of the pricing problems. Rather than enumerating all ofthe variables for the master problem ab initio, a specialised variant of the simplexalgorithm is employed which operates with a small subset of the variables at anyone time. The restricted master problem is optimised over the current active set ofvariables; new variables which price out suitably to enter the basis are added; andthis procedure continues until no new variables which price out to enter the basiscan be found. As in the standard simplex algorithm, this signifies that an optimalsolution has been found. In general an optimal solution will be found withouthaving to generate explicitly all of the variables of the master problem.

How are variables which price out to enter the current basis of the restrictedmaster problem generated? By defining suitable objective functions for the pric-ing problems, based on the dual values of the restricted master problem, we cancontrive for the objective value at each extreme point to be the reduced cost—orprice—of the master problem variable associated with the extreme point. Solvingthe modified pricing problems generates basic feasible solutions of maximum re-duced cost (for maximisation problems). If the objective value at an extreme pointis positive, then the associated master problem variable is added to the masterproblem; if the minimum objective value over all extreme points is negative, thenno master problem variable exists to improve the current master problem solution.

The computational advantage of Dantzig-Wolfe decomposition arises from per-forming a significant amount of the computational work in the pricing problems.Based on the subsystem blocks of the original coefficient matrix, they are an orderof magnitude smaller than the original problem, and thus in general easier to solve.The pricing problems are also independent of one another and thus may be solvedconcurrently on suitable hardware. Against this must be set the potentially hugesize of the master problem, with many more variables—though fewer constraints—than the original problem. As we shall show, the feasible region of the masterproblem is more complex than that of the original problem and the solution pathmay be longer. Numerical problems may result from dynamically generating thevariables of the master problem. Thus there are many factors which may influencethe performance of the algorithm and is difficult to assess its merits without solvingpractical problems. And that is what DecOpt is for.


Figure C.2: Primal block angular structure in an LP problem.

C.5 Identifying a Suitable Block Structure in aProblem

Dantzig-Wolfe decomposition is suitable for problems where, if a relatively smallnumber of constraints were removed, the problem would fall apart into a numberof independent problems. In other words, it is possible to re-order the rows andcolumns in the coefficient matrix such that, apart from a few rows, the matrix isblock diagonal. Thus, after re-ordering, the matrix can be expressed as shown infigure C.2, where the constraints (rows) are partitioned into global constraints I0and blocks of constraints I1, I2, . . . , IK ; and the variables (columns) are partitionedinto subsets J0, J1, . . . , JK . A and B are the blocks of non-zero matrix coefficients;the T are the vectors of the constraint types (each vector made up of ‘=’, ‘≤’, ‘≥’and non-binding signs); and the b are the constant right hand sides (RHS) of theconstraints. So by removing the global constraints I0, the problem decomposes intoK independent problems with constraints

Bkxk Tk bk.

The global constraints are sometimes referred to as common, linking or masterconstraints, and include the objective function.

In practice, identifying a suitable block structure requires a little work. It isbest to work with the model file, since it is in the model that the structures areapparent. But it is not simply a case of partitioning out the sets of variables andconstraints as they appear in the model file. Indeed, it is normally the case thateach set of variables or constraints as defined in the model needs to be broken upand partitioned among the different blocks according to the indices of variable orconstraint sets.

To find a suitable block structure, look for natural self contained blocks in themodel. For instance, you may have a time-stage model in which the individualtime stages are independent apart from relatively few global constraints, such aslinking constraints to maintain inventory from one period to the next, or globalconstraints on the total amount of some scarce resource available over all timeperiods. Taking out these global constraints (labelled I0 above), you will be leftwith a series of independent problems, i.e., all variables and constraints refer toone and only one time period. Put these variables and constraints into subsetsJ1, . . . , JK and I1, . . . , IK according to the time period that they refer to. Finally

C.6. PREPARING THE PROBLEM DATA FOR DECOPT 189

if there are any variables that refer only to the global constraints, label them masterproblem variables and put them into subset J0.

Block structure may arise out of many different natural partitions in the modelother than time stages, such as, for instance, through considering different plantsor locations, or different products. Typically a model will contain several suchsuitable block structures. Often a good technique for identifying them is to lookfor an index, such as one representing time periods or plants, that is shared bymany of the variables and constraints in the problem. You can then define the kth

block as follows: the variables Jk are those that refer to the kth index (e.g., the kth

time period); and the constraints Ik are those that refer only to the variables Jk,i.e., constraints that include only variables with the kth index (e.g., the kth timeperiod). All remaining variables J0 and constraints I0 make up the master problem.

Sometimes a problem will naturally decompose along different variable sets andconstraint sets in the model. This is the case when a problem can be thought ofas two or more different models joined by a few constraints. It is important toemphasise that there is no single correct decomposition, and a problem may bedecomposed in several ways, all valid for Dantzig-Wolfe decomposition. Formallya valid block structure for Dantzig-Wolfe decomposition is the following. Eachpricing problem k must comprise a set of variables Jk, and a set of constraints Ikthat refer only to variables in Jk. All variables J0 and constraints I0 not includedin any pricing problem make up the master problem.

You might ask what makes a good decomposition. One could say that it de-pends solely on the performance of the decomposition algorithm to solve differentinstances of the problem decomposed in that way. For the decomposition algorithmto be of any practical use, it is important that the time taken to solve the pricingproblems per algorithm cycle be an order of magnitude greater than the time takento solve the master problem, complete with some proposals, per algorithm cycle.Of course this measure will only become apparent after trying out the algorithmon the problem.

More usefully there are some practical guidelines. Certainly you should aimfor relatively few global constraints—the fewer the better. You should also try toconstruct the pricing problems so that they are well formed problems in their ownright, for it is by solving the pricing problems rather than the master problem thatdecomposition makes a saving.

C.6 Preparing the Problem Data for DecOpt

In this section we describe the matrix files required by DecOpt, given a problem witha valid primal block angular structure already identified. The DecMod tool has beendeveloped to automate this process (see the DecMod User Manual in appendix D),so it is unlikely you will need to construct the matrix files yourself. However, if youare constructing the matrix files in some other way, or wish to understand whatDecMod must produce, you will find this section useful.

Given a valid block structure for a particular problem, as identified by the setsI0, I1, . . . , IK of constraints and J1, J2, . . . , JK of variables, it is a simple matter todefine the matrices that should be prepared for DecOpt. DecOpt requires a separatematrix file for the master problem and each of the K pricing problems.

The master problem matrix is:


J0 Type RHSI0: A0 T0 b0

Here A0 is the block of matrix coefficients for the master variables J0 in the globalconstraints I0; T0 is the vector of constraint types for the global constraints; andb0 is the RHS vector for the global constraints. The objective function must beincluded as the first global constraint.

It is very important that all of the global constraints I0 are included in themaster problem matrix file, even if there are no matrix coefficients. You can ac-complish this easily in an MPS matrix file by including each global constraint inthe ROWS section, even if it is not referenced in the COLUMNS section.

The matrix for pricing problem k is:

Jk Type RHSI0: Ak T0 b0Ik: Bk Tk bk

Here Ak is the block of matrix coefficients for variables Jk in the global constraintsI0; Bk is the block of matrix coefficients for variables Jk in constraints Ik; Tk isthe vector of constraints types for the constraints Ik; and bk is the RHS vectorfor constraints Ik. The type and RHS of the global constraints I0 in the pricingproblem matrix files are actually irrelevant—DecOpt automatically converts theseconstraints to be non-binding with zero RHS when the matrices are input—but toprevent any ambiguity, use the type and RHS of the global constraints from theoriginal or master problem.

Again, it is very important that all of the global constraints I0 are includedin each pricing problem matrix file (even if a particular global constraint has nomatrix coefficients in a pricing problem), that they are the first constraints in thematrix file, and are given in the same order as the master problem matrix file.DecOpt simply uses the row numbers to identify the global constraints, and doesnot check the row names.

Place the matrices in separate MPS or LP matrix files, with .mat, .mps or .lpsuffices. You may include upper and lower bounds. The decomposition algorithmsolves LP problems only; thus if you include any MIP specifications, they will beused merely to generate a LP relaxation.

C.7 Starting DecOpt

You must supply one mandatory argument to start DecOpt:

decopt problem

This argument, the problem name, is used as a basis for the names of most problemspecific files used by DecOpt. It may include a drive and/or directory specification(the directory component) following the syntax of the operating system on yourcomputer. With the exception of names for files related to individual matrices, allfilenames are constructed automatically by appending a suffix to the problem name.For instance, DecOpt will first look for the initialisation file problem.ini and willsend output to the log file problem.log. Thus if problem includes a drive and/ordirectory component then all filenames will refer to that drive and directory. Files

C.8. CONTROLLING DECOPT 191

pertaining to individual matrices have names that distinguish between the matrices,as explained in section C.12 below.

You can also supply any number of optional augments to DecOpt to set controlvariables:

decopt problem -control=value

The control name is preceded by a minus sign, and no space is allowed after theminus sign or around the equals sign. The next section gives details of optionalarguments.

C.8 Controlling DecOpt

You control the behaviour of DecOpt using control variables. With a few exceptions,noted below, these are pre-set to default values that have been found to give bestresults on a variety of problems. However, you must set certain key controls, and itis sometimes desirable to tune some of the other controls for a specific application.

There are several ways to set control variables. Initially they are set to defaultvalues, as given section C.10. DecOpt then checks for initialisation files,

decopt.ini (in the current working directory)problem.ini

and, if they exist, control variables listed in the files are set to the new valueindicated. Those in problem.ini take precedence. Next, any optional argumentspassed when calling DecOpt are processed, which take precedence over previoussettings. Finally, the values of certain control variables are checked and altered ifnecessary. This last procedure applies particularly to the variables controlling thesize of the proposal buffers, which are automatically reset if they have been left atzero. See section C.9.2 below for details.

An initialisation file is a text file comprising separate lines of the form:

control=value

or, when no value is required, simply:

control

There must be no space on either side of the equals sign. The control variablesrequiring no value are listed as having a void type in (section C.10).

Control variables supplied as optional arguments when calling DecOpt take asimilar format, the only difference being that the control name is preceded by aminus sign:

decopt problem -control=value -control

As command line options take complete precedence, they are especially useful whenexperimenting to override normal settings listed in one of the initialisation files.

You must use the controls to set the number of problems and the sense ofoptimisation. For instance, a basic initialisation file to do this might be:

nprob=6max


This states that the complete problem is made up of 6 separate problems, i.e., amaster problem and 5 pricing problems, and that the problem is to be maximised.

Alternatively, the same effect can be achieved using command line options asfollows:

decopt problem -nprob=6 -max

A complete list of control variables is given in section C.10.

C.9 Features

C.9.1 Proposal Generation

Generating multiple proposals from each pricing problem on each cycle often re-sults in a smaller number of decomposition cycles, and a reduced time to solvethe problem. Multiple proposals are generated from basic feasible points encoun-tered by the simplex algorithm before the optimal solution of the pricing problemis reached. Proposals generated before an optimal solution is obtained are interme-diate proposals; those generated at an optimal or unbounded solution are terminalproposals. However, generating too many intermediate proposals can cause its ownproblems—lots of similar proposals in the master problem offers little benefit inreducing the number of cycles, and may hinder progress by increasing the amountof work required to solve the master problem and creating numerical instability inthe master problem.

We use a set of heuristics called a proposal generation strategy to generate mul-tiple proposals whilst controlling the number and similarity of the proposals to limitill effects. DecOpt supports several mechanisms for controlling multiple proposalgeneration: limiting the number of proposals generated from each pricing problem;preventing two proposals being generated within a minimum number of simplexpivots; preventing a proposal being generated without a minimum percentage im-provement in the reduced cost over the previous proposal, both in order to avoidsimilar proposals; limiting the number of proposals generated from all pricing prob-lems during one cycle; and limiting the number of pricing problems solved duringone cycle. These mechanisms are controlled by a set of parameters, which allowthe user to control each mechanism. A proposal generation strategy is specified byvalues for the control variables, which we indicate here in typewriter style.

A check is made to generate a proposal every propfreq pivots during the opti-misation of a pricing problem, and also when the final optimal solution is attained.(A value of zero for propfreq disables the generation of intermediate proposals.) Aproposal is generated if the solution is feasible, and the reduced cost of the solution,when viewed as a prospective variable in the master problem, is below the reducedcost criterion propcostmax. Each time a proposal is generated, propcostmax isincreased by propcost%. (A value of zero for propcost disables the increase.)Proposal generation continues until optimality is obtained, or propgcol proposals,of total length (number of non-zero coefficients) propgelt, have been generated.The pricing problem is always solved to optimality (unless unbounded), so that thefinal objective value can be used in calculating a new dual bound if possible.

The proposal generation frequency, propfreq, is initialised with a commonvalue and then varies independently for each pricing problem using our own in-novative scheme. We define the observed proposal frequency to be the number of

C.9. FEATURES 193

simplex pivots taken to re-optimise the pricing problem after the first proposal isgenerated, divided by propgcol − 1, where propgcol is the number of proposalsallowed from a single pricing problem. It is the value of propfreq that would haveresulted in propgcol proposals being generated evenly during re-optimisation ofthe pricing problem. At the end of each cycle propfreq is updated using the mostrecent observed proposal frequency. A decay factor weights the influence of themost recent frequency with past frequencies. The new value of propfreq is:

(1− propfreqdcay)× propfreq

+ propfreqdcay× (observed proposal frequency).

A value of zero for propfreqdcay means that propfreq will remain constant; whilsta value of one means that propfreq will be updated each cycle with no memory.Intermediate values give combinations of these two extreme policies. An additionalcontrol, propfreqmin, allows a minimum value for propfreq to be specified.

If a pricing problem is found to be unbounded, a different mechanism is em-ployed to generate multiple proposals. There are often many unbounded rays em-anating from the same basic solution, or within close proximity, and generating asingle unbounded ray in this case means it can take several cycles to fathom theunboundedness. Hence, once an extreme ray proposal has been generated, we fixthe vector causing the unboundedness to zero, and resume optimisation and pro-posal generation. Several unbounded vectors may be found and fixed to zero in thisway. At the end of the cycle, the fixed vectors are freed up again.

Each pricing problem is considered in turn, until cycsubmax pricing problemshave been solved, or the total number or length of proposals generated during thecycle exceeds propicol and propielt respectively. (A value of zero for cycsubmaxmeans all pricing problems are solved on each cycle.) In parallel DecOpt, these limitsapply independently to the set of pricing problems held on each slave process. Onthe next cycle, the pricing problem phase resumes at the next pricing problem insequence, so that the pricing problems get equal treatment on average.

At the beginning of the algorithm, one proposal is generated for each pricingproblem before entering the column generation algorithm proper, by solving thepricing problem with its original objective function. This gives some useful side-effects: the feasibility of each pricing problem is checked at the beginning, andthe proposals generated are used as part of a simple proposal crash, by makingeach basic on its respective convexity row, in place of the convexity row artificialvariable.

C.9.2 Proposal Buffers

The proposal generation buffer, or g-buffer, stores the proposals generated from onepricing problem in a cycle until work finishes on that pricing problem. Proposals arethen transferred to the intermediate proposal buffer, or i-buffer, which holds themuntil the pricing problem phase of a cycle is finished. Proposals are then loadedinto the master problem matrix, where the space designated for that purpose isknown as the master proposal buffer, or m-buffer. All buffers hold the proposals insparse column-wise format. In parallel DecOpt, the proposals collected in the locali-buffer on a slave process are sent to the i-buffer on the master process at the endof a cycle, before being loaded into the master problem matrix.


The size of these buffers thus determines how many proposals are generatedand held during each cycle of the algorithm. For normal operation, the prescribedrules will set the sizes automatically. Sometimes, the average number of proposalelements allowed per proposal is too high, and occupies excessive memory. If youknow that a proposal will have a certain number or certain percentage of non-zeroelements, use the propnelt or propdens controls to limit the space set aside forproposal elements. This is particularly important in problems with many globalconstraints.

The size of the master proposal buffer is normally determined by the purgefrequency, the non-basic proposal purge criterion and the size of the intermediateproposal buffer. An additional control, propbas, allows additional space for a fixednumber of proposals, to allow for a basis consisting of proposals only. This controlnormally takes the value of the number of constraints in the master problem, butit may be necessary to increase this value if the master proposal buffer fills up intrial runs.

The sizes of all buffers may also be set directly, using the controls propgcol,propicol, propmcol, propgelt, propielt and propmelt. See section C.10 fordetails of all these controls and the automatic rules used to set them.

C.9.3 Proposal Management

As the column generation algorithm proceeds, proposals are added to the restrictedmaster problem in order to build up a representation of the full master problem ona path to its optimal solution. If proposals are deleted, there is a chance that thesame proposal will have to be regenerated in a later cycle, thus delaying progress.Conversely, progress can also be delayed by the presence of many proposals in themaster problem, as they can be expensive to price out. (Proposals are typicallydense, much more dense than columns in practical LPs.)

As with proposal generation, DecOpt provides a set of parameters to define aproposal management strategy, a strategy for managing the persistence of proposalsin the master problem. A check is made every purgfreq cycles to delete (or purge)proposals from the master problem, or whenever the number and length of propos-als in the master problem, including those just generated, exceeds propmcol andpropmelt respectively. Although there is no guarantee that a proposal will not berequired again, we offer two methods to measure redundancy: the number of cyclesthat a proposal has been non-basic, and the reduced cost of a proposal. A proposalis condemned if it has been out of the basis for purgnbas or more cycles, and itsreduced cost is above a criterion level, purgcostmax. (Either test may be disabled.)Proposals that are basic or generated during the previous pricing problem phaseare not considered for deletion. The purge is repeated with relaxed criteria untilthe number and length of proposals remaining are within the specified limits.

C.9.4 Algorithm Termination

The algorithm terminates when one of the following events occurs:

1. No proposals are generated on a cycle, which proves that the current solutionfor the master problem is optimal, or, if infeasible, no feasible solution exists.

2. The master problem is unbounded.

C.9. FEATURES 195

3. A pricing problem is infeasible.

4. Once a feasible solution has been found, the algorithm generates both primaland dual bounds as it progresses, with the optimal value lying between thetwo bounds. The difference between the two bounds is called the dualitygap. The algorithm halts once the duality gap is less than optabs or if thepercentage duality gap divided by the primal value is less than optrel.

5. The decomposition cycle (maxcyc) or time (maxtim) limits are reached.

6. You can interrupt optimisation by pressing control-C. DecOpt continues untila convenient place is found to halt, and then saves its state to file, so thatoptimisation can be resumed at a later session (see below). Pressing control-C again results in immediate termination and the creation of a save file isaborted.

Setting the save control causes the state of DecOpt to be saved to file when ithalts in any of the latter three cases, so that optimisation can be resumed in alater session. All information relevant to the state of the optimiser is written to asave file: either problem.jbs (binary save file format) or problem.jas (text savefile format). Use the restore control to resume optimisation from a save file.Together with the save file, you will need the original matrix files and any relevantinitialisation files or command line options: the matrix data and control values arenot included in the save file. Control values do not have to be set to their previousvalues when resuming optimisation.

C.9.5 Solution Output

Provided that a feasible solution is found and the isol control is set, the finalsolution is recorded in a number of standard xpress-mp solution files, one for eachmatrix, with suitable extensions according to the type of file. The isol control is abitmap so that any combination of solution files may be produced. Simply add upthe values to obtain the solution files that you require: ASCII solution files (.asc);formatted print solution files (.prt); and basis files (.bss). A value of zero meansno solution files are produced and the final dual feasible solution is not convertedinto a primal feasible solution (phase 3 ). Please consult the Xpress-MP ReferenceManual for details of these solution files.

C.9.6 Message Output

The output of DecOpt is designed for the practitioner and expert alike: a compre-hensive set of messages are graded into levels, and the level of messages outputto the console and log file can be varied independently to suit your needs. Themessages allow you to analyse the operation of DecOpt, especially those areas likeproposal generation where different strategies can be applied. In addition, a set ofperformance statistics are output on demand.

The message output includes a listing of the values of all the control variables,summary statistics of the problem as input, an optimisation log showing progresstowards a solution, the result of the problem, and statistics summarising the per-formance of the algorithm. This output is directed both to the screen (console) andto the log file, with filename problem.log. The amount of information displayed


can be varied by adjusting the two message level controls msgcon and msglog, forthe screen (console) and log file respectively.

The output levels include

−1 no output0 banner, error messages1 warnings2 controls, problem statistics, decomposition cycle log, solution sum-

mary, performance statistics3 decomposition pricing problem log4 proposal generation log5 proposal magnitude summary, proposal management log6 solver optimisation log7 proposal check log8 progress messages9 full xpress-mp output

Values of 10 and above provide debugging information. Caution should be taken ifusing them as the amount of output can be excessive.

C.9.7 Parallel DecOpt

The parallel implementation of DecOpt uses the coarse grained distributed paral-lelism offered by the independence of the pricing problems in the pricing problemphase. The master process, the DecOpt program which you start up, itself initiates anumber (nslave) of slave processes, which run a cut down version of DecOpt calledDecSlave. The master process allocates the pricing problems evenly among theslaves and itself, with a slight bias towards the slave processes. The master processretains the master problem, and co-ordinates the slave processes by broadcastingthe prices obtained by solving the master problem, and receiving the proposalsgenerated on the slave processes in return. Thus the parallelism springs directlyfrom the algorithm itself, and echoes the decentralised planning methods proposedfor the decomposition algorithm when it was first devised.

It is clear that the maximum saving obtainable using this algorithmic paral-lelism is equal to the total time taken to solve all of the pricing problems in sequence,less the greatest time taken to solve any one of the pricing problems, summed overcycles. If there are fewer processors than pricing problems, then the potential sav-ing can be calculated in a similar way. The time taken to solve a problem with pprocessors can be approximated as

Tp = tm + dk/pets

where tm is the average time taken to solve the master problem on each cycle, tsis the average time taken to solve one pricing problem on each cycle, and k is thenumber of pricing problems. This ignores the overhead due to parallel optimisationand the possibility that the decomposition algorithm may take a different path tothe optimal solution.

C.10. DIRECTORY OF CONTROLS 197

The speed-up and efficiency are then

Sp = T1/Tp

=1 + kts/tm

1 + dk/pets/tmEp = Sp/p

=1 + kts/tm

p+ pdk/pets/tm

≈ 1 +R

p+R

where R = kts/tm. The final approximation ignores the granularity of the paral-lelism. Observe that if R� 0, the efficiency will be close to one and the speed-upnear linear, whereas if R ≈ 0, the efficiency Ep ≈ 1/p and the speed-up non-existent. So based on the results of a serial run, you can judge if a problem is likelyto benefit from parallel optimisation.

C.10 Directory of Controls

The full set of controls for DecOpt are listed in the table below. The first columnof the table gives the name of the control to be used in an initialisation file or onthe command line. The second column lists the type of value that the variabletakes: void means that the variable takes no value; otherwise the value may be ashort—integer, up to ±32000; long—integer, up to ±2×109; double—real number,up to ±1037; or character string as indicated. The third column gives the defaultvalue of the variable. The final column gives a description of the control variable.See section C.9 for a more extended description of DecOpt features.

nprob [short; default value 0] Number of matrices to be input; equals number ofpricing problems plus one. You must use this control to set the number ofmatrices in your problem.

nslave [short; default value 0] Number of slave processes in parallel optimisation.Set to zero for serial optimisation.

min [void; default value off] Minimise the problem.

max [void; default value off] Maximise the problem.

save [void; default value on] Save algorithm status to file if unfinished when termi-nated, to resume in a later session (using the restore control). This controlis automatically switched on if optimisation is interrupted by Ctrl-C. Thealgorithm status cannot be saved when parallel optimisation is used.

restore [void; default value off] Restore algorithm status from file (created usingthe save control) and resume optimisation. The algorithm status cannot berestored when parallel optimisation is used.

savfmt [short; default value 0] Save file format. 0: binary (.jbs); 1: text (.jas).

assemble [void; default value off] Assemble and output full problem matrix fromthe individual matrix files and exit without optimising.


msglog [short; default value 3] Message level for log file.

msgcon [short; default value 2] Message level for the console (screen output).

matfmt [short; default value 0] Matrix input file format. 0: text (.mat, .mps or.lp); 1: binary (.bif).

xp frefmt [long; default value xpress-mp default] xpress-mp MPS matrix fileformat. 0: fixed; 1: free; −1: automatic detection. (Refer to the FREFMTcontrol in the Xpress-MP Reference Manual.)

matnam [string; default value empty] Matrix file name, to be used in place of theproblem name file component. The directory component of the problem nameis automatically prefixed, and a matrix identifier and matrix file suffix areautomatically appended. If it is empty after processing all control variables,the filename component of the problem name is used. See section C.12.

icrash [short; default value 3] Crash control bitmap. 0: none; 1: master convexityconstraint crash; 2: master global constraint crash.

iscale [short; default value 18] Scaling control bitmap. 0: none; 1: proposal col-umn scaling; 2: convexity row scaling; 16: re-scale complete master problemmatrix after proposal purge.

maxcyc [long; default value 1 000 000] Maximum number of Dantzig-Wolfe algo-rithm cycles.

maxtim [long; default value 43 200] Maximum time, in seconds. Note 43 200 seconds= 12 hours.

optrel [double; default value−1.0] Percentage duality gap criterion for ε-optimality.The decomposition algorithm generates primal and dual bounds as it pro-gresses; the difference between the two bounds is the duality gap. Optimisa-tion halts when the percentage duality gap falls below the value of optrel.The test is disabled if the value is negative.

optabs [double; default value −1.0] Absolute duality gap criterion for ε-optimality.The decomposition algorithm generates primal and dual bounds as it pro-gresses; the difference between the two bounds is the duality gap. Optimi-sation halts when the absolute duality gap falls below the value of optabs.The test is disabled if the value is negative.

isol [short; default value 0] Compute and output full primal solution to file(s)indicated by bitmap. 0: no primal solution generated; otherwise output to 1:ASCII solution files (.asc); 2: formatted print solution files (.prt); 4: basis files(.bss). The primal solution cannot be computed when parallel optimisationis used.

istats [short; default value 1] Produce performance statistics. 0: off; 1: on.

cycsubmax [short; default value 0] Maximum number of pricing problems to solveeach cycle. A value of 0 means all pricing problems will be solved (unlessother controls prevent this).

C.10. DIRECTORY OF CONTROLS 199

cycdual [short; default value 1] Solve pricing problems to optimality each cycle,to enable a new dual bound to be generated. If this control is switched off,optimality of a pricing problem will halt when a full quote of proposals hasbeen generated. It is automatically enabled if the xpress-mp presolve isused. 0: off; 1: on.

propfreq [long; default value 0] Initial proposal generation frequency. A value ofzero means that intermediate proposals are never generated.

propfreqmin [short; default value 10] Minimum proposal generation frequency.

propfreqdcay [double; default value 0.5] Proposal generation frequency exponen-tial decay factor. A value of 0 sets the proposal generation frequency of apricing problem to be the observed proposal generation frequency of the samepricing problem on the last cycle, i.e., historical values are ignored. As thevalue increases from 0 to 1, more weight is given to historic values over recentobserved values. A value of 1 means the frequency stays at its initial value,i.e., the observed value is ignored.

propcostmax [double; default value 0.0001] Maximum reduced cost of proposals.Only proposals with reduced cost lower than this value will be generated.(The reduced costs in maximisation problems are negated, so that profitablereduced costs are always negative.)

propcost [double; default value 0.0] Proposal generation percentage cost factor.Each time a proposal is generated, the maximum reduced cost is updatedto be the reduced cost of the proposal plus this percentage. The maximumreduced cost is reset for each pricing problem on each cycle. A negative valuedisables this control.

propfixunb [short; default value 1] Fix unbounded proposals to zero in a pricingproblem and continue generating proposals. The fixed proposals are freed atthe end of the cycle.

propelttol [double; default value 10−8] Zero tolerance on proposal coefficients.Coefficients with an absolute value less than this value are disregarded.

propdens [short; default value 100] Average percentage density of non-zero pro-posal coefficients. This control may be used to size the proposal buffers.

propnelt [long; default value 0] Average number of proposal coefficients per pro-posal. This control may be used to size the proposal buffers. If 0, it will beset to (the number of rows in a proposal) × propdens.

propbas [long; default value 100] Number of basic proposals in the master problem.This control helps to size the proposal buffer in the master problem. Spaceis allocated for basic proposals plus a quota of new proposals remaining atthe last purge and those generated since the last purge. If 0, it will be set tothe number of rows in the master problem.

purgfreq [short; default value 4] Purge frequency. Additional purges may be per-formed if space is exhausted in the master problem proposal buffer.

purgnbas [short; default value 1] Non-basic proposal purge criterion. When purg-ing proposals, consider first those that have been non-basic for at least thisnumber of cycles. A value of zero disables this purge criterion. The valuemay be temporarily lowered if enough space cannot be released.


purgcostmax [double; default value 0.0] Reduced cost proposal purge criterion.When purging proposals, consider first those that have a reduced cost largerthan this value. (The reduced costs in maximisation problems are negated, sothat unprofitable reduced costs are always positive.) A value of zero disablesthis purge criterion. The value may be temporarily lowered if enough spacecannot be released.

propgcol [long; default value 2] Maximum number of proposals in proposal gen-eration buffer.

propgelt [long; default value 0] Maximum number of elements in proposal gener-ation buffer. If 0, it will be set to the propgcol × propnelt.

propicol [long; default value 0] Maximum number of proposals in proposal inter-mediate buffer. If 0, it will be set to cycsubmax × propgcol.

propielt [long; default value 0] Maximum number of elements in proposal inter-mediate buffer. If 0, it will be set to propicol × propnelt.

propmcol [long; default value 0] Maximum number of proposals in the masterproblem. If 0, it will be set to propbas + (purgnbas + purgfreq + 25) ×propicol.

propmelt [long; default value 0] Maximum number of proposal elements in themaster problem. If 0, it will be set to the propmcol × propnelt.

dualtol [double; default value 10−12] Zero tolerance on dual variables. Dual vari-ables with an absolute value less than this value are disregarded when revisingthe objective function.

optgaptol [double; default value 10−6] Zero tolerance on the objective value. Therelative duality gap is not calculated if the gap between the primal objectivevalue and dual bound includes a value closer to zero than optgaptol.

allocrhstol [double; default value 10−6] Zero tolerance on proposal activities.Proposal activities with an absolute value less than this value are disregardedwhen allocating RHS of global constraints.

lpmaster1 [short; default value 3] xpress-mp algorithm control for use when opti-mising an infeasible master problem. 1: xpress-mp default; 2: dual simplex;3: primal simplex; 4: Newton barrier.

lpmaster2 [short; default value 3] xpress-mp algorithm control for use when op-timising a feasible master problem. 1: xpress-mp default; 2: dual simplex;3: primal simplex; 4: Newton barrier.

lpsub [short; default value 3] xpress-mp algorithm control for use when optimisinga pricing problem. 1: xpress-mp default; 2: dual simplex; 3: primal simplex;4: Newton barrier.

xp ifpres [long; default value 0 xpress-mp presolve control. 0: off; 1: on. (Referto the IFPRES control in the Xpress-MP Reference Manual.)]

xp iphsf [long; default value −1] xpress-mp pricing control. −1: use partialpricing; 0: automatic choice; 1: use devex pricing. (Refer to the IPHSFcontrol in the Xpress-MP Reference Manual.)

C.11. ERROR MESSAGES 201

xp ifscal [long; default value xpress-mp default] xpress-mp matrix scaling con-trol bitmap. (Refer to the IFSCAL control in the Xpress-MP Reference Man-ual.)

xp fastopt [long; default value 0] xpress-mp re-inversion control. 0: always re-invert matrix at re-optimisation; 1: use existing basis representation if pos-sible.

xp prtmsg [long; default value xpress-mp default] xpress-mp message output. 0:disabled; 1: enabled.

xp itrlog [long; default value xpress-mp default] xpress-mp simplex log fre-quency for the master problem. A log is displayed every xp itrlog sim-plex iterations when solving the master problem, in addition to the standardDecOpt cycle log.

C.11 Error Messages

If an error occurs, DecOpt halts with an error number and message. Error num-bers over 1000 indicate that the error occurred inside xpress-mp. Provided thatxp prtmsg= 1, an accompanying xpress-mp error message will be printed. Non-drastic xpress-mp errors will give an xosl function return code of 1032 togetherwith a more informative xpress-mp error code. xosl function return codes otherthan 1032 signal serious xpress-mp errors such as dongle, security or memoryerrors.

If you cannot remedy the error then please contact the author and report therelevant error numbers.

C.12 Matrix File Names

Most files used by DecOpt have names derived from the problem name by appendinga particular file suffix (see section C.7). Files specific to the individual matrices—the master problem matrix and each pricing problem matrix—are exceptions to thisrule because more than one file is required. For instance, there must be a matrixfile to input each matrix in the problem (each with a suffix .mat), and any solutiongenerated can be saved to separate xpress-mp solution files for each matrix, suchas ASCII files, with extension .asc, formatted print files, with extension .prt, and/orbasis files, with extension .bss.

DecOpt constructs the names of matrix-specific files by concatenating the prob-lem name with a matrix identifier number, plus of course a suitable file suffix suchas .mat, .asc, .bss, etc:

problem matrix identifier file suffix

The matrix identifier is in hex and runs in sequence: 0, 1, 2, . . . , 9, a, b, . . . , e,f, 10, 11, . . . , 19, 1a, 1b, . . . , ff. The master problem has identifier 0; the pricingproblems have the identifiers 1, 2, 3, etc.

For example, suppose DecOpt was called as follows:

decopt dyna


the names of the matrix files are constructed as follows:

dyna0.matdyna1.matdyna2.matdyna3.mat. . .

These will all be looked for in the local directory. If instead DecOpt was called:

decopt c:\mats\dyna

then the matrix files sought are:

c:\mats\dyna0.matc:\mats\dyna1.matc:\mats\dyna2.matc:\mats\dyna3.mat. . .

and indeed all files DecOpt deals with will be located in the directory c:\mats.It is also possible to name the matrices differently from the problem name. If

you set the control matnam (by default it is an empty string), matrix-specific filenames are specified thus:

directory name matrix name matrix identifier file suffix

where matrix name is the value of the control matnam. The directory name com-prises the drive and/or directory specification in the problem name (if any). (Inother words, the value of matnam is used to replace the filename component of theproblem name.)

As an example, suppose we call DecOpt with:

decopt c:\mats\dynamic -matnam=dyna

the initialisation file and log file have filenames

c:\mats\dynamic.inic:\mats\dynamic.log

while the matrix filenames are

c:\mats\dyna0.matc:\mats\dyna1.matc:\mats\dyna2.matc:\mats\dyna3.mat. . .

The ability to redefine matnam is useful if the problem name is too long to accom-modate the hex suffix, or if you wish to clearly distinguish the suffix, for instance,by including an underscore ‘ ’ in the matrix names, or even if you wish to store thematrices in a different directory to the initialisation file.

Appendix D

DecMod User Manual

For completeness, this manual repeats some material from the main body of thethesis.

D.1 Introduction and Background

DecMod is a tool to help exploit block structure in a linear programming (LP) prob-lem. It takes a standard algebraic formulation of a problem written in the xpress-

mp Modelling Language, complete with data or references to data in external datasources, and a specification of the block structure. The block structure (decomposi-tion) specification is supplied in a separate text file in a format that allows a naturaland concise specification of an arbitrary block structure in a matrix. It also uses asimilar notation to the formulation of the problem in xpress-mp, making it easyfor you to write decomposition files. DecMod picks out the various blocks identifiedin the problem in preparation for the application of a decomposition algorithm tothe problem. DecMod thus allows you to concentrate on identifying suitable blockstructures, and to be able to quickly and easily assess the application of decompo-sition solvers, without spending time and effort extracting the block structure fromproblems. It removes one of the hurdles towards applying decomposition and otherblock structure exploitation algorithms as general solution techniques.

The motivation for DecMod arose from the need to automate the process of con-structing the master problem and pricing problem matrices required for an imple-mentation of the Dantzig-Wolfe decomposition algorithm DecOpt (see the separateDecOpt User Manual). We wanted to experiment with different block structureswith some of the problems we had available in standard xpress-mp model formu-lations. The simplest way to supply a problem to our Dantzig-Wolfe optimiser isin the form of separate matrices for the master problem and each of the pricingpricing problems, and it soon became apparent that constructing these matriceswas a non-trivial task, which would prove a serious handicap if decomposition wereto be applied on anything more than an ad hoc scale.

With toy problems, it is easy enough to construct the matrices by hand by edit-ing the matrix for the complete problem. Alternatively, the algebraic formulationfor the original problem (often loosely called the model) may be manipulated tomake it produce the various matrices in turn. However neither of these methodscan be applied to large commercial models because the amount of manual work

203

204 APPENDIX D. DECMOD USER MANUAL

Figure D.1: DecMod input and output.

required is time consuming, difficult to automate and error prone. Fortunately, atabout the same time, Dash Optimization were developing xpress-emosl, a com-bined modelling and optimisation subroutine library, which provided the base forwhat was to become DecMod. emosl incorporates sophisticated means to access thevarious entities in the problem, i.e., the data, variables and constraints, using thenotation of the model. It also provides advanced facilities to manipulate matrices.emosl underlies DecMod and is called upon to pull out specified entities from theoriginal problem, so that they can be organised blocks and output in a form suitablefor a decomposition algorithm code.

DecMod is now much more general than originally implemented, and is notlimited to the primal block angular structure associated with Dantzig-Wolfe de-composition. You may specify any block structure, such as the dual block angularstructure associated with Benders decomposition, staircase structures, nested blockstructures, such as those exploited by nested Dantzig-Wolfe or nested Benders de-composition, and hybrid structures exploited by cross or primal-dual decomposition.

The output of DecMod can be varied to suit your needs. DecMod has beendesigned to communicate block structure to optimisers. This can be done eitherby producing separate matrices for each block, or by producing a matrix for thecomplete problem in which the rows and columns are ordered by block, or byindicating (by means of a secondary file) in which block each of the rows andcolumns of the standard matrix for the complete problem lie. DecMod can outputthe block structure in any one of these ways and it should be relatively simple tosupport other formats by modifying the export routines. The default behaviour is toproduce separate matrices, and it this behaviour that is discussed in the examples.

The structure of this manual is as follows. We first illustrate the use of DecModby means of an extended example. This gives the best description of what DecModdoes and how you use it. The remainder of the manual features a comprehensivereference to DecMod, to which you should refer for specific details.

D.2 Tutorial: Dantzig-Wolfe Decomposition

In this section we present a detailed example of the use of DecMod, showing howseveral block structures suitable for Dantzig-Wolfe decomposition (primal blockangular structures) can be extracted from a model in readiness for the applicationof a Dantzig-Wolfe optimiser.

D.2. TUTORIAL: DANTZIG-WOLFE DECOMPOSITION 205

Figure D.2: Primal block angular structure in an LP problem.

D.2.1 Theoretical Background

Dantzig-Wolfe decomposition is suitable for problems where, if a relatively smallnumber of constraints were removed, the problem would fall apart into a numberof independent problems. In other words, it is possible to re-order the rows andcolumns in the coefficient matrix such that, apart from a few rows, the matrix isblock diagonal. Thus, after re-ordering, the matrix can be expressed as shown infigure D.2, where the constraints (rows) are partitioned into global constraints I0and blocks of constraints I1, I2, . . . , IK ; and the variables (columns) are partitionedinto subsets J0, J1, . . . , JK . A and B are the blocks of non-zero matrix coefficients;the T are the vectors of the constraint types (each vector made up of ‘=’, ‘≤’, ‘≥’and non-binding signs); and the b are the constant right hand sides (RHS) of theconstraints. So by removing the global constraints I0, the problem decomposes intoK independent problems with constraints

Bkxk Tk bk.

The global constraints are sometimes referred to as common, linking or masterconstraints, and include the objective function.

In practice, identifying a suitable block structure requires a little work. It isbest to work with the model file, since it is in the model that the structures areapparent. But it is not simply a case of partitioning out the sets of variables andconstraints as they appear in the model file. Indeed, it is normally the case thateach set of variables or constraints as defined in the model needs to be broken upand partitioned among the different blocks according to the indices of variable orconstraint sets.

To find a suitable block structure, look for natural self contained blocks in themodel. For instance, you may have a time-stage model in which the individualtime stages are independent apart from relatively few global constraints, such aslinking constraints to maintain inventory from one period to the next, or globalconstraints on the total amount of some scarce resource available over all timeperiods. Taking out these global constraints (labelled I0 above), you will be leftwith a series of independent problems, i.e., all variables and constraints refer toone and only one time period. Put these variables and constraints into subsetsJ1, . . . , JK and I1, . . . , IK according to the time period that they refer to. Finallyif there are any variables that refer only to the global constraints, label them masterproblem variables and put them into subset J0.


Block structure may arise out of many different natural partitions in the modelother than time stages, such as, for instance, through considering different plantsor locations, or different products. Typically a model will contain several suchsuitable block structures. Often a good technique for identifying them is to lookfor an index, such as one representing time periods or plants, that is shared bymany of the variables and constraints in the problem. You can then define the kth

block as follows: the variables Jk are those that refer to the kth index (e.g., the kth

time period); and the constraints Ik are those that refer only to the variables Jk,i.e., constraints that include only variables with the kth index (e.g., the kth timeperiod). All remaining variables J0 and constraints I0 make up the master problem.

Sometimes a problem will naturally decompose along different variable sets andconstraint sets in the model. This is the case when a problem can be thought ofas two or more different models joined by a few constraints. It is important toemphasise that there is no single correct decomposition, and a problem may bedecomposed in several ways, all valid for Dantzig-Wolfe decomposition. Formallya valid block structure for Dantzig-Wolfe decomposition is the following. Eachpricing problem k must comprise a set of variables Jk, and a set of constraints Ikthat refer only to variables in Jk. All variables J0 and constraints I0 not includedin any pricing problem make up the master problem.

You might ask what makes a good decomposition. One could say that it de-pends solely on the performance of the decomposition algorithm to solve differentinstances of the problem decomposed in that way. For the decomposition algorithmto be of any practical use, it is important that the time taken to solve the pricingproblems per algorithm cycle be an order of magnitude greater than the time takento solve the master problem, complete with some proposals, per algorithm cycle.Of course this measure will only become apparent after trying out the algorithmon the problem.

More usefully there are some practical guidelines. Certainly you should aimfor relatively few global constraints—the fewer the better. You should also try toconstruct the pricing problems so that they are well formed problems in their ownright, for it is by solving the pricing problems rather than the master problem thatdecomposition makes a saving.

D.2.2 A Production Planning Problem

Consider a simple production planning model over a set of months t ∈ T and aset of facilities f ∈ F , based on the Coco problem in the Xpress-MP User Guide.In each month at each facility a number of raw materials m ∈ M are purchased,and manufactured into end products p ∈ P which are sold. Storage of both ma-terials and products is possible at all facilities. The activities and material flowsin the model are pictured in figure D.3, and table D.1 presents an algebraic modelformulation. Decision variables are denoted in lower case; data in upper case; andthe names of constraints in mixture of upper and lower case. A subscript on theconstraint name indicates that the constraint applies for all values of the subscript.

Aside from the mass balance equations (PBal and MBal), there are limits onthe total sales of each product in each period (MxSell), the total stock of productsat each facility in each period (MxPStock), the total production at each facility ineach period (MxMake), the total stock of materials at each facility in each period(MxMStock), and the total purchases of each material in each period (MxBuy).


buym,f,t↓


makep,f,t↓


↓sellp,f,t

Figure D.3: Activities and material flows in the production planning model.



−∑



mft CBUY mt buymft

Subject to


−∑



MxPStockft :∑



MxMStockft :∑




Table D.1: An algebraic model of the production planning problem.


Sales of products generate revenue, whilst all other activities incur costs. Theobjective is to maximise profit.

CONSTRAINTS









Table D.2: An xpress-mp model of the production planning problem.

An xpress-mp model of the problem is included in the DecMod release (filepplan.mod). The equations are contained within the CONSTRAINTS section of themodel, which is reproduced in table D.2. We look for block structure based on oneof the index sets in the model: p, m, f, t.

D.2.3 Decomposition by Facility

Let us first consider decomposing the problem by the facility index. Later on wewill consider other decompositions based on product and time period, and thiswill allow us to compare the application and performance of the algorithm on thedifferent decompositions and to assess the quality of the decompositions.

Inspecting the model we note that the objective function MaxProfit, constraintsMxSell and constraints MxBuy are not indexed by facilities (f=1–NF). All otherconstraints and all variables are indexed by one facility f. Furthermore, thoseconstraints that are indexed by facility f, feature only variables indexed by the samesingle facility f. Rearranging the rows and columns of the constraint matrix, andhighlighting the non-zero matrix elements, we can confirm that these observationsimply a suitable block structure (see figure D.4).


Figure D.4: Facility based block structure. (The shaded parts representnon-zero coefficients.)

Figure D.5: Initial master problem matrix. (Note that there are no variablesin the initial matrix, only constraints.)


Figure D.6: Pricing problem matrix.

Our Dantzig-Wolfe solver DecOpt requires that the problem be split into severalmatrices before the application of the solver. The master problem comprises globalconstraints and variables that don’t fit neatly into the block structure pattern. (Inthis case there are no variables in the master problem.) Then for each block in theblock structure, we have a pricing problem comprising the variables and constraintsthat apply only to that block, and also the portion of the global constraints thatinclude variables in the block. Figure D.5 illustrates the master problem matrix andfigure D.6 illustrates the pricing problem matrix, for an arbitrary pricing problem f.Observe that each pricing problem, f=1–NF, has the same variables and constraints,just differing by the index f.

D.2.4 Block Structure Specification for Facility Decomposi-tion

Now that we have given the matrices required by our solver it is easy enough tospecify the matrices by writing lists of the variables and constraints in each matrix.This in fact is what the block structure specification (or decomposition) file contains,and in table D.3 we reproduce one such file to pick out the facility decomposition.The file defines two sets of blocks, corresponding to the master problem and theset of pricing problems.

The first set of blocks has no dummy variables, and so contains just oneblock, the master problem. The master problem contains the objective functionMaxProfit and two sets of constraints: MxSell and MxBuy; and no variables.(Dantzig-Wolfe decomposition is a column generation algorithm, and so variableswill be generated dynamically for the master problem as the algorithm proceeds.)







END

Table D.3: DecMod specification for the facility decomposition.

Both the constraint sets included are included in their entirety: no restrictions(ranges) have been placed on any of their indices. Observe that MaxProfit has noindices in the model file while MxSell has two, and this is reflected in the decom-position file.

The second set of blocks has one dummy variable defined: f, with a range of 1 toNF, and so contains NF blocks, one for each f in the range 1 to NF, corresponding tothe pricing problems. Each block contains several sets of variables and constraints,including the global constraints of the master problem. The other variables andconstraints all have one index restricted to the value f. What this means is that inblock f=1, variables and constraints with the f index restricted to 1 are included.In block f=2, variables and constraints with the f index restricted to 2 are included.And so on, for f all the way to NF. Thus the variables and constraints are partitionedout amongst the pricing problems, with each pricing problem including a subsetof each set of variables and constraints. Observe again that the number of indicesspecified on each model entity matches the number of indices specified in the modelfile.


D.2.5 Different Block Structures

It is possible to pick out at least two more block structures suitable for Dantzig-Wolfe decomposition, one based on the product index p and one based on the timeindex t. Tables D.4 and D.5 show decomposition files for a product block structureand time period block structure respectively.

! Production planning problem; product decomposition

BLOCKS master ! Master problemCONSTRAINTS ! Global constraintsMaxProfitMBal(*,*,*)MxPStock(*,*)MxMake(*,*)MxMStock(*,*)MxBuy(*,*)VARIABLES ! Master problem variablesmstock(*,*,*)buy(*,*,*)

BLOCKS sub ! Pricing problemsp=1:NP ! 1 for each product indexCONSTRAINTS ! Global constraintsMaxProfitMBal(*,*,*)MxPStock(*,*)MxMake(*,*)MxMStock(*,*)MxBuy(*,*)CONSTRAINTS ! Pricing roblem constraintsPBal(p,*,*)MxSell(p,*)VARIABLES ! Pricing problem variablessell(p,*,*)pstock(p,*,*)make(p,*,*)

END

Table D.4: DecMod specification for the product decomposition.

These structures are less natural than the one based on the facility index, andmore constraints and variables are included in the master problem.

The time decomposition illustrates that occasionally it is necessary (or desir-able) to include entities with different index values from the majority of entities.For example, we decided that pricing problem t should contain variable mstock(and, as a consequence, constraint MxMStock) restricted to time index t+1 ratherthan t. This was really a matter of individual preference, as either result in a

D.3. REQUIREMENTS AND INSTALLATION 213

! Production planning problem; time period decomposition

BLOCKS master ! Master problemCONSTRAINTS ! Global constraintsMaxProfitPBal(*,*,*)MBal(*,*,*)

VARIABLES ! Global variablespstock(*,*,1)mstock(*,*,1)

BLOCKS sub ! Pricing problemst=1:NT ! 1 for each time period

CONSTRAINTS ! Global constraints MaxProfitPBal(*,*,*)MBal(*,*,*)

CONSTRAINTS ! Pricing problem constraintsMxSell(*,t)MxPStock(*,t+1)MxMake(*,t)MxMStock(*,t+1)MxBuy(*,t)

VARIABLES ! Pricing problem variablessell(*,*,t)pstock(*,*,t+1)make(*,*,t)mstock(*,*,t+1)buy(*,*,t)

END

Table D.5: DecMod specification for the 1 time period decomposition.

valid block structure. (Using index t, we might include mstock and MxMStock fort=NT+1 rather than t=1 in the master problem.) Both make sense in the realworld context of the problem as well, as mstock(*,*,t) represents the amount ofstock at the beginning of period t, while mstock(*,*,t) represents the amount ofstock at the end of period t.

D.3 Requirements and Installation

DecMod is available on a variety of platforms including 32 bit Windows (e.g., 95, 98,NT, 2000) and Solaris 2. Potentially it can be ported to any platform supported byxpress-mp. You will need to obtain the software and a licence of emosl Release 12.Subsequent releases may also be supported.

Install DecMod into a new (‘decmod’) directory. In addition you will probablywant a separate (‘data’) directory for your problem models and decomposition files.


To run DecMod, you need the following emosl files:

Windows Unixxpress.pwd xpress.pwdmp-opt.msg mp-opt.msgmp-model.msg mp-model.msgxpopt32.dllemosl32.dllqelib.dllxpodbc.dllxpressnh.ini

Keep them together in the same directory—it is best to copy them into your decmoddirectory. In Windows, add the directory containing the DLLs to the Windows path.In Unix, set the environment variable XPRESS to point to the directory containingthe message and password files.

The memory allocated to the xpress-emosl Model Builder component is thevalue of the XPMEMMOD environment variable, and if this is zero or not set, a defaultvalue is used.

D.4 Starting DecMod

DecMod takes one or two command line arguments. The first specifies the modelfile and the second, if present, specifies the block structure specification file (thedecomposition file):

decmod model [decomp]

If the second argument is missing then it is assumed to be the same as the firstargument. Neither filename may contain a suffix, and the suffices .mod for themodel file and .dec for the decomposition file will be appended automatically. Thesefilenames may contain a drive and/or directory specification, for example:

decmod c:\decomp\models\pplan c:\decomp\decomps\pplanf

The two arguments model and decomp are used to construct the filenames of allfiles used by DecMod (see section D.7), and we refer to them throughout the manual.

Additionally, any number of optional arguments may be included to set controlvariables. Each optional argument is preceded by a minus sign and is of the form:

-control=integer expression

Here control must be the name of one of the control variables listed in section D.8;and integer expression must be an integer constant (see section D.5.2). Note thatthere must be no spaces after the minus sign or around the equals sign (in factanywhere in the argument).

For example, to run DecMod so that a model error log is produced and up to1,000,000 non-zero matrix elements are allowed in any one block, use:

decmod pplan pplanf -moderr=1 -mspace=1000000

If DecMod is called with no arguments at all then a list of the valid control variablesand default values is given.

D.5. THE BLOCK STRUCTURE SPECIFICATION LANGUAGE 215

D.5 The Block Structure Specification Language

You specify the block structure apparent in the model (apparent that is to you, notto DecMod) via the decomposition file (also known as the block structure specifica-tion file). This file is a text file with the suffix .dec.

The decomposition syntax has been designed to be powerful, intuitive and easyto use. Powerful, in that it allows any block structure to be specified; intuitive,in that it follows the syntax of the xpress-mp Modelling Language in which themodel is written; and easy to use, in that blocks following the same pattern maybe specified at once and parameters may be used from model parameter and datavalues or derived from them using simple arithmetic. In this comprehensive sectionwe describe the syntax.

D.5.1 Sections and Cards

Each line (or card) in the file is treated separately, and the lines are grouped intosections. A line beginning with a recognised keyword (a header card) signals thestart of a new section, and subsequent lines (data cards) contain the data for thatsection, which continues until the next section keyword is encountered. Blank lines,or lines containing comments only, are ignored.

There are five valid sections, for which the keywords are SETTINGS, BLOCKS,CONSTRAINTS, VARIABLES and END. The END section contains no data; the keywordsignals the end of the file and any further lines are ignored. The sections aredescribed in sections D.5.4–D.5.8.

Comments may be placed in the file following an exclamation mark ‘!’, andlast until the end of the line. Comments may be placed anywhere in the file andare ignored.

D.5.2 Expressions

In the following sections, we define the notation integer expression and range thatrefer to two types of expressions used in the decomposition file.

An integer expression is defined in a similar way to most programming lan-guages.

• An integer constant, a dummy variable previously defined in the currentBLOCKS section, a model parameter (defined using the LET keyword in themodel), or a single element of an model data table (defined using the TABLESkeyword in the model) is an integer expression.Examples: 14, -3, t, NPROD, TAB(1,3), PROD(NPROD-3).If the model entity is real valued, the integer part of the value is used.Note that when referring to an indexed data table in an expression, you mustensure that you supply enough indices to specify one element of a data table.The range of elements signified by TAB(1,*) is not acceptable.

• If e is an integer expression, (e) is an integer expression.Examples: (14), (-3), (NPROD), (TAB(1,3)), (5+3).

• If e1 and e2 are integer expressions, so are e1+e2, e1-e2, e1*e2 and e1/e2.Note that e1/e2 means integer division, so, for example, 10/3 evaluates to 3.Examples: 1+5, 5/3+4-1, NPROD+TAB(I+3,2*I-4).


The usual precedence rules apply.A range expression is used to define the range of dummy variables and the

subset of sets of variables and sets of constraints that are to be included in blocks.A range expression is defined in one of the following ways:

• If e1 and e2 and are integer expressions, then e1:e2 is a range expression.Examples: 1:5, 1:NPROD, TAB(1):TAB(2), 5*(T-1)+1:5*T.

• If e is an integer expressions, then e is a range expression, and is shorthandfor the single valued range e:e.Examples: 1, NPROD, TAB(1).Note that this differs from the xpress-mp Modelling Language, in which therange signified by a single value e is shorthand for the range starting at eand continuing until the upper limit on the index is reached.

• The wild card range expression ‘*’ means no limit (on an index of a model en-tity). This range can only be used when specifying variables and constraintsin a block; it cannot of course define the range of a dummy variable.Examples: Supply(*,t), Demand(*,t,*,1).

D.5.3 Model Entities

When specifying model entities—either data tables or symbols in expressions orvariables or constraints to include in blocks—you must include the correct numberof indices.

• An entity which has no indices is specified without parentheses.Examples: NPROD, MaxProfit, Cost.

• An entity which has one or more indices is specified using the correct numberof indices, surrounded by parentheses. If when specifying constraints or vari-ables, no restriction is to be placed on any indices, use the wild card range‘*’.Examples: TAB(1), COST(3,5,8), x(s,*), Demand(*,t,3,*).

D.5.4 The SETTINGS Section

Keyword SETTINGS

Purpose Sets DecMod control variables—an alternative to using command linearguments.

Format

SETTINGScontrol=integer expressioncontrol=integer expression. . .

Examples

1. We set the maximum space allowed for each block matrix.


SETTINGSrspace=5000 ! Allow 5,000 rows in each block,cspace=10000 ! 10,000 columns,mspace=60000 ! and 60,000 matrix non-zeros.

2. We set the maximum space allowed for each block matrix using param-eters from the model.

SETTINGSrspace=1 + NP + 2*NTcspace=2*NT + 3 *NPmspace=6*(2*NT + 3 *NP)

3. We allow the creation of an optimiser log, which will contain the sizesand any error or warning messages when creating the matrix for eachblock.

SETTINGSoptlog=1

Description The SETTINGS section is used to set control variables, and is particu-larly useful for setting those that define the maximum sizes of any one block.See section D.8 for a full listing of control variables and their default values.Note that for many applications the default values will be sufficient.

In data cards, control must be a recognised control variable name. Theinteger expression may include reference to data entities in the model, andindeed one advantage of using a SETTINGS section over command line argu-ments (section D.4) is that reference can be made to model data entities.

Control variables apply globally, to each set of blocks. (It is not possible toset different matrix size controls for each set of blocks.) If any control is setmore than once, the final value is used, and settings supplied as commandline arguments override those given in SETTINGS sections.

D.5.5 The BLOCKS Section

Keyword BLOCKS

Purpose Defines a set of blocks to be extracted from the problem.

Format

BLOCKS [block name]dummy variable=rangedummy variable=range. . .

CONSTRAINTS sectionVARIABLES section

Examples


1. We define a set of blocks labelled master comprising just one block.Included in the block are two entire constraints, Profit and Supply,and one set of variables, store, for which the second index value isrestricted to the value 1.

BLOCKS master ! Define master problem matrixCONSTRAINTS ! Master problem constraintsProfitSupply(*,*)

VARIABLES ! Master problem variablesstore(*,1)

2. We define a set of blocks labelled sub comprising one block for eachvalue of t in the range 1 to NT. Included in each block are two setsof constraints, Profit, and the subset of Demand in which the secondindex is fixed to t; and two sets of variables, sell, again with thesecond index restricted to , and store, with the second index fixed tot+1. Note that the Profit constraint is included in each block in theset sub, while the set of constraints Demand is partitioned between theset of blocks.

BLOCKS sub ! Define subproblem matricest=1:NT

CONSTRAINTS ! Subproblem constraintsProfitDemand(*,t)

VARIABLES ! Subproblem variablessell(*,t)store(*,t+1)

3. We define an unnamed set of blocks comprising one block for each setof 6 time periods. To simplify the list of constraints and variables, wedefine two extra dummy variables, l and u, giving the first and lasttime periods in each block respectively.As in the previous example, we include two sets of constraints and twosets of variables in each block. We use the dummy variables l and uto restrict Demand and sell to time periods in the range l to u, andstore to time periods in the range l+1 to u+1.

BLOCKS ! Define a set of blockst=1:NT/6 ! 1 block for each 6 time periodsl=6*(t-1)+1 ! First time period in blocku=6*t ! Last time period in block

CONSTRAINTSProfitDemand(*,l:u)

VARIABLESsell(*,l:u)store(*,l+1:u+1)


Description The BLOCKS section is used to define a set of blocks. A set of blocksmust be contiguous in the decomposition file. The set may optionally begiven a name, which may be used to help construct the filenames of thematrices created (see section D.7). Block names also enhance any messagesgenerated if any errors are encountered when reading the decomposition file.Any number of dummy variables may be defined on subsequent cards, untilthe first CONSTRAINTS or VARIABLES section begins. Each dummy variable isdefined by a range, and a separate block is generated for each combinationof values over all the dummy variables. Thus the dummy variables implicitlydefine the number of blocks in the set of blocks. The range for dummyvariables cannot be undefined (i.e., using the wild card range ‘*’) but maybe a single value. The blocks are created by looping over the last dummyvariable defined, then incrementing the second to last dummy variable andlooping over the last one again, and so on, so that the first dummy variabledefined is looped over once only in an outer loop. Dummy variables mayrefer to dummy variables previously defined in the block. When generatingthe block, any reference to a dummy variable is replaced by the value of thedummy variable current for that block.The names of dummy variables may be any alpha-numeric token, includingunderscores, but no other characters or spaces. The first character must notbe a digit. Lower and upper case characters are distinguished. Redefinitionof a dummy variable in the same set of blocks is not allowed, but names maybe re-used in other BLOCKS sections. If a dummy variable name co-incideswith a model entity, the dummy variable masks the model entity for theduration of the block.Once all dummy variables have been defined, CONSTRAINTS and VARIABLESsections (qqv) specify the constraints and variables to be included in the setof blocks. It is possible to include none, one or several CONSTRAINTS and/orVARIABLES sections in each set of blocks. All CONSTRAINTS and VARIABLESsections encountered following a BLOCKS keyword are included in that set ofBLOCKS until another BLOCKS keyword is encountered.

D.5.6 The CONSTRAINTS Section

Keyword CONSTRAINTS

Purpose Specifies sets of constraints to be included in the current set of blocks.

Format

CONSTRAINTSmodel constraint [(range [, range [, . . . ]])]. . .

Examples

1. We include two sets of constraints in the current set of blocks: the singleconstraint Profit and the subset of constraints Demand for which thesecond index takes the value t, where t is a dummy variable defined inthe current BLOCKS section.


CONSTRAINTS ! Subproblem constraintsProfitDemand(*,t)

2. This example is equivalent to the first example. It makes no differencewhether the constraints are included in one CONSTRAINTS section ofseveral CONSTRAINTS sections, provided they all occur in a single BLOCKSsection. All CONSTRAINTS and VARIABLES sections refer to the currentset of blocks until a new BLOCKS section is encountered.

CONSTRAINTS ! Subproblem constraintsProfit

CONSTRAINTS ! More subproblem constraintsDemand(*,t)

Description The CONSTRAINTS section is used to list sets of constraints to beincluded in the current set of blocks. It cannot be used before a set of blockshas been defined using a BLOCKS section (qv).

The data cards in a CONSTRAINTS section list constraints from the modelthat are to be included in the current set of BLOCKS. The name of each setof constraints must correspond exactly to the name given in the model, andthe names are case sensitive. The reference to a set constraints must includethe correct number of indices, corresponding to the number of indices in themodel (see section D.5.3).

Restrictions may be placed on the indices of a set of constraints to include asubset of the constraints only in a particular block. These restrictions takethe form of ranges on the indices of the constraint. Use the wild card range(‘*’) leave a particular index unrestricted. Restricted ranges may be constant,refer to model data entities, or refer to dummy variables; see section D.5.2.

D.5.7 The VARIABLES Section

Keyword VARIABLES

Purpose Specifies sets of variables to be included in the current set of blocks.

Format

VARIABLESmodel variable [(range [, range [, . . . ]])]. . .

Examples

1. We include two sets of variables in the current set of blocks: the subsetof variables sell for which the second index takes the value t, and thesubset of constraints store for which the second index takes the valuet+1, where t is a dummy variable defined in the current BLOCKS section.

D.6. EXPORTING THE PARTITIONED PROBLEM 221

VARIABLES ! Subproblem variablessell(*,t)store(*,t+1)

The VARIABLES section is analogous to the CONSTRAINTS section sofurther examples are not included here.

Description The VARIABLES section is used to list sets of variables to be includedin the current set of blocks. It cannot be used before a set of blocks has beendefined using a BLOCKS section (qv).

The data cards in a VARIABLES section list variables from the model thatare to be included in the current set of BLOCKS. The name of each set ofvariables must correspond exactly to the name given in the model, and thenames are case sensitive. The reference to a set variables must include thecorrect number of indices, corresponding to the number of indices in themodel (see section D.5.3).

Restrictions may be placed on the indices of a set of variables to include asubset of the variables only in a particular block. These restrictions take theform of ranges on the indices of the variable. Use the wild card range (‘*’)leave a particular index unrestricted. Restricted ranges may be constant,refer to model data entities, or refer to dummy variables; see section D.5.2.

D.5.8 The END Section

Keyword END

Purpose Must be included to signal the end of the decomposition file.

Format

END

Description The END keyword is used to signal the end of the decomposition file.It must be included and any further cards are ignored.

D.6 Exporting the Partitioned Problem

DecMod permits the partitioned problem to be output in several export modes, forcompatibility with various different optimisers. This section describes the differentexport modes, which may be selected using the export control (see section D.8).Hopefully one of the modes suits you—if not, please contact the author as it maybe possible to add new export modes to DecMod.

Throughout this section, as indeed throughout the entire manual, the namesof files used by DecMod are given in terms of the two input arguments supplied toDecMod, model and decomp (see section D.4).

Partition the Problem into Separate Matrix Files The problem is partitionedinto blocks specified by the decomposition, and each block is output to a sep-arate matrix file. This is the default export mode, export=0.


Various different file naming conventions may be used for the matrix filescreated, selected using the matnam control. These are described below.If using DecMod to format a problem for use with the Dantzig-Wolfe optimiserDecOpt, use this export mode 0 together with matrix file name conventionmatnam=0.

Re-Order the Problem by Block The problem is re-ordered into blocks speci-fied by the decomposition, and the entire problem is output to a single matrixfile ordered by block. The rows in the matrix will be grouped according toblock, with the rows specified in the first block being listed first in the ma-trix, followed by the rows from the second block, and so on. Similarly, whenthe columns are listed, those from the first block will appear first, then thosefrom the second block, and so on. If any row or column appears in morethan one block it is output only once, in the first block in which it was in-cluded. If a row or column does not appear in any block, then it will not beincluded. Only those matrix elements that appear in one of the blocks willbe included, i.e., the row and column identifying the position in the matrixwhere the element occurs must appear together in the same block, otherwiseit will not be included.The matrix file is named decomp.mat. This export mode is selected usingexport=1.

Output a Block Index File The rows and columns in the matrix are assignedto a block according to the decomposition specification. The blocks are num-bered 0, 1, 2, . . . , according to the order in which they appear in the specifi-cation. The entire problem is output to a single matrix file, using the defaultordering of xpress-mp given the model file, accompanied by a block indexfile. The block index file lists the rows and columns of the matrix with aninteger value giving the index of the block in which it belongs.The matrix file is named decomp.mat; the block index file is named decomp.blk.This export mode is selected using export=2.

D.6.1 File Naming Conventions

File naming conventions are relevant to export mode 0 only. The matnam controlallows different conventions to be used when naming the set of matrix files producedunder export mode 0.

matnam=0: The matrix filename for a block is constructed by concatenating thedecomposition name, the block sequence number, and the suffix .mat:

decomp block id .mat

All of the blocks are numbered (block id) in one sequence 0, 1, 2, . . . , (in hex)according to the order in which they appear in the specification. This is the filenaming convention used by the Dantzig-Wolfe solver DecOpt.

For example, suppose DecMod was called using:

decmod c:\model\pplan c:\decomp\pplanf

Suppose further that the decomposition file contained two sets of blocks defined asfollows:

D.6. EXPORTING THE PARTITIONED PROBLEM 223

BLOCKS master ! Define master problem matrix! CONSTRAINTS and VARIABLES sections

BLOCKS sub ! Define subproblem matricest=1:NT

! CONSTRAINTS and VARIABLES sections

The first set of blocks, master, contains just one individual block, so one matrix isgenerated with filename c:\decomp\pplanf0.mat. The second set of blocks, sub,contains NT individual blocks, and so NT matrices are generated with filenames c:\decomp\pplanf1.mat, c:\decomp\pplanf2.mat, c:\decomp\pplanf3.mat, etc.

matnam=1: The matrix filename for a block is constructed by concatenatingthe directory component of the decomposition name, the block name, the blocksequence number within the set of blocks, and the suffix .mat:

decomp dir block name block id .mat

The directory component of the decomposition name includes any drive and direc-tory specification in the decomp argument supplied to DecMod, in accordance withthe operating system on your computer. The block name (block name) is the namegiven to the set of blocks in the BLOCKS header card. The blocks are numbered(block id) in sequence 0, 1, 2, . . . , (in hex) according to the order in which theyappear in the specification, with the count re-starting at zero for each set of blocks.

For example, suppose DecMod was applied as above. The first set of blocks,master, contains just one individual block, so one matrix is generated with filenamec:\decomp\master0.mat. The second set of blocks, sub, contains NT individualblocks, and so NT matrices are generated with filenames c:\decomp\sub0.mat, c:\decomp\sub1.mat, c:\decomp\sub2.mat, etc.

matnam=2: The matrix filename for a block is constructed by concatenating thedirectory component of the decomposition name, the block name, the value of eachdummy variable use to create the individual block, and the suffix .mat:

decomp dir block name [dummy1 [dummy2 [ . . . ]]] .mat

The directory component of the decomposition name includes any drive and direc-tory specification in the decomp argument supplied to DecMod, in accordance withthe operating system on your computer. The block name (block name) is the namegiven to the set of blocks in the BLOCKS header card. The dummy values are writtenin decimal notation, preceded by a zero if necessary to make at least 2 digits. Ifthere are no dummy variables in the set of blocks, then no dummy variable valuesare included in the filename.

For example, suppose DecMod was applied as above. The first set of blocks,master, contains just one individual block, with no dummy variables, so one matrixis generated with filename c:\decomp\master.mat. The second set of blocks, sub,contains NT individual blocks, with one dummy variable t, and so NT matricesare generated with filenames c:\decomp\sub01.mat, c:\decomp\sub02.mat, c:\decomp\sub03.mat, etc.


D.7 Files Used by DecMod

In this section we describe the files that are read and written by DecMod. In thefollowing list, input or output signify whether DecMod reads or creates the file; whilstthe filename is given in terms of the two input arguments supplied to DecMod, modeland decomp (see section D.4 for details of the input arguments).

Model file [input; filename: model.mod] xpress-mp model describing the prob-lem.

Decomposition file [input; filename: decomp.dec] Specification of the decompo-sition (block structure) to be used to partition the problem, written usingthe syntax described in section D.5.

Matrix file(s) [output; filename: see section D.6] Either a set of matrix files, eachcontaining one block of the problem after it has been partitioned using thedecomposition, or a single matrix file, containing the complete problem re-ordered into blocks according to the decomposition, or a single matrix file,in its original order, to accompany the block index file.

The types and names of the output files produced are controlled by theexport control, and are fully described in section D.6.

Block index file [output; filename: decomp.blk] Block index file, listing for eachrow and column in the matrix file the index of the block in which it belongs,according to the decomposition. This file is only output under export mode 2;see section D.6.

Model log file [output; filename: model.lst] xpress-mp model log, containingany errors, warnings and other information and output, such as model PRINTstatements, produced whilst parsing the model.

Model error file [output; filename: model.err] xpress-mp model error log, con-taining any errors and warnings produced whilst parsing the model. Notethat these messages are all included in the model log file. This file is en/disabledusing the moderr control (see section D.8).

Optimiser log file [output; filename: model.opt] xpress-mp optimiser log, con-taining any errors, warnings and other log information produced whilst cre-ating the matrices. This file is en/disabled using the optlog control (seesection D.8).

D.8 Control Variables

There are a small number of control variables you can use control certain aspectsof DecMod. The following table gives a complete list. A complete list of controlvariables is also displayed by calling DecMod with no arguments.

Control variables may be set through command line arguments (see section D.4)or in a SETTINGS section in the decomposition file (see section D.5.4). If a controlvariable is set both in a command line argument and in the decomposition file, thecommand line setting takes precedence.

D.8. CONTROL VARIABLES 225

export [integer; default value 0] Export mode: 0 output each block in the decom-position to a separate matrix file; 1 output the complete problem to a singlematrix file, ordered by block; 2 output the problem to a single matrix filewithout re-ordering it, together with a block index file, giving the block indexof each constraint and variable in the matrix. See section D.6 for full details.

matnam [integer; default value 0] Matrix file naming convention. See section D.6for the options available. This control is only relevant to export mode 0.

matexp [integer; default value 2] Matrix export options: (bitmap) 0 no options; 1scaled; 2 high-precision MPS.The matexp control allows vary how the blocks are exported to matrix files.It is a bitmap, meaning that you should add up the values of each option youwish to select to obtain the the value of the control. For example, matexp=3means that the matrix will be scaled and high-precision.The scaling option allows you to extract and export a consistent set of scaledsub-matrices. Of course, any solution obtained applying a solver to this setof matrices will need to be unscaled for it to be a valid solution to the originalproblem. Currently there is no automatic method to do this, and nor is itpossible to obtain the row and column scaling factors used. Scaling may helpto overcome some numerical difficulties with badly scaled problems.The high precision option outputs a high-precision MPS matrix, and mustbe selected if the single ordered matrix version is being used. No error issignalled if it is not.

rspace [integer; default value 0] Maximum number of rows in any one block.

cspace [integer; default value 0] Maximum number of columns in any one block.

mspace [integer; default value 0] Maximum number of matrix non-zero elements inany one block.The rspace, cspace and mspace controls allow you to specify the maximumamount of space that will be required in the matrices corresponding to anysingle matrix. By default, if the values are zero, then DecMod allows space forthe entire original matrix. (This may not be enough if using export option1, as variables and constraints that appear in several blocks are countedeach time they appear.) If memory is limited, and DecMod is having troublehandling a large problem then it may be possible to overcome certain memoryproblems by setting these controls to be the maximum size of a block matrix,provided this size is significantly smaller than the original matrix. If you getthe error message ‘Not enough space to store matrix data’ then you need tore-run DecMod and increase the appropriate control.

moderr [integer; default value 0] 1 if an xpress-mp model error log file to bewritten; 0 otherwise. See section D.7 for details.

optlog [integer; default value 0] 1 if an xpress-mp optimiser log file to be written;0 otherwise. See section D.7 for details.


Bibliography

[ALR81] D. C. Ament, Etienne Loute, and M. W. Remmelswaal, LIFTprogram reference manual (provisional), CORE, UniversiteCatholique de Louvain, Louvain-la-Neuve, circa 1981.

[Ame81] D. C. Ament, An implementation of the LIFT algorithm, Doc-toraal scriptie, Econometrisch Instituut, Erasmus Universiteit,Rotterdam, March 1981.

[And98] Erling D. Andersen, Private communication, 1998.

[Bea68] E. Martin L. Beale, Mathematical programming in practice,Topics in Operational Research, Sir Isaac Pitman & Sons, Lon-don, 1968.

[Bea71] E. Martin L. Beale, Sparseness in linear programming, LargeSparse Sets of Linear Equations (J. K. Reid, ed.), AcademicPress, London, 1971, pp. 1–15.

[Bea96] J. E. Beasley, Advances in linear and integer programming,Oxford University Press, 1996.

[Ben62] J. F. Benders, Partitioning procedures for solving mixed-variables programming problems, Numerishe Mathematik 4(1962), 238–252.

[BFM97] Ralf Borndorfer, Carlos E. Ferreira, and Alexander Martin,Matrix decomposition by branch-and-cut, Technical report SC97-14, Konrad-Zuse-Zentrum fur Informationstechnik, Berlin,1997.

[BFM98] Ralf Borndorfer, Carlos E. Ferreira, and Alexander Martin,Decomposing matrices into blocks, SIAM Journal on Optimiza-tion 9 (1998), no. 1, 236–269.

[BGHR77] M. Benichou, J. M. Gauthier, G. Hentges, and G. Ribiere, Theefficient solution of large-scale linear programming problems—some algorithmic techniques and computational results, Math-ematical Programming 13 (1977), 280–322.

227

228 BIBLIOGRAPHY

[BHS65] E. Martin L. Beale, P. A. B. Hughes, and R. E. Small, Expe-riences in using a decomposition program, Computing Journal8 (1965), 13–18.

[Bix92] Robert E. Bixby, Implementing the simplex method: The initialbasis, ORSA Journal on Computing 4 (1992), no. 3, 287–284.

[Bix94] Robert E. Bixby, Progress in linear programming, ORSA Jour-nal on Computing 6 (1994), no. 1, 15–22.

[BJ98] Francisco Barahona and David Jensen, Plant location withminimum inventory, Mathematical Programming 83 (1998),no. 1, 101–111.

[BJN+94] Cynthia Barnhart, Ellis L. Johnson, George L. Nemhauser,Martin W. P. Savelsbergh, and Pamela H. Vance, Branch andprice: Column generation for solving huge integer programs,Mathematical Programming: State of the Art (J. R. Birge andK. G. Murty, eds.), International Symposium on MathematicalProgramming, 1994.

[BM82] Johannes Bisschop and Alex Meeraus, On the development ofthe general algebraic modeling system in a strategic planningenvironment, Mathematical Programming Study 20 (1982), 1–29.

[CDM78] Harlan P. Crowder, Ron S. Dembo, and John M. Mulvey, Re-porting computational experiments in mathematical program-ming, Mathematical Programming 15 (1978), 316–329.

[CF96] D. Chang and Emmanuel Fragniere, SPLITDAT and DE-COMP: Two new GAMS I/O subroutines to handle mathe-matical programming problems with an automated decomposi-tion procedure, Logilab technical report 1997.5, Department ofManagement Studies, University of Geneva, 102 Bd Carl-Vogt,CH-1211 Geneve 4, 28 August 1996.

[Chv83] V. Chvatal, Linear programming, Freeman, New York, 1983.

[CL96] Pierre Chardaire and A. Lisser, Simplex and interior point spe-cialised algorithms for solving non-oriented multi-commodityflow problems, Technical report, School of Information Sys-tems, University of East Anglia, Norwich, 1 August 1996, toappear in Operations Research.

[CLF98] Christian Condevaux-Lanloy and Emmanuel Fragniere, SET-STOCH: A tool for multistage stochastic programming with re-course, Logilab technical report, Department of Management

BIBLIOGRAPHY 229

Studies, University of Geneva, 102 Bd Carl-Vogt, CH-1211Geneve 4, 6 August 1998.

[CN96] Jordi Castro and Narcis Nabona, An implementation of linearand nonlinear multicommodity network flows, European Jour-nal of Operational Research 92 (1996), 37–53.

[Cor99] Cecile Cordier, Development and experimentation with a bra-nch & cut system for mixed integer programming, Ph.D. thesis,Department d’Ingenierie Mathematique, Universite Catholiquede Louvain, Louvain-la-Neuve, March 1999.

[Dan63] George B. Dantzig, Linear programming and extensions, Prin-ceton University Press, Princeton, New Jersey, 1963.

[Dan77] George B. Dantzig, Large-scale systems optimization with ap-plication to energy, Technical report SOL 77-3, Systems Op-timization Laboratory, Department of Operations Research,Stanford University, Stanford, California, April 1977.

[Dan83] George B. Dantzig, Reminiscences about the origins of linearprogramming, Mathematical Programming: The State of theArt (A. Bachem, M. Groetschel, and B. Korte, eds.), SpringerLecture Notes, Springer Verlag, Berlin, 1983, pp. 78–86.

[Das99a] Dash Associates, Blisworth House, Blisworth, Northants NN73BX, XPRESS-MP reference manual, release 11 ed., 1999.

[Das99b] Dash Associates, Blisworth House, Blisworth, Northants NN73BX, XPRESS-MP user guide, release 11 ed., 1999.

[Das00] Dash Associates, Blisworth House, Blisworth, Northants NN73BX, XPRESS-MP reference manual, release 12 supplemented., 2000.

[DJ79] Y. M. I. Dirickx and L. Peter Jennergren, Systems analysisby multilevel methods: With applications to economics andmanagement, chapter 4, Numerical Experiences with Dantzig-Wolfe Decomposition, John Wiley & Sons, Chichester, 1979.

[dMVDH99] Olivier du Merle, Daniel Villeneuve, Jacques Desrosiers, andPierre Hansen, Stabilized column generation, Discrete Mathe-matics 194 (1999), 229–237.

[DT01] Robert C. Daniel and James R. Tebboth, A tightly integratedmodelling and optimisation library: A new framework for rapidalgorithm development, Annals of Operations Research 104(2001), 313–333.

230 BIBLIOGRAPHY

[DW60] George B. Dantzig and Philip Wolfe, Decomposition principlefor linear programs, Operations Research 8 (1960), 101–111.

[DW61] George B. Dantzig and Philip Wolfe, The decomposition algo-rithm for linear programs, Econometrica 29 (1961), 767–778.

[Ent89] Robert Entriken, The parallel decomposition of linear pro-grams, Ph.D. thesis and technical report SOL 89-17, SystemsOptimization Laboratory, Department of Operations Research,Stanford University, Stanford, California, November 1989.

[FG97] Robert Fourer and David Gay, Proposals for stochastic pro-gramming in AMPL modeling language, presented at the In-ternational Symposium on Mathematical Programming, Lau-sanne; see http://iems.nwu.edu/˜4er/, 27 August 1997.

[FG00] Robert Fourer and J. W. Gregory, Linear programming FAQ,See ftp://rtfm.mit.edu or http://www.mcs.anl.gov/home/otc/faq/linear-programming-faq.html in /pub/usenet/sci.answers/linear-programming-faq, 2000.

[FG01] Emmanuel Fragniere and Jacek Gondzio, Optimization mod-eling languages, Handbook of Applied Optimization (MauricoG. C. Resende and Panos M. Pardalos, eds.), Oxford Univer-sity Press, New York, October 2001.

[FGS98] Emmanuel Fragniere, Jacek Gondzio, and Robert Sarkissian,Customised block structures in algebraic modelling languages:The stochastic programming case, Proceedings of the CE-FES/IFAC98 (Berlin) (Sean Holly, ed.), Springer Verlag, 1998.

[FGS01] Emmanuel Fragniere, Jacek Gondzio, and Robert Sarkissian,Efficient management of multiple sets to extract complex struc-tures from mathematical programs, Annals of Operations Re-search 104 (2001), 67–87.

[FGSV00] Emmanuel Fragniere, Jacek Gondzio, Robert Sarkissian, andJean-Philippe Vial, Structure exploiting tool in algebraic mod-elling languages, Management Science 46 (2000), 1145–1158.

[FGV00] Emmanuel Fragniere, Jacek Gondzio, and Jean-Philippe Vial,Building and solving large-scale stochastic programs on an af-fordable distributed computing system, Annals of OperationsResearch 99 (2000), 167–187, (we referred to an earlier editionof the paper entitled A planning model with one million sce-narios solved on an affordable parallel machine, dated 1998).

BIBLIOGRAPHY 231

[FH98] Michael C. Ferris and Jeffrey D. Horn, Partitioning mathemat-ical programs for parallel solution, Mathematical Programming80 (1998), 35–61.

[Fou82] Robert Fourer, Solving staircase linear programs by the simplexmethod, 1: Inversion, Mathematical Programming 23 (1982),274–313.

[Fou83] Robert Fourer, Modelling languages versus matrix generatorsfor linear programming, ACM Transactions on MathematicalSoftware 9 (1983), no. 2, 143–183.

[Fou96a] Robert Fourer, Software for optimisation: A buyer’s guide part1, INFORMS Computer Science Technical Section Newsletter17 (1996), no. 1, 14–17.

[Fou96b] Robert Fourer, Software for optimisation: A buyer’s guide part2, INFORMS Computer Science Technical Section Newsletter17 (1996), no. 2, 3–4, 9–10.

[Fra97] Antonio Frangioni, Description of the PPRN file format formulticommodity min cost flow problems, Technical report, Op-erations Research Group, Dipartimento di Informatica, Uni-versita di Pisa, Pisa, 10 April 1997, obtained through [Fra00].

[Fra00] Antonio Frangioni, Multicommodity problems, See http://www.di.unipi.it/di/groups/optimize/Data/MMCF.html,2000.

[Gay97] David M. Gay, Electronic mail distribution of linear program-ming test problems, Technical report, Lucent Bell Laboratories,Murray Hill, New Jersey 07974, 1 February 1997, an earlierversion was published in the MPS COAL Newsletter, 13:10–12, December 1985.

[GBD+] A. Geist, A. Beguelin, J. Dongarra, W. Jiang, R. Manchek,and V. Sunderam, PVM: Parallel virtual machine—a users’guide and tutorial for networked parallel computing, Availableat http://www.netlib.org/pvm3/book/pvm-book.html.

[Geo70] Arthur M. Geoffrion, Elements of large-scale mathematical pro-gramming: Parts I and II, Management Science 16 (1970),no. 11, 652–691.

[GG98] William P. Gardiner and George Gettinby, Experimental designtechniques in statistical practice: A practical software-basedapproach, Horwood Series in Mathematics and Applications,Horwood, Chichester, 1998.

232 BIBLIOGRAPHY

[GH93] S. Kingsley Gnanendran and James K. Ho, Load balancing inthe parallel optimization of block-angular programs, Mathemat-ical Programming 62 (1993), no. 1, 41–67.

[GHVZ93] J.-L. Goffin, Alain Haurie, Jean-Philippe Vial, and D. L. Zhu,Using central prices in the decomposition of linear programs,European Journal of Operational Research 64 (1993), 393–409.

[GI95] Horand I. Gassmann and Alice M. Ireland, Scenario formula-tion in an algebraic modelling language, Annals of OperationsResearch 59 (1995), 45–75.

[GI96] Horand I. Gassmann and Alice M. Ireland, On the automaticformulation of stochastic linear programs, Annals of Opera-tions Research 64 (1996), 83–112.

[GKT51] D. Gale, H. W. Kuhn, and A. W. Tucker, Linear programmingand the theory of games, Activity Analysis of Production andAllocation (Tjalling C. Koopmans, ed.), Cowles CommissionMonograph 13, John Wiley & Sons, New York, 1951, proceed-ings of the Conference on Linear Programming, Chicago, 20–24 June 1949.

[Gla73] C. Roger Glassey, Nested decomposition and multi-stage linearprograms, Management Science 20 (1973), no. 3, 282–292.

[GM92] Harvey J. Greenberg and Frederic H. Murphy, A comparison ofmathematical programming modeling systems, Annals of Oper-ations Research 38 (1992), 177–238.

[GPS00] Christelle Gueret, Christian Prins, and Marc Sevaux, Program-mation lineaire, Editions Eyrolles, Paris, 2000, in French.

[GR77] D. Goldfarb and J. K. Reid, A practicable steepest edge simplexalgorithm, Mathematical Programming 12 (1977), 361–371.

[Gre92] Harvey J. Greenberg, Intelligent analysis support for linearprograms, Computers and Chemical Engineering 16 (1992),no. 7, 659–673.

[Gri80] Michael D. Grigoriadis, Partitioning methods for block diago-nal linear systems and programs, Technical report, IBM Cor-poration, 1133 Westchester Avenue, White Plains, New York10604, July 1980.

[GS96] Jacek Gondzio and Robert Sarkissian, Column generation witha primal-dual method, Logilab technical report 1996.6, Depart-ment of Management Studies, University of Geneva, 102 BdCarl-Vogt, CH-1211 Geneve 4, 27 June 1996.

BIBLIOGRAPHY 233

[GSV97] Jacek Gondzio, Robert Sarkissian, and Jean-Philippe Vial, Us-ing interior point method for the master problem in a decom-position approach, European Journal of Operational Research101 (1997), 577–587.

[GSV01] Jacek Gondzio, Robert Sarkissian, and Jean-Philippe Vial,Parallel implementation of a central decomposition method forsolving large-scale planning problems, Computational Opti-mization and Applications 19 (2001), 5–29.

[GV99] Jacek Gondzio and Jean-Philippe Vial, Warm start and ε-subgradients in cutting plane scheme for block-angular linearprograms, Computational Optimization and Applications 14(1999), 17–36.

[Har73] Paula Harris, Pivot selection methods of the Devex code, Math-ematical Programming 5 (1973), 1–28.

[Him73] David M. Himmelblau (ed.), Decomposition of large scale prob-lems, North-Holland, Amsterdam, 1973.

[HL80] James K. Ho and Etienne Loute, A comparative study of twomethods for staircase linear programs, ACM Transactions onMathematical Software 6 (1980), no. 1, 17–30.

[HL81] James K. Ho and Etienne Loute, An advanced implementationof the Dantzig-Wolfe decomposition algorithm for linear pro-gramming, Mathematical Programming 20 (1981), 303–326.

[HL83] James K. Ho and Etienne Loute, Computational experiencewith advanced implementation of decomposition algorithms forlinear programming, Mathematical Programming 27 (1983),283–290.

[HL86] James K. Ho and Etienne Loute, On the degree of decomposi-tion in linear programming, Informatica 7 (1986), no. 3, 337–348.

[HLS88] James K. Ho, Tak C. Lee, and R. P. Sundarraj, Decompositionof linear programs using parallel computation, MathematicalProgramming 42 (1988), 391–405.

[HLSvdV79] James K. Ho, Etienne Loute, Yves Smeers, and E. van deVoort, The use of decomposition techniques for large-scale lin-ear programming energy models, Energy Policy (A. Strub, ed.),IPC Press, London, 1979, special issue on Energy Models forthe European Community, pp. 94–101.

234 BIBLIOGRAPHY

[HM74] James K. Ho and Alan S. Manne, Nested decomposition fordynamic models, Mathematical Programming 6 (1974), 121–140.

[HM00] Julian A. J. Hall and Ken I. M. McKinnon, Hyper-sparsity inthe revised simplex method and how to exploit it, MS 00-015,Department of Mathematics and Statistics, University of Ed-inburgh, The King’s Buildings, Edinburgh, EH9 3JZ, August2000, submitted to SIAM Journal of Optimization.

[Ho74] James K. Ho, Nested decomposition of large scale linear pro-grams with the staircase structure, Ph.D. thesis and technicalreport SOL 74-4, Systems Optimization Laboratory, Depart-ment of Operations Research, Stanford University, Stanford,California, 1974.

[Ho78] James K. Ho, Implementation and application of a nesteddecomposition algorithm, Computers and Mathematical Pro-gramming (W. W. White, ed.), (US) National Bureau of Stan-dards, 1978, special publication no. 502, pp. 21–30.

[Ho84] James K. Ho, Convergence behaviour of decomposition algo-rithms for linear programs, Operations Research Letters 3(1984), no. 2, 91–94.

[Ho87] James K. Ho, Recent advances in the decomposition approachto linear programming, Mathematical Programming Study 31(1987), 119–128.

[HS89] James K. Ho and R. P. Sundarraj, DECOMP: An implemen-tation of Dantzig-Wolfe decomposition for linear programming,Lecture Notes in Economics and Mathematical Systems, no.338, Springer Verlag, New York, 1989.

[JL92] Kimberly L. Jones and Irvin J. Lustig, Input format for multi-commodity flow problems, Technical report, Program in Statis-tics and Operations Research, Department of Civil Engineer-ing and Operations Research, Princeton University, Princeton,New Jersey, November 1992, obtained through [Fra00].

[JLFP93] Kimberly L. Jones, Irvin J. Lustig, Judith M. Farvolden, andWarren B. Powell, Multicommodity network flows: The impactof formulation on decomposition, Mathematical Programming62 (1993), 95–117.

[KL70] Brian W. Kernighan and S. Lin, An efficient heuristic proce-dure for partitioning graphs, The Bell System Technical Jour-nal (1970), 291–307.

BIBLIOGRAPHY 235

[KN91] K. Kim and J. Larry Nazareth, The decomposition principleand algorithms for linear programming, Linear Algebra and itsApplications 152 (1991), 119–133.

[Kut73] Gary P. Kutcher, On decomposing price-endogenous models,Multi-level Planning: Case Studies in Mexico (Louis M. Gorensand Alan S. Manne, eds.), North-Holland, Amsterdam, 1973.

[Las70] Leon S. Lasdon, Optimization theory for large systems, Mac-millan Series in Operations Research, Macmillan, New York,1970.

[Lou97] Etienne Loute, Private communication, 1997.

[Lus89] Irvin J. Lustig, An analysis of an available set of linear pro-gramming test problems, Computers and Operations Research16 (1989), no. 2, 173–184.

[Mad73] Oli B. G. Madsen, The connection between decomposition algo-rithms and the optimal degree of decomposition, In Himmelblau[Him73], pp. 241–250.

[Mel98] David Mellor, High Performance PVM, Parallel Computer Sys-tems, Coventry; see http://www.parasys.co.uk, 1998.

[Mon01] Douglas C. Montgomery, Design and analysis of experiments,5th ed., John Wiley & Sons, New York, 2001.

[MT99] Ruben Kirkeby Martinson and Jørgen Tind, An interior pointmethod in Dantzig-Wolfe decomposition, Computers and Op-erations Research 26 (1999), 1195–1216.

[Naz84] J. Larry Nazareth, Numerical behaviour of LP algorithms basedupon the decomposition principle, Linear Algebra and its Ap-plications 57 (1984), 181–189.

[NW88] George L. Nemhauser and Laurence A. Wolsey, Integer andcombinatorial optimization, Wiley Interscience Series in Dis-crete Mathematics and Optimization, John Wiley & Sons, NewYork, 1988.

[OH68] William Orchard-Hays, Advanced linear-programming comput-ing techniques, McGraw-Hill, New York, 1968.

[OH72] William Orchard-Hays, Do-it-yourself-decomposition, unpub-lished manuscript, circa 1972.

236 BIBLIOGRAPHY

[OH73] William Orchard-Hays, Practical problems in LP decomposi-tion and a standardized phase I decomposition as a tool forsolving large scale problems, In Himmelblau [Him73], pp. 153–166.

[Pow98] PowerGen plc, Private communication, 1998.

[RL82] R. L. Rardin and B. W. Lin, Test problems for computationalexperiments—issues and techniques, Evaluating MathematicalProgramming Techniques (J. M. Mulvey, ed.), Springer Verlag,Berlin, 1982, proceedings of a conference held at the NationalBureau of Standards, Boulder, Colorado, 5–6 January 1981.

[Sav97] Martin W. P. Savelsbergh, A branch-and-price algorithm forthe generalised assignment problem, Operations Research 45(1997), 831–841.

[Sch86] Alexander Schrijver, Theory of linear and integer program-ming, Wiley Interscience Series in Discrete Mathematics andOptimization, John Wiley & Sons, Chichester, 1986.

[SR95] Ramesh Sharda and Gayatri Rampal, Algebraic modeling onPCs, OR/MS Today 23 (1995), no. 3, 58–63.

[SS93] David Steiger and Ramesh Sharda, LP modeling languages forpersonal computers: A comparison, Annals of Operations Re-search 43 (1993), 195–216.

[Tho97] Robin T. Thompson, Fast sequential and parallel methods forsolving multistage stochastic linear programmes, Ph.D. thesis,Department of Mathematics, University of Essex, Colchester,April 1997.

[Tom75a] John A. Tomlin, On scaling linear programming problems,Mathematical Programming Study 4 (1975), 146–166.

[Tom75b] John A. Tomlin, User’s guide for LPM1, Systems Optimiza-tion Laboratory, Department of Operations Research, StanfordUniversity, Stanford, California, 1975.

[TW92] John A. Tomlin and James S. Welch, Mathematical program-ming systems, Computing (E. G. Coffman et al., eds.), Hand-books in OR and MS, vol. 3, North-Holland, New York, 1992,pp. 561–601.

[VW96] Francois Vanderbeck and Laurence A. Wolsey, An exact algo-rithm for IP column generation, Operations Research Letters19 (1996), no. 4, 151–159.

BIBLIOGRAPHY 237

[Wil93] H. Paul Williams, Model building in mathematical program-ming, 3rd revised ed., John Wiley & Sons, Chichester, 1993.

[WK71] Roman L. Weil and Paul C. Kettler, Rearranging matrices toblock-angular form for decomposition (and other) algorithms,Management Science 18 (1971), no. 1, 98–108.

[WR74] H. Paul Williams and A. C. Redwood, A structured linear pro-gramming model in the food industry, Operational ResearchQuarterly 25 (1974), 517–527.

[Wri97] Stephen J. Wright, Primal-dual interior point methods, SIAM,1997.

Documents

A Computational Study of Dantzig-Wolfe Decomposition · additional novel ideas. We use DecOptto evaluate several computational strategies in a consistent computational framework,