SCALABLE ALGORITHMS FOR PARALLEL
TREE SEARCH
by
Yan Xu
Presented to the Graduate and Research Committee
of Lehigh University
in Candidacy for the Degree of
Doctor of Philosophy
in
Industrial and Systems Engineering
Lehigh University
October 2007
Approved and recommended for acceptance as a dissertation in partial fulfillment of
the requirements for the degree of Doctor of Philosophy.
Date
Dr. Theodore K. Ralphs
Dissertation Advisor
Accepted Date
Committee:
Dr. Theodore K. Ralphs, Chairman
Dr. Laszlo Ladányi
Dr. Jeffrey T. Linderoth
Dr. George R. Wilson
ii
Acknowledgments
First of all, I must express my sincere appreciation to my advisor Theodore Ralphs for
his wonderful guidance that made this work possible. I would also like to thank Laszlo
Ladányi, Jeffrey Linderoth and George Wilson for sharing their brilliant ideas on many
subjects with me, and also for their consistent support for completing this work. Thanks
are due to Matthew Saltzman for letting me use the cluster at Clemson University and
for contributions to CHiPPS.
So many people made my life easier at Lehigh. Here are some names come to
my mind: Rita Frey, Kathy Rambo, Minzhou Jing, Rui Kuang, Qing Ye, Peiling Wu,
Jianbiao Pan, Tianhao Wu, Jinglong Ban, Qunhong Tang, Shangyuan Luo, Wei Wang,
Menal Guzelsoy, Ashutosh Mahajan, Svetlana Oshkai, Mu Qing, and Ying Rong. Thank
you all for the advices and accommodations that you generously provided to me.
I have wonderful co-workers at SAS, who tolerated my frequently vacations at
Lehigh. Special thanks to Alexander Andrianov, Manoj Chari, Hao Cheng, Matthew
Galati, Jennie Hu, Trevor Kearney, Radhika Kulkarni, and Benhao Wang for the under-
standing and support.
Although my parents are in China, I feel they are here with me everyday. My wife
Kun Yang disagreed with me on quite a few things, fortunately, she firmly supported
iii
me to complete this research. It is almost impossible to forget to mention my daughter
Emily Xu, who always competes with me to play games on the laptop that I used to
write this thesis. You bring so much happiness to me. I love you all very much.
iv
Contents
Acknowledgments iii
Contents v
List of Tables x
List of Figures xii
Abstract 1
1 Introduction 4
1.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1.1 Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
1.1.2 Tree Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
1.2 Tree Search Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2.1 General Framework . . . . . . . . . . . . . . . . . . . . . . . . 9
1.2.2 Branch and Bound . . . . . . . . . . . . . . . . . . . . . . . . 13
1.3 Classes of Tree Search Problems . . . . . . . . . . . . . . . . . . . . . 16
1.3.1 Mathematical Programming Problems . . . . . . . . . . . . . . 16
v
1.3.2 Constraint Satisfaction Problems . . . . . . . . . . . . . . . . . 18
1.4 Parallel Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
1.4.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
1.4.2 Parallel Computers . . . . . . . . . . . . . . . . . . . . . . . . 21
1.4.3 Parallel Programming Tools . . . . . . . . . . . . . . . . . . . 28
1.4.4 Parallel Algorithm Paradigms . . . . . . . . . . . . . . . . . . 32
1.4.5 Scalability Analysis . . . . . . . . . . . . . . . . . . . . . . . 35
1.4.6 Performance Measures . . . . . . . . . . . . . . . . . . . . . . 37
1.4.7 Termination Detection . . . . . . . . . . . . . . . . . . . . . . 41
1.5 Scalability Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
1.5.1 Levels of Parallelism . . . . . . . . . . . . . . . . . . . . . . . 42
1.5.2 Parallel Algorithm Phases . . . . . . . . . . . . . . . . . . . . 43
1.5.3 Synchronization and Handshaking . . . . . . . . . . . . . . . . 44
1.5.4 Knowledge Sharing . . . . . . . . . . . . . . . . . . . . . . . . 45
1.5.5 Implementation Paradigms . . . . . . . . . . . . . . . . . . . . 46
1.5.6 Task Granularity . . . . . . . . . . . . . . . . . . . . . . . . . 47
1.5.7 Static Load Balancing . . . . . . . . . . . . . . . . . . . . . . 47
1.5.8 Dynamic Load Balancing . . . . . . . . . . . . . . . . . . . . 49
1.6 Previous Frameworks and Applications . . . . . . . . . . . . . . . . . 50
1.7 Research Objectives and Thesis Outline . . . . . . . . . . . . . . . . . 54
2 A Framework for Parallel Tree Search 56
2.1 Implementation Paradigm . . . . . . . . . . . . . . . . . . . . . . . . . 57
2.1.1 The Master-Hub-Worker Paradigm . . . . . . . . . . . . . . . . 57
2.1.2 Process Clustering . . . . . . . . . . . . . . . . . . . . . . . . 58
vi
2.2 Knowledge Management . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.2.1 Knowledge Objects . . . . . . . . . . . . . . . . . . . . . . . . 60
2.2.2 Knowledge Pools . . . . . . . . . . . . . . . . . . . . . . . . . 61
2.2.3 Knowledge Brokers . . . . . . . . . . . . . . . . . . . . . . . . 62
2.3 Knowledge Sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.4 Task Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
2.4.1 Multi-level Task Management System . . . . . . . . . . . . . . 65
2.4.2 Key Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
2.5 Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
2.5.1 Static Load Balancing . . . . . . . . . . . . . . . . . . . . . . 74
2.5.2 Dynamic Load Balancing . . . . . . . . . . . . . . . . . . . . 79
2.6 Class Hierarchy of ALPS . . . . . . . . . . . . . . . . . . . . . . . . . 86
2.7 Developing Applications . . . . . . . . . . . . . . . . . . . . . . . . . 88
2.7.1 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
2.7.2 Example: The KNAP Application . . . . . . . . . . . . . . . . 89
3 A Framework for Parallel Integer Programming 95
3.1 Branch and Cut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.1.1 Branching Methods . . . . . . . . . . . . . . . . . . . . . . . . 96
3.1.2 Search Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . 102
3.1.3 Valid Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . 105
3.1.4 Primal Heuristics . . . . . . . . . . . . . . . . . . . . . . . . . 112
3.2 Knowledge Management . . . . . . . . . . . . . . . . . . . . . . . . . 113
3.2.1 Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113
3.2.2 Branching Information . . . . . . . . . . . . . . . . . . . . . . 114
vii
3.2.3 Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
3.3 Task Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116
3.3.1 BiCePS Task Management . . . . . . . . . . . . . . . . . . . . 116
3.3.2 BLIS Task Management . . . . . . . . . . . . . . . . . . . . . 116
3.4 The Class Hierarchies of BiCePS and BLIS . . . . . . . . . . . . . . . 118
3.4.1 BiCePS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118
3.4.2 BLIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
3.5 Developing Applications . . . . . . . . . . . . . . . . . . . . . . . . . 130
3.5.1 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130
3.5.2 Example: The Vehicle Routing Problem . . . . . . . . . . . . . 131
4 Computational Results 137
4.1 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.2 Test Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138
4.3 Overall Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141
4.3.1 Solving Moderately Difficult Knapsack Instances . . . . . . . . 142
4.3.2 Solving Difficult Knapsack Instances . . . . . . . . . . . . . . 146
4.3.3 Solving Generic MILP Instances . . . . . . . . . . . . . . . . . 148
4.3.4 Solving VRP Instances . . . . . . . . . . . . . . . . . . . . . . 151
4.4 Comparison with SYMPHONY . . . . . . . . . . . . . . . . . . . . . 152
4.5 The Effect of the Master-Hub-Worker Paradigm . . . . . . . . . . . . . 154
4.6 Choice of Static Load Balancing Scheme . . . . . . . . . . . . . . . . 155
4.7 The Effect of Dynamic Load Balancing Schemes . . . . . . . . . . . . 156
4.7.1 Inter-cluster Dynamic Load Balancing . . . . . . . . . . . . . . 158
4.7.2 Intra-cluster Dynamic Load Balance . . . . . . . . . . . . . . . 158
viii
4.8 The Impact of Dedicated Hubs . . . . . . . . . . . . . . . . . . . . . . 159
4.9 The Impact of Problem Properties . . . . . . . . . . . . . . . . . . . . 160
4.10 The Effect of Sharing Pseudocosts . . . . . . . . . . . . . . . . . . . . 161
4.11 The Impact of Hardware on Performance . . . . . . . . . . . . . . . . 163
4.12 The Effectiveness of Differencing . . . . . . . . . . . . . . . . . . . . 166
4.13
