Click here to load reader

SCALABLE ALGORITHMS FOR PARALLEL TREE ted/files/papers/YanXuDissertation07.pdf · PDF file SCALABLE ALGORITHMS FOR PARALLEL TREE SEARCH by Yan Xu ... Minzhou Jing, Rui Kuang, Qing

  • View
    1

  • Download
    0

Embed Size (px)

Text of SCALABLE ALGORITHMS FOR PARALLEL TREE ted/files/papers/YanXuDissertation07.pdf · PDF...

  • SCALABLE ALGORITHMS FOR PARALLEL

    TREE SEARCH

    by

    Yan Xu

    Presented to the Graduate and Research Committee

    of Lehigh University

    in Candidacy for the Degree of

    Doctor of Philosophy

    in

    Industrial and Systems Engineering

    Lehigh University

    October 2007

  • Approved and recommended for acceptance as a dissertation in partial fulfillment of

    the requirements for the degree of Doctor of Philosophy.

    Date

    Dr. Theodore K. Ralphs

    Dissertation Advisor

    Accepted Date

    Committee:

    Dr. Theodore K. Ralphs, Chairman

    Dr. Laszlo Ladányi

    Dr. Jeffrey T. Linderoth

    Dr. George R. Wilson

    ii

  • Acknowledgments

    First of all, I must express my sincere appreciation to my advisor Theodore Ralphs for

    his wonderful guidance that made this work possible. I would also like to thank Laszlo

    Ladányi, Jeffrey Linderoth and George Wilson for sharing their brilliant ideas on many

    subjects with me, and also for their consistent support for completing this work. Thanks

    are due to Matthew Saltzman for letting me use the cluster at Clemson University and

    for contributions to CHiPPS.

    So many people made my life easier at Lehigh. Here are some names come to

    my mind: Rita Frey, Kathy Rambo, Minzhou Jing, Rui Kuang, Qing Ye, Peiling Wu,

    Jianbiao Pan, Tianhao Wu, Jinglong Ban, Qunhong Tang, Shangyuan Luo, Wei Wang,

    Menal Guzelsoy, Ashutosh Mahajan, Svetlana Oshkai, Mu Qing, and Ying Rong. Thank

    you all for the advices and accommodations that you generously provided to me.

    I have wonderful co-workers at SAS, who tolerated my frequently vacations at

    Lehigh. Special thanks to Alexander Andrianov, Manoj Chari, Hao Cheng, Matthew

    Galati, Jennie Hu, Trevor Kearney, Radhika Kulkarni, and Benhao Wang for the under-

    standing and support.

    Although my parents are in China, I feel they are here with me everyday. My wife

    Kun Yang disagreed with me on quite a few things, fortunately, she firmly supported

    iii

  • me to complete this research. It is almost impossible to forget to mention my daughter

    Emily Xu, who always competes with me to play games on the laptop that I used to

    write this thesis. You bring so much happiness to me. I love you all very much.

    iv

  • Contents

    Acknowledgments iii

    Contents v

    List of Tables x

    List of Figures xii

    Abstract 1

    1 Introduction 4

    1.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

    1.1.1 Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

    1.1.2 Tree Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

    1.2 Tree Search Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . 9

    1.2.1 General Framework . . . . . . . . . . . . . . . . . . . . . . . . 9

    1.2.2 Branch and Bound . . . . . . . . . . . . . . . . . . . . . . . . 13

    1.3 Classes of Tree Search Problems . . . . . . . . . . . . . . . . . . . . . 16

    1.3.1 Mathematical Programming Problems . . . . . . . . . . . . . . 16

    v

  • 1.3.2 Constraint Satisfaction Problems . . . . . . . . . . . . . . . . . 18

    1.4 Parallel Computing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

    1.4.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

    1.4.2 Parallel Computers . . . . . . . . . . . . . . . . . . . . . . . . 21

    1.4.3 Parallel Programming Tools . . . . . . . . . . . . . . . . . . . 28

    1.4.4 Parallel Algorithm Paradigms . . . . . . . . . . . . . . . . . . 32

    1.4.5 Scalability Analysis . . . . . . . . . . . . . . . . . . . . . . . 35

    1.4.6 Performance Measures . . . . . . . . . . . . . . . . . . . . . . 37

    1.4.7 Termination Detection . . . . . . . . . . . . . . . . . . . . . . 41

    1.5 Scalability Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

    1.5.1 Levels of Parallelism . . . . . . . . . . . . . . . . . . . . . . . 42

    1.5.2 Parallel Algorithm Phases . . . . . . . . . . . . . . . . . . . . 43

    1.5.3 Synchronization and Handshaking . . . . . . . . . . . . . . . . 44

    1.5.4 Knowledge Sharing . . . . . . . . . . . . . . . . . . . . . . . . 45

    1.5.5 Implementation Paradigms . . . . . . . . . . . . . . . . . . . . 46

    1.5.6 Task Granularity . . . . . . . . . . . . . . . . . . . . . . . . . 47

    1.5.7 Static Load Balancing . . . . . . . . . . . . . . . . . . . . . . 47

    1.5.8 Dynamic Load Balancing . . . . . . . . . . . . . . . . . . . . 49

    1.6 Previous Frameworks and Applications . . . . . . . . . . . . . . . . . 50

    1.7 Research Objectives and Thesis Outline . . . . . . . . . . . . . . . . . 54

    2 A Framework for Parallel Tree Search 56

    2.1 Implementation Paradigm . . . . . . . . . . . . . . . . . . . . . . . . . 57

    2.1.1 The Master-Hub-Worker Paradigm . . . . . . . . . . . . . . . . 57

    2.1.2 Process Clustering . . . . . . . . . . . . . . . . . . . . . . . . 58

    vi

  • 2.2 Knowledge Management . . . . . . . . . . . . . . . . . . . . . . . . . 60

    2.2.1 Knowledge Objects . . . . . . . . . . . . . . . . . . . . . . . . 60

    2.2.2 Knowledge Pools . . . . . . . . . . . . . . . . . . . . . . . . . 61

    2.2.3 Knowledge Brokers . . . . . . . . . . . . . . . . . . . . . . . . 62

    2.3 Knowledge Sharing . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

    2.4 Task Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

    2.4.1 Multi-level Task Management System . . . . . . . . . . . . . . 65

    2.4.2 Key Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

    2.5 Load Balancing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

    2.5.1 Static Load Balancing . . . . . . . . . . . . . . . . . . . . . . 74

    2.5.2 Dynamic Load Balancing . . . . . . . . . . . . . . . . . . . . 79

    2.6 Class Hierarchy of ALPS . . . . . . . . . . . . . . . . . . . . . . . . . 86

    2.7 Developing Applications . . . . . . . . . . . . . . . . . . . . . . . . . 88

    2.7.1 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

    2.7.2 Example: The KNAP Application . . . . . . . . . . . . . . . . 89

    3 A Framework for Parallel Integer Programming 95

    3.1 Branch and Cut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

    3.1.1 Branching Methods . . . . . . . . . . . . . . . . . . . . . . . . 96

    3.1.2 Search Strategy . . . . . . . . . . . . . . . . . . . . . . . . . . 102

    3.1.3 Valid Inequalities . . . . . . . . . . . . . . . . . . . . . . . . . 105

    3.1.4 Primal Heuristics . . . . . . . . . . . . . . . . . . . . . . . . . 112

    3.2 Knowledge Management . . . . . . . . . . . . . . . . . . . . . . . . . 113

    3.2.1 Bounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

    3.2.2 Branching Information . . . . . . . . . . . . . . . . . . . . . . 114

    vii

  • 3.2.3 Objects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

    3.3 Task Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

    3.3.1 BiCePS Task Management . . . . . . . . . . . . . . . . . . . . 116

    3.3.2 BLIS Task Management . . . . . . . . . . . . . . . . . . . . . 116

    3.4 The Class Hierarchies of BiCePS and BLIS . . . . . . . . . . . . . . . 118

    3.4.1 BiCePS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118

    3.4.2 BLIS . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

    3.5 Developing Applications . . . . . . . . . . . . . . . . . . . . . . . . . 130

    3.5.1 Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

    3.5.2 Example: The Vehicle Routing Problem . . . . . . . . . . . . . 131

    4 Computational Results 137

    4.1 Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

    4.2 Test Suite . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 138

    4.3 Overall Scalability . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

    4.3.1 Solving Moderately Difficult Knapsack Instances . . . . . . . . 142

    4.3.2 Solving Difficult Knapsack Instances . . . . . . . . . . . . . . 146

    4.3.3 Solving Generic MILP Instances . . . . . . . . . . . . . . . . . 148

    4.3.4 Solving VRP Instances . . . . . . . . . . . . . . . . . . . . . . 151

    4.4 Comparison with SYMPHONY . . . . . . . . . . . . . . . . . . . . . 152

    4.5 The Effect of the Master-Hub-Worker Paradigm . . . . . . . . . . . . . 154

    4.6 Choice of Static Load Balancing Scheme . . . . . . . . . . . . . . . . 155

    4.7 The Effect of Dynamic Load Balancing Schemes . . . . . . . . . . . . 156

    4.7.1 Inter-cluster Dynamic Load Balancing . . . . . . . . . . . . . . 158

    4.7.2 Intra-cluster Dynamic Load Balance . . . . . . . . . . . . . . . 158

    viii

  • 4.8 The Impact of Dedicated Hubs . . . . . . . . . . . . . . . . . . . . . . 159

    4.9 The Impact of Problem Properties . . . . . . . . . . . . . . . . . . . . 160

    4.10 The Effect of Sharing Pseudocosts . . . . . . . . . . . . . . . . . . . . 161

    4.11 The Impact of Hardware on Performance . . . . . . . . . . . . . . . . 163

    4.12 The Effectiveness of Differencing . . . . . . . . . . . . . . . . . . . . 166

    4.13

Search related