New Optimization Algorithms in Physics_1

8/11/2019 New Optimization Algorithms in Physics_1

1/10

New Optimization

Algorithms in Physics

Edited byAlexander K. Hartmann and Heiko Rieger

New Optimization Algorithms in Physics.Edited by Alexander K. Hartmann, Heiko Rieger

Copyright c 2004 Wiley-VCH Verlag GmbH & Co. KGaA

ISBN: 3-527-40406-6


2/10

New Optimization

Algorithms in Physics

Edited byAlexander K. Hartmann and Heiko Rieger


3/10

Editors

Alexander K.Hartmann

Universitt Gttingen, Germany

[email protected]

Heiko Rieger

Universitt des Saarlandes, Germany

[email protected]

Cover Picture

Short artificial peptide (U. H. E. Hansmann);

Dense configuration of polydisperse hard discs (W.

Krauth); Cut Polytope for the complete graph withN=3 (F. Liers et al); Factor graph for 3-SAT (R.

Zecchina)

This book was carefully produced.Nevertheless,

editors,authors and publisher do not warrant the

information contained therein to be free of errors.Readers are advised to keep in mind that state-

ments,data, illustrations, procedural details orother items may inadvertently be inaccurate.

Library of Congress Card No.: applied for

British Library Cataloging-in-Publication Data:

A catalogue record for this book is available fromthe British Library

Bibliographic information published by

Die Deutsche Bibliothek

Die Deutsche Bibliothek lists this publication inthe Deutsche Nationalbibliografie; detailed bibli-

ographic data is available in the Internet at.

2004 WILEY-VCH Verlag GmbH & Co. KGaA,

Weinheim

All rights reserved (including those of translationinto other languages). No part of this book may

be reproduced in any form nor transmitted or

translated into machine language without writtenpermission from the publishers. Registered

names, trademarks, etc. used in this book, evenwhen not specifically marked as such, are not to

be considered unprotected by law.

Printed in the Federal Republic of Germany

Printed on acid-free paper

Composition Uwe Krieg,Berlin

Printing Strauss GmbH, MrlenbachBookbinding Litges & Dopf Buchbinderei

GmbH, HeppenheimISBN 3-527-40406-6


4/10

Contents

List of Contributors XI

1 Introduction

(A.K. Hartmann and H. Rieger

) 1

Part I Applications in Physics 5

2 Cluster Monte Carlo Algorithms

(W. Krauth) 7

2.1 Detailed Balance and a priori Probabilities . . . . . . . . . . . . . . . . . . . 7

2.2 The Wolff Cluster Algorithm for the Ising Model . . . . . . . . . . . . . . . 10

2.3 Cluster Algorithm for Hard Spheres and Related Systems . . . . . . . . . . . 12

2.4 Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.4.1 Phase Separation in Binary Mixtures . . . . . . . . . . . . . . . . . . 16

2.4.2 Polydisperse Mixtures . . . . . . . . . . . . . . . . . . . . . . . . . 18

2.4.3 Monomer-Dimer Problem . . . . . . . . . . . . . . . . . . . . . . . 19

2.5 Limitations and Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

3 Probing Spin Glasses with Heuristic Optimization Algorithms

(O.C. Martin) 23

3.1 Spin Glasses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.1.1 Motivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

3.1.2 The Ising Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

3.1.3 Models of Spin Glasses . . . . . . . . . . . . . . . . . . . . . . . . . 243.1.4 Some Challenges . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.2 Some Heuristic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.2.1 General Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

3.2.2 Variable Depth Search . . . . . . . . . . . . . . . . . . . . . . . . . 32

3.2.3 Genetic Renormalization Algorithm . . . . . . . . . . . . . . . . . . 36

3.3 A Survey of Physics Results . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.3.1 Convergence of the Ground-state Energy Density . . . . . . . . . . . 41

3.3.2 Domain Walls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.3.3 Clustering of Ground States . . . . . . . . . . . . . . . . . . . . . . 42

3.3.4 Low-energy Excitations . . . . . . . . . . . . . . . . . . . . . . . . 42


5/10

VI Contents

3.3.5 Phase Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

3.4 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

4 Computing Exact Ground States of Hard Ising Spin Glass Problems byBranch-and-cut

(F. Liers, M. Jnger, G. Reinelt, and G. Rinaldi) 47

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47

4.2 Ground States and Maximum Cuts . . . . . . . . . . . . . . . . . . . . . . . 48

4.3 A General Scheme for Solving Hard Max-cut Problems . . . . . . . . . . . . 51

4.4 Linear Programming Relaxations of Max-cut . . . . . . . . . . . . . . . . . 55

4.5 Branch-and-cut . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

4.6 Results of Exact Ground-state Computations . . . . . . . . . . . . . . . . . . 62

4.7 Advantages of Branch-and-cut . . . . . . . . . . . . . . . . . . . . . . . . . 65

4.8 Challenges for the Years to Come . . . . . . . . . . . . . . . . . . . . . . . 66References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

5 Counting States and Counting Operations

(A. Alan Middleton) 71

5.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

5.2 Physical Questions about Ground States . . . . . . . . . . . . . . . . . . . . 72

5.2.1 Homogeneous Models . . . . . . . . . . . . . . . . . . . . . . . . . 72

5.2.2 Magnets with Frozen Disorder . . . . . . . . . . . . . . . . . . . . . 73

5.3 Finding Low-energy Configurations . . . . . . . . . . . . . . . . . . . . . . 75

5.3.1 Physically Motivated Approaches . . . . . . . . . . . . . . . . . . . 755.3.2 Combinatorial Optimization . . . . . . . . . . . . . . . . . . . . . . 76

5.3.3 Ground-state Algorithm for the RFIM . . . . . . . . . . . . . . . . . 78

5.4 The Energy Landscape: Degeneracy and Barriers . . . . . . . . . . . . . . . 80

5.5 Counting States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82

5.5.1 Ground-state Configuration Degeneracy . . . . . . . . . . . . . . . . 83

5.5.2 Thermodynamic State . . . . . . . . . . . . . . . . . . . . . . . . . 85

5.5.3 Numerical Studies of Zero-temperature States . . . . . . . . . . . . . 86

5.6 Running Times for Optimization Algorithms . . . . . . . . . . . . . . . . . . 91

5.6.1 Running Times and Evolution of the Heights . . . . . . . . . . . . . 92

5.6.2 Heuristic Derivation of Running Times . . . . . . . . . . . . . . . . 945.7 Further Directions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

6 Computing the Potts Free Energy and Submodular Functions

(J.-C. Angls dAuriac) 101

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

6.2 The Potts Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

6.2.1 Definition of the Potts Model . . . . . . . . . . . . . . . . . . . . . . 102

6.2.2 Some Results for Non-random Models . . . . . . . . . . . . . . . . . 103

6.2.3 The Ferromagnetic Random Bond Potts Model . . . . . . . . . . . . 103


6/10

Contents VII

6.2.4 High Temperature Development . . . . . . . . . . . . . . . . . . . . 103

6.2.5 Limit of an Infinite Number of States . . . . . . . . . . . . . . . . . 104

6.3 Basics on the Minimization of Submodular Functions . . . . . . . . . . . . . 105

6.3.1 Definition of Submodular Functions . . . . . . . . . . . . . . . . . . 1056.3.2 A Simple Characterization . . . . . . . . . . . . . . . . . . . . . . . 105

6.3.3 Examples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

6.3.4 Minimization of Submodular Function . . . . . . . . . . . . . . . . . 106

6.4 Free Energy of the Potts Model in the Infinite q-Limit . . . . . . . . . . . . . 107

6.4.1 The Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

6.4.2 The Auxiliary Problem . . . . . . . . . . . . . . . . . . . . . . . . . 108

6.4.3 The Max-flow Problem: the Goldberg and Tarjan Algorithm . . . . . 111

6.4.4 About the Structure of the Optimal Sets . . . . . . . . . . . . . . . . 111

6.5 Implementation and Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . 112

6.5.1 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1126.5.2 Example of Application . . . . . . . . . . . . . . . . . . . . . . . . 114

6.5.3 Evaluation of the CPU Time . . . . . . . . . . . . . . . . . . . . . . 114

6.5.4 Memory Requirement . . . . . . . . . . . . . . . . . . . . . . . . . 114

6.5.5 Various Possible Improvements . . . . . . . . . . . . . . . . . . . . 115

6.6 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

Part II Phase Transitions in Combinatorial Optimization Problems 119

7 The Random 3-satisfiability Problem: From the Phase Transition to the Efficient

Generation of Hard, but Satisfiable Problem Instances

(M. Weigt) 121

7.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

7.2 Random 3-SAT and the SAT/UNSAT Transition . . . . . . . . . . . . . . . . 122

7.2.1 Numerical Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

7.2.2 Using Statistical Mechanics . . . . . . . . . . . . . . . . . . . . . . 124

7.3 Satisfiable Random 3-SAT Instances . . . . . . . . . . . . . . . . . . . . . . 127

7.3.1 The Naive Generator . . . . . . . . . . . . . . . . . . . . . . . . . . 129

7.3.2 Unbiased Generators . . . . . . . . . . . . . . . . . . . . . . . . . . 130

7.4 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 136

8 Analysis of Backtracking Procedures for Random Decision Problems

(S. Cocco, L. Ein-Dor, and R. Monasson) 139

8.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139

8.2 Phase Diagram, Search Trajectories and the Easy SAT Phase . . . . . . . . . 143

8.2.1 Overview of Concepts Useful to DPLL Analysis . . . . . . . . . . . 144

8.2.2 Clause Populations: Flows, Averages and Fluctuations . . . . . . . . 145

8.2.3 Average-case Analysis in the Absence of Backtracking . . . . . . . . 147

8.2.4 Occurrence of Contradictions and Polynomial SAT Phase . . . . . . . 150


7/10

VIII Contents

8.3 Analysis of the Search Tree Growth in the UNSAT Phase . . . . . . . . . . . 153

8.3.1 Numerical Experiments . . . . . . . . . . . . . . . . . . . . . . . . 153

8.3.2 Parallel Growth Process and Markovian Evolution Matrix . . . . . . 155

8.3.3 Generating Function and Large-size Scaling . . . . . . . . . . . . . . 158

8.3.4 Interpretation in Terms of Growth Process . . . . . . . . . . . . . . . 1618.4 Hard SAT Phase: Average Case and Fluctuations . . . . . . . . . . . . . . . 164

8.4.1 Mixed Branch and Tree Trajectories . . . . . . . . . . . . . . . . . . 164

8.4.2 Distribution of Running Times . . . . . . . . . . . . . . . . . . . . . 165

8.4.3 Large Deviation Analysis of the First Branch in the Tree . . . . . . . 167

8.5 The Random Graph Coloring Problem . . . . . . . . . . . . . . . . . . . . . 171

8.5.1 Description of DPLL Algorithm for Coloring . . . . . . . . . . . . . 171

8.5.2 Coloring in the Absence of Backtracking . . . . . . . . . . . . . . . 172

8.5.3 Coloring in the Presence of Massive Backtracking . . . . . . . . . . 173

8.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179

9 New Iterative Algorithms for Hard Combinatorial Problems

(R. Zecchina) 183

9.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183

9.2 Combinatorial Decision Problems, K-SAT and the Factor Graph Representation185

9.2.1 Random K-SAT . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

9.3 Growth Process Algorithm: Probabilities, Messages and Their Statistics . . . 190

9.4 Traditional Message-passing Algorithm: Belief Propagation as Simple Cavity

Equations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192

9.5 Survey Propagation Equations . . . . . . . . . . . . . . . . . . . . . . . . . 1949.6 Decimating Variables According to Their Statistical Bias . . . . . . . . . . . 195

9.7 Conclusions and Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . 197

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199

Part III New Heuristics and Interdisciplinary Applications 203

10 Hysteretic Optimization

(K.F. Pl) 205

10.1 Hysteretic Optimization for Ising Spin Glasses . . . . . . . . . . . . . . . . . 206

10.2 Generalization to Other Optimization Problems . . . . . . . . . . . . . . . . 21410.3 Application to the Traveling Salesman Problem . . . . . . . . . . . . . . . . 221

10.4 Outlook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 224

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

11 Extremal Optimization

(S. Boettcher) 227

11.1 Emerging Optimality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227

11.2 Extremal Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

11.2.1 Basic Notions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 228

11.2.2 EO Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 230


8/10

Contents IX

11.2.3 Extremal Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . 231

11.2.4 Rank Ordering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232

11.2.5 Defining Fitness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234

11.2.6 Distinguishing EO from other Heuristics . . . . . . . . . . . . . . . 235

11.2.7 Implementing EO . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

11.3 Numerical Results for EO . . . . . . . . . . . . . . . . . . . . . . . . . . . . 238

11.3.1 Early Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239

11.3.2 Applications of EO by Others . . . . . . . . . . . . . . . . . . . . . 242

11.3.3 Large-scale Simulations of Spin Glasses . . . . . . . . . . . . . . . . 243

11.4 Theoretical Investigations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 246

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 249

12 Sequence Alignments

(A.K. Hartmann) 253

12.1 Molecular Biology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253

12.2 Alignments and Alignment Algorithms . . . . . . . . . . . . . . . . . . . . . 259

12.3 Low-probability Tail of Alignment Scores . . . . . . . . . . . . . . . . . . . 266

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 271

13 Protein Folding in Silico the Quest for Better Algorithms

(U.H.E. Hansmann) 275

13.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275

13.2 Energy Landscape Paving . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277

13.3 Beyond Global Optimization . . . . . . . . . . . . . . . . . . . . . . . . . . 280

13.3.1 Parallel Tempering . . . . . . . . . . . . . . . . . . . . . . . . . . . 28013.3.2 Multicanonical Sampling and Other Generalized-ensemble Techniques 283

13.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288

13.4.1 Helix Formation and Folding . . . . . . . . . . . . . . . . . . . . . . 288

13.4.2 Structure Predictions of Small Proteins . . . . . . . . . . . . . . . . 290

13.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293

References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 293

Index 297


9/10

List of Contributors

Jean-Christian Angls dAuriac, Ch. 6

Centre de Recherches sur les Tres Basses Tem-

peratures

BP 166, F-38042 Grenoble, France

e-mail: [email protected]

Stefan Boettcher, Ch. 11

Stefan Boettcher

Physics Department

Emory University

Atlanta, GA 30322; USA


www.physics.emory.edu/faculty/boettcher

Simona Cocco, Ch. 8

Laboratoire de Dynamique des Fluides Com-

plexes

3 rue de lUniversit

67000 Strasbourg, France


Liat Ein-Dor, Ch. 8

Department of Physics of Complex Systems

Wezmann Institute

Rehovot 76100, Israel

and

Laboratoire de Physique Thorique de lENS

24 rue Lhomond

75005 Paris, France

Ulrich H. E. Hansmann, Ch. 13

Department of Physics

Michigan Technological University

Houghton

MI 49931-1295, USA


Alexander K. Hartmann, Ch. 1, 12

Institut fr Theoretische Physik

Universitt Gttingen

Tammanstrae 1

D-37077 Gttingen, Germany

e-mail:

[email protected]

Michael Jnger, Ch. 4

Institut fr Informatik

Universitt zu Kln

Pohligstrae 1

D-50969 Kln, Germany


Werner Krauth, Ch. 2

CNRS-Laboratoire de Physique Statistique

Ecole Normale Suprieure24, rue Lhomond

F-75231 Paris Cedex 05, France


Frauke Liers, Ch. 4


Universitt zu Kln

Pohligstrae 1

D-50969 Kln, Germany


New Optimization Algorithms in Physics.Edited by Alexander K. Hartmann, Heiko Rieger

Copyright c 2004 Wiley-VCH Verlag GmbH & Co. KGaA

ISBN: 3-527-40406-6


10/10

XII List of Contributors

Olivier C. Martin, Ch. 3

LPTMS, Universit Paris-Sud

Orsay Cedex 91405, France


A. Alan Middleton, Ch. 5

Department of Physics

Syracuse University

Syracuse, NY, USA


Remi Monasson, Ch. 8Laboratoire de Physique Thorique de lENS

24 rue Lhomond

75005 Paris, France


and

Laboratoire de Physique Thorique

3 rue de lUniversit67000 Strasbourg, France

Kroly F. Pl, Ch. 10

Department of Theoretical Physics

Institute of Nuclear Research of the

Hungarian Academy of Sciences

Bem tr 18/c

H-4026 Debrecen, Hungary


Gerhard Reinelt, Ch. 4


Ruprecht-Karls-Universitt Heidelberg

Im Neuenheimer Feld 368

D-69120 Heidelberg, Germany

e-mail:

[email protected]

Heiko Rieger, Ch. 1

Theoretische Physik

Universitt des Saarlandes

D-66041 Saarbrcken, Germany


Giovanni Rinaldi, Ch. 4

Istituto di Analisi dei Sistemi ed Informatica

Antonio Ruberti - CNR

Viale Manzoni, 30

00185 Roma, Italy


Martin Weigt, Ch. 7

Institut fr Theoretische Physik

Universitt GttingenTammanstrae 1

D-37077 Gttingen, Germany


Riccardo Zecchina, Ch. 9

International Centre for Theoretical Physics

(ICTP)

Strada Costiera, 11 P.O.Box 586

I-34100 Trieste, Italy


Documents

New Optimization Algorithms in Physics_1