Embed Size (px)
Citation preview
International Series in OperationsResearch & Management Science
Volume 146
Series Editor:Frederick S. HillierStanford University, CA, USA
Special Editorial Consultant:Camille C. PriceStephen F. Austin State University, TX, USA
For further volumes:http://www.springer.com/series/6161
Michel Gendreau · Jean-Yves PotvinEditors
Handbook of Metaheuristics
Second Edition
123
EditorsMichel GendreauDepartement de mathematiques et de
genie industrielEcole Polytechnique de Montreal, and
Centre interuniversitaire de recherchesur les reseaux d’entreprise
la logistique et le transportC.P. 6079, succ. Centre-villeMontreal, QC H3C 3A7, [email protected]
Jean-Yves PotvinDepartement d’informatique et de
recherche operationnelleUniversite de Montreal, and Centre
interuniversitaire de recherchesur les reseaux d’entreprise
la logistique et le transportC.P. 6128, succ. Centre-villeMontreal, QC H3C 3J7, [email protected]
ISBN 978-1-4419-1663-1 e-ISBN 978-1-4419-1665-5DOI 10.1007/978-1-4419-1665-5Springer New York Dordrecht Heidelberg London
Library of Congress Control Number: 2010933095
c© Springer Science+Business Media, LLC 2003, 2010All rights reserved. This work may not be translated or copied in whole or in part without the writtenpermission of the publisher (Springer Science+Business Media, LLC, 233 Spring Street, New York,NY 10013, USA), except for brief excerpts in connection with reviews or scholarly analysis. Use inconnection with any form of information storage and retrieval, electronic adaptation, computersoftware, or by similar or dissimilar methodology now known or hereafter developed is forbidden.The use in this publication of trade names, trademarks, service marks, and similar terms, even ifthey are not identified as such, is not to be taken as an expression of opinion as to whether or notthey are subject to proprietary rights.
Printed on acid-free paper
Springer is part of Springer Science+Business Media (www.springer.com)
A nos epouses Johanne et Lynne et a nosenfants Catherine, Laurent, Gabrielle,Stephanie et Simon.
Preface
The first edition of the Handbook of Metaheuristics was published in 2003 underthe editorship of Fred Glover and Gary A. Kochenberger. Given the numerous de-velopments observed in the field of metaheuristics in recent years, it appeared thatthe time was ripe for a second edition of the Handbook. For different reasons, Fredand Gary were unable to accept Springer’s invitation to prepare this second edi-tion and they suggested that we should take over the editorship responsibility of theHandbook. We are deeply honored and grateful for their trust.
As stated in the first edition, metaheuristics are “solution methods that orches-trate an interaction between local improvement procedures and higher level strate-gies to create a process capable of escaping from local optima and performing arobust search of a solution space.” Although this broad characterization still holdstoday, many new and exciting developments and extensions have been observed inthe last few years. We think in particular to hybrids, which take advantage of thestrengths of each of their individual metaheuristic components to better explore thesolution space. Hybrids of metaheuristics with other optimization techniques, likebranch-and-bound, mathematical programming or constraint programming are alsoincreasingly popular. On the front of applications, metaheuristics are now used tofind high-quality solutions to an ever-growing number of complex, ill-defined real-world problems, in particular combinatorial ones.
This second edition of the Handbook of Metaheuristics, through its 21 chapters,is designed to provide a broad coverage of the concepts, implementations, and ap-plications in this important field of optimization. We were glad to get a positive re-sponse from renowned experts for each chapter. They either accepted to revise andupdate their chapter from the first edition or to write brand new ones. The Hand-book now includes updated chapters on the best known metaheuristics, includingsimulated annealing, tabu search, variable neighborhood search, scatter search andpath relinking, genetic algorithms, memetic algorithms, genetic programming, antcolony optimization, multi-start methods, greedy randomized adaptive search proce-dure, guided local search, hyper-heuristics, and parallel metaheuristics. It also con-tains three new chapters on large neighborhood search, artificial immune systems,and hybrid metaheuristics. The last four chapters are devoted to more general issues
vii
viii Preface
related to the field of metaheuristics, namely reactive search, stochastic search, fit-ness landscape analysis, and performance comparison. A few chapters from the firstedition were discarded, as they appear to be less relevant.
We think that this Handbook will be a great reference for researchers and grad-uate students, as well as practitioners. Each presentation, although exhibiting in-evitable stylistic differences, adheres to some common principles which results instand-alone chapters that can be read individually.
We are grateful to all authors for taking the time to write the chapters that ap-pear in this Handbook. We are also very grateful to Fred Hillier, Neil Levine, andMatthew Amboy of Springer for their encouragements, support, and patience at thedifferent stages of production of this book.
Montreal, Canada Michel GendreauMarch 2010 Jean-Yves Potvin
Preface to First Edition
Metaheuristics, in their original definition, are solution methods that orchestrate aninteraction between local improvement procedures and higher level strategies to cre-ate a process capable of escaping from local optima and performing a robust searchof a solution space. Over time, these methods have also come to include any proce-dures that employ strategies for overcoming the trap of local optimality in complexsolution spaces, especially those procedures that utilize one or more neighborhoodstructures as a means of defining admissible moves to transition from one solutionto another, or to build or destroy solutions in constructive and destructive processes.
The degree to which neighborhoods are exploited varies according to the typeof procedure. In the case of certain population-based procedures, such as geneticalgorithms, neighborhoods are implicitly (and somewhat restrictively) defined byreference to replacing components of one solution with those of another, by vari-ously chosen rules of exchange popularly given the name of “crossover.” In otherpopulation-based methods, based on the notion of path relinking, neighborhoodstructures are used in their full generality, including constructive and destructiveneighborhoods as well as those for transitioning between (complete) solutions. Cer-tain hybrids of classical evolutionary approaches, which link them with local search,also use neighborhood structures more fully, though apart from the combinationprocess itself. Meanwhile, “single thread” solution approaches, which do not un-dertake to manipulate multiple solutions simultaneously, run a wide gamut that notonly manipulate diverse neighborhoods but incorporate numerous forms of strate-gies ranging from thoroughly randomized to thoroughly deterministic, depending onthe elements such as the phase of search or (in the case of memory-based methods)the history of the solution process.1
1 Methods based on incorporating collections of memory-based strategies, invoking forms of mem-ory more flexible and varied than those used in approaches such as tree search and branch andbound, are sometimes grouped under the name Adaptive Memory Programming. This term, whichoriginated in the tabu search literature where such adaptive memory strategies were first introducedand continue to be the primary focus, is also sometimes used to encompass other methods that havemore recently adopted memory-based elements.
ix
x Preface to First Edition
A number of the tools and mechanisms that have emerged from the creationof metaheuristic methods have proved to be remarkably effective, so much so thatmetaheuristics have moved into the spotlight in recent years as the preferred lineof attack for solving many types of complex problems, particularly those of a com-binatorial nature. While metaheuristics are not able to certify the optimality of thesolutions they find, exact procedures (which theoretically can provide such a cer-tification, if allowed to run long enough)2 have often proved incapable of findingsolutions whose quality is close to that obtained by the leading metaheuristics—particularly for real-world problems, which often attain notably high levels of com-plexity. In addition, some of the more successful applications of exact methodshave come about by incorporating metaheuristic strategies within them. These out-comes have motivated additional research and application of new and improvedmetaheuristic methodologies.
This handbook is designed to provide the reader with a broad coverage of theconcepts, themes, and instrumentalities of this important and evolving area of opti-mization. In doing so, we hope to encourage an even wider adoption of metaheuristicmethods for assisting in problem solving and to stimulate research that may lead toadditional innovations in metaheuristic procedures.
The handbook consists of 19 chapters. Topics covered include scatter search,tabu search, genetic algorithms, genetic programming, memetic algorithms, variableneighborhood search, guided local search, GRASP, ant colony optimization, simu-lated annealing, iterated local search, multi-start methods, constraint programming,constraint satisfaction, neural network methods for optimization, hyper-heuristics,parallel strategies for metaheuristics, metaheuristic class libraries, and A-teams.This family of metaheuristic chapters, while not exhaustive of the many approachesthat have sprung into existence in recent years, encompasses the critical strategicelements and their underlying ideas that represent the state of the art of modernmetaheuristics.
This book is intended to provide the communities of both researchers and prac-titioners with a broadly applicable, up-to-date coverage of metaheuristic method-ologies that have proven to be successful in a wide variety of problem settings andthat hold particular promise for success in the future. The various chapters serve asstand-alone presentations giving both the necessary underpinnings as well as prac-tical guides for implementation. The nature of metaheuristics invites an analyst tomodify basic methods in response to problem characteristics, past experiences, andpersonal preferences, and the chapters in this handbook are designed to facilitatethis process as well.
2 Some types of problems seem quite amenable to exact methods, particularly to some of themethods embodied in the leading commercial software packages for mixed integer programming.Yet even by these approaches the “length of time” required to solve many problems exactly appearsto exceed all reasonable measure, including in some cases measures of astronomical scale. It hasbeen conjectured that metaheuristics succeed where exact methods fail because of their ability touse strategies of greater flexibility than permitted to assure that convergence will inevitably beobtained.
Preface to First Edition xi
The authors who have contributed to this volume represent leading figures fromthe metaheuristic community and are responsible for pioneering contributions to thefields they write about. Their collective work has significantly enriched the fieldof optimization in general and combinatorial optimization in particular. We are es-pecially grateful to them for agreeing to provide the first-rate chapters that appearin this handbook. We would also like to thank our graduate students, Gyung Yungand Rahul Patil, for their assistance. Finally, we would like to thank Gary Folvenand Carolyn Ford of Kluwer Academic Publishers for their unwavering support andpatience throughout this project.
Fred GloverGary A. Kochenberger
Contents
Preface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii
Preface to First Edition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Contributors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
1 Simulated Annealing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1Alexander G. Nikolaev and Sheldon H. Jacobson
2 Tabu Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41Michel Gendreau and Jean-Yves Potvin
3 Variable Neighborhood Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61Pierre Hansen, Nenad Mladenovic, Jack Brimberg and Jose A. MorenoPerez
4 Scatter Search and Path-Relinking: Fundamentals, Advances,and Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87Mauricio G.C. Resende, Celso C. Ribeiro, Fred Glover and RafaelMartı
5 Genetic Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109Colin R. Reeves
6 A Modern Introduction to Memetic Algorithms . . . . . . . . . . . . . . . . . . 141Pablo Moscato and Carlos Cotta
7 Genetic Programming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185William B. Langdon, Robert I. McKay and Lee Spector
8 Ant Colony Optimization: Overview and Recent Advances . . . . . . . . . 227Marco Dorigo and Thomas Stutzle
xiii
xiv Contents
9 Advanced Multi-start Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 265R. Martı, J. Marcos Moreno-Vega, and A. Duarte
10 Greedy Randomized Adaptive Search Procedures: Advances,Hybridizations, and Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283Mauricio G.C. Resende and Celso C. Ribeiro
11 Guided Local Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321Christos Voudouris, Edward P.K. Tsang and Abdullah Alsheddy
12 Iterated Local Search: Framework and Applications . . . . . . . . . . . . . . 363Helena R. Lourenco, Olivier C. Martin and Thomas Stutzle
13 Large Neighborhood Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 399David Pisinger and Stefan Ropke
14 Artificial Immune Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 421Julie Greensmith, Amanda Whitbrook and Uwe Aickelin
15 A Classification of Hyper-heuristic Approaches . . . . . . . . . . . . . . . . . . 449Edmund K. Burke, Matthew Hyde, Graham Kendall, Gabriela Ochoa,Ender Ozcan, and John R. Woodward
16 Metaheuristic Hybrids . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 469Gunther R. Raidl, Jakob Puchinger and Christian Blum
17 Parallel Meta-heuristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497Teodor Gabriel Crainic and Michel Toulouse
18 Reactive Search Optimization: Learning While Optimizing . . . . . . . . 543Roberto Battiti and Mauro Brunato
19 Stochastic Search in Metaheuristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . 573Walter J. Gutjahr
20 An Introduction to Fitness Landscape Analysis and Cost Modelsfor Local Search . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 599Jean-Paul Watson
21 Comparison of Metaheuristics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 625John Silberholz and Bruce Golden
Subject Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 641
Contributors
Uwe AickelinThe University of Nottingham, Nottingham, UKe-mail: [email protected]
Abdullah AlsheddyUniversity of Essex, Colchester, UKe-mail: [email protected]
Roberto BattitiUniversita di Trento, Trento, Italye-mail: [email protected]
Christian BlumUniversitat Politecnica de Catalunya, Barcelona, Spaine-mail: [email protected]
Jack BrimbergRoyal Military College of Canada, Kingston, ON, Canadae-mail: [email protected]
Mauro BrunatoUniversita di Trento, Trento, Italye-mail: [email protected]
Edmund K. BurkeThe University of Nottingham, Nottingham, UKe-mail: [email protected]
Carlos CottaUniversidad de Malaga, Malaga, Spaine-mail: [email protected]
xv
xvi Contributors
Teodor Gabriel CrainicUniversite du Quebec a Montreal and CIRRELT, Montreal, QC, Canadae-mail: [email protected]
Marco DorigoUniversite Libre de Bruxelles, Brussels, Belgiume-mail: [email protected]
Abraham DuarteUniversidad Rey Juan Carlos, Madrid, Spaine-mail: [email protected]
Michel GendreauEcole Polytechnique de Montreal and CIRRELT, Montreal, QC, Canadae-mail: [email protected]
Fred GloverUniversity of Colorado and OptTek Systems, Inc., Boulder, CO, USAe-mail: [email protected]
Bruce GoldenUniversity of Maryland, College Park, MD, USAe-mail: [email protected]
Julie GreensmithThe University of Nottingham, Nottingham, UKe-mail: [email protected]
Walter J. GutjahrUniversity of Vienna, Vienna, Austriae-mail: [email protected]
Pierre HansenEcole des Hautes Etudes Commerciales and GERAD, Montreal, QC, Canadae-mail: [email protected]
Matthew HydeThe University of Nottingham, Nottingham, UKe-mail: [email protected]
Sheldon H. JacobsonUniversity of Illinois, Urbana, IL, USAe-mail: [email protected]
Graham KendallThe University of Nottingham, Nottingham, UKe-mail: [email protected]
William B. LangdonUniversity College London, London, UKe-mail: [email protected]
Contributors xvii
Helena R. LourencoUniversitat Pompeu Fabra, Barcelona, Spaine-mail: [email protected]
J. Marcos Moreno-VegaUniversidad de La Laguna, La Laguna, Santa Cruz de Tenerife, Spaine-mail: [email protected]
Rafael MartıUniversidad de Valencia, Valencia, Spaine-mail: [email protected]
Olivier C. MartinUniversite Paris-Sud, Orsay, Francee-mail: [email protected]
Robert I. McKaySeoul National University, Seoul, Koreae-mail: [email protected]
Nenad MladenovicBrunel University-West London, Uxbridge, UKe-mail: [email protected]
Jose A. Moreno PerezUniversidad de La Laguna, La Laguna, Santa Cruz de Tenerife, Spaine-mail: [email protected]
Pablo MoscatoThe University of Newcastle, Callaghan, NSW Australiae-mail: [email protected]
Alexander G. NikolaevUniversity at Buffalo, Buffalo, NY, USAe-mail: [email protected]
Gabriela OchoaThe University of Nottingham, Nottingham, UKe-mail: [email protected]
Ender OzcanThe University of Nottingham, Nottingham, UKe-mail: [email protected]
David PisingerTechnical University of Denmark, Lyngby, Denmarke-mail: [email protected]
Jean-Yves PotvinUniversite de Montreal and CIRRELT, Montreal, QC, Canadae-mail: [email protected]
xviii Contributors
Jakob PuchingerUniversity of Melbourne, Melbourne, Australiae-mail: [email protected]
Gunther R. RaidlVienna University of Technology, Vienna, Austriae-mail: [email protected]
Colin R. ReevesCoventry University, Coventry, UKe-mail: [email protected]
Mauricio G.C. ResendeAT&T Labs Research, Florham Park, NJ, USAe-mail: [email protected]
Celso C. RibeiroUniversidade Federal Fluminense, Niteroi, RJ, Brazile-mail: [email protected]
Stefan RopkeTechnical University of Denmark, Lyngby, Denmarke-mail: [email protected]
John SilberholzUniversity of Maryland, College Park, MD, USAe-mail: [email protected]
Lee SpectorHampshire College, Amherst, MA, USAe-mail: [email protected]
Thomas StutzleUniversite Libre de Bruxelles, Brussels, Belgiume-mail: [email protected]
Michel ToulouseCIRRELT, Montreal, QC, Canadae-mail: [email protected]
Edward P.K. TsangUniversity of Essex, Colchester, UKe-mail: [email protected]
Christos VoudourisBT Group plc, Ipswich, UKe-mail: [email protected]
Jean-Paul WatsonSandia National Laboratories, Albuquerque, NM, USAe-mail: [email protected]
Contributors xix
Amanda WhitbrookThe University of Nottingham, Nottingham, UKe-mail: [email protected]
John R. WoodwardThe University of Nottingham, Nottingham, UKe-mail: [email protected]
