SUSTAINABLE EVOLUTIONARY ALGORITHMS AND
SCALABLE EVOLUTIONARY SYNTHESIS OF DYNAMIC
SYSTEMS
By
Jianjun Hu
A DISSERTATION
Submitted to
Michigan State University
in partial fulfillment of the requirements
for the degree of
DOCTOR OF PHILOSOPHY
Department of Computer Science and Engineering
2004
ABSTRACT
SUSTAINABLE EVOLUTIONARY ALGORITHMS AND SCALABLE
EVOLUTIONARY SYNTHESIS OF DYNAMIC SYSTEMS
By
Jianjun Hu
This dissertation concerns the principles and techniques for scalable evolutionary computation
to achieve better solutions for larger problems with more computational resources. It suggests
that many of the limitations of existent evolutionary algorithms, such as premature
convergence, stagnation, loss of diversity, lack of reliability and efficiency, are derived from the
fundamental convergent evolution model, the oversimplified “survival of the fittest”
Darwinian evolution model. Within this model, the higher the fitness the population achieves,
the more the search capability is lost. This is also the case for many other conventional search
techniques.
The main result of this dissertation is the introduction of a novel sustainable evolution
model, the Hierarchical Fair Competition (HFC) model, and corresponding five sustainable
evolutionary algorithms (EA) for evolutionary search. By maintaining individuals in
hierarchically organized fitness levels and keeping evolution going at all fitness levels, HFC
transforms the conventional convergent evolutionary computation model into a sustainable
search framework by ensuring a continuous supply and incorporation of low-level building
blocks and by culturing and maintaining building blocks of intermediate levels with its
assembly-line structure. By reducing the selection pressure within each fitness level while
maintaining the global selection pressure to help ensure exploitation of good building blocks
found, HFC provides a good solution to the explore vs. exploitation dilemma, which implies
its wide applications in other search, optimization, and machine learning problems and
algorithms.
The second theme of this dissertation is an examination of the fundamental principles
and related techniques for achieving scalable evolutionary synthesis. It first presents a survey of
related research on principles for handling complexity in artificially designed and naturally
evolved systems, including modularity, reuse, development, and context evolution. Limitations
of current genetic programming based evolutionary synthesis paradigm are discussed and
future research directions are outlined. Within this context, this dissertation investigates two
critical issues in topologically open-ended evolutionary synthesis, using bond-graph-based
dynamic system synthesis as benchmark problems. For the issue of balanced topology and
parameter search in evolutionary synthesis, an effective technique named Structure Fitness
Sharing (SFS) is proposed to maintain topology search capability. For the representation issue
in evolutionary synthesis, or more specifically the function set design problem of genetic
programming, two modular set approaches are proposed to investigate the relationship
between representation, evolvability, and scalability.
Copyright by
Jianjun Hu
2004
v
To my wife Fangfang, mom, dad, and sister
for their support and their pride of each progress
I made during this dissertation research
vi
ACKNOWLEDGEMENTS
It is a risky enterprise to propose that a new sustainable evolution model is needed to
achieve scalable evolutionary computation by replacing the “survival of the fittest”
model held as a tenet for more than three decades. At the end of this expedition, I am
indebted to many people for their inspirations, ideas, encouragement, support, and
love that have endowed me with energy, courage, and enthusiasms.
Foremost, I’d like to thank my parents who have been always proud of their son
for each progress he has made in study and research. I am also grateful to my
parents-in-law for bringing up their lovely daughter who stays with me while they are
left alone themselves.
My greatest gratitude goes to my advisor Erik Goodman, who has always been
my ultimate source of inspiration, encouragement, and support during this hard
journey. It is his challenges to my ideas, his vision of the fundamentals of
evolutionary computation, and his insights in our numerous discussions that help to
shape this dissertation. I also owe my thanks to my previous supervisor in China,
Professor Shuchun Wang for guiding me into this exciting area of computational
intelligence.
I would also like to thank my dissertation committee, Bill Punch, Charles Ofria,
Ronald Rosenberg, and Charles MacCluer for sparing their precious time to serve on
my committee and giving valuable comments and suggestions. Their help is not
limited to the dissertation itself, but also includes many other aspects, such as advice
concerning my job talk presentation style.
vii
I am very happy to have worked with an exciting research group, the GPBG
group at the MSU genetic algorithm research lab (GARAGe). I owe my special
thanks to Dr. Kisung Seo for his help all through this three-year NSF project, to Dr.
Ronald Rosenberg for his exemplary high standard of research, and to Zhun Fan for
many interesting discussions.
Finally, I am especially indebted to my wife, Fang Han for her deep love,
understanding, and no-effort-sparing support for my research, without which this
dissertation is impossible. I am proud and fortunate enough to have such an
encouraging and wonderful woman to share a happy life available only after her
presence.
viii
TABLE OF CONTENTS
LIST OF TABLES............................................................................................................ xi
LIST OF FIGURES........................................................................................................ xiii
1 INTRODUCTION ....................................................................................................... 1
1.1 Motivation and Background...................................................................................................2
1.2 Problem Statement...................................................................................................................5
1.3 Proposed Evolution Models and Techniques.....................................................................8
1.4 Contributions and Significance........................................................................................... 10
1.5 Organization of the Dissertation........................................................................................ 12
2 BACKGROUND AND RELATED WORK.............................................................. 14
2.1 Overview of Evolutionary Algorithms.............................................................................. 14
2.1.1 The Darwinian Root of Evolutionary Algorithms .............................................. 15
2.1.2 Genetic Algorithms ................................................................................................... 16
2.1.3 Genetic programming ............................................................................................... 21
2.1.4 Evolution Strategies and Evolutionary Programming ........................................ 24
2.1.5 Common Framework, Issues and Pitfalls ............................................................. 27
2.2 Previous Work on Sustainable Evolutionary Computation .......................................... 28
2.2.1 Previous Approaches to Balance Exploration and Exploitation ...................... 29
2.2.2 Previous Approaches to Sustainable Evolution by Improving Evolvability..38
2.2.3 A Common Root of Tragedies: the Enigma of Premature Stagnation........... 41
2.3 Overview of Topologically Open-Ended Evolutionary Synthesis............................... 44
2.3.1 Topologically Open-Ended Synthesis: a Common Framework....................... 44
2.3.2 Direct Encoding Techniques................................................................................... 47
2.3.3 Indirect Encoding/Generative Encoding Techniques ....................................... 50
2.3.4 Artificial Embryology................................................................................................ 52
2.4 Previous Work To Improve Scalability of Evolutionary Synthesis ............................. 54
2.5 Summary ................................................................................................................................. 56
3 TOPOLOGICALLY OPEN-ENDED SYNTHESIS OF DYNAMIC SYSTEMS
BY GENETIC PROGRAMMING..................................................................................58
3.1 Brief History of Dynamic System Design Synthesis....................................................... 58
3.2 The Representatio
