Click here to load reader

SUSTAINABLE EVOLUTIONARY ALGORITHMS AND SCALABLE ... · PDF file SUSTAINABLE EVOLUTIONARY ALGORITHMS AND SCALABLE EVOLUTIONARY SYNTHESIS OF DYNAMIC SYSTEMS By Jianjun Hu This dissertation

  • View
    1

  • Download
    0

Embed Size (px)

Text of SUSTAINABLE EVOLUTIONARY ALGORITHMS AND SCALABLE ... · PDF file SUSTAINABLE EVOLUTIONARY...

  • SUSTAINABLE EVOLUTIONARY ALGORITHMS AND SCALABLE EVOLUTIONARY SYNTHESIS OF DYNAMIC

    SYSTEMS

    By

    Jianjun Hu

    A DISSERTATION

    Submitted to Michigan State University

    in partial fulfillment of the requirements for the degree of

    DOCTOR OF PHILOSOPHY

    Department of Computer Science and Engineering

    2004

  • ABSTRACT

    SUSTAINABLE EVOLUTIONARY ALGORITHMS AND SCALABLE EVOLUTIONARY SYNTHESIS OF DYNAMIC SYSTEMS

    By

    Jianjun Hu

    This dissertation concerns the principles and techniques for scalable evolutionary computation

    to achieve better solutions for larger problems with more computational resources. It suggests

    that many of the limitations of existent evolutionary algorithms, such as premature

    convergence, stagnation, loss of diversity, lack of reliability and efficiency, are derived from the

    fundamental convergent evolution model, the oversimplified “survival of the fittest”

    Darwinian evolution model. Within this model, the higher the fitness the population achieves,

    the more the search capability is lost. This is also the case for many other conventional search

    techniques.

    The main result of this dissertation is the introduction of a novel sustainable evolution

    model, the Hierarchical Fair Competition (HFC) model, and corresponding five sustainable

    evolutionary algorithms (EA) for evolutionary search. By maintaining individuals in

    hierarchically organized fitness levels and keeping evolution going at all fitness levels, HFC

    transforms the conventional convergent evolutionary computation model into a sustainable

    search framework by ensuring a continuous supply and incorporation of low-level building

    blocks and by culturing and maintaining building blocks of intermediate levels with its

    assembly-line structure. By reducing the selection pressure within each fitness level while

    maintaining the global selection pressure to help ensure exploitation of good building blocks

    found, HFC provides a good solution to the explore vs. exploitation dilemma, which implies

  • its wide applications in other search, optimization, and machine learning problems and

    algorithms.

    The second theme of this dissertation is an examination of the fundamental principles

    and related techniques for achieving scalable evolutionary synthesis. It first presents a survey of

    related research on principles for handling complexity in artificially designed and naturally

    evolved systems, including modularity, reuse, development, and context evolution. Limitations

    of current genetic programming based evolutionary synthesis paradigm are discussed and

    future research directions are outlined. Within this context, this dissertation investigates two

    critical issues in topologically open-ended evolutionary synthesis, using bond-graph-based

    dynamic system synthesis as benchmark problems. For the issue of balanced topology and

    parameter search in evolutionary synthesis, an effective technique named Structure Fitness

    Sharing (SFS) is proposed to maintain topology search capability. For the representation issue

    in evolutionary synthesis, or more specifically the function set design problem of genetic

    programming, two modular set approaches are proposed to investigate the relationship

    between representation, evolvability, and scalability.

  • Copyright by Jianjun Hu 2004

  • v

    To my wife Fangfang, mom, dad, and sister

    for their support and their pride of each progress I made during this dissertation research

  • vi

    ACKNOWLEDGEMENTS

    It is a risky enterprise to propose that a new sustainable evolution model is needed to

    achieve scalable evolutionary computation by replacing the “survival of the fittest”

    model held as a tenet for more than three decades. At the end of this expedition, I am

    indebted to many people for their inspirations, ideas, encouragement, support, and

    love that have endowed me with energy, courage, and enthusiasms.

    Foremost, I’d like to thank my parents who have been always proud of their son

    for each progress he has made in study and research. I am also grateful to my

    parents-in-law for bringing up their lovely daughter who stays with me while they are

    left alone themselves.

    My greatest gratitude goes to my advisor Erik Goodman, who has always been

    my ultimate source of inspiration, encouragement, and support during this hard

    journey. It is his challenges to my ideas, his vision of the fundamentals of

    evolutionary computation, and his insights in our numerous discussions that help to

    shape this dissertation. I also owe my thanks to my previous supervisor in China,

    Professor Shuchun Wang for guiding me into this exciting area of computational

    intelligence.

    I would also like to thank my dissertation committee, Bill Punch, Charles Ofria,

    Ronald Rosenberg, and Charles MacCluer for sparing their precious time to serve on

    my committee and giving valuable comments and suggestions. Their help is not

    limited to the dissertation itself, but also includes many other aspects, such as advice

    concerning my job talk presentation style.

  • vii

    I am very happy to have worked with an exciting research group, the GPBG

    group at the MSU genetic algorithm research lab (GARAGe). I owe my special

    thanks to Dr. Kisung Seo for his help all through this three-year NSF project, to Dr.

    Ronald Rosenberg for his exemplary high standard of research, and to Zhun Fan for

    many interesting discussions.

    Finally, I am especially indebted to my wife, Fang Han for her deep love,

    understanding, and no-effort-sparing support for my research, without which this

    dissertation is impossible. I am proud and fortunate enough to have such an

    encouraging and wonderful woman to share a happy life available only after her

    presence.

  • viii

    TABLE OF CONTENTS LIST OF TABLES............................................................................................................ xi

    LIST OF FIGURES........................................................................................................ xiii

    1 INTRODUCTION ....................................................................................................... 1 1.1 Motivation and Background...................................................................................................2 1.2 Problem Statement...................................................................................................................5 1.3 Proposed Evolution Models and Techniques.....................................................................8 1.4 Contributions and Significance........................................................................................... 10 1.5 Organization of the Dissertation........................................................................................ 12

    2 BACKGROUND AND RELATED WORK.............................................................. 14

    2.1 Overview of Evolutionary Algorithms.............................................................................. 14 2.1.1 The Darwinian Root of Evolutionary Algorithms .............................................. 15 2.1.2 Genetic Algorithms ................................................................................................... 16 2.1.3 Genetic programming ............................................................................................... 21 2.1.4 Evolution Strategies and Evolutionary Programming ........................................ 24 2.1.5 Common Framework, Issues and Pitfalls ............................................................. 27

    2.2 Previous Work on Sustainable Evolutionary Computation .......................................... 28 2.2.1 Previous Approaches to Balance Exploration and Exploitation ...................... 29 2.2.2 Previous Approaches to Sustainable Evolution by Improving Evolvability..38 2.2.3 A Common Root of Tragedies: the Enigma of Premature Stagnation........... 41

    2.3 Overview of Topologically Open-Ended Evolutionary Synthesis............................... 44 2.3.1 Topologically Open-Ended Synthesis: a Common Framework....................... 44 2.3.2 Direct Encoding Techniques................................................................................... 47 2.3.3 Indirect Encoding/Generative Encoding Techniques ....................................... 50 2.3.4 Artificial Embryology................................................................................................ 52

    2.4 Previous Work To Improve Scalability of Evolutionary Synthesis ............................. 54 2.5 Summary ................................................................................................................................. 56

    3 TOPOLOGICALLY OPEN-ENDED SYNTHESIS OF DYNAMIC SYSTEMS BY GENETIC PROGRAMMING..................................................................................58

    3.1 Brief History of Dynamic System Design Synthesis....................................................... 58 3.2 The Representatio