Institute of Intelligent Power Electronics IPE Page1
Introduction to Basics of Genetic Algorithms Docent Xiao-Zhi Gao
Department of Electrical Engineering and Automation
[email protected]
Slide 2
Institute of Intelligent Power Electronics IPE Page2 Background
of Genetic Algorithms Genetic Algorithms (GA) are optimization
methods based on ideas of natural selection and evolutionary
processes [Holland 75] GAs unique characteristics Derivative free
Stochastic/Random and Global optimization Parallel search GA are
applied in optimization of complex systems with little
information
Slide 3
Institute of Intelligent Power Electronics IPE Page3 Charles
Darwin (1809-1882)
Slide 4
Institute of Intelligent Power Electronics IPE Page4 The Origin
of Species by Darwin [1859]
Slide 5
Institute of Intelligent Power Electronics IPE Page5 Species
Evolution and Natural Selection
Slide 6
Institute of Intelligent Power Electronics IPE Page6 Evolution
of Human Beings?
Slide 7
Institute of Intelligent Power Electronics IPE Page7 Evolution
of Human Beings?
Slide 8
Institute of Intelligent Power Electronics IPE Page8 Darwinian
Paradigm Reproduction Competition SelectionSurvive
Slide 9
Institute of Intelligent Power Electronics IPE Page9 John Henry
Holland: Father of Genetic Algorithms John Henry Holland (born on 2
February 1929) is a Professor of Electrical Engineering and
Computer Science at the University of Michigan, Ann Arbor Book
Adaptation in Natural and Artificial Systems (1975)
http://en.wikipedia.org/wiki/Jo hn_Henry_Holland
Slide 10
Institute of Intelligent Power Electronics IPE Page10
Conceptual Genetic Algorithms
Slide 11
Institute of Intelligent Power Electronics IPE Page11 Procedure
of Genetic Algorithms 1. Encode some possible/random solutions into
bit strings (chromosomes), e.g, 1010011, and construct a population
(group of chromosomes) Float (real-valued) encoding is also
possible 2. Evaluate these chromosomes, and calculate their
fitnesses (i.e., how good they are) Based on criteria function
(fitness function ) 3. Create new chromosomes by using mutation and
crossover operators on the existing chromosomes Mutation and
crossover probabilities
Slide 12
Institute of Intelligent Power Electronics IPE Page12
Chromosome, Genes and Genomes
Slide 13
Institute of Intelligent Power Electronics IPE Page13
Historical Terms in GA Binary strings: Genetic Algorithms
Real-valued vectors : Evolution Strategies Finite state machines:
Evolutionary Programming LISP trees: Genetic Programming etc.
Slide 14
Institute of Intelligent Power Electronics IPE Page14 Mutation
Operator Mutation Probability
Slide 15
Institute of Intelligent Power Electronics IPE Page15 Crossover
Operator Crossover Probability Two-Point Crossover One-Point
Crossover
Slide 16
Institute of Intelligent Power Electronics IPE Page16 Procedure
of Genetic Algorithms 4. Evaluate new chromosomes, and calculate
their fitnesses 5. Select the mostly fitted chromosomes using
roulette selection scheme Retain size of population fixed Better
fitted chromosomes must have higher possibilities of survival
Chromosome with the best fitness is always kept 6. Repeat above
steps until a pre-set criteria is met
Slide 17
Institute of Intelligent Power Electronics IPE Page17 Roulette
Selection Scheme
Slide 18
Institute of Intelligent Power Electronics IPE Page18 Flow
Chart of Genetic Algorithms
Slide 19
Institute of Intelligent Power Electronics IPE Page19 Structure
of Genetic Algorithms Best Chromosomes
Slide 20
Institute of Intelligent Power Electronics IPE Page20 Typical
Behavior of GA
Slide 21
Institute of Intelligent Power Electronics IPE Page21 Typical
Run: Progression of Fitness
Slide 22
Institute of Intelligent Power Electronics IPE Page22 GA as
Problem Solvers: Goldbergs 1989 View
Slide 23
Institute of Intelligent Power Electronics IPE Page23 GA as
Problem Solvers: Michalewicz 1996 View
Slide 24
Institute of Intelligent Power Electronics IPE Page24 An
Example: Traveling Salesman Problem (TSP) The Traveling Salesman
Problem: Find a tour of a given set of cities so that each city is
visited only once the total distance traveled is minimized
Slide 25
Institute of Intelligent Power Electronics IPE Page25
Representation
Slide 26
Institute of Intelligent Power Electronics IPE Page26
Crossover
Slide 27
Institute of Intelligent Power Electronics IPE Page27
Mutation
Slide 28
Institute of Intelligent Power Electronics IPE Page28 TSP
Example: 30 Cities
Slide 29
Institute of Intelligent Power Electronics IPE Page29 Solution
i (Distance = 941)
Slide 30
Institute of Intelligent Power Electronics IPE Page30 Solution
j (Distance = 800)
Slide 31
Institute of Intelligent Power Electronics IPE Page31 Solution
k (Distance = 652)
Slide 32
Institute of Intelligent Power Electronics IPE Page32 Best
Solution (Distance = 420)
Slide 33
Institute of Intelligent Power Electronics IPE Page33 Overview
of Performance
Slide 34
Institute of Intelligent Power Electronics IPE Page34 An
Example: Maximization of Peak Function Using GA The peak function
is defined as: It has two inputs: x and y The peak function is a
highly nonlinear function, and has multiple local minima
Maximization of peak function is hard
Slide 35
Institute of Intelligent Power Electronics IPE Page35 Peak
Function
Slide 36
Institute of Intelligent Power Electronics IPE Page36 Initial
Populations
Slide 37
Institute of Intelligent Power Electronics IPE Page37
Populations After Five Generations
Slide 38
Institute of Intelligent Power Electronics IPE Page38
Populations After 10 Generations
Slide 39
Institute of Intelligent Power Electronics IPE Page39
Performance of GA Optimization
Slide 40
Institute of Intelligent Power Electronics IPE Page40 Some GA
Applications
Slide 41
Institute of Intelligent Power Electronics IPE Page41 Reference
David Fogel (born in 1964) is a pioneer in evolutionary computation
(Ph.D. from University of California, San Diego in 1992). He is
currently the CEO of the Natural Selection, Inc.
Slide 42
Institute of Intelligent Power Electronics IPE Page42 Reference
Evolutionary Computation: Toward a New Philosophy of Machine
Intelligence by David Fogel, 2005
Slide 43
Institute of Intelligent Power Electronics IPE Page43 Fusion of
GA with Neural Networks GA can be employed to optimize the
parameters and structures of neural networks Parameter optimization
Structure optimization (how to encode?) GA-based optimization is a
universal solution Independent of problems to be solved
Slide 44
Institute of Intelligent Power Electronics IPE Page44 An
Example: GA-based Optimization of Elman Neural Network [Gao 00]
Initial outputs of context nodes in Elman neural network play an
important role in network prediction accuracy Hybrid training of
Elman neural network consists of two parts Gradient descent
algorithm for weights Genetic algorithms for initial context nodes
outputs
Slide 45
Institute of Intelligent Power Electronics IPE Page45 Elman
Neural Network
Slide 46
Institute of Intelligent Power Electronics IPE Page46 Training
Procedure of Elman Neural Network with BP Learning
Slide 47
Institute of Intelligent Power Electronics IPE Page47
GA-involved Optimization Process of Initial Context Nodes Outputs
PE: Prediction Error
Slide 48
Institute of Intelligent Power Electronics IPE Page48 Motor
Fault Diagnosis Using Elman Neural Network with GA-aided
Training
Slide 49
Institute of Intelligent Power Electronics IPE Page49
Conclusions Basic knowledge and typical applications of GA are
introduced Numerous variants of GA exist [Fogel 06] GA can be used
to handle data mining GA-based optimization is a quite time
consuming procedure Suitable for simulations on high-speed parallel
processing computers Fast GA are being investigated