Genetic Programming

Genetic ProgrammingGenetic Programming

Using Simulated Natural Selection to Using Simulated Natural Selection to Automatically Write ProgramsAutomatically Write Programs


John Koza, Stanford UniversityJohn Koza, Stanford University Principal proponent of GPPrincipal proponent of GP Has obtained human-competitive results in a Has obtained human-competitive results in a

number of problem domainsnumber of problem domains Reproduced existing patentsReproduced existing patents Created new patentable designsCreated new patentable designs

Has written extensively on GPHas written extensively on GP Four volume set on Genetic ProgrammingFour volume set on Genetic Programming Numerous papers on the GPNumerous papers on the GP


Basic AlgorithmBasic Algorithm Create a population of programsCreate a population of programs Each program attempts to solve a set of problems Each program attempts to solve a set of problems

in a “training set.”in a “training set.” Program fitness is determined by success in Program fitness is determined by success in

solving training setsolving training set More fit members have better chance to produce More fit members have better chance to produce

offspring in the next generationoffspring in the next generation Offspring are produced using some form of Offspring are produced using some form of

crossovercrossover

Tree Structure of Genetic ProgramsTree Structure of Genetic Programs

Various structures are used to represent Various structures are used to represent genetic programs, but tree structures are the genetic programs, but tree structures are the most well known.most well known. Nonterminal nodes are functions that take their Nonterminal nodes are functions that take their

children as parameters.children as parameters.

+

21

Tree Structure Tree Structure

Terminal Nodes, the nodes that make up the Terminal Nodes, the nodes that make up the leaves of a program tree, provide data to the leaves of a program tree, provide data to the program.program. ConstantsConstants Parameterless functionsParameterless functions InputsInputs

Genetic Program ComponentsGenetic Program Components

Terminal SetTerminal Set Work as set of primitive data typesWork as set of primitive data types

ConstantsConstants Parameterless functionsParameterless functions Input ValuesInput Values

Function setFunction set Set of available functionsSet of available functions Often tailored specifically for the needs of the Often tailored specifically for the needs of the

program domain.program domain.

Initializing the PopulationInitializing the Population

The following two parameters are specifiedThe following two parameters are specified Maximum depth of a program treeMaximum depth of a program tree Maximum number of nodes in a program treeMaximum number of nodes in a program tree

Three methods in common use (Koza)Three methods in common use (Koza) FullFull

Nonterminals are used to build a complete tree up to the Nonterminals are used to build a complete tree up to the leaf nodes, which are then completely populated with leaf nodes, which are then completely populated with terminals. Every tree is grown to maximum depth and terminals. Every tree is grown to maximum depth and has the maximum number of nodes allowed.has the maximum number of nodes allowed.

Initializing the Population Initializing the Population (continued)(continued)

Three methods in common use (Koza)Three methods in common use (Koza) GrowGrow

The root node is chosen from the function setThe root node is chosen from the function set All nodes not at maximum depth are chosen randomly.All nodes not at maximum depth are chosen randomly.

Growth for a branch ends when a terminal is chosen.Growth for a branch ends when a terminal is chosen. Trees can have irregular shapes.Trees can have irregular shapes.

Nodes at the maximum depth are chosen from the Nodes at the maximum depth are chosen from the terminal set only.terminal set only.

Initializing the Population Initializing the Population (continued)(continued)

Three methods in common use (Koza)Three methods in common use (Koza) Ramped Half and HalfRamped Half and Half

M is the max depth of deepest partition in the M is the max depth of deepest partition in the populationpopulation

The population is separated into M partitionsThe population is separated into M partitions The ith partition, (i ranges from 0 to M-1) has a max depth of The ith partition, (i ranges from 0 to M-1) has a max depth of

M – i.M – i.

Half of each partition is populated with grow, the other Half of each partition is populated with grow, the other half is populated with full.half is populated with full.

Genetic Operators: CrossoverGenetic Operators: Crossover

CrossoverCrossover Randomly select a node in the motherRandomly select a node in the mother Randomly select a node in the fatherRandomly select a node in the father Swap the two nodes along with their subtreesSwap the two nodes along with their subtrees

Crossover ExampleCrossover Example+

1

*

2

-

2

-

13

/

4

power

abs2

-7

Parent 1 Parent 2

+

*

2

-

2

-

power

abs2

-7

1

13

/

4

Child 1 Child 2

Genetic Operations: MutationGenetic Operations: Mutation

MutationMutation Randomly select a node in the program treeRandomly select a node in the program tree Remove that node and its subtreeRemove that node and its subtree Replace the node with a new subtree, using the Replace the node with a new subtree, using the

same method used to initially instantiate the same method used to initially instantiate the population.population.

Typically, mutation is applied to a small number of Typically, mutation is applied to a small number of offspring after crossover. offspring after crossover.

Mutation ExampleMutation Example+

1 3

*

2 4

+

+

1

*

2

Left subtree is randomly selected for mutation.

-

*2

7 4

The entire subtree is replaced

Fitness-based SelectionFitness-based Selection

Gives “graded and continuous feedback about Gives “graded and continuous feedback about how well a program performs on the training how well a program performs on the training set” (Banzhaf et. al.)set” (Banzhaf et. al.) Standardized FitnessStandardized Fitness

Fitness scores are transformed so that 0 is the fitness of Fitness scores are transformed so that 0 is the fitness of the most fit member.the most fit member.

Normalized FitnessNormalized Fitness Fitness is transformed to values that always are between Fitness is transformed to values that always are between

0 and 1.0 and 1.

Different Selection AlgorithmsDifferent Selection Algorithms

GA ScenarioGA Scenario Same as that used in Genetic AlgorithmsSame as that used in Genetic Algorithms

Create gene pool by selecting parents based on fitnessCreate gene pool by selecting parents based on fitness Next generation completely replaces current generationNext generation completely replaces current generation

ES ScenarioES Scenario Same as used in Evolutionary StrategiesSame as used in Evolutionary Strategies

Generate children firstGenerate children first Apply fitness function to parents and childrenApply fitness function to parents and children Select the next generation from children (and possibly parents too)Select the next generation from children (and possibly parents too) Selection pressure can be tuned by adjusting the ratio of the Selection pressure can be tuned by adjusting the ratio of the

number of offspring to the number of parents.number of offspring to the number of parents.

Selection PressureSelection Pressure

Ratio of the best individual’s selection Ratio of the best individual’s selection probability to the average selection probabilityprobability to the average selection probability

MostFitSelectionProbability / MostFitSelectionProbability / AverageFitSelectionProbabilityAverageFitSelectionProbability

The larger this ratio, the greater the selection The larger this ratio, the greater the selection pressure.pressure.

Sample Fitness MeasuresSample Fitness Measures

Error FitnessError Fitness The sum of the absolute value of the differences between The sum of the absolute value of the differences between

the computed result and the desired result.the computed result and the desired result.

||1

i

n

iip opf

Where:

fp is the fitness of the pth individual in the population

oi is the desired output for the ith example in the training set

pi is the output from the pth individual on the ith example in the training set

* Squaring the expressing (pi-oi) can provide larger penalties for errors.

Fitness Measures can be as Varied as Fitness Measures can be as Varied as the Applicationsthe Applications

ExamplesExamples Number of correct solutionsNumber of correct solutions Number of wins competing against other members Number of wins competing against other members

of the population.of the population. Number of errors navigating a mazeNumber of errors navigating a maze Time required to solve a puzzleTime required to solve a puzzle

Truncation or (Truncation or (µ, µ, λλ) Selection) Selection

A number of parents (A number of parents (µ) are allowed to breed µ) are allowed to breed and produce (and produce (λλ) children. The µ best children ) children. The µ best children are used to produce the next generation.are used to produce the next generation. A variation, A variation, ((µ + µ + λλ) selection includes the parents ) selection includes the parents

in those considered for selection into the next in those considered for selection into the next generation. generation.

Ranking SelectionRanking Selection

Selection Based on Fitness OrderSelection Based on Fitness Order The members of the population are ranked from The members of the population are ranked from

best to worst.best to worst. The selection probability is assigned based on the The selection probability is assigned based on the

rank.rank.

Tournament SelectionTournament Selection

Select a subset of the population (the Select a subset of the population (the tournament size) randomly.tournament size) randomly.

More fit (winning) individuals are used to More fit (winning) individuals are used to generate replacements for less fit (losing) generate replacements for less fit (losing) individuals.individuals. Accelerates processing time (compared with full Accelerates processing time (compared with full

competition)competition) Facilitates parallel processingFacilitates parallel processing

The Basic GP Algorithm (from The Basic GP Algorithm (from Banzhaf, et. al)Banzhaf, et. al)

Define the terminal setDefine the terminal set Define the function setDefine the function set Define the fitness functionDefine the fitness function Define parameters such as population size, Define parameters such as population size,

maximum individual size, crossover maximum individual size, crossover probability, selection method, and termination probability, selection method, and termination criterioncriterion

Generational GPGenerational GP

Like what we have seen in GALike what we have seen in GA New generation completely replaces the previous New generation completely replaces the previous

generation.generation. Initialize the populationInitialize the population Evaluate the individual programsEvaluate the individual programs Until a new population is fully populated, repeatUntil a new population is fully populated, repeat

Select an individual or individuals in the population using selection Select an individual or individuals in the population using selection algorithmalgorithm

Perform genetic operations on the selected individual or individualsPerform genetic operations on the selected individual or individuals Insert the result of the genetic operations into the new populationInsert the result of the genetic operations into the new population

Best individual is the resulting program.Best individual is the resulting program.

Steady State GPSteady State GP

There are no generationsThere are no generations1.1. Initialize the populationInitialize the population

2.2. Randomly choose a subset of the population to take part Randomly choose a subset of the population to take part in the tournamentin the tournament

3.3. Evaluate the fitness value of each competitor in the Evaluate the fitness value of each competitor in the tournament.tournament.

4.4. Select the winner or winners from the competitors in the Select the winner or winners from the competitors in the tournament using the selection algorithm.tournament using the selection algorithm.

5.5. Apply genetic operators to the winner or winners of the Apply genetic operators to the winner or winners of the tournamenttournament

Steady State GP (continued)Steady State GP (continued)

6.6. Replace the losers in the tournament with the results Replace the losers in the tournament with the results of the application of the genetic operators to the of the application of the genetic operators to the winners of the tournament.winners of the tournament.

7.7. Repeat steps 2-6 until the termination criterion is met.Repeat steps 2-6 until the termination criterion is met.

IntronsIntrons

Code sections (functions) that provide no real Code sections (functions) that provide no real value for the problem at handvalue for the problem at hand Introns do not directly affect the fitness of the Introns do not directly affect the fitness of the

individual.individual. e.g., j = j + 0 or j = j * 1e.g., j = j + 0 or j = j * 1 Early and middle sections of GP runs might Early and middle sections of GP runs might

include 40-60% introns.include 40-60% introns. Later in the run, introns begin to dominate the Later in the run, introns begin to dominate the

code.code. Introns growth is exponential!Introns growth is exponential!

Why GP Introns EmergeWhy GP Introns Emerge

Children tend to be less fit than parentsChildren tend to be less fit than parents Crossover and mutation can be extremely Crossover and mutation can be extremely

destructivedestructive Introns reduce the destructive effects of genetic Introns reduce the destructive effects of genetic

operatorsoperators Parents generate introns when it is easier to protect Parents generate introns when it is easier to protect

what they already can do, through the creation of what they already can do, through the creation of introns, than improve on what they are currently introns, than improve on what they are currently doing.doing.

Effective FitnessEffective Fitness

Function of at least two factorsFunction of at least two factors The fitness of the parentThe fitness of the parent Likelihood that genetic operators will affect the Likelihood that genetic operators will affect the

fitness of the parent’s childrenfitness of the parent’s children

Effects of IntronsEffects of Introns

Introns may have differing effects before and Introns may have differing effects before and after exponential growth of introns beginsafter exponential growth of introns begins

Different systems may generate different types Different systems may generate different types of introns with different probabilities.of introns with different probabilities.

The extent to which genetic operatos are The extent to which genetic operatos are destructive in their effect is likely to be a very destructive in their effect is likely to be a very important initial condition in intron growth.important initial condition in intron growth.

Mutation and crossover may affect different Mutation and crossover may affect different types of introns differently.types of introns differently.

Problems Caused by IntronsProblems Caused by Introns

Run stagnation (no progress)Run stagnation (no progress) Poor results (do nothing code)Poor results (do nothing code) Drain on memory and CPU time (storing and Drain on memory and CPU time (storing and

executing unnecessary code)executing unnecessary code)

Possible Beneficial Effects of IntronsPossible Beneficial Effects of Introns

Introns might serve to isolate useful code Introns might serve to isolate useful code blocksblocks This facilitates the building block model by This facilitates the building block model by

protecting useful building blocks from disruptionprotecting useful building blocks from disruption

Methods of Handling IntronsMethods of Handling Introns

Reduce the destructiveness of genetic Reduce the destructiveness of genetic operatorsoperators Reducing destructive crossover to 0 results in hill Reducing destructive crossover to 0 results in hill

climbingclimbing Attach fitness penalty to the length of the Attach fitness penalty to the length of the

program.program. Change the fitness functionChange the fitness function

Provides the GP with a way to improve that is Provides the GP with a way to improve that is better than just insulating the current best solution.better than just insulating the current best solution.

ReferencesReferences

Genetic Programming, An IntroductionGenetic Programming, An Introduction Wolfgang Banzhaf, Peter Nordin, Robert E. Keller, Wolfgang Banzhaf, Peter Nordin, Robert E. Keller,

Frank D. FranconeFrank D. Francone Genetic Programming TutorialGenetic Programming Tutorial

John Koza, Gecco 2005John Koza, Gecco 2005 Genetic Programming: The MovieGenetic Programming: The Movie

John KozaJohn Koza

Documents

Genetic Programming