Upload
esmond-rich
View
227
Download
0
Tags:
Embed Size (px)
Citation preview
Meta- Heuristic Algorithms
Ant Colony
Genetic Algorithm
Simulated Annealing
Ant Colony Optimization
ACO Concept• Ants (blind) navigate from nest to food source• Shortest path is discovered via pheromone trails
– each ant moves at random– pheromone is deposited on path– ants detect lead ant’s path, inclined to follow– more pheromone on path increases probability of path
being followed
ACO System• Virtual “trail” accumulated on path segments• Starting node selected at random• Path selected at random
– based on amount of “trail” present on possible paths from starting node
– higher probability for paths with more “trail”
• Ant reaches next node, selects next path• Continues until reaches starting node• Finished “tour” is a solution
ACO System, cont.• A completed tour is analyzed for optimality• “Trail” amount adjusted to favor better solutions
– better solutions receive more trail– worse solutions receive less trail– higher probability of ant selecting path that is part of a
better-performing tour
• New cycle is performed• Repeated until most ants select the same tour
on every cycle (convergence to solution)
ACO System, cont.• Often applied to TSP (Travelling Salesman
Problem): shortest path between n nodes• Algorithm in Pseudocode:
– Initialize Trail– Do While (Stopping Criteria Not Satisfied) – Cycle Loop
• Do Until (Each Ant Completes a Tour) – Tour Loop• Local Trail Update• End Do• Analyze Tours• Global Trail Update
– End Do
Background
• Discrete optimization problems difficult to solve
• “Soft computing techniques” developed in past ten years:– Genetic algorithms (GAs)
• based on natural selection and genetics
– Ant Colony Optimization (ACO)• modeling ant colony behavior
Background, cont.
• Developed by Marco Dorigo (Milan, Italy), and others in early 1990s
• Some common applications:– Quadratic assignment problems– Scheduling problems– Dynamic routing problems in networks
• Theoretical analysis difficult– algorithm is based on a series of random decisions
(by artificial ants)– probability of decisions changes on each iteration
Implementation
Ant Algorithms
Ant Algorithms
Implementation
• Can be used for both Static and Dynamic Combinatorial optimization problems
• Convergence is guaranteed, although the speed is unknown– Value– Solution
The Algorithm
• Ant Colony Algorithms are typically use to solve minimum cost problems.
• We may usually have N nodes and A undirected arcs
• There are two working modes for the ants: either forwards or backwards.
• Pheromones are only deposited in backward mode.
The Algorithm
• The ants memory allows them to retrace the path it has followed while searching for the destination node
• Before moving backward on their memorized path, they eliminate any loops from it. While moving backwards, the ants leave pheromones on the arcs they traversed.
The Algorithm
• The ants evaluate the cost of the paths they have traversed.
• The shorter paths will receive a greater deposit of pheromones. An evaporation rule will be tied with the pheromones, which will reduce the chance for poor quality solutions.
The Algorithm
• At the beginning of the search process, a constant amount of pheromone is assigned to all arcs. When located at a node i an ant k uses the pheromone trail to compute the probability of choosing j as the next node:
• where is the neighborhood of ant k when in node i.
0
ki
ij kik
ilij l N
ki
if j Np
if j N
kiN
The Algorithm
• When the arc (i,j) is traversed , the pheromone value changes as follows:
• By using this rule, the probability increases that forthcoming ants will use this arc.
kij ij
The Algorithm
• After each ant k has moved to the next node, the pheromones evaporate by the following equation to all the arcs:
• where is a parameter. An iteration is a completer cycle involving ants’ movement, pheromone evaporation, and pheromone deposit.
(1 ) , ( , )ij ijp i j A
(0,1]p
Steps for Solving a Problem by ACO
1. Represent the problem in the form of sets of components and transitions, or by a set of weighted graphs, on which ants can build solutions
2. Define the meaning of the pheromone trails3. Define the heuristic preference for the ant while
constructing a solution4. If possible implement a efficient local search algorithm
for the problem to be solved.5. Choose a specific ACO algorithm and apply to problem
being solved6. Tune the parameter of the ACO algorithm.
Applications
Efficiently Solves NP hard Problems• Routing
– TSP (Traveling Salesman Problem)– Vehicle Routing– Sequential Ordering
• Assignment– QAP (Quadratic Assignment Problem)– Graph Coloring– Generalized Assignment– Frequency Assignment– University Course Time Scheduling
43
52
1
Applications• Scheduling
– Job Shop– Open Shop– Flow Shop– Total tardiness (weighted/non-weighted)– Project Scheduling– Group Shop
• Subset– Multi-Knapsack– Max Independent Set– Redundancy Allocation– Set Covering– Weight Constrained Graph Tree partition– Arc-weighted L cardinality tree– Maximum Clique
Applications
• Other– Shortest Common Sequence– Constraint Satisfaction– 2D-HP protein folding– Bin Packing
• Machine Learning– Classification Rules– Bayesian networks– Fuzzy systems
• Network Routing– Connection oriented network routing– Connection network routing– Optical network routing
Genetic Algorithms
Part I: GA Theory
What are genetic algorithms?
How to design a genetic algorithm?
Genetic Algorithm Is Not...
Gene coding...
Genetic Algorithm Is...
…Computer algorithm
That resides on principles of genetics and evolution
Instead of Introduction...
•Hill climbing
locallocal
globalglobal
Instead of Introduction…(2)
•Multi-climbers
Instead of Introduction…(3)
•Genetic algorithm
I am not at the top.I am not at the top.My high is better!My high is better!
I am at the I am at the toptop
Height is ...Height is ...
I will continueI will continue
Instead of Introduction…(3)
•Genetic algorithm - few microseconds after
GA Concept• Genetic algorithm (GA) introduces the principle of evolution
and genetics into search among possible solutions to given problem.
• The idea is to simulate the process in natural systems. • This is done by the creation within a machine
of a population of individuals represented by chromosomes, in essence a set of character strings,that are analogous to the DNA,that we have in our own chromosomes.
Survival of the Fittest• The main principle of evolution used in GA
is “survival of the fittest”.• The good solution survive, while bad ones
die.
Nature and GA...
NatureNature Genetic algorithmGenetic algorithm
ChromosomeChromosome StringString
GeneGene CharacterCharacter
LocusLocus String positionString position
GenotypeGenotype PopulationPopulation
PhenotypePhenotype Decoded structureDecoded structure
The History of GA
• Cellular automata – John Holland, university of Michigan, 1975.
• Until the early 80s, the concept was studied theoretically.
• In 80s, the first “real world” GAs were designed.
Algorithmic Phases
Initialize the populationInitialize the population
Select individuals for the mating poolSelect individuals for the mating pool
Perform crossoverPerform crossover
Insert offspring into the populationInsert offspring into the population
The EndThe End
Perform mutationPerform mutation
yesyes
nono
Stop?Stop?
Designing GA... How to represent genomes? How to define the crossover operator? How to define the mutation operator? How to define fitness function? How to generate next generation? How to define stopping criteria?
Representing Genomes...
RepresentationRepresentation ExampleExample
stringstring 1 0 1 1 1 0 0 11 0 1 1 1 0 0 1
array of stringsarray of strings http avala yubc net ~apopovichttp avala yubc net ~apopovic
tree - genetic programmingtree - genetic programming>>
bbxorxor
oror
cc
bbaa
Crossover
• Crossover is concept from genetics.• Crossover is sexual reproduction.• Crossover combines genetic material from
two parents,in order to produce superior offspring.
• Few types of crossover:– One-point– Multiple point.
One-point Crossover
Parent #1Parent #1 Parent #2Parent #2
00
11
55
33
55
44
77
66
77
66
22
44
22
33
00
11
One-point Crossover
Parent #1Parent #1 Parent #2Parent #2
00
11
55
33
55
44
77
66
77
66
22
44
22
33
00
11
Mutation
• Mutation introduces randomness into the population.
• Mutation is asexual reproduction.• The idea of mutation
is to reintroduce divergence into a converging population.
• Mutation is performed on small part of population,in order to avoid entering unstable state.
Mutation...
11 11 00 11 00 1100 00
00 11 00 11 00 1100 11
11 00
00 11
ParentParent
ChildChild
About Probabilities...• Average probability for individual to crossover
is, in most cases, about 80%.• Average probability for individual to mutate
is about 1-2%.• Probability of genetic operators
follow the probability in natural systems.• The better solutions reproduce more often.
Fitness Function
• Fitness function is evaluation function,that determines what solutions are better than others.
• Fitness is computed for each individual.• Fitness function is application depended.
Selection• The selection operation copies a single individual,
probabilistically selected based on fitness, into the next generation of the population.
• There are few possible ways to implement selection:– “Only the strongest survive”
• Choose the individuals with the highest fitness for next generation
– “Some weak solutions survive”• Assign a probability that a particular individual
will be selected for the next generation• More diversity• Some bad solutions might have good parts!
Selection - Survival of The Strongest
0.930.93 0.510.51 0.720.72 0.310.31 0.120.12 0.640.64
Previous generationPrevious generation
Next generationNext generation
0.930.93 0.720.72 0.640.64
Selection - Some Weak Solutions Survive
0.930.93 0.510.51 0.720.72 0.310.31 0.120.12 0.640.64
Previous generationPrevious generation
Next generationNext generation
0.930.93 0.720.72 0.640.64 0.120.12
0.120.12
Mutation and Selection...
Phenotype
D
Phenotype
D
Phenotype
D
SelectionSelection MutationMutation
Solution distributionSolution distribution
Stopping Criteria• Final problem is to decide
when to stop execution of algorithm.• There are two possible solutions
to this problem: – First approach:
• Stop after production of definite number of generations
– Second approach: • Stop when the improvement in average fitness
over two generations is below a threshold
GA Vs. Ad-hoc Algorithms
Genetic AlgorithmGenetic Algorithm Ad-hoc AlgorithmsAd-hoc Algorithms
SpeedSpeed
Human workHuman work
ApplicabilityApplicability
PerformancePerformance
SlowSlow * * Generally fastGenerally fast
MinimalMinimal Long and exhaustiveLong and exhaustive
GeneralGeneralThere are problems There are problems
that cannot be solved analyticallythat cannot be solved analytically
ExcellentExcellent DependsDepends
* Not necessary!* Not necessary!
Problems With Gas
• Sometimes GA is extremely slow, and much slower than usual algorithms
Advantages of Gas• Concept is easy to understand.• Minimum human involvement.• Computer is not learned how to use existing solution,
but to find new solution!• Modular, separate from application• Supports multi-objective optimization• Always an answer; answer gets better with time !!!• Inherently parallel; easily distributed• Many ways to speed up and improve a GA-based
application as knowledge about problem domain is gained• Easy to exploit previous or alternate solutions
GA: An Example - Diophantine Equations
•Diophantine equation (n=4):
A*x + b*y + c*z + d*q = s
•For given a, b, c, d, and s - find x, y, z, q
•Genome:
)X, y, z, p = (
xx yy zz qq
GA:An Example - Diophantine Equations(2)
•Crossover
•Mutation
( 1, 2, 3, 4 )( 1, 2, 3, 4 )
( 5, 6, 7, 8 )( 5, 6, 7, 8 )
( 1, 6, 3, 4 )( 1, 6, 3, 4 )
( 5, 2, 7, 8 )( 5, 2, 7, 8 )
( 1, 2, 3, 4 )( 1, 2, 3, 4 ) ( 1, 2, 3, 9 )( 1, 2, 3, 9 )
GA:An Example - Diophantine Equations(3)• First generation is randomly generated of numbers
lower than sum (s).• Fitness is defined as absolute value of difference
between total and given sum:
Fitness = abs ( total - sum ) ,
• Algorithm enters a loop in which operators are performed on genomes: crossover, mutation, selection.
• After number of generation a solution is reached.
Some Applications of Gas
GAGAInternet searchInternet search
Data miningData mining
Software guided circuit designSoftware guided circuit designControl systems designControl systems design
Stock prize predictionStock prize prediction
Path findingPath finding Mobile robotsMobile robotssearchsearch
OptimizationOptimization
Trend spottingTrend spotting
Part II: Applications of GAs
GA and the internet
GA and image segmentation
GA and system design
Simulated Annealing
Simulated Annealing(1)
• What is annealing?• Process of slowly cooling down a
compound or a substance• Slow cooling let the substance flow around thermodynamic equilibrium
• Molecules get obtimum conformation
contraction : cause stress
Simulated Annealing(2)
• What is simulated annealing?
– Stochastic optimization method– Simulates slow cooling of annealing process– Implemented by Metropolis algorithm, Monte Carlo
step– Analogy
• Current thermodynamic state = solution• Energy equation = objective function• Ground state = global optimum• Temperature T = ???
Simulated Annealing(3)
• Charateristics
– Take some upnhill steps to escape the local minimum
– Instead of picking the best move, it picks a random move
– If the move improves the situation, it is executed.
Otherwise, move with some probability less than 1.
Simulated Annealing(4)
• Simple Iterative Algorithm1. find a path p
2. make p’, a variation of p
3. if p’ is better than p, keep p’ as p
4. goto 2
• Metropolis Algorithm– 3’ : if (p’ is better than p) or ( within Prob), then keep
p’ as p
• Simulated Annealing– T is reduced to 0 by schedule as time passes
Simulated Annealing(5)• Algorithm
function Simulated-Annealing(problem, schedule) returns a solution stateinputs: problem, a problemlocal variables: current, a node
next, a nodeT, a “temperature” controlling the probability of downward
steps
current Make-Node(Initial-State[problem])for t1 to infinity do
T schedule[t]if T=0 then return currentnext a randomly selected successor of currentDE Value[next] – Value[current]if DE>0 then currentnextelse currentnext only with probability eDE/T
Simulated Annealing(6)
• Schedule • Determines rate at which the temperature is
lowered • Lowers T slowly enough, the algorithm will
find a global optimum
• Temperature T : control parameter• Used to determine the probability• High T : large changes • Low T : small changes
Simulated Annealing(7)
p
E0
1
eE/T
• Probability distribution
Simulated Annealing(8)
moves
T0
Tfn
T(n)
• Schedule example
Simulated Annealing(9)
higher probability of escaping local maxima
Little chance of escaping local maxima, but local maxima may be good enough in practical problmes.
Simulated Annealing(10)
• To avoid of entrainment in local minima
– Anealing schedule : by trial and error• Choice of initial temperature• How many iterations are performed at each
temperature• How much the temperature is decremented
at each step as cooling proceeds
Simulated Annealing(11)
• Difficulties
– Determination of parameters• Schedule T• # of moves at each temperature
– If cooling is too slow• Too much time to get solution
– If cooling is too rapid• Solution may not be the global optimum
Simulated Annealing(12)
• Application
– VLSI layout
– Signal and image processing
– Task allocation
– Network design
– Molecular analysis
Application in CSP
• Can be solved by IIA– Allow states with unsatisfied constraints operators reassign
variable values
• Heuristic repair method– Assigning values to all the variables– Applying modification operators which simply assign a
different value to a variable
• Min-conflicts heuristic– Choose value that violates the fewest constraints i.e., hillclimb with h(n) = total number of violated constraints– Surprisingly effective (ex:million-queens in less than 50
steps)
Min-conflicts heuristics
• Example : 4-Queens– States : 4 queens in 4 columns(44 = 256
states)– Operators : move queen in column– Goal test : no attacks– Evaluation : h(n) = number of attacks
h = 5 h = 2 h = 0
Summary
• Iterative improvement algorithms – Keep only single state in memory – Can get stuck on local maxima– Simulated annealing to escape local maxima
• Complete, optimal given a long enough cooling schedule
• For CSP – Variable and value ordering heuristics can provide
huge performance gains – Current algorithms often solve very large problems
very quickly