73
Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Embed Size (px)

Citation preview

Page 1: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Meta- Heuristic Algorithms

Ant Colony

Genetic Algorithm

Simulated Annealing

Page 2: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Ant Colony Optimization

Page 3: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

ACO Concept• Ants (blind) navigate from nest to food source• Shortest path is discovered via pheromone trails

– each ant moves at random– pheromone is deposited on path– ants detect lead ant’s path, inclined to follow– more pheromone on path increases probability of path

being followed

Page 4: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

ACO System• Virtual “trail” accumulated on path segments• Starting node selected at random• Path selected at random

– based on amount of “trail” present on possible paths from starting node

– higher probability for paths with more “trail”

• Ant reaches next node, selects next path• Continues until reaches starting node• Finished “tour” is a solution

Page 5: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

ACO System, cont.• A completed tour is analyzed for optimality• “Trail” amount adjusted to favor better solutions

– better solutions receive more trail– worse solutions receive less trail– higher probability of ant selecting path that is part of a

better-performing tour

• New cycle is performed• Repeated until most ants select the same tour

on every cycle (convergence to solution)

Page 6: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

ACO System, cont.• Often applied to TSP (Travelling Salesman

Problem): shortest path between n nodes• Algorithm in Pseudocode:

– Initialize Trail– Do While (Stopping Criteria Not Satisfied) – Cycle Loop

• Do Until (Each Ant Completes a Tour) – Tour Loop• Local Trail Update• End Do• Analyze Tours• Global Trail Update

– End Do

Page 7: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Background

• Discrete optimization problems difficult to solve

• “Soft computing techniques” developed in past ten years:– Genetic algorithms (GAs)

• based on natural selection and genetics

– Ant Colony Optimization (ACO)• modeling ant colony behavior

Page 8: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Background, cont.

• Developed by Marco Dorigo (Milan, Italy), and others in early 1990s

• Some common applications:– Quadratic assignment problems– Scheduling problems– Dynamic routing problems in networks

• Theoretical analysis difficult– algorithm is based on a series of random decisions

(by artificial ants)– probability of decisions changes on each iteration

Page 9: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Implementation

Page 10: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Ant Algorithms

Page 11: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Ant Algorithms

Page 12: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Implementation

• Can be used for both Static and Dynamic Combinatorial optimization problems

• Convergence is guaranteed, although the speed is unknown– Value– Solution

Page 13: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

The Algorithm

• Ant Colony Algorithms are typically use to solve minimum cost problems.

• We may usually have N nodes and A undirected arcs

• There are two working modes for the ants: either forwards or backwards.

• Pheromones are only deposited in backward mode.

Page 14: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

The Algorithm

• The ants memory allows them to retrace the path it has followed while searching for the destination node

• Before moving backward on their memorized path, they eliminate any loops from it. While moving backwards, the ants leave pheromones on the arcs they traversed.

Page 15: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

The Algorithm

• The ants evaluate the cost of the paths they have traversed.

• The shorter paths will receive a greater deposit of pheromones. An evaporation rule will be tied with the pheromones, which will reduce the chance for poor quality solutions.

Page 16: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

The Algorithm

• At the beginning of the search process, a constant amount of pheromone is assigned to all arcs. When located at a node i an ant k uses the pheromone trail to compute the probability of choosing j as the next node:

• where is the neighborhood of ant k when in node i.

0

ki

ij kik

ilij l N

ki

if j Np

if j N

kiN

Page 17: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

The Algorithm

• When the arc (i,j) is traversed , the pheromone value changes as follows:

• By using this rule, the probability increases that forthcoming ants will use this arc.

kij ij

Page 18: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

The Algorithm

• After each ant k has moved to the next node, the pheromones evaporate by the following equation to all the arcs:

• where is a parameter. An iteration is a completer cycle involving ants’ movement, pheromone evaporation, and pheromone deposit.

(1 ) , ( , )ij ijp i j A

(0,1]p

Page 19: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Steps for Solving a Problem by ACO

1. Represent the problem in the form of sets of components and transitions, or by a set of weighted graphs, on which ants can build solutions

2. Define the meaning of the pheromone trails3. Define the heuristic preference for the ant while

constructing a solution4. If possible implement a efficient local search algorithm

for the problem to be solved.5. Choose a specific ACO algorithm and apply to problem

being solved6. Tune the parameter of the ACO algorithm.

Page 20: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Applications

Efficiently Solves NP hard Problems• Routing

– TSP (Traveling Salesman Problem)– Vehicle Routing– Sequential Ordering

• Assignment– QAP (Quadratic Assignment Problem)– Graph Coloring– Generalized Assignment– Frequency Assignment– University Course Time Scheduling

43

52

1

Page 21: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Applications• Scheduling

– Job Shop– Open Shop– Flow Shop– Total tardiness (weighted/non-weighted)– Project Scheduling– Group Shop

• Subset– Multi-Knapsack– Max Independent Set– Redundancy Allocation– Set Covering– Weight Constrained Graph Tree partition– Arc-weighted L cardinality tree– Maximum Clique

Page 22: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Applications

• Other– Shortest Common Sequence– Constraint Satisfaction– 2D-HP protein folding– Bin Packing

• Machine Learning– Classification Rules– Bayesian networks– Fuzzy systems

• Network Routing– Connection oriented network routing– Connection network routing– Optical network routing

Page 23: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Genetic Algorithms

Page 24: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Part I: GA Theory

What are genetic algorithms?

How to design a genetic algorithm?

Page 25: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Genetic Algorithm Is Not...

Gene coding...

Page 26: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Genetic Algorithm Is...

…Computer algorithm

That resides on principles of genetics and evolution

Page 27: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Instead of Introduction...

•Hill climbing

locallocal

globalglobal

Page 28: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Instead of Introduction…(2)

•Multi-climbers

Page 29: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Instead of Introduction…(3)

•Genetic algorithm

I am not at the top.I am not at the top.My high is better!My high is better!

I am at the I am at the toptop

Height is ...Height is ...

I will continueI will continue

Page 30: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Instead of Introduction…(3)

•Genetic algorithm - few microseconds after

Page 31: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

GA Concept• Genetic algorithm (GA) introduces the principle of evolution

and genetics into search among possible solutions to given problem.

• The idea is to simulate the process in natural systems. • This is done by the creation within a machine

of a population of individuals represented by chromosomes, in essence a set of character strings,that are analogous to the DNA,that we have in our own chromosomes.

Page 32: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Survival of the Fittest• The main principle of evolution used in GA

is “survival of the fittest”.• The good solution survive, while bad ones

die.

Page 33: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Nature and GA...

NatureNature Genetic algorithmGenetic algorithm

ChromosomeChromosome StringString

GeneGene CharacterCharacter

LocusLocus String positionString position

GenotypeGenotype PopulationPopulation

PhenotypePhenotype Decoded structureDecoded structure

Page 34: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

The History of GA

• Cellular automata – John Holland, university of Michigan, 1975.

• Until the early 80s, the concept was studied theoretically.

• In 80s, the first “real world” GAs were designed.

Page 35: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Algorithmic Phases

Initialize the populationInitialize the population

Select individuals for the mating poolSelect individuals for the mating pool

Perform crossoverPerform crossover

Insert offspring into the populationInsert offspring into the population

The EndThe End

Perform mutationPerform mutation

yesyes

nono

Stop?Stop?

Page 36: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Designing GA... How to represent genomes? How to define the crossover operator? How to define the mutation operator? How to define fitness function? How to generate next generation? How to define stopping criteria?

Page 37: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Representing Genomes...

RepresentationRepresentation ExampleExample

stringstring 1 0 1 1 1 0 0 11 0 1 1 1 0 0 1

array of stringsarray of strings http avala yubc net ~apopovichttp avala yubc net ~apopovic

tree - genetic programmingtree - genetic programming>>

bbxorxor

oror

cc

bbaa

Page 38: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Crossover

• Crossover is concept from genetics.• Crossover is sexual reproduction.• Crossover combines genetic material from

two parents,in order to produce superior offspring.

• Few types of crossover:– One-point– Multiple point.

Page 39: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

One-point Crossover

Parent #1Parent #1 Parent #2Parent #2

00

11

55

33

55

44

77

66

77

66

22

44

22

33

00

11

Page 40: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

One-point Crossover

Parent #1Parent #1 Parent #2Parent #2

00

11

55

33

55

44

77

66

77

66

22

44

22

33

00

11

Page 41: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Mutation

• Mutation introduces randomness into the population.

• Mutation is asexual reproduction.• The idea of mutation

is to reintroduce divergence into a converging population.

• Mutation is performed on small part of population,in order to avoid entering unstable state.

Page 42: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Mutation...

11 11 00 11 00 1100 00

00 11 00 11 00 1100 11

11 00

00 11

ParentParent

ChildChild

Page 43: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

About Probabilities...• Average probability for individual to crossover

is, in most cases, about 80%.• Average probability for individual to mutate

is about 1-2%.• Probability of genetic operators

follow the probability in natural systems.• The better solutions reproduce more often.

Page 44: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Fitness Function

• Fitness function is evaluation function,that determines what solutions are better than others.

• Fitness is computed for each individual.• Fitness function is application depended.

Page 45: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Selection• The selection operation copies a single individual,

probabilistically selected based on fitness, into the next generation of the population.

• There are few possible ways to implement selection:– “Only the strongest survive”

• Choose the individuals with the highest fitness for next generation

– “Some weak solutions survive”• Assign a probability that a particular individual

will be selected for the next generation• More diversity• Some bad solutions might have good parts!

Page 46: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Selection - Survival of The Strongest

0.930.93 0.510.51 0.720.72 0.310.31 0.120.12 0.640.64

Previous generationPrevious generation

Next generationNext generation

0.930.93 0.720.72 0.640.64

Page 47: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Selection - Some Weak Solutions Survive

0.930.93 0.510.51 0.720.72 0.310.31 0.120.12 0.640.64

Previous generationPrevious generation

Next generationNext generation

0.930.93 0.720.72 0.640.64 0.120.12

0.120.12

Page 48: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Mutation and Selection...

Phenotype

D

Phenotype

D

Phenotype

D

SelectionSelection MutationMutation

Solution distributionSolution distribution

Page 49: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Stopping Criteria• Final problem is to decide

when to stop execution of algorithm.• There are two possible solutions

to this problem: – First approach:

• Stop after production of definite number of generations

– Second approach: • Stop when the improvement in average fitness

over two generations is below a threshold

Page 50: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

GA Vs. Ad-hoc Algorithms

Genetic AlgorithmGenetic Algorithm Ad-hoc AlgorithmsAd-hoc Algorithms

SpeedSpeed

Human workHuman work

ApplicabilityApplicability

PerformancePerformance

SlowSlow * * Generally fastGenerally fast

MinimalMinimal Long and exhaustiveLong and exhaustive

GeneralGeneralThere are problems There are problems

that cannot be solved analyticallythat cannot be solved analytically

ExcellentExcellent DependsDepends

* Not necessary!* Not necessary!

Page 51: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Problems With Gas

• Sometimes GA is extremely slow, and much slower than usual algorithms

Page 52: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Advantages of Gas• Concept is easy to understand.• Minimum human involvement.• Computer is not learned how to use existing solution,

but to find new solution!• Modular, separate from application• Supports multi-objective optimization• Always an answer; answer gets better with time !!!• Inherently parallel; easily distributed• Many ways to speed up and improve a GA-based

application as knowledge about problem domain is gained• Easy to exploit previous or alternate solutions

Page 53: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

GA: An Example - Diophantine Equations

•Diophantine equation (n=4):

A*x + b*y + c*z + d*q = s

•For given a, b, c, d, and s - find x, y, z, q

•Genome:

)X, y, z, p = (

xx yy zz qq

Page 54: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

GA:An Example - Diophantine Equations(2)

•Crossover

•Mutation

( 1, 2, 3, 4 )( 1, 2, 3, 4 )

( 5, 6, 7, 8 )( 5, 6, 7, 8 )

( 1, 6, 3, 4 )( 1, 6, 3, 4 )

( 5, 2, 7, 8 )( 5, 2, 7, 8 )

( 1, 2, 3, 4 )( 1, 2, 3, 4 ) ( 1, 2, 3, 9 )( 1, 2, 3, 9 )

Page 55: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

GA:An Example - Diophantine Equations(3)• First generation is randomly generated of numbers

lower than sum (s).• Fitness is defined as absolute value of difference

between total and given sum:

Fitness = abs ( total - sum ) ,

• Algorithm enters a loop in which operators are performed on genomes: crossover, mutation, selection.

• After number of generation a solution is reached.

Page 56: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Some Applications of Gas

GAGAInternet searchInternet search

Data miningData mining

Software guided circuit designSoftware guided circuit designControl systems designControl systems design

Stock prize predictionStock prize prediction

Path findingPath finding Mobile robotsMobile robotssearchsearch

OptimizationOptimization

Trend spottingTrend spotting

Page 57: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Part II: Applications of GAs

GA and the internet

GA and image segmentation

GA and system design

Page 58: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Simulated Annealing

Page 59: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Simulated Annealing(1)

• What is annealing?• Process of slowly cooling down a

compound or a substance• Slow cooling let the substance flow around thermodynamic equilibrium

• Molecules get obtimum conformation

contraction : cause stress

Page 60: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Simulated Annealing(2)

• What is simulated annealing?

– Stochastic optimization method– Simulates slow cooling of annealing process– Implemented by Metropolis algorithm, Monte Carlo

step– Analogy

• Current thermodynamic state = solution• Energy equation = objective function• Ground state = global optimum• Temperature T = ???

Page 61: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Simulated Annealing(3)

• Charateristics

– Take some upnhill steps to escape the local minimum

– Instead of picking the best move, it picks a random move

– If the move improves the situation, it is executed.

Otherwise, move with some probability less than 1.

Page 62: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Simulated Annealing(4)

• Simple Iterative Algorithm1. find a path p

2. make p’, a variation of p

3. if p’ is better than p, keep p’ as p

4. goto 2

• Metropolis Algorithm– 3’ : if (p’ is better than p) or ( within Prob), then keep

p’ as p

• Simulated Annealing– T is reduced to 0 by schedule as time passes

Page 63: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Simulated Annealing(5)• Algorithm

function Simulated-Annealing(problem, schedule) returns a solution stateinputs: problem, a problemlocal variables: current, a node

next, a nodeT, a “temperature” controlling the probability of downward

steps

current Make-Node(Initial-State[problem])for t1 to infinity do

T schedule[t]if T=0 then return currentnext a randomly selected successor of currentDE Value[next] – Value[current]if DE>0 then currentnextelse currentnext only with probability eDE/T

Page 64: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Simulated Annealing(6)

• Schedule • Determines rate at which the temperature is

lowered • Lowers T slowly enough, the algorithm will

find a global optimum

• Temperature T : control parameter• Used to determine the probability• High T : large changes • Low T : small changes

Page 65: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Simulated Annealing(7)

p

E0

1

eE/T

• Probability distribution

Page 66: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Simulated Annealing(8)

moves

T0

Tfn

T(n)

• Schedule example

Page 67: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Simulated Annealing(9)

higher probability of escaping local maxima

Little chance of escaping local maxima, but local maxima may be good enough in practical problmes.

Page 68: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Simulated Annealing(10)

• To avoid of entrainment in local minima

– Anealing schedule : by trial and error• Choice of initial temperature• How many iterations are performed at each

temperature• How much the temperature is decremented

at each step as cooling proceeds

Page 69: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Simulated Annealing(11)

• Difficulties

– Determination of parameters• Schedule T• # of moves at each temperature

– If cooling is too slow• Too much time to get solution

– If cooling is too rapid• Solution may not be the global optimum

Page 70: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Simulated Annealing(12)

• Application

– VLSI layout

– Signal and image processing

– Task allocation

– Network design

– Molecular analysis

Page 71: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Application in CSP

• Can be solved by IIA– Allow states with unsatisfied constraints operators reassign

variable values

• Heuristic repair method– Assigning values to all the variables– Applying modification operators which simply assign a

different value to a variable

• Min-conflicts heuristic– Choose value that violates the fewest constraints i.e., hillclimb with h(n) = total number of violated constraints– Surprisingly effective (ex:million-queens in less than 50

steps)

Page 72: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Min-conflicts heuristics

• Example : 4-Queens– States : 4 queens in 4 columns(44 = 256

states)– Operators : move queen in column– Goal test : no attacks– Evaluation : h(n) = number of attacks

h = 5 h = 2 h = 0

Page 73: Meta- Heuristic Algorithms Ant Colony Genetic Algorithm Simulated Annealing

Summary

• Iterative improvement algorithms – Keep only single state in memory – Can get stuck on local maxima– Simulated annealing to escape local maxima

• Complete, optimal given a long enough cooling schedule

• For CSP – Variable and value ordering heuristics can provide

huge performance gains – Current algorithms often solve very large problems

very quickly