Ant Algorithms - Simulated Ant Colonies for Optimization Problemsdbauer/stud/dbauer_AntAlgorith… · · 2006-07-30Real Ant Colonies Artiﬁcial Ant Colonies Summary Ant Algorithms

Real Ant ColoniesArtificial Ant Colonies

Summary

Ant AlgorithmsSimulated Ant Colonies for Optimization Problems

Daniel Bauer ([email protected])

July 6, 2006

Daniel Bauer ([email protected]) Ant Algorithms


Summary

Topics

1 Real Ant ColoniesBehaviour of Real AntsPheromones

2 Artificial Ant ColoniesSimulating AntsS-ACOThe Ant SystemGeneral ACO MetaheuristicAnts and the Agent view

3 Summary



Summary

Behaviour of Real AntsPheromones

Introduction

Observation:

Ants living in colonies take the shortest route e.g. from theanthill to a source of foodThis way commonly known ant highways emerge

But

Ants are blindAnts have very limited communication capabilities

How can complex behaviour emerge?



Summary


Behaviour of Real Ants

Anthill

Sugar



Summary


Double Bridging Experiments

Deneubourg, Gross et al. 1989, 1990

anthill sugar

30 cm

30 cm

anthill sugar



Summary


Pheromones

Communication regarding path is performed by use of pheromonetrails:

Isolated ants move randomly (exploring the environment)

A moving ant lays down portions of pheromone (chemicalmarker substance) and leaves a trail

Other ants discovering pheromone follow the strongest trailmore probably and reinforces the trail with it’s own pheromone

The more ants follow a trail, the more attractive it becomesto follow



Summary


Pheromones

Simple model: An ant follows a path with a probabilityproportional to the amount of pheromone on it.



Summary


So far...

Individual ants are blind and dumb

Ants’ complex behaviour emerges from social interaction

Communication is performed solely by pheromone trails

Ant colonies are able to solve certain problems

How can we exploit this mechanism in computation?



Summary

Simulating AntsS-ACOThe Ant SystemGeneral ACO MetaheuristicAnts and the Agent view

Simulating Ants

Idea: Use simulated ant colonies to find shortest path in graphs.

source (anthill)

destination (source of food)

1 2 2

12 1 3

1

3 3 3

31 2 1 1

3 1 1

Simulated ants live in a descrete time environmentThey move from one node to an adjacent one in one stepThey leave a pheromone trail on the arc dependent on thepath costs. (This is problematic, why?)



Summary


Path Memory

source (anthill)

destination (source of food)

1 2 2

12 1 3

1

3 3 3

31 2 1 1

3 1 1

Problem: Cycles may occur

Solution: Ants have memory to store their path



Summary


Backward Movement and Pheromone evaporation

Path memory also allows the ant to return to the source onthe same way

Idea:

Ants find forward solutions probabilistically without puttingdown pheromoneIf a solution was found (the ant reaches the destination) theydeterministically move backward along their pathWhile in backward mode they leave pheromone on arcsdepending on the value of the solution

We want pheromones to evaporate after some time

This Algorithm is called S-ACO



Summary


S-ACO - Terminology

Graph: G = (N,A)

each arc (i , j) ∈ A is associated with it’s artificial pheromonetrail τij

τij is proportional to the utility of using the arc (i , j) to builda good solution. Choose update: Ck= 1P

pathcosts for ant k

Neighbourhood of ant k in node i: N ki

all nodes which are adjacent to i but not the one where theant came from

Pheromone evaporation parameter ρ ∈ (0, 1]



Summary


S-ACO - Algorithm

Initialize pheromone trails ∀(i , j) ∈ Aτij ← 1In each turn:

for each ant k:If k is in forward mode:

Go from current node i to j with probabilitypk

ij =τijP

l∈Nki

τilif j ∈ N k

i else pkij = 0

Store arc used and it’s costs to k’s path memorySwitch to backward mode if destination is reached, eliminatecycles from path memory

If k is in backward mode:

Go back one step deterministically (look up previous used arcin the path memory)Update pheromone on the path for an ant k:τij → τij + Ck

Switch to forward mode if source is reached.

Calculate pheromone evaporation:∀(i , j) ∈ A : τij ← τij − (1− ρ) · τij

until convergenceDaniel Bauer ([email protected]) Ant Algorithms


Summary


S-ACO - Example

S-ACO applied to the double bridging experiment

1

2

2

1

1

2

2

1

15 ants 15 ants

30 ants

1

2

2

1

30 ants

20 ants10 ants



Summary


Performance of S-ACO

Convergence cannot be guaranteed for all problems

Performance of the Algorithms is highly dependent onParameters

Only for a great number of ants a solution is found quickly inmore complex problems

Pheromone evaporation is important to avoid stagnationbehaviour

There are polynomial time algorithms which find shortest pathmore efficiently

S-ACO can be seen as an didactic example, not as an applicablealgorithm.



Summary


Combinatorical Optimization Problems

Main idea: From a set of entities find a subset that optimizesa cost function while meeting constraints.

Examples:

graph coloringjob schedulingknapsack problemtravelling salesman

CO-Problems tend to be NP-hard (all of the above problemsare NP-Complete)

Most of the time we can just approximate optimal solutions



Summary


Heuristics and Metaheuristics

Heuristics are empirical or rule-of-thumb informations used toguide searches for solutions

Example: always prefer the shortest arc when searching ashortest path

Metaheuristics are heuristics that can be applied in a range ofproblems

ACO can be seen as a metaheuristic for optimization problems

other examples: local search, simulated annealing



Summary


The Ant System (AS)

The Ant System (AS) (Dorigo et al. 1996) was one of the firstACO algorithms

It can be seen as a generalization/extension of S-ACO

It was first applied to the Traveling Salesman Problem



Summary


Traveling Salesman Problem

12

3

31

2

22

1

Given a set of n cities

In TSP a solution is any circular tour including all cities

find the shortest circular tour

formalization: completely connected graph with annotatedpath costs (distance) G=(N,A)

Idea: Let ants generate tours, then update pheromone trailsdependent on length of tour



Summary


The Ant System - Start Cities and Number of Ants

In the completely connected graph representing a TSP there is nounique source

Ants can start from an arbitrary city

Ants in city i : bi

Total number of ants: m=∑n

i=1 bi



Summary


The Ant System cont.

Ants perform cycle detection on-line

Path memory is called tabu list: tabuk

Allowed successor cities: A = N\tabuk

All ants move simultaniously

After each cycle of the algorithm the shortest path isremembered



Summary


The Ant System - Move Probability

Ants in the AS are not totally blind

They use the shortest arc distance as a heuristic

probability is calculated by integrating weightet pheromonetrail and heuristic

heuristic arc value of arc (i , j) ∈ A: ηij = 1/dij

Probability for ant k of choosing arc (i , j):

if j ∈ Akj : pk

ij =ταij ·η

βijP

l∈Akl τα

il ·ηβil

else pkij = 0

α and β are parameters to be choosen.



Summary


The Ant System - Pheromone Update and Evaporation

Pheromone is not initialized with 1, but with a small constantc

If the ants have completed a tour each (a cycle is finished)the pheromone is updated.

The update process is performed for all ants simultaniously

Evaporation is included in the update process

Update for all arcs (i , j) ∈ A: τij ← ρ · τij + ∆τij

with ∆τij =∑

k∈ants ∆τkij

and pheromone laid on arc (i,j) if visited by ant k (else ∆τij = 0):∆τk

ij = QPpathcosts of ant k



Summary


The Ant System - Algorithm

Initialize pheromone trails ∀(i , j) ∈ Aτij ← cPlace the m ants on the n nodes, write start city of ant k to tabuk

In each cycle:

for each ant k find a route:repeat

Go from current node i to j with probability pkij

Add city to tabuk

until |tabuk | = n (route was completed)

for all ants k calculate total route costs.for all ants k and all arcs (i , j) calculate ∆τk

ij

Calculate pheromone change ∆τij for all arcs (i,j)Perform pheromone update (i , j): τij ← ρ · τ ij + ∆τij

move all ants k to their starting city (from tabuk(n) totabuk(1)remember shortest routeempty tabu lists

until convergence (all ants take same route or user defined no. ofcycles)Output shortest route



Summary


The Ant System - Parameters

Performance of the Ant System depends on the choice ofparameters

Number of ants: m

Initial pheromone trail: c

Weighting of shortest path heuristic β and pheromone α

Evaporation rate: ρ

A constant related to the quantity of pheromone placed: Q



Summary


The Ant System - Experimental Results

Best values for ρ, α and β were found in experiments on a 30 cityTSP (maximal 5000 cycles were performed):α β ρ

1 5 0.5development of best tour length:



Summary


The Ant System - Experimental Results cont.

Same values were applied to a simple 10 city problem.



Summary


The Ant System vs. other heuristics

Comparision of performance of different general purpose heuristicsin the 30 city problem.

Performance compared to other TSP-tailored heuristics:



Summary


Structure of the ACO Metaheuristic

All ACO Algorithms share the same structure:

ConstructAntSolutions

UpdatePheromones

DemonActions (optional) (e.g. remember shortest path...)



Summary


How to apply ACO to other optimization problems

Since TSP is NP-hard any NP Problem can be reduced to ittheoretically

In practice it might be far easier to find a way to apply ACOmetaheuristics directly to a certain problem

We might even find a general procedure how to apply ACO toany CO-Problem



Summary


Combinatorical Optimization Problems again

All Combinatorical Optimization Problems share the samestructure:

From a set of entities find a subset that optimizes acost/value function while meeting constraints.

General aproach: Generate subsets by finding path through agraph (called construction graph)

If a solution was found (what we call a solution depends onthe problem) evaluate it

Then update pheromone trails



Summary


Multiple Knapsack Problem

Given a knapsack with a number m of resourceconstraints j (e.g. room, weight...) with capacity aj

and a set of items i ∈ I each with a value vi andrecource requirements rj i

Find a subset S of I that maximizes∑

i∈S vi subject tothe constraints∀j :

∑i∈S rij ≤ aj

Application of ACO by Leguizamon and Michalewicz(1999)



Summary


Multiple Knapsack Problem - Graph

Each item is a node in the totally connected Graph G=(I,A)

Item nodes are annotated their value and their resourcerequirements

While building solutions ants are only allowed to proceed tonodes which do not violate the constraints.

If no allowed successor node was found the ant has found asolution



Summary


Multiple Knapsack Problem - Pheromone and Heuristics

Pheromone Trails are associated with items, not with arcs.∆τi = q ·

∑i∈S vi (weighting constant q ∈ (0, 1])

Heuristics: items with high value and low requirements shouldbe prefered

average tightness of resource constraints r i = 1m ·

∑mj=1

aj

rij

Heuristic ηi = vir i



Summary


Are Simulated Ants Agents?

Ants presented so far do not correspond to our definition of anagent

Simulated ants are not intelligent at all

Simulated ants are not self interested

The environment is far to simple to be called environment



Summary


Routing in Computer Networks

Ants are processes traveling betweenmachinesThey might also have to perform othertasks (get offers, buy items...)They are self interested, they want to findthe shortest pathIf in backward mode they leavepheromone on each machine as a hint forants with the same destination.The environment they live in is highlycomplex (and changes dynamically)⇒ they can be called mobile agents(which inherit properties of ants)



Summary


Parallel Processing

Ants can move sequentially (in a loop) or syncroniously

Ants have process character

ACO algorithms can be parallelized

fine grained parallelization: Individual ants have their ownprocessorcoarse grained parallelization: Parts of the population havetheir own processor



Summary

Advantages of ACO Algorithms

ACO is robust and versatile

It can be applied to a number of optimization problemsIt can be applied to different versions of the problemApplication can be done in a straightforward way

Algorithms shows good performance compared to othermetaheuristics

The population based approach can be exploitet in parallelimplementations



Summary

Summary

Behaviour of natural populations can be an inspiration for newalgorithms

Algorithms based on artificial ant colonies can solve shortestpath problems or provide a meta heuristic in a range ofoptimization problems

Their performance depends on the choice of severalparameters



Summary

References

M. Dorigo et al., The Ant System: Optimization by acolony of cooperating agents, IEEE Transactions onSystems, Man, and CyberneticsPart B, Vol.26, No.1, 1996,pp.1-13

M. Dorigo, T. Sttzle. Ant Colony Optimization. MIT Press,Cambridge, 2004.

M. Dorigo, Ant Algorithms Solve Difficult OptimizationProblems, 2001(http://iridia.ulb.ac.be/ meta/newsite/downloads/Dorigo-

ECAL2001.pdf)


Documents

Ant Algorithms - Simulated Ant Colonies for Optimization Problemsdbauer/stud/dbauer_AntAlgorith… · · 2006-07-30Real Ant Colonies Artiﬁcial Ant Colonies Summary Ant Algorithms