1

Topic 7: Stigmergy, Swarm Intelligence and Ant Algorithms

stigmergy swarm intelligence ant algorithms

AntNet: routing AntSystem: TSP

2

Some natural Swarm Intelligence systems Ants

find the shortest path to food go on raiding parties to gather it carry it cooperatively make cemeteries sort their brood by size etc.

Termites build nests with complex features like

fortified chambers spiral air vents fungus gardens etc.

Bees gather pollen with high efficiency, exploiting the nearest richest food source first.

Geese coordinate takeoff and landing flight patterns etc.

Fish / Birds /… swarms

Human societies economies ?

3

Swarm Intelligence

“Swarm Intelligence (SI) is the property of a system whereby the collective behaviors of (unsophisticated) agents interacting locally with their environment cause coherent functional global patterns to emerge.”

characteristics of a swarm: distributed, no central control or data source no (explicit) model of the environment perception of environment, i.e. sensing ability to change environment

problem solving is emergent behaviour

4

5

6

How Do Social Insects Coordinate Their Behaviour?

communication is necessary

two types of communication direct

antennation, trophallaxis (food or liquid exchange),mandibular contact, visual contact, chemical contact, etc.

indirect two individuals interact indirectly when one of them modifies the

environment and the other responds to the new environment at a later time

called stigmergy

7

Stigmergy

“La coordination des taches, la regulation des constructions ne dependent pas directement des oeuvriers, mais des constructions elles-memes. L’ouvrier ne dirige pas son travail, il est guidé par lui. C’est à cette stimulation d’un type particulier que nous donnons le nom du STIGMERGIE (stigma, piqure; ergon, travail, oeuvre = oeuvre stimulante).” Grassé P. P., 1959

[“The coordination of tasks and the regulation of constructions does not depend directly on the workers, but on the constructions themselves. The worker does not direct his work, but is guided by it. It is to this special form of stimulation that we give the name STIGMERGY (stigma, sting; ergon, work, product of labour = stimulating product of labour).”]

8

Example of Stigmergy: Trail Following and Ants Foraging Behaviour

while walking, ants and termites may deposit a pheromone on the ground follow with high probability pheromone trails they sense on the

ground

Nest

Food

9

Example of Stigmergy:Ant cemeteries in nature

simple behaviour rules for ants wander around

if you find a dead antpick it up with probability inversely proportional

to the number of other dead ants nearby if you are carrying a dead ant

put it down with probability directly proportionalto the number of other dead ants nearby

10

StigmergyStimulation of workers

by the performance

they have achieved Grassé P. P., 1959

11

Stigmergy

Communication through marks in the environment … Marks serve as a shared memory for the agents Promotes loose coupling Robust

Stigmergy ~ action + sign: Agents put marks in the environment (inform other agents about issues of

interest) other agents perceive these marks (influence their behavior) manipulate marks other agents perceive the marks ….etc.

Marks can be static (environment does not manipulate marks over time)

e.g., flags dynamic (environment manipulates marks over time)

e.g., pheromones, gradient fields

12

Ants Foraging BehaviourExample: The Double Bridge Experiment

15 cm

Nest Food

A

B0

25

50

75

100

0 5 10 15 20 25 30

Time (minutes)

% o

f pa

ssa

ge

s

% Passages A % Passages B

Simple bridge % of ant passages on the two branches

Goss et al., 1989, Deneubourg et al., 1990

13

Food foraging in nature

14

Asymmetric Binary Bridge Experiment

15

Some Results

r is the length ratio among the two bridges

0

50

100

0-20 20-40 40-60 60-80 80-100

% of traffic on the short branch

0

50

100

0-20 20-40 40-60 60-80 80-100

% of traffic on the short branch

r = 1

0

50

100

0-20 20-40 40-60 60-80 80-100

% of traffic on the short branch

r = 2

r = 2

Short branch added later

16

“Artificial” Stigmergy

Indirect communication mediated by modifications of environmental states which are only locally accessible by the communicating agents

Dorigo & Di Caro, 1999

Characteristics of artificial stigmergy: Indirect communication Local accessibility

17

What Are Ant Algorithms?

Ant algorithms are multi-agent systems that exploit artificial stigmergy as a means for coordinating artificial ants for the solution of computational problems

examples1. AntNet: network routing

2. AntSystem: TSP

18

AntNet: An Ant Algorithm for Routing in Packet-Switched Networks (e.g. Internet)

the routing problem build routing table at each node

costs are dynamic adaptive routing is hard

routing table for node kdestination 5 2 1 …next node 3 3 8

19

AntNet data structure

routing table probabilities of choosing each neighbor nodes for each possible

final destination

trips vector contains statistics about ants’ trip times from current node k to

each destination node d (means and variances)

destination nodeneighbour node d

…n

…

P(k,n,d)

20

Artificial Ants in a Network

?Probabilistic rule to

choose the path

Pheromone traildepositing

Source

Destination

Memory

21

Simple AntNet Algorithm

ants are launched regularlyfrom each node to randomly chosen destination

ants build their paths probabilistically with probability function of artificial pheromone values heuristic values

ants memorize visited nodes and incurred costs

once destination is reached, ants deterministically retrace their path backwards, updating pheromone trails that is a function of the quality of the solution they generated

refresh + evaporation !!

22

AntNet Routing - Agents

Two kinds of Agents Forward Ant

explores the network and collects information when reaches the destination, changes into backward ant

Backward Ant goes back in the same path as forward ant update routing tables for all the nodes in the path

23

AntNet: F-Ants and B-Ants

F-ants collect implicit and explicit information on paths and traffic load

implicit through arrival rate at destination explicit by storing experienced trip times

share queues with data packets B-ants

backpropagate fast use higher priority queues

Backward Ant

1 2 3 4

Forward Ant

(N1,T1)

(N2,T2)

(N1,T1)Forward Ant

(N1,T1)

(N2,T2)

(N3,T3)Forward Ant

(N1,T1)

(N2,T2)

(N3,T3)

(N4, T4)Forward Ant

Backward AntBackward AntBackward Ant

24

Using Pheromone and Memoryto Choose the Next Node

Memory of visited nodes

i

ijd

ird

j

k

ikd

r

25

Ants’ Probabilistic Transition Rule

ijd is the amount of pheromone trail on edge (i,j,d)

Jik is the set of feasible nodes ant k positioned on

node i can move to

26

Using Pheromone and Heuristicto Choose the Next Node

Memory of visited nodes

ij

k

r

ikd ; ikd

ijd ; ijd

ird ; ird

stored in pheromone table

a heuristic evaluation of link (i,j,d)which introduces problem specific information

27

Using Pheromone and Heuristicto Choose the Next Node

Memory of visited nodes

ij

k

r

ikd ; ikd

ijd ; ijd

ird ; ird

stored in pheromone table

a heuristic evaluation of link (i,j,d)which introduces problem specific information:for AntNet: proportional to the inverse of link (i,j) queue length

28

B-ants update routing tables: if B-ant has source node d and goes from node n to k

Increase the probability of the channel that backward ant comes fromP(k,n,d) = P(k,n,d) + R.(1 – P(k,n,d))

Decrease the probability of the other channels P(k,i,d) = P(k,i,d) - R.(1 – P(k,i,d)) for all i != n

R (0 < R <= 1), function of- T: time experienced by F-ant- m: avg time for same destination memorized in trips table- sigma: std. deviation for same destination memorized in trips table

destination nodeneighbour node d

…n

…

P(k,n,d)

29

AntNet evaluation

extensive tests and experiments American NSF net Japanese NTT net artificial networks

simple graphs grid networks

in short: good results similar throughput compared to other routing algorithms remarkably smaller avg. packet delay

30

Ant Colony Optimization (ACO)

TSP has been a popular problem for the ACO models.

- several reasons why TSP is chosen …

Key concepts: Positive feedback

build a solution using local solutions, by keeping good solutions in memory.

Negative feedback want to avoid premature convergence, evaporate the pheromone.

Time scale number of runs are also critical.

31

Traveling Salesman Problem (TSP)

32

Ant Optimization of TSP

1. seed all paths with initial pheromone

2. ants make a tour of all cities ant probabilistically selects next city to visit considering distance and

pheromone strength on each path

3. ants deposit pheromone on their path with an intensity inversely proportional to the length of their tour

4. pheromone decays with each time step

5. repeat steps 2 - 4 until some threshold is met

33

A simple TSP example

A

ED

C

B

1

[]

4

[]

3

[]

2

[]

5

[]

dAB =100;dBC = 60…;dDE =150

34

Iteration 1

A

ED

C

B1

[A]

5

[E]

3

[C]

2

[B]

4

[D]

35

How to build next sub-solution?

A

ED

C

B1

[A]

1

[A]

1

[A]1

[A]

1

[A,D]

otherwise 0

allowed j if k

kallowedk

ikik

ijij

kij

][)]t([

][)]t([

)t(p

36

Iteration 2

A

ED

C

B3

[C,B]

5

[E,A]

1

[A,D]

2

[B,C]

4

[D,E]

37

Iteration 3

A

ED

C

B

4

[D,E,A]

5

[E,A,B]

3

[C,B,E]

2

[B,C,D]

1

[A,D,C]

38

Iteration 4

A

ED

C

B4

[D,E,A,B]

2

[B,C,D,A]

5

[E,A,B,C]

1

[A,D,C,E]

3

[C,B,E,D]

39

Iteration 5

A

ED

C

B

1

[A,D,C,E,B]

3

[C,B,E,D,A]

4

[D,E,A,B,C]

2

[B,C,D,A,E]

5

[E,A,B,C,D]

40

Path and Pheromone Evaluation

1

[A,D,C,E,B]

5

[E,A,B,C,D]

L1 =300

otherwise 0

tour )j,i(ifL

Q

kkj,i

L2 =450

L3 =260

L4 =280

L5 =420

2

[B,C,D,A,E]

3

[C,B,E,D,A]

4

[D,E,A,B,C]

5B,A

4B,A

3B,A

2B,A

1B,A

totalB,A

41

End of First Run

All ants die

New ants are born

Save Best Tour (Sequence and length)

stopping criteriastagnationmax.runs

42

Ant cycle for TSP

1. tabu list

2. random walk

3. priorities

4. backtracing and considering edge length

55

3322

11 1144

11

43

Tabu List

n : number of cities = one city tourm : number of antsk : ant indexs : member in tabu list (current city)

for each iteration in the ant algorithm cycle:insert town in „visited node list“

for (int k = 1 ; k <= m ; k++) { insert town of ant k in Tabuk(s) }

44

Random Walk & Priorities

i,j : edge between nodes i,jnij : 1/distance(i,j) : weight for marking : weight for node „neighbourhoodness“

0,][)]([

][)]([)( elseallowedjif

nt

nttp

kkij

kallowedlilil

ijij

probability of ant k for going from city i to city j

from ant routing table:

allowedk : unvisited cities for ant k

45

Backtracing

m

k

k

ijij1

ijijij tnt )()(

i,j : edge between nodes i,jij (t): marking at time t : evaporation rate (t,t+n)

marking after tour = evaporation_rate * marking_old + marking_delta

marking_delta = sum of markings of all ants k who passed (i,j)

0,),( elseTourjiifLQ k

k

k

ij

Q/Lk : const/tour-length of ant kWith Tabu List, ant routing table:

46

Ant System (Ant Cycle) Dorigo [1] 1991

t = 0; NC = 0; τij(t)=c for ∆τij=0Place the m ants on the n nodes

Update tabuk(s)

Compute the length Lk of every antUpdate the shortest tour found

=For every edge (i,j)Compute

For k:=1 to m do

Initialize

Choose the city j to move to. Use probability

Tabu list management

Move k-th ant to town j. Insert town j in tabuk(s)

Set t = t + n; NC=NC+1; ∆τij=0 NC<NCmax

&& not stagn.

Yes

End

No

Yes

ijijij )t()nt(

otherwise 0

by tabu describedtour k)j,i(ifL

Q

kkj,i

kijijij :

otherwise 0

allowed j if k

kallowedk

ikik

ijij

kij

][)]t([

][)]t([

)t(p

47

10-Cities-Problem CCA0

AntSystem marking distribution at beginning and after 100 cycles

48

Oliver30 Problem

cycles: 342length : 420

= 1 = 5 = 0.5

49

Advantages: positive feedback accounts for rapid discovery of good

solutions distributed computation avoids premature

convergence the greedy heuristic helps find acceptable solution in

the early solution in the early stages of the search process

the collective interaction of a population of agents

Disadvantages: slower convergence than other heuristics performed poorly for TSP problems larger than 75

cities

50

Improvements to AS: Ant Colony System (ACS)

ACS (pheromone update)

)t()t()1()1t( bestijijij

update pheromone trail while building the solution ants eat pheromone on the trail local search added before pheromone update

51

Research Summary

Ant System (AS) [Dorigo, Maniezzo & Colorni] Original implementation of ant-inspired optimization

Ant Colony System (ACS) [Dorigo & Gambardella] Ants probabilistically choose exploitation or biased exploration

Max-Min Ant System (MMAS) [Stutzle & Hoos] Only ant with best tour deposits pheromone Range of pheromone limited to given interval Pheromone levels initialized to max levels

52

Research Summary

refinements

Pheromone Trail Smoothing (PTS) increases pheromone on all segments in proportion to the difference from

the max value

Ranked Ants n ants with best paths deposit pheromone based on rank

Elite Ants best path per cycle receives additional pheromone

Modified 3-Opt iteratively swap order of three-city sub-paths such that total path length

decreases

53

Result Comparison

PTS = Pheromone Trail SmoothingMMAS = MaxMin AntSystemACS = Ant Colony SystemAS = Ant System10,000 cycles per run

Instance Optimal MMAS+pts MMAS ACS eil51 426 427.1 427.6 428.1 kroA100 21282 21291.6 21320.3 21420.0 d198 15780 15956.8 15972.5 16054.0

Instance AS-Rank AS-Rank +pts AS-Elite AS-Elite +pts AS eil51 434.5 428.8 428.3 427.4 437.3 kroA100 21746.0 21394.9 21522.8 21431.9 22471.4 d198 16199.1 16025.2 16205.0 16140.8 16702.1 [Stutzle & Hoos, 1999]

54

General ACO

a stochastic construction procedure probabilistically build a solution iteratively adding solution components to partial solutions

heuristic information pheromone trail

reinforcement modify the problem representation at each iteration

ants work concurrently and independently collective interaction via indirect communication leads

to good solutions

55

Conclusion general ACO work well on static problems like TSP, but hard to

beat specialist algorithmsbut most of all … ants are “dynamic” optimizers inherently capable of dealing with dynamism

Other Application Domains for Ants manufacturing control peer-2-peer active networking …

similar principles artificial stigmergy / feedback through pheromones information decay (pheromone evaporation) probabilistic choice in path following …