View
4
Download
0
Category
Preview:
Citation preview
COMP473: Advanced Algorithms
Mark Dras
Department of ComputingMacquarie University
2007
Mark Dras (MacqU) COMP473: Advanced Algorithms 1 / 75
Insect-based Agents
Outline
1 Insect-based Agents
2 Ant Colony OptimisationTravelling SalesmanVertex CoverQuadratic Assignment ProblemTelecom NetworksGraph Partitioning
Mark Dras (MacqU) COMP473: Advanced Algorithms 2 / 75
Insect-based Agents
Agents
different types of agents
we’re looking a low-level ones with simple behaviours
want the solution to emerge from interactions, rather than beingexplicitly programmed
Mark Dras (MacqU) COMP473: Advanced Algorithms 3 / 75
Insect-based Agents
Game of Life
invented by John Conway, published in Martin Gardner’s“Mathematical Games” column in 1970
based on a 2D grid (usually a torus: wraparound edges) dividedinto cells
a cell has 8 neighbours, and can be alive or dead
Game of Life rules:
if a live cell has fewer than two live neighbours, it dies (of loneliness)if a live cell has more than three neighbours, it dies of overcrowdingif a dead cell has exactly three neighbours, it will come alive nextiteration
Mark Dras (MacqU) COMP473: Advanced Algorithms 4 / 75
Insect-based Agents
Game of Life
look athttp://www.math.com/students/wonders/life/life.html
types of structures that appear:
‘still life’ (stable objects): block, beehive, boat, ship, loaf‘oscillators’ (periodic objects): blinker, toad
not all structures are easily predictable
r-pentamino: 5 adjacent squaresdoesn’t become stable until turn 1103
Mark Dras (MacqU) COMP473: Advanced Algorithms 5 / 75
Insect-based Agents
Emergence
unexpected / unprogrammed behaviour occurs at the global level
you only program the low-level behaviour, but some interestinghigh-level or overall behaviour is producedin r-pentaminos there are complex, self-organising patterns thataren’t explicitly programmedthese patterns emerge from a lower-level specification
Mark Dras (MacqU) COMP473: Advanced Algorithms 6 / 75
Insect-based Agents
Emergence
can be seen in economics
Adam Smith’s ‘invisible hand’economy just made up of individuals interested in their own income,consumption and savingbut at the macro level, economy moves toward equilibrium as ifguided by invisible hand
can be seen in ecology
sine behaviour of predator-prey is (maybe) emergent
Mark Dras (MacqU) COMP473: Advanced Algorithms 7 / 75
Insect-based Agents
Emergence
two definitions of emergence
position 1: “emergence is just a word, like ‘random’, to cover upthe fact that we don’t know the underlying relationships”
position 2: “emergence is a kind of process in which the systemreturns robustly to an attractor that is an irresistible mathematicalnecessity of the dynamics that define it”
Mark Dras (MacqU) COMP473: Advanced Algorithms 8 / 75
Insect-based Agents
Cellular Automata
a generalisation of the Game of Life
definition
a cell can have a value from a given set (say, 0 or 1)its state at time t + 1 is determined by the state of its neighboursand itself at time t
a 1D CA is just a string
rules define how substrings change
relationship to L-systems, dragon curves, etc
Mark Dras (MacqU) COMP473: Advanced Algorithms 9 / 75
Insect-based Agents
Cellular Automata
rules for a 1D CA
1 1 1 → 01 1 0 → 11 0 1 → 01 0 0 → 10 1 1 → 10 1 0 → 00 0 1 → 10 0 0 → 0
consider string 01001100011011110111
can define different size neighbourhoods, different dimensions ofCA, different values for state
Mark Dras (MacqU) COMP473: Advanced Algorithms 10 / 75
Insect-based Agents
Key Idea #1: Self-Organisation
structures appear at the global level of a system from interactionsamong its lower-level components
rules specifying interaction among lower-level components arepurely local
relies on four basic ingredients
positive feedbacknegative feedbackamplification of fluctuationsmultiple interactions
Mark Dras (MacqU) COMP473: Advanced Algorithms 11 / 75
Insect-based Agents
Key Idea #1: Self-Organisation
1. positive feedback
bees can transmit information about a nectar source by dancingbee can choose to dance to indicate direction and distance tosource; or can continue foraging, or abandon food source andbecome an uncommitted followerhas been shown experimentally that a bee has a relatively highprobability of dancing for a good source and of abandoning a badonekey ideas are recruitment (getting other bees to come by dancing)and reinforcement (increasing the effect through multiple bees)similar story for ants laying pheromone trails
Mark Dras (MacqU) COMP473: Advanced Algorithms 12 / 75
Insect-based Agents
Key Idea #1: Self-Organisation
2. negative feedback
necessary to avoid just fixating on first found solutionin general can stem from saturation, exhaustion, competitionin bee or ant foraging, can stem from limited number of availableforagers, satiation, food source exhaustion, crowding at the foodsource, competition between food sources
Mark Dras (MacqU) COMP473: Advanced Algorithms 13 / 75
Insect-based Agents
Key Idea #1: Self-Organisation
3. amplification of fluctuations
random walks, errors, random task-switching, etchelps insects to avoid getting stuck in locally but not globally bestsolution
Mark Dras (MacqU) COMP473: Advanced Algorithms 14 / 75
Insect-based Agents
Key Idea #1: Self-Organisation
4. multiple interactions
not just one single bee dancing one single time
Mark Dras (MacqU) COMP473: Advanced Algorithms 15 / 75
Insect-based Agents
Key Idea #2: Stigmergy
individuals can interact directly
by antennation, trophallaxis (food or liquid exchange), mandibularcontact, visual contact
individuals can interact indirectly
one modifies the environment and the other responds to the newenvironmentthis is a type of stigmergy
e.g. termites
don’t have an overall plan for building nestsdon’t communicate directly even locally regarding shape of nesta stimulating configuration triggers the response of a termiteworker: cyclic
Mark Dras (MacqU) COMP473: Advanced Algorithms 16 / 75
Insect-based Agents
Ant Systems
inspired by various ant behaviours
foragingdivision of labour and task allocationbrood sorting and cemetary organising
uses key idea of pheromone for stigmergy
this is a chemical that ants laycan accumulate and evaporate
Mark Dras (MacqU) COMP473: Advanced Algorithms 17 / 75
Ant Colony Optimisation
Outline
1 Insect-based Agents
2 Ant Colony OptimisationTravelling SalesmanVertex CoverQuadratic Assignment ProblemTelecom NetworksGraph Partitioning
Mark Dras (MacqU) COMP473: Advanced Algorithms 18 / 75
Ant Colony Optimisation Travelling Salesman
Outline
1 Insect-based Agents
2 Ant Colony OptimisationTravelling SalesmanVertex CoverQuadratic Assignment ProblemTelecom NetworksGraph Partitioning
Mark Dras (MacqU) COMP473: Advanced Algorithms 19 / 75
Ant Colony Optimisation Travelling Salesman
TSP: Ant System
the earliest version (1992)
ants build a solution by moving on the problem graph from one cityto another until they complete a tour
during an iteration of the AS algorithm each ant k (k = 1 . . .m)builds a tour executing n = |V | steps in which a probabilistictransition rule is applied
iterations are indexed by t , 1 ≤ t ≤ tmax
Mark Dras (MacqU) COMP473: Advanced Algorithms 20 / 75
Ant Colony Optimisation Travelling Salesman
TSP: Ant System
for each ant, the transition from city i to city j depends on:1 whether or not the city has already been visited: ants keep a record
(TABU LIST), which is emptied between tours2 VISIBILITY, defined as the inverse of the distance, ηij = 1/dij ;
represents the heuristic desirability of choosing city j when in city i ,and is static
3 amount of virtual PHEROMONE TRAIL τij(t); represents the learneddesirability of choosing city j when in city i
Mark Dras (MacqU) COMP473: Advanced Algorithms 21 / 75
Ant Colony Optimisation Travelling Salesman
TSP: Ant System
random proportional TRANSITION RULE for ant k is given by
pkij (t) =
[τij(t)]α ·[ηij ]β
P
l∈Jki[τil (t)]α·[ηil ]β
if j ∈ Jki
0 otherwise
whereJk
i is the set of cities left to visit when at city iα, β are weighting parameters
if α = 0, the rule corresponds to the simple greedy algorithm; ifβ = 0, only pheromone amplification is at work, and initial(probably non-optimal) candidates will just be reinforced
Mark Dras (MacqU) COMP473: Advanced Algorithms 22 / 75
Ant Colony Optimisation Travelling Salesman
TSP: Ant System
after completing a tour, each ant k lays a quantity of pheromone∆τ k
ij (t) on each edge (i , j) it has used, the amount depending onhow it has performed, giving the PHEROMONE LAYING RULE
∆τ kij (t) =
{
Q/Lk (t) if (i , j) ∈ T k(t)0 otherwise
whereT k(t) is the tour done by ant k at iteration tLk(t) is the length of T k(t)Q is a parameter (same order of magnitude as optimal tour length)
Mark Dras (MacqU) COMP473: Advanced Algorithms 23 / 75
Ant Colony Optimisation Travelling Salesman
TSP: Ant System
pheromones also decay to avoid stagnation, giving the followingGLOBAL PHEROMONE UPDATE RULE
τij(t + 1)← (1− ρ) · τij(t) + ∆τij(t)
whereρ is the coefficient of decay, 0 ≤ ρ < 1∆τij(t) =
∑mk=1 ∆τ k
ij (t)m is the number of ants
number of ants m is also an important parameter: too many wouldreinforce suboptimal trails, too few would not lay enoughpheromone to find good paths
Mark Dras (MacqU) COMP473: Advanced Algorithms 24 / 75
Ant Colony Optimisation Travelling Salesman
TSP: Ant System
original results weren’t so good
worked comparably with general heuristics (simulated annealing,tabu search), but outperformed by specialised heuristics
however, it worked differently from other heuristics
it keeps track of (with non-zero levels of pheromone) alternativegood solutionstypically, it doesn’t fix on one optimal solution: it will shuffle betweenseveralthis suggests it could be useful for dynamic problems
Mark Dras (MacqU) COMP473: Advanced Algorithms 25 / 75
Ant Colony Optimisation Travelling Salesman
TSP: Ant Colony System (ACS)
based on four modifications of Ant System:
a different transition rulea different pheromone trail update rulethe use of local updates of pheromone trail to favour explorationthe use of a candidate list to restrict the choice of the next city tovisit
Mark Dras (MacqU) COMP473: Advanced Algorithms 26 / 75
Ant Colony Optimisation Travelling Salesman
TSP: ACS
transition rule modified to allow explicitly for exploration
an ant k chooses a city j to move to following the rule
j =
{
arg maxu∈Jki
{
[τiu(t)] · [ηiu]β}
if q ≤ q0
J otherwise
whereq is uniform random over [0,1]q0 is a tunable parameter, 0 ≤ q0 ≤ 1J is a city randomly selected according to a given probability
Mark Dras (MacqU) COMP473: Advanced Algorithms 27 / 75
Ant Colony Optimisation Travelling Salesman
TSP: ACS
probability for determining city J:
pkiJ(t) =
[τiJ(t)]·[ηiJ ]βP
l∈Jki[τil(t)]·[ηil ]β
if j ∈ Jki
0 otherwise
note that this is very similar to Ant System’s rule
q ≤ q0 corresponds to exploitation of available knowledgeq > q0 corresponds to more exploration
tuning of q0 corresponds to tuning of temperature in simulatedannealing
Mark Dras (MacqU) COMP473: Advanced Algorithms 28 / 75
Ant Colony Optimisation Travelling Salesman
TSP: ACS
pheromone trail update modified by restricting global trail updateonly to best ant
in addition, whereas in Ant System pheromone was added to alledges, in ACS it is added only to edges in best tour since start oftrial
τij(t + 1)← (1− ρ) · τij(t) + ∆τij(t)
where(i , j)’s are those edges belonging to T +, the optimal tour∆τij(t) = 1/L+
L+ is the length of T +
Mark Dras (MacqU) COMP473: Advanced Algorithms 29 / 75
Ant Colony Optimisation Travelling Salesman
TSP: ACS
there is also a local update of pheromone trail: when, whileperforming a tour, ant k is in city i and selects city j ∈ J k
i , thepheromone concentration of (i , j) is updated by the formula
τij(t)← (1− ρ) · τij(t) + ρ · τ0
whereτ0 is the initial value of pheromone trails (experimentally found tobe usefully set to τ0 = (n · Lnn)
1, where n is the number of citiesand Lnn is the optimal tour produced by the nearest neighbourheuristic)
this means that when an ant visits an edge, pheromonediminishes, encouraging exploration
Mark Dras (MacqU) COMP473: Advanced Algorithms 30 / 75
Ant Colony Optimisation Travelling Salesman
TSP: ACS Results
tested on TSPLIB graphs
results by best tour length and number of iterations
ACS ACS GA GA EP EP SA SAbest # iter best # iter best # iter best # iter
Eil50 425 1830 428 25000 426 100000 443 68512Eil75 535 3480 545 80000 542 325000 580 173250KroA100 21282 4820 21761 103000 n/a n/a n/a n/a
Mark Dras (MacqU) COMP473: Advanced Algorithms 31 / 75
Ant Colony Optimisation Travelling Salesman
TSP: ACS modifications
it’s been demonstrated that heuristic methods generally need toincorporate a local optimiser
this takes the solution at the end of a tour and drives it to its localoptimum
ACS combined with local search (3-opt)
results are compared with winner of First International Contest onEvolutionary Optimization, and are quite comparable
Mark Dras (MacqU) COMP473: Advanced Algorithms 32 / 75
Ant Colony Optimisation Travelling Salesman
Local search: n-opt
in 2-opt, a 2-opt (or 2-exchange) neighbourhood of a candidatesolution s is defined as the set of all candidate solutions s ′ that canbe obtained by exchanging two pairs of edges in all possible ways
e.g. taking edges (x1, x2) and (x3, x4), s′ then consists of all of theedges in s but replacing the two selected by (x1, x3) and (x2, x4)
in 3-opt, at most three edges are replaced
in 2.5-opt, a 2-opt move is performed, and at the same time theinsertion of a vertex is evaluated
Mark Dras (MacqU) COMP473: Advanced Algorithms 33 / 75
Ant Colony Optimisation Vertex Cover
Outline
1 Insect-based Agents
2 Ant Colony OptimisationTravelling SalesmanVertex CoverQuadratic Assignment ProblemTelecom NetworksGraph Partitioning
Mark Dras (MacqU) COMP473: Advanced Algorithms 34 / 75
Ant Colony Optimisation Vertex Cover
VC: Ant Colony Optimization
from Shyu et al (2001)
question: how much tailoring is needed to apply the ant systemideas to another problem?
this is the reason ant systems are generally referred to as ameta-heuristic
the major difference with TSP (in terms of ant movement) is thatthe set of nodes constituting the VC isn’t guaranteed to form apath
therefore, from the graph G = 〈V ,E〉 we construct the completegraph Gc = 〈V ,Ec〉, where Ec = {(i , j) | i , j ∈ V}
Mark Dras (MacqU) COMP473: Advanced Algorithms 35 / 75
Ant Colony Optimisation Vertex Cover
VC: ACO
note that we still need to be able to reconstruct the original graphfrom Gc, so we define a function
ψk (r , j) =
{
1, if (r , j) ∈ E ;0, if (r , j) ∈ Ec − E
the function ψ is also used by the ants when deciding whether toput a vertex in the VC
the second major difference is that we want to select a subset ofvertices rather than of edges
hence, in pheromone laying, pheromone is placed on verticesrather than on edges
other differences are just in the specific definition of behaviourrules
Mark Dras (MacqU) COMP473: Advanced Algorithms 36 / 75
Ant Colony Optimisation Vertex Cover
VC: ACO
the transition rule is similar to TSP
pkj =
1, if q < q0 and j = arg maxr∈Ak{τrη
βrk};
0, if q < q0 and j 6= arg maxr∈Ak{τrη
βrk};
τjηβ
jkP
r∈Akτr η
β
rk
, if q ≥ q0,
whereAk denotes the set of accessible vertices for ant k ,ηjk is the heuristic goodness of node j ,τj is the amount of pheromone on node j ,β is a rational number greater than zero.
Mark Dras (MacqU) COMP473: Advanced Algorithms 37 / 75
Ant Colony Optimisation Vertex Cover
VC: ACO
as noted before, heuristic goodness will depend on whether theedges are ‘real’ or not
ηjk =
∑
(r ,j)∈Ecψk (r , j)
w(j).
note that this heuristic is dynamic
because each time we place a node j in the vertex cover we set forall r , ψk (r , j) = 0, the heuristic goodness for all adjacent nodes rchanges (cf. static heuristic goodness for TSP)
Mark Dras (MacqU) COMP473: Advanced Algorithms 38 / 75
Ant Colony Optimisation Vertex Cover
VC: ACO
the global update rule (again, for pheromone decay)
τi =
{
(1− ρ)τi + τ0ρ∆τi , if i ∈ V ′c
(1− ρ)τi , otherwise
where∆τi = 1
P
j∈V ′c
w(j) ,
V ′c are the nodes in the current best solution.
Mark Dras (MacqU) COMP473: Advanced Algorithms 39 / 75
Ant Colony Optimisation Vertex Cover
VC: ACO
the local update rule
τi = (1− ϕ)τi + ϕτ0
whereτ0 = |V |∗(|V |−|V ′
c |)P
j∈V ′c
w(j)
ϕ is an evaporation rate specific to the local update, 0 ≤ ϕ < 1
Mark Dras (MacqU) COMP473: Advanced Algorithms 40 / 75
Ant Colony Optimisation Vertex Cover
VC: ACO Results
the comparison is with a simple approximation algorithm, greedydegree; simulated annealing; and tabu search
note that Shyu et al had weighted nodes, so the problem is slightlymore general; numbers here represent total weight
n m GD SA TS ACO OPT10 30 449.0 451.8 451.8 431.3 431.315 60 818.5 763.4 763.4 729.0 726.220 80 987.7 927.9 927.9 905.4 898.025 100 1236.9 1148.0 1148.0 1107.4 1106.9
Mark Dras (MacqU) COMP473: Advanced Algorithms 41 / 75
Ant Colony Optimisation Vertex Cover
VC: ACO Results
in addition, times for ACO were significantly faster (e.g. for 25vertex instance, 0.02 CPU seconds vs 0.08 for SA and 0.078 forTS)
also, scales up to at least moderate-sized problems (300 vertices),in terms of both accuracy and speed
Mark Dras (MacqU) COMP473: Advanced Algorithms 42 / 75
Ant Colony Optimisation Quadratic Assignment Problem
Outline
1 Insect-based Agents
2 Ant Colony OptimisationTravelling SalesmanVertex CoverQuadratic Assignment ProblemTelecom NetworksGraph Partitioning
Mark Dras (MacqU) COMP473: Advanced Algorithms 43 / 75
Ant Colony Optimisation Quadratic Assignment Problem
Quadratic Assignment Problem
what is it?
a set of n activities has to be assigned to n locationsthere is an n × n matrix D of distances dij between locationsthere is an n × n matrix of flows fhk between activity h and activity kan assignment is a permutation π of {1, . . . , n}, where π(i) is theactivity assigned to location iobjective function:
C(π) =
n∑
i,j=1
dij fπ(i)π(j)
the optimal permutation is given by:
πopt = arg minπ∈Π(n)
C(π)
Mark Dras (MacqU) COMP473: Advanced Algorithms 44 / 75
Ant Colony Optimisation Quadratic Assignment Problem
QAP Characteristics
was originally formulated (1957) to describe assignment offacilities to locations, with the flows being things like transfers ofdata, material, people, . . .
also used for Steiner Wiring Problem (placing components on acomputer backboard so that total length of interconnected wiring isminimised)
also NP-hard
“harder” than e.g. vertex cover
any α-approximation solution is also NP-completeproblems where n > 15 are intractable for exact solutions, evenwhen using cutting planes or branch-and-bound
Mark Dras (MacqU) COMP473: Advanced Algorithms 45 / 75
Ant Colony Optimisation Quadratic Assignment Problem
QAP Example
consider a randomly generated problem of order 4:
D =
0 10 4 210 0 6 4
4 6 0 12 4 1 0
,F =
0 3 8 33 0 2 48 2 0 53 4 5 0
consider also, of the 4! permutations, the following two:
π1 = (1)(2)(3)(4)π2 = (12)(34)
Mark Dras (MacqU) COMP473: Advanced Algorithms 46 / 75
Ant Colony Optimisation Quadratic Assignment Problem
QAP Example
C(π1) = d1,1f1,1 + d1,2f1,2 + d1,3f1,3 + d1,4f1,4+d2,1f2,1 + d2,2f2,2 + d2,3f2,3 + d2,4f2,4+d3,1f3,1 + d3,2f3,2 + d3,3f3,3 + d3,4f3,4+d4,1f4,1 + d4,2f4,2 + d4,3f4,3 + d4,4f4,4+
= 202
C(π2) = d1,1f2,2 + d1,2f2,1 + d1,3f2,4 + d1,4f2,3+d2,1f1,2 + d2,2f1,1 + d2,3f1,4 + d2,4f1,3+d3,1f4,2 + d3,2f4,1 + d3,3f4,4 + d3,4f4,3+d4,1f3,2 + d4,2f3,1 + d4,3f3,4 + d4,4f3,3+
= 210
also have to check other 22 permutations to get lowest
Mark Dras (MacqU) COMP473: Advanced Algorithms 47 / 75
Ant Colony Optimisation Quadratic Assignment Problem
QAP: Ant System
graph representation is similar to TSP
pheromone, representing the learned desirability of settingπ(i) = j , is also deposited on edges
need to derive a definition for heuristic goodness
consider vectors d̄ (‘distance potentials’) and f̄ (‘flow potentials’)obtained by
di =n
∑
j=1
dij {i = 1, . . . , n}
fh =n
∑
k=1
fhk {h = 1, . . . , n}
potentials represent total distance from a given node, and total flowfrom a given node; we’re interested in ‘barycentric nodes’ (smallesttotal distance: most central) and nodes with the greatest flow
Mark Dras (MacqU) COMP473: Advanced Algorithms 48 / 75
Ant Colony Optimisation Quadratic Assignment Problem
QAP: Ant System Example
from previous example, vectors of potential are:
d =
1620117
,F =
149
1512
location 4 is central, 2 is peripheral; activity 3 is the most active, 2is least
basic heuristic, min-max coupling rule: attach high flow potentialsto low distance potentials
Mark Dras (MacqU) COMP473: Advanced Algorithms 49 / 75
Ant Colony Optimisation Quadratic Assignment Problem
QAP: Ant System Example
construct coupling matrix B = d̄ · f̄ T to define heuristic goodness
B =
224 144 240 192280 180 300 240154 99 165 13298 63 105 84
apply min-max coupling to get bolded values
activity 3 w location 4; 1 w 3; 4 w 1; 2 w 2
Mark Dras (MacqU) COMP473: Advanced Algorithms 50 / 75
Ant Colony Optimisation Quadratic Assignment Problem
QAP: Ant System
heuristic goodness is then ηij = bij = di · fjthe rest is the same as for TSP:
pkij (t) =
[τij(t)]α ·[ηij ]β
P
l∈Jki[τil (t)]α·[ηil ]β
if j ∈ Jki
0 otherwise
where Jki is the set of feasible activities (i.e. the set of activities
ant k has yet to assign)
difference with TSP is that ant starts at location with lowestdistance potential, and probabilistically chooses the best activity;then at the next step the second best location is chosen, etc
Mark Dras (MacqU) COMP473: Advanced Algorithms 51 / 75
Ant Colony Optimisation Telecom Networks
Outline
1 Insect-based Agents
2 Ant Colony OptimisationTravelling SalesmanVertex CoverQuadratic Assignment ProblemTelecom NetworksGraph Partitioning
Mark Dras (MacqU) COMP473: Advanced Algorithms 52 / 75
Ant Colony Optimisation Telecom Networks
Telecommunications Networks
basic problem: routing
a mechanism that allows information transmitted over a network tobe directed from a source to a destination through a sequence ofintermediate switching/buffering nodesnecessary because a fully connected network is too costlyaim is to direct traffic in a way that maximises network performancewhile minimising costs (e.g. rate of call rejection, throughput,packet delay, . . . )
previous problems have been static
network is both spatially distributed and dynamic: nodes or linkscan fail or become congested
Mark Dras (MacqU) COMP473: Advanced Algorithms 53 / 75
Ant Colony Optimisation Telecom Networks
Networks: Ant-Based Control
an adaptive routing algorithm based on ants that modify therouting policy at every node in a telephone network by depositingvirtual pheromone trail on routing table entries
ants’ goal is to build, and adapt to load changes at run time, routingtables so that network performance is maximisedperformance is measured by rate of incoming calls which areaccepted (calls can be rejected depending on availability ofresources at call time)
Mark Dras (MacqU) COMP473: Advanced Algorithms 54 / 75
Ant Colony Optimisation Telecom Networks
Networks: ABC
telephone network is represented by a graph with N nodes and Ebidirectional links
each node i has ki neighbours, and is characterised by capacity Ci ,spare capacity Si , and a routing table Ri = [r i
n,d (t)]ki ,N−1 (i.e. ki
rows, N − 1 cols)links have infinite capacityCi represents the maximum number of connections node i canestablish, and Si represents the percentage of capacity stillavailable for new connectionsrows in R are neighbour nodes and cols are destinationsthe value r i
n,d (t) is a probability that a given ant, whose destinationis node d , be routed from node i to neighbour node n
Mark Dras (MacqU) COMP473: Advanced Algorithms 55 / 75
Ant Colony Optimisation Telecom Networks
Networks: ABC
ants are launched, and move, probabilistically
the ant is deleted when it reaches its destinationa node’s routing table is updated after an ant visits it: pheromone isdeposited corresponding to the neighbour node the ant came from,and from its source node
calls (unlike ants) use routing tables deterministically: at set-uptime a route from node s to node d is built by choosingsequentially the neighbour node with the highest score(probability)
then decrease capacity of each node on route by fixed amount
calls are rejected if a node in the route under construction has noremaining capacity
Mark Dras (MacqU) COMP473: Advanced Algorithms 56 / 75
Ant Colony Optimisation Telecom Networks
Networks: ABC
an ant, whose destination is node 2 and whose source is node 5, arrives at node4 from node 5
col 2 gives ant probabilities of movement; col 5 is updated
/. -,() *+2
����
�??
???
/. -,() *+1
????
?
/. -,() *+3
/. -,() *+5
����
�
/. -,() *+4
1 2 3 51 0.8 0.3 0.1 0.13 0.1 0.4 0.8 0.15 0.1 0.3 0.1 0.8
Mark Dras (MacqU) COMP473: Advanced Algorithms 57 / 75
Ant Colony Optimisation Telecom Networks
Networks: ABC
math definitions: let s be the ant source node, i − 1 be the node it justcame from, i the current node at time t
entry r ii−1,s is reinforced while other entries r i
n,s(t), n 6= i − 1, in the samecol decay
r ii−1,s(t + 1) =
r ii−1,s(t) + δr
1 + δr
r in,s(t + 1) =
r in,s(t)
1 + δr, n 6= i − 1
where δr is a reinforcement parameter depending on ant characteristicssuch as age
in the previous slide’s example, the updated probabilities for node 5 aretherefore 0.1/(1 + δr), 0.1/(1 + δr), (0.8 + δr)/(1 + δr)
Mark Dras (MacqU) COMP473: Advanced Algorithms 58 / 75
Ant Colony Optimisation Telecom Networks
Networks: ABC
comments:
important assumption: any specific path connecting s and d mustexhibit approx the same level of congestion in both directions; OKfor telephone, questionable for packet-switchingamount of reinforcement δr depends on how well ant is performing;can use age T as a proxy (as delay ages ants)
δr =aT
+ b
age is manipulated by imposing a delay D on ants based on sparecapacity S
D = c · ed ·S
random noise is also added to avoid paths being ‘frozen’
Mark Dras (MacqU) COMP473: Advanced Algorithms 59 / 75
Ant Colony Optimisation Telecom Networks
Networks: ABC
five comparisons
fixed routing scheme based on shortest pathalgorithm based on mobile agents by Appleby and Stewardan improved version of mobile agentsABCABC with noise (works like the q0 parameter of ACS)
Mark Dras (MacqU) COMP473: Advanced Algorithms 60 / 75
Ant Colony Optimisation Telecom Networks
Networks: ABC
avg call fail std devshortest path 12.57 2.16mobile agents 9.19 0.78improved mobile agents 4.22 0.77ABC without noise 1.79 0.54ABC with noise 1.99 0.54
Mark Dras (MacqU) COMP473: Advanced Algorithms 61 / 75
Ant Colony Optimisation Telecom Networks
Networks: AntNet
differences from ABC:
the way ants collect and distribute information was designed so thatAntNet can be applied to both connection-oriented andconnectionless networks“ants collect information which is used to build local parametricmodels of the network status”, which are used to computereinforcements to change probabilistic routing tablesAntNet has been tested on a packet-switching network model, andits performance (throughput and delivery delays) has beencompared with well-known routing algorithmstest model based on continuous-time discrete event simulation,rather than sequential discrete iterations simulation
Mark Dras (MacqU) COMP473: Advanced Algorithms 62 / 75
Ant Colony Optimisation Telecom Networks
Networks: AntNet
uses same characteristics as ABC: s, d , Ri = [r in,d(t)]ki ,N−1 (the
latter thought of as desirability of a node)
in addition, define a set of estimates for trip times from currentnode i to all other nodes d (averages and variances)
Γi = {µi→d , σ2i→d}
two types of ants, FORWARD ANTS Fs→d and BACKWARD ANTS
Bd→s
forward ants move from source to dest at the same priority as datapackets (hence experience queues and traffic load)backward ants move in reverse, and backpropagate info alongcorresponding forward ant’s path, using high-priority queues
Mark Dras (MacqU) COMP473: Advanced Algorithms 63 / 75
Ant Colony Optimisation Telecom Networks
Networks: AntNet
forward ants Fs→d are launched at regular intervals from everynode s to random d
hops to neighbours are chosen randomly and proportionally todesirability, storing memory on a stackif a cycle is detected, cycle’s nodes are popped from ant’s stackand memory is destroyedwhen Fs→d reaches d , it generates a backward ant Bd→s and dies
Mark Dras (MacqU) COMP473: Advanced Algorithms 64 / 75
Ant Colony Optimisation Telecom Networks
Networks: AntNet
Bd→s follows path of Fs→d
updates Γi with (good) elapsed times stored in stack memoryupdates Ri by incrementing r i
i−1,d (t) and decreasing other r in,d (t)
increments r based on trip times Ti→d , best trip time Wi→d ,‘correcting term’
r ∝ (1−Wi→d/Ti→d) + ∆(σ,W )
other r in,d (t) decremented to normalise probabilities
Mark Dras (MacqU) COMP473: Advanced Algorithms 65 / 75
Ant Colony Optimisation Telecom Networks
Networks: AntNet
results compared with
open shortest path first (OPSF), current official Internet routingalgorithmsophisticated version of asynchronous distributed Bellman-Fordalgorithm (BF) with dynamic cost metricshortest path first (SPF) with dynamic cost metricQ-routing algorithm of Boyan and Littmanpredictive Q-routingdaemon, which knows instantaneously the states of all queues ofall nodes (as upper bound)
better than all, quite close to daemon
Mark Dras (MacqU) COMP473: Advanced Algorithms 66 / 75
Ant Colony Optimisation Graph Partitioning
Outline
1 Insect-based Agents
2 Ant Colony OptimisationTravelling SalesmanVertex CoverQuadratic Assignment ProblemTelecom NetworksGraph Partitioning
Mark Dras (MacqU) COMP473: Advanced Algorithms 67 / 75
Ant Colony Optimisation Graph Partitioning
Graph Partitioning
previous problems used foraging metaphor from ants
this one uses ability of ants to sort
seen in brood sorting (sorting out different larvae into several piles)or cemetery organisationagain, building is by recognising local configurations
Mark Dras (MacqU) COMP473: Advanced Algorithms 68 / 75
Ant Colony Optimisation Graph Partitioning
Graph Partitioning
problem is as presented in Bonabeau et al
start with an undirected unweighted graph G = 〈V ,E〉,V = {vi | i = 1, . . . ,N}, with adjacency matrix A = [aij ]
each vertex is initially assigned a set of random coordinates in Rn
the task is to change the vertices’ coordinates in Rn such that
1 clusters (cliques?) present in the graph are located in the sameportion of space
2 intercluster edges are minimised3 different clusters are clearly separated
Mark Dras (MacqU) COMP473: Advanced Algorithms 69 / 75
Ant Colony Optimisation Graph Partitioning
Graph Partitioning
not sure what problem this is (not minimum k -clustering, notminimum clique partition, not . . . )
this partitioning is one way of laying out graphs
Mark Dras (MacqU) COMP473: Advanced Algorithms 70 / 75
Ant Colony Optimisation Graph Partitioning
Graph Partitioning
definition of typical solution approach:
let (xim)m=1,...,n be the coordinate of vertex i in Rn, and
X = [xim](i=1,...,N)(M=1,...,n) the N × n matrix describing thecoordinates of all vertices
want to minimise quadratic cost function
C =
N∑
i=1
N∑
j=1
n∑
m=1
aij(xim − xjm)
under the constraint X T X = Inrelies heavily on matrix diagonalisation, which can be costlycomputationally
Mark Dras (MacqU) COMP473: Advanced Algorithms 71 / 75
Ant Colony Optimisation Graph Partitioning
GP: Ant-based model
embedded in a plane
vertices move in the plane and are attracted to or rejected byclusters
distance metric is
d(vi , vj ) =|D(ρ(vi ), ρ(vj ))|
|ρ(vi )|+ |ρ(vj )|
whereρ(vi) = {vj ∈ V ; aij 6= 0} ∪ {vi}D(A,B) is the symmetric difference of two sets A,B
when two vertices vi , vj have many common adjacent nodes, D issmall, hence also d
Mark Dras (MacqU) COMP473: Advanced Algorithms 72 / 75
Ant Colony Optimisation Graph Partitioning
GP: Ant-based model
also need to define probabilities of picking up and depositingvertices; assume agent is located at site r at time t and findsvertex vi
‘local density’ is
f (vi) =
(
1s2
P
vj∈Neigh(s×s)(r)
[1 − d(vi , vj)] if f > 0
0 otherwise
if all s2 sites around r are occupied, then f (vi) = 1 and the vertexshould not be picked up; if none are occupied, f (vi) = 0
picking up and dropping probabilities are
pp(vi ) =
„
k1
k1 + f (vi )
«2
, pd(vi ) =
„
f (vi )
k2 + f (vi )
«2
Mark Dras (MacqU) COMP473: Advanced Algorithms 73 / 75
Ant Colony Optimisation Graph Partitioning
GP: Ant-based model
at time of book, no quantitive comparison available
for visual, check out Fig 4.10 in Bonabeau et al
Mark Dras (MacqU) COMP473: Advanced Algorithms 74 / 75
Ant Colony Optimisation Graph Partitioning
Criticisms of ACO
from Pablo Moscato (of Memetic Algorithms)
need to solve two problems
original problemsetting of parameters
‘curse of dimensionality’
trying to guide search by sampling some kind of underlying optimaldistributionalso, near-optimal solutions may have very similar costs
Mark Dras (MacqU) COMP473: Advanced Algorithms 75 / 75
Recommended