Ant Colony Optimization: an introduction

Daniel Chivilikhin03.04.2013

2

Outline

1. Biological inspiration of ACO2. Solving NP-hard combinatorial problems3. The ACO metaheuristic4. ACO for the Traveling Salesman Problem

3

Outline

1. Biological inspiration of ACO2. Solving NP-hard combinatorial problems3. The ACO metaheuristic4. ACO for the Traveling Salesman Problem

4

Biological inspiration: from real to artificial ants

5

Ant colonies• Distributed systems of social insects• Consist of simple individuals• Colony intelligence >> Individual intelligence

6

Ant Cooperation• Stigmergy – indirect communication

between individuals (ants)• Driven by environment modifications

7

Denebourg’s double bridge experiments

• Studied Argentine ants I. humilis• Double bridge from ants to food source

8

Double bridge experiments: equal lengths (1)

9

Double bridge experiments: equal lengths (2)

• Run for a number of trials• Ants choose each branch ~ same number

of trials

10

Double bridge experiments: different lengths (2)

11

Double bridge experiments: different lengths (2)

• The majority of ants follow the short path

12

Outline

1. Biological inspiration of ACO2. Solving NP-hard combinatorial problems3. The ACO metaheuristic4. ACO for the Traveling Salesman Problem

13

Solving NP-hard combinatorialproblems

14

Combinatorial optimization• Find values of discrete variables• Optimizing a given objective function

15

Combinatorial optimization

Π = (S, f, Ω) – problem instance• S – set of candidate solutions• f – objective function• Ω – set of constraints• – set of feasible solutions (with

respect to Ω)• Find globally optimal feasible solution s*

SS ~

16

NP-hard combinatorial problems

• Cannot be exactly solved in polynomial time

• Approximate methods – generate near-optimal solutions in reasonable time

• No formal theoretical guarantees• Approximate methods = heuristics

17

Approximate methods• Constructive algorithms• Local search

18

Constructive algorithms• Add components to solution incrementally• Example – greedy heuristics: add solution component with best heuristic

estimate

19

Local search• Explore neighborhoods of complete

solutions• Improve current solution by local changes

– first improvement– best improvement

20

What is a metaheuristic?• A set of algorithmic concepts• Can be used to define heuristic methods• Applicable to a wide set of problems

21

Examples of metaheuristics• Simulated annealing• Tabu search• Iterated local search• Evolutionary computation• Ant colony optimization• Particle swarm optimization• etc.

22

Outline

1. Biological inspiration of ACO2. Solving NP-hard combinatorial problems3. The ACO metaheuristic4. ACO for the Traveling Salesman Problem

23

The ACO metaheuristic

24

ACO metaheuristic• A colony of artificial ants cooperate in

finding good solutions• Each ant – simple agent• Ants communicate indirectly using

stigmergy

25

Combinatorial optimization problem mapping (1)

• Combinatorial problem (S, f, Ω(t))• Ω(t) – time-dependent constraints

Example – dynamic problems• Goal – find globally optimal feasible

solution s*• Minimization problem• Mapped on another problem…

26

Combinatorial optimization problem mapping (2)

• C = {c1, c2, …, cNc} – finite set of components

• States of the problem:

X = {x = <ci, cj, …, ch, …>, |x| < n < +∞}

• Set of candidate solutions: XS

27

Combinatorial optimization problem mapping (3)

• Set of feasible states:

• We can complete into a solution satisfying Ω(t)

• Non-empty set of optimal solutions:

XX ~

Xx ~

SXS ~*

28

Combinatorial optimization problem mapping (4)

S* X~S

X• X – states

• S – candidate solutions

• – feasible states

• S* – optimal solutions

X~

29

Combinatorial optimization problem mapping (5)

• Cost g(s, t) for each • In most cases – g(s, t) ≡ f(s, t) • GC = (C, L) – completely connected graph• C – set of components• L – edges fully connecting the

components (connections)• GC – construction graph

SsSs ~

30

Combinatorial optimization problem mapping (last )

• Artificial ants build solutions by performing randomized walks on GC(C, L)

31

Construction graph

• Each component ci or connection lij have associated: heuristic information pheromone trail

32

Heuristic information• A priori information about the problem• Does not depend on the ants• On components ci – ηi

• On connections lij – ηij

• Meaning: cost of adding a component to the current

solution

33

Pheromone trail• Long-term memory about the entire search

process• On components ci – τi

• On connections lij – τij

• Updated by the ants

34

Artificial ant (1)• Stochastic constructive procedure• Builds solutions by moving on GC

• Has finite memory for:– Implementing constraints Ω(t)– Evaluating solutions– Memorizing its path

35

Artificial ant (2)• Has a start state x• Has termination conditions ek

• From state xr moves to a node from the neighborhood – Nk(xr)

• Stops if some ek are satisfied

36

Artificial ant (3)• Selects a move with a probabilistic rule

depending on: Pheromone trails and heuristic information of

neighbor components and connections Memory Constraints Ω

37

Artificial ant (4)• Can update pheromone on visited

components (nodes)• and connections (edges)• Ants act:

Concurrently Independently

38

The ACO metaheuristic

While not doStop():ConstructAntSolutions()

UpdatePheromones() DaemonActions()

39

ConstructAntSolutions• A colony of ants build a set of solutions• Solutions are evaluated using the

objective function

40

UpdatePheromones• Two opposite mechanisms:

Pheromone deposit Pheromone evaporation

41

UpdatePheromones: pheromone deposit

• Ants increase pheromone values on visited components and/or connections

• Increases probability to select visited components later

42

UpdatePheromones: pheromone evaporation

• Decrease pheromone trails on all components/connections by a same value

• Forgetting – avoid rapid convergence to suboptimal solutions

43

DaemonActions• Optional centralized actions, e.g.:

Local optimization Ant elitism (details later)

44

ACO applications• Traveling salesman• Quadratic assignment• Graph coloring• Multiple knapsack• Set covering• Maximum clique• Bin packing• …

45

Outline

1. Biological inspiration of ACO2. Solving NP-hard combinatorial problems3. The ACO metaheuristic4. ACO for the Traveling Salesman

Problem

46

ACO for the Traveling Salesman Problem

47

Traveling salesman problem• N – set of nodes (cities), |N| = n• A – set of arcs, fully connecting N• Weighted graph G = (N, A)• Each arc has a weight dij – distance• Problem:

Find minimum length Hamiltonian circuit

48

TSP: construction graph• Identical to the problem graph• C = N• L = A• states = set of all possible partial tours

49

TSP: constraints

• All cities have to be visited• Each city – only once• Enforcing – allow ants only to go to new

nodes

50

TSP: pheromone trails• Desirability of visiting city j directly after i

51

TSP: heuristic information

• ηij = 1 / dij

• Used in most ACO for TSP

52

TSP: solution construction• Select random start city• Add unvisited cities iteratively• Until a complete tour is built

53

ACO algorithms for TSP• Ant System• Elitist Ant System• Rank-based Ant System• Ant Colony System• MAX-MIN Ant System

54

Ant System: Pheromone initialization

• Pheromone initializationΤij = m / Cnn,

where: m – number of ants Cnn – path length of nearest-neighbor

algorithm

55

Ant System: Tour construction

• Ant k is located in city i

• is the neighborhood of city i• Probability to go to city :

kiNl

ilil

ijijkijp

kiN

kiNj

56

Tour construction: comprehension

• α = 0 – greedy algorithm• β = 0 – only pheromone is at work

quickly leads to stagnation

kiNl

ilil

ijijkijp

57

Ant System: update pheromone trails – evaporation

Evaporation for all connections∀(i, j) ∈ L:

τij ← (1 – ρ) τij,

ρ ∈[0, 1] – evaporation rate

Prevents convergence to suboptimal solutions

58

Ant System: update pheromone trails – deposit

• Tk – path of ant k• Ck – length of path Tk

• Ants deposit pheromone on visited arcs:

Ljim

k

kijijij

,,1

otherwise

TjiC kkkij ,0

,,/1

59

Elitist Ant System• Best-so-far ant deposits pheromone on each

iteration:

bestij

m

k

kijijij e

1

otherwise

TjiC bestbestbestij ,0

,,/1

60

Rank-based Ant System• Rank all ants• Each ant deposits amounts of pheromone

proportional to its rank

61

MAX-MIN Ant System

1. Only iteration-best or best-so-far ant deposits pheromone

2. Pheromone trails are limited to the interval [τmin, τmax]

62

Ant Colony System• Differs from Ant System in three points:

More aggressive tour construction rule Only best ant evaporates and deposits

pheromone Local pheromone update

63

Ant Colony System

otherwiseSystemAntlike

qqifj ililNl k

i

,

,maxarg 0

1.Tour Construction

2. Local pheromone update:

τij ← (1 – ξ)τij + ξτ0,

64

Comparing Ant System variants

65

State of the art in TSP• CONCORDE http://www.tsp.gatech.edu/

concorde.html • Solved an instance of 85900 cities• Computation took 286-2719 CPU days!

66

Current ACO research activity• New applications• Theoretical proofs

67

Further reading• M. Dorigo, T. Stützle. Ant Colony

Optimization. MIT Press, 2004.• http://iridia.ulb.ac.be/~mdorigo/ACO/

68

Next time…• Some proofs of ACO convergence

69

Thank you!

Any questions?

This presentation is available at:http://rain.ifmo.ru/~chivdan/presentations/

Daniel Chivilikhin [mailto: chivdan@gmail.com]

70

Used resources• http://teamaltman.com/wp-content/uploads

/2011/06/Uncertainty-Ant-Apple-1024x1024.jpg • http://myrealestatecoach.files.wordpress.com/2012/04/ant.jpg• http://ars.els-cdn.com/content/image/1-s2.0-S1568494613000264-gr3.jpg• http://www.theorie.physik.uni-goettingen.de/forschung/ha/talks/stuetzle.pdf • http://www.buyingandsellingwebsites.com/wp-content/uploads/2011/12/Ant-150.jpg• http://moodle2.gilbertschools.net/moodle/file.php/1040/Event-100_Days/Ant_Hormiga

.gif• http://4.bp.blogspot.com/_SPAe2p8Y-kg/TK3hQ8BUMtI/AAAAAAAAAHA/p75K-GcT_

oo/s1600/ant+vision.GIF• http://1.bp.blogspot.com/-tXKJQ4nSqOY/UQZ3vabiy_I/AAAAAAAAAjA

/IK8jtlhElqk/s1600/16590492-illustration-of-an-ant-on-a-white-background.jpg