Termite: A Swarm Intelligent Routing Algorithm for Mobile

Termite:A Swarm Intelligent Routing Algorithmfor Mobile Wireless Ad-Hoc Networks

Martin RothWireless Intelligent Systems LaboratoryCornell UniversityIthaca, New YorkUSA

Overview

IntroductionMobile Wireless Ad-Hoc NetworksSwarm Intelligence

TermiteMotivation and IntroductionPerformance Evaluation

Pheromone Model AnalysisPheromone Behavior in MANET ModelParameter Heuristics

Pheromone Update SchemesEnhancementsSimulations

ConclusionFuture WorkContributions

Overview






Introduction to Ad-Hoc Networks

A Mobile WirelessAd-Hoc Network is:A network of mobile nodes that communicate via a wireless channel, cooperatively forwarding information

Ad-Hoc Routing Problem

Network Topology is very unstable

Node movementChanges in Communications ChannelNode loss

Node deathLoss of Power

How can data be reliably moved from Source to Destination?

Maximize Data GoodputMinimize Control Traffic

Typical Solutions to Ad-Hoc Routing

Ad-Hoc Routing exists since the early 1990s

DSR, AODV, ZRP, TORA, DSDV, PARO, etc…Same general approach

Flood to collect informationGraph theoretical computationRoute!

Drawbacks of typical approachesControl traffic for path recoveryPath maintenance requires complicated rulesHigh mobility causes too much control traffic; network becomes unusable

!!!

***

Swarm Intelligent Routing

Instead of building explicit paths,Let each path have a probability of being used

Link failures may be immediately circumvented by choosing the next most probable path

Control traffic is avoided

Swarm Intelligent (SI) Routing is a method for building probabilistic paths between nodes based on simple rules

Simple probability update rules avoid algorithm complexity

Swarm Intelligence

A framework for designing distributed algorithmsOriginally derived from models of social insect behavior

System of many independent agents obeying:Positive Feedback

Reinforce good solutions

Negative FeedbackRemove old solutions

RandomnessAllows many solutions to be tested simultaneously

Multiple InteractionsSpreads local informationGenerates global behavior from local interactions

Swarm Intelligence:Ant Foraging

[1] http://iridia.ulb.ac.be/~mdorigo/ACO/RealAnts.html

Overview


TermitePrevious Work and IntroductionPerformance Evaluation




A Brief History ofSwarm Intelligent Routing

Ant Based Control1996

AntNet1997

Cooperative Asymmetric Forwarding1998

Virtual-Wavelength Path Routing1999

Probabilistic Emergent Routing Algorithm2002

Mobile Ants-Based Routing2002

Multiple Ant Colony Optimization2002

Ant-like Routing Algorithm2002

Ad-hoc Networking with Swarm Intelligence2004

Termite

TermiteSwarm intelligent routing algorithm formobile wireless ad-hoc networksDesigned with swarm intelligence framework

Reduce control trafficCollect network information from overheard packets

Reduce complexity of an individual nodeModel all routing functions mathematically

Increase robustness to changes in network topologyEmergent routing behavior from node interactions

Termite:A SI MANET Routing Algorithm

Each node has a Pheromone TableList of neighbors and destinations

Packet arrives at APrevious hop: B

Source Pheromone decaysPheromone, γ, is addedto link with BRoute more likely through BDestination → Source

Destination

Source

A

BC

E

S DD

BC

?

Pheromone Table

The Mathematics behind Termite

Pheromone AccountingClassic pheromone update model

Continuous Pheromone Decay

Forwarding Equation

Pheromone Threshold, KPheromone Sensitivity, F

( )( )∑

∈

+

+=

nNj

Fndj

Fndin

diKP

KPp

,

,.

( )τnsttnsi

nsi

n ePPNi −−⋅←∈∀ ,,,γ+← n

srnsr PP ,,

normalize!

Destinations

Nei

ghbo

rs

Route Management

Route Management is the finding, repairing, and updating of pathsRoute Discovery

Route Request (RREQ)Random walk to find destination

Not floodingRoute Reply (RREP)

Routed to the source according to theForwarding Equation

Same as data packetsControl packets are forwarded with high priority

Route RepairLost neighbor removed from Pheromone TablePacket resent on a remaining link

Neighbors are gained by overhearing transmissions

Termite Simulations:Goodput vs. Node Mobility

Termite Simulations:Mean Packet Path Length

Termite Conclusions

Large data packets explore the networkNot energy efficientNot bandwidth efficientOnly a constant fraction of control packets needed

Nodes maintain knowledge of most destinationsBasic routing decisions can be made withoutresorting to control trafficRoutes can be repaired automatically

Overview






Motivation

The Termite concept has been shown to workHow can performance be improved?

Start with an analytical approach

Questions to Answer:How does pheromone behave on a link?

What parameters are involved?What are the global effects of this behavior?

How do local parameters affect global behavior?

The Model

[2] D. Subramanian, P. Druschel, J. Chen, Ants and Reinforcement Learning: A Case Study in Routing in Dynamic Networks, Proceedings of the International Joint Conference on Artificial Intelligence, 1997.

Analysis Approach

Single link caseGain intuition onrelevant parameters

Two link systemExplore system behavior

Single Link Pheromone

Packets arrive with poisson distributionExpected pheromone over time is calculated

A scale invariant parameter is found!

−

−+−

=−

βµ

βµ β

τ

11)( 0PetEP

t

τλλβ+

=

( )τ

τλµβ

µ +=

−=1

EP

Pheromone Filter

Linear filter analogyPheromone arrival equation…

…can be expressed as a summation…

…which can be expressed as a convolution…

…of a one-tap infinite impulse response (IIR) filterThe pheromone update rule is a time dependant linear IIR filter

Commonly used for averagingBut this filter is unnormalized

∑ −

=−+=

1

00 )()( n

iin inPnEP βγβ

)()()( nBnnEP ∗= γ nnB β=)(

( )( )( ) µµµ τττ +⋅⋅+⋅+= −−− nttt eeePnP K210)(

-3 -2 -1 0 1 2 30

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2Frequency Response of IIR Filter (β = 0.5)

Normalized Frequency

Gai

n

Two Link Pheromone

Write a system of equations to describe the pheromone on links

Numerically find the steady state solution

All pheromone ends up on the better link

As expected…

γ Pheromone Filter

γ Pheromone Filter is the classic ant model

And the maximum mean link pheromone

Per link pheromone can be determined

( )0,0

~ µλτ

τλλ

A

ABABP

+=

0,0,0,0 µλλ

τλλ B

ABA

BA

AB pPP

+⋅

+

←

( )τnsttnsi

nsi

n ePPNi −−⋅←∈∀ ,,,γ+← n

srnsr PP ,,

100

101

102

0

0.5

1

1.5

2

2.5

time [# packet arrivals]

Phe

rom

one

on L

ink

Link Pheromone vs. Time (γ Pheromone Filter)

Dominant Link @ A (µ1 = 1)

Lesser Link @ A (µ0 = 0.99)

Dominant Link @ B (µ1 = 1)

Lesser Link @ B (µ0 = 0.99)

Normalized γ Pheromone Filter

Joint Decay IIR Filter is a normalized γ pheromone filter

And the maximum mean link pheromone

Per link pheromone can be determined

( )( ) 0,0

~ µτλλτλλ

AB

ABABP +

+=

( )τnsttnsi

nsi

n ePPNi −−⋅←∈∀ ,,,( )[ ] γτ ⋅−+← −− srttn

srnsr ePP ,1,,

100

101

102

0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

time [# packet arrivals]

Phe

rom

one

on L

ink

Link Pheromone vs. Time (Joint Decay IIR Averaging Filter)

Dominant Link @ A (µ1 = 1)

Lesser Link @ A (µ0 = 0.99)

Dominant Link @ B (µ1 = 1)

Lesser Link @ B (µ0 = 0.99)

0,0,0

,0,0,0 1 µ

τλλ

λλ

τλλ B

AAB

BA

BBABA

BA

AB p

pp

PP

+−

+⋅

+

←

How to Optimize Performance

Independent variables must be identifiedThe only controllable parameter isthe pheromone decay rate, τ

Packet arrival rate, λ, is givenNetwork environment changes on its own accord

Performance can only be optimized through manipulation of τ

On the Pheromone Decay Rate

Decay rate intuitionτ > τ*, information about network is lost too quicklyτ < τ*, too much information is retained

Order of magnitude studies to confirm decay rate trendPerformance improves with larger τ as network becomes more volatile

1 2 3 4 5 6 7 8 9 100

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5Pheromone Decay Rate (τ) vs. Speed

Node Speed (m/s)

Bes

t S

uite

d P

hero

mon

e D

ecay

Rat

e

Joint Decay IIR Filterγ Pheromone FilterpDijkstra

Decay Rate Heuristic:Maximum Pheromone

Link pheromone should decay within network correlation time

Maximum pheromone is known!

KeP ct =− τ~

Ke PFct =⋅⋅+ − γτµττλ

2

Ke PFct =⋅⋅++ − γτµ

τλτλ2

0 1 2 3 4 5 6 70

1

2

3

4

5

6

7

8

9

10

Instance Rate (sec-1)

Phe

rom

one

Dec

ay R

ate,

τ

τ Heuristic vs. Network Instance Rate

γ Pheromone FilterJoint Decay IIR Filter

Decay Rate Heuristic:Filter Cutoff

-3 -2 -1 0 1 2 30

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2Frequency Response of IIR Filter (β = 0.5)

Normalized Frequency

Gai

n

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 20

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

Instance Rate (sec-1)

Phe

rom

one

Dec

ay R

ate,

τ

τ Heuristic vs. Network Event Instance Rate

γ Pheromone FilterNormalized γ Pheromone FilterPheromone Filter Cutoff (-0.5dB)Pheromone Filter Cutoff (-3dB)Pheromone Filter Cutoff (-10dB)

Pheromone update can be viewed as a simple averaging filer

Decay rate can be adjusted to control the cutoff point of the filter

Depends also on packet arrival rate

Ultimately too complicated for a serious implementation

ωτλβω je

B −⋅−=

),(11)(

Overview






Motivation

How well do update strategies actually work?What is the global effect of a pheromone update method?

Data goodputDelivery efficiency

What effect do parameters have onglobal performance?

Exploration vs. Exploitation

Termite Enhancements

Several improvements are made to the routingTrue Continuous Pheromone Decay

Decay based on pheromone observation interarrival timesDecay on packet arrival and forwarding

Source Pheromone Repel [3]

R: pheromone repel constant

( )( )∑

∈

−

−

=

nNj

Rnsj

ndj

Rnsi

ndi

dipp

ppp

,,

,,,ˆ

[3] G. N. Varela, M. C. Sinclair, Ant Colony Optimization for Virtual Wavelength Path Routing and Wavelength Allocation, Proceedings of the Congress of Evolutionary Computation, 1999.

Pheromone Update Methods

Traditional pheromone based approachesγ Pheromone FilterNormalized γ Pheromone FilterClassic Pheromone FilterProbabilistic DijkstraAnt-Based Control + X

Independent link utility estimationIIRBox

Control casesOracleRandom Routing

Data Goodput vs. Node Speed

1 2 3 4 5 6 7 8 9 100.8

0.82

0.84

0.86

0.88

0.9

0.92

0.94

0.96

0.98

1Goodput vs. Speed

Node Speed (m/s)

Goo

dput

BoxIIRJoint Decat IIR FilterOracleγ Pheromone FilterClassic Pheromone FilterpDijkstraRandom RoutingABC + pDijkstraABC + γ Pheromone FilterABC + IIR2

Path Inefficiency vs. Node Speed

1 2 3 4 5 6 7 8 9 101

1.5

2

2.5

3

3.5Average Path Overshoot Ratio vs. Speed

Node Speed

Ave

rage

Pat

h O

vers

hoot

Rat

io


Delivery Efficiency vs. Node Speed

1 2 3 4 5 6 7 8 9 100.2

0.25

0.3

0.35

0.4

0.45

0.5

0.55

0.6

0.65Goodput/Average Path Overshoot Ratio vs. Speed

Node Speed (m/s)

Goo

dput

/ A

vera

ge P

ath

Ove

rsho

ot R

atio


Pheromone Accouting:Performance Evaluation

γ pheromone filter achieves best goodputThe algorithm nature intended!

Normalized γ pheromone filter achievesbest overall performance

Less pheromone allows faster adaptationIndependent link estimators perform badly

Box, IIRInformation loss on all links must be accounted for

The Oracle does not perform perfectly

Overview



Pheromone Model AnalysisPheromone Behavior in MANET ModelReinforcement LearningParameter Heuristics


AardvarkResimulation

ConclusionFuture Work

Conclusion

Created SI Termite routing algorithmUses Pheromone Based MetricRandom Walk for Route DiscoveryLocal Retransmit instead of Local Repair

Developed Pheromone Decay Rate HeuristicsMaximum PheromoneFilter Cutoff

Developed Pheromone Accounting SchemesNormalized γ Pheromone FilterProbabilistic Bellman-Ford

Updated Termite with new FeaturesContinuous Pheromone DecaySource Pheromone RepelUpdated Pheromone Accounting Schemes

Future Work

TermiteAdapt Termite to asymmetric networks

Use Cooperative-Asymmetric Forwarding (CAF) [4]

Use flooding for route discovery?Further explore operational limits

RoutingSystem of linear systems

Extend the linear filter analogyNetwork correlation time

Swarm IntelligenceFormalization of information and information loss

[4] M. Heusse, D. Snyers, S. Guerin, P. Kuntz, Adaptive Agent-Driven Routing and Load Balancing in Communication Networks, 1998.

Publications

M. Roth, S. Wicker,Ad-Hoc Networking with Swarm Intelligence,in preparation, 2005.M. Roth, S. Wicker,Termite: Updates and Comparisons in Swarm Intelligent Ad-Hoc Networking,in preparation, 2005.M. Roth, S. Wicker,Performance Evaluation of Pheromone Update Schemesin Swarm Intelligent MANETs,Under Review, 2005.M. Roth, S. Wicker,Asymptotic Pheromone Behavior in Swarm Intelligent MANETs:An Analytical Analysis of Routing Behavior,Sixth IFIP IEEE International Conference on Mobile and Wireless Communications Networks (MWCN), 2004. M. Roth, S. Wicker,Termite: Ad-Hoc Networking with Stigmergy,IEEE Globecom 2003, 2003.M. Roth, S. Wicker,Termite: Emergent Ad-Hoc Networking,The Second Mediterranean Workshop on Ad-Hoc Networks, 2003.

Thank you!

Questions?

Documents

Termite: A Swarm Intelligent Routing Algorithm for Mobile