- 1 -

DRIFT: Efficient Message Ordering in Ad Hoc Networks

Stefan Pleisch

Joint work with

Thomas Clouser, Mikhail Nesterenko, André Schiper

EPFL, Kent State University

SRDS 2006

- 2 -

Background: Ad Hoc Networks• Communication is ad hoc, i.e., no fixed communication

infrastructure• Two nodes can communicate if

– they are within transmission range, or– there is a sequence of intermediate nodes that can forward the

message from the source to the destination (set up by a routing protocol)

• Communication is by broadcast (to all neighbors), or by point-to-point (to one neighbor, but still a broadcast)

• Shared transmission media and low bandwidth leads to interference and message collisions, and thus to message losses– Hidden terminal problem (Allan 1993)

• Maintaining routing information may not be feasible if nodes are mobile or have limited resources– flooding is an effective mechanism of reaching all nodes

in the network without routing information• a source broadcasts a message to its neighbors• every node rebroadcasts the message once

- 3 -

The Need for Total Ordering in Ad Hoc Networks

• Consider a temporary military sensor network deployed to protect an extended asset– communication is multihop and ad hoc

– directives are periodically issued by mobile operators

– commands must be delivered in the same order at each sensor node to prevent conflicting behavior of different network regions

requires Total Order Multicast

• More applications from traditional fixed networks find their way to wireless networks

• Properties of ad hoc networks make total ordering of messages challenging

- 4 -

Outline

• Total Order Multicast

• Lamport’s Total Order Multicast Algorithm

• Virtual Flooding

• DRIFT Description

• Example

• Simulation and Experimentation

• Conclusion and Future Work

- 5 -

Total Order Multicast

• Communication primitives– TO-multicast is invoked to send a message to all the nodes of the

multicast group; executed by a source node– TO-deliver is executed to convey the message to the application;

executed by a destination node

• Problem: Ensure all messages TO-multicast by any source are TO-delivered in the same order at all destinations

- 6 -

Related Work• Many total order multicasts in fixed networks, see ACM Survey by

Defago et al. 2003– sequencer-based: single sequencer decides ordering

• asymmetric load on the network– privilege-based: token-holder establishes order

• asymmetric load on the network• expensive route maintenance among the token users

– destination: ordering by agreement among destinations• expensive when man destinations

– communication history ordering: based on message timestamps

• Few algorithm in ad hoc networks– sequencer-based in single hop networks

• Bartoli 1998 (Mobile Networks and Applications)• Anastati et al 1999 (SRDS’99)

– privilege-based• Malpani et al. (IEEE ToMC)

– communication history • Prakash et al. (ICDCS’97), dependency on fixed infrastructure

– Lou et al. (IEEE ToMC) use probabilistic guarantees

- 7 -

Lamport’s Total Order Multicast Algorithm• CACM 1978, belongs to the class of communication history ordering

• Delivers messages based on the causal order of multicasts– causal relation establishes a partial ordering– total order achieved by deterministically ordering concurrent messages (e.g. by

source id)

• Assumes FIFO communication channels and reliable messaging, no failures

• Uses logical clocks (Lamport’s clocks) for capturing causal relationship between multicast messages

• Source nodes update their logical clock– prior to sending a multicast– when the source receives a multicast message it has not received before

• Delivery rule: Node n can TO-deliver a particular message m only after it receives a message with a higher or equal timestamp from every source

• Disadvantage: the delivery rate of all destinations depends on the sending rate of the source that multicasts least frequently

- 8 -

Virtual Flooding

• Node attaches data to an unrelated message it has to broadcast

• Example

(1) Node a sends m

(2) Node b forwards m

(3) Node c attaches virtually flooded message and forwards m

(4) Node d forwards combined message

(5) Node e forwards combined message

Node a has message to physically flood

Node c has a message to virtually flood

(1) (2)

(3) (4)

a b c d e a b c d e

a b c d e a b c d e

(5)

a b c d e Physically flooded msgVirtually flooded msg

Local broadcast

- 9 -

DRIFT: Key Concepts

• Combines virtual flooding with Lamport’s algorithm to achieve lower delivery latencies

• Virtual flooding propagates the latest logical clock of each source

• For node n to TO-deliver message m it is sufficient that n learns that it will not receive a message from any source with a timestamp less than or equal to m’s timestamp

• DRIFT assumes reliable flooding, as e.g. in DELUGE (Hui and Culler, Sensys’04)

- 10 -

Example

a cb

lc = 0

sn = 0

lc = 0

sn = 0

RcvdSN = [0,0]

Seen = { }

RcvdSN = [0,0]

Seen = { }

RCVD = { }

DRIFT DLVD = { }

TOF DLVD = { }

RcvdSN = [0,0]

Seen = { }

Initial condition – a and c are sources, b is a destination

Note: TOF is Lamport’s algorithm

- 11 -

Example

a cb

lc = 1

sn = 1

lc = 0

sn = 0

RcvdSN = [1,0]

Seen = {}

RcvdSN = [0,0]

Seen = { }

RCVD = { }

DRIFT DLVD = { }

TOF DLVD = { }

RcvdSN = [0,0]

Seen = {}

a TO-multicasts <am1, a, 1, 1, {<a, 1, 1>}>Source Logical clock (lc)

Sequence number (sn)

- 12 -

Example

a cb

lc = 1

sn = 1

lc = 0

sn = 0

RcvdSN = [1,0]

Seen = {}

RcvdSN = [1,0]

Seen = {<a,1,1>}

RCVD = {am1}

DRIFT DLVD = {}

TOF DLVD = {}

RcvdSN = [0,0]

Seen = {}

b updates RcvdSN and Seen

b has received <am1, a, 1, 1, {<a, 1, 1>}>

- 13 -

Example

a cb

lc = 1

sn = 1

lc = 0

sn = 0

RcvdSN = [1,0]

Seen = {}

RcvdSN = [1,0]

Seen = {<a,1,1>}

RCVD = {am1}

DRIFT DLVD = {}

TOF DLVD = {}

RcvdSN = [0,0]

Seen = {}

b rebroadcasts <am1, a, 1, 1, {<a, 1, 1>}>

- 14 -

Example

a cb

lc = 1

sn = 1

lc = 2

sn = 0

RcvdSN = [1,0]

Seen = {}

RcvdSN = [1,0]

Seen = {<a,1,1>}

RCVD = {am1}

DRIFT DLVD = {}

TOF DLVD = {}

RcvdSN = [1,0]

Seen = {<a,1,1>}

c has received <am1, a, 1, 1, {<a, 1, 1>}>

- 15 -

Example

a cb

lc = 1

sn = 1

lc = 2

sn = 0

RcvdSN = [0,0]

Seen = {}

RcvdSN = [1,0]

Seen = {<a,1,1>}

RCVD = {am1}

DRIFT DLVD = {}

TOF DLVD = {}

RcvdSN = [1,0]

Seen = {<a,1,1>}

c rebroadcasts <am1, a, 1, 1, {<a, 1, 1>, <c, 2, 0>}>

- 16 -

Example

a cb

lc = 1

sn = 1

lc = 2

sn = 0

RcvdSN = [0,0]

Seen = {}

RcvdSN = [1,0]

Seen = {<a,1,1>, <c,2,0>}

RCVD = {am1}

DRIFT DLVD = {am1}

TOF DLVD = {}

RcvdSN = [1,0]

Seen = {<a,1,1>}

b TO-delivers am1 with DRIFT, but not with TOF

b receives <am1, a, 1, 1, {<a, 1, 1>, <c, 2, 0>}>

- 17 -

Example

a cb

lc = 1

sn = 1

lc = 3

sn = 1

RcvdSN = [1,0]

Seen = {}

RcvdSN = [1,0]

Seen = {<a,1,1>, <c,2,0>}

RCVD = {am1}

DRIFT DLVD = {am1}

TOF DLVD = {}

RcvdSN = [1,0]

Seen = {<a,1,1>}

c TO-multicasts <cm1, c, 3, 1, {<a, 1, 1>, <c, 3, 1>}>

- 18 -

Example

a cb

lc = 4

sn = 1

lc = 3

sn = 1

RcvdSN = [1,1]

Seen = {<c,3,1>}

RcvdSN = [1,1]

Seen = {<a,1,1>, <c,2,0>,<c,3,1>, <a,4,1> }

RCVD = {am1, cm1}

DRIFT DLVD = {am1, cm1}

TOF DLVD = {am1, cm1}

RcvdSN = [1,1]

Seen = {<a,1,1>}

- 19 -

Simulation -- Setup• Using Java-based JiST/SWANS network simulator

(v1.0.4)– Developed by Rimon Barr, 2004 (http://jist.ece.cornell.edu)– Applications written for real deployment can be ported to the

simulation environment and be placed under variety of simulated scenarios

• Communication is by broadcast as defined by IEEE 802.11b, transmission range is set to 88m

• Setup– 100 nodes in a field of 400x400m– Nodes are stationary– Nodes start up at random times and positions. Each node floods

20 messages (128Bytes) at regular interval (= base rate) through the entire field

- 20 -

Simulation – Results and Metric

• Message loss due to hidden terminal problem minimized due to low base rates

• Results: Average of 20 runs with 95% confidence interval

• Delivery latency – the time needed to TO-deliver a message after it was received at a destination

• We compare the performance of– Total Order multicast with Flooding only (TOF) , Lamport’s Algorithm

– Total Order multicast with Virtual Flooding (TOVF), DRIFT using physically flooded multicast messages as virtual flooding carriers

=> Speedup = latencyTOF / latencyTOVF

• Unless otherwise noted – the measurement for TOF and TOVF are taken in the same experimental trial

- 21 -

Rate Delay• Speedup with different base rates• Varying rate delay

• The delivery latency is impacted by– base rate – the rate at which

the source TO-multicast messages

– rate delay – the relative difference in the base rate between the sources

• To evaluate the effect of relative rate differences, for each source i we set the TO-multicast rate as follows

TO-multicastRatei = baseRate + (i rateDelay)

- 22 -

Position of Flooding Source (1/2)

Single source [0,0]

Average

Max

• One to four sources physically flood messages at aggregated frequency 1s-1

• Other nodes use only virtual flooding

• Varying positions of (physical) flooding source

• All nodes are destinations

order of network diameter

- 23 -Single source [350,350]

Average

Max

2 sources [0,0], [600,600]

Average

Max

4 sources in 4 corners

Average

Max

Position of Flooding Source (2/2)

- 24 -

Gossiping

• Number of virtual flooding entries per message

• Base rate 10s• Overhead increases

linearly• Speedup decreases

with increasing number of transmitted tuples

• Applications need to chose a trade-off point between overhead and speedup

- 25 -

Experimental Setup• We evaluate the performance of DRIFT using

BenchNet, our wireless sensor testbed

• Setup: 16 MICA2 motes, arranged in a 4x4 grid, running the TinyOS operating system and programmed using the nesC programming language

• Each mote is connected via its communication port to an Ethernet programming board – which allows monitoring applications and the motes to communicate

• 4 interior nodes are sources, all nodes are destinations

• Each source multicasts 10 messages• Base rate of 30 seconds• Reliable 1-hop communication

emulated• TOF and TOVF implemented

separately

- 26 -

Conclusion and Future Work

• DRIFT uses virtual flooding to propagate logical clock information

• We demonstrated through simulation and experimentation the effectiveness of DRIFT as a total order multicast delivery mechanism for ad hoc networks

• Although based on flooding, DRIFT can be modified to work with structured routing mechanisms

• Virtual flooding can be used to propagate data of any type

• Future work – measure different failure scenarios, especially failures of sources

– scale the experimental evaluation up to many nodes

- 27 -

Questions