Tiny Networking

Tiny Networking

David Culler University of California, Berkeley

Intel Research Berkeley

http://webs.cs.berkeley.edu

11/14/2002 NEC Networking

Vast Networks of Tiny Devices• Past 25 years of internet technology built up around powerful

dedicated devices that are carefully configured and very stable– local high-power wireless subnets at the edges– 1-1 communication between named computers

• Here, ...

• every little node is potentially a router• work together in application specific ways• collections of data defined by attributes• connectivity is highly variable• must self-organize to manage topology, routing, etc• and for power savings, radios may be off 99% of the time


Directed Diffusion Concept

• Nodes express ‘interest’ in data with certain attributes (sinks)

• Establishes gradient from sources

Estrin, Govindan, Heideman


Directed Diffusion Concept

• Nodes express ‘interest’ in data with certain attributes (sinks)

• Establishes gradient from sources

• Sources generate data

• Useful paths reinforced, others suppressed

• in-network aggregation

• nested queries


Huge Design Space: Application

• Traffic– any-to-any, all-to-one, one-to-all, collection of sub-groups, ...– steady low-BW steam of readings/findings, bursts of data, periodic logs,

...

• Duration– years (hazard alarm), months (field season), when the big red button is

pressed

• Available infrastructure– power, base-stations, location

• Mobility, Stationary– all fixed, all mobile, mixture, changing environment

• Placement / Physical Topology / Scale– arbitrary, controlled, unknown

• Network Services– time synchronization, localization, proximity, ...

=> Best way to get a networking solution now is to define your application context


Design Space: Underlying Technology

• Link (Radio) Technology– range, control of signal strength, noise tolerance– channel capacity, coding, error rates– single channel, multi-channel, tunable

• Device Density, Failures over time• MAC (media access control)

– channel sensing, back-off, protocols, collision behavior, link-level acks– time synch, energy aware

• Power management & Energy Constraints– scheduling, functional allocation

• Transmission Rate Control• Topology Formation

– hierarchical spine, geographic, static, dynamic

• Routing– single path, multipath, passive participation– explicit vs implicit nbhd detection


Losing the forest for the trees

• Systems side– clever fix to one particular aspect, but only looking at it in

sliver of the design space

• Theory side– assuming cell coverage is a disk of radius r with sharp

boundary

– within radius Pconnect = 1, outside Pconnect = 0

– eg., unit disk-graphs for routing, maximal independent sets, min. dominating sets, leader election

• Wireless communication between small devices is inherently noisy, unpredictable, & non-deterministic

– Radio signal fades with distance

– Interference

– Multipath (reflections)

– Collisions

– Mobility or not


Surge Demo


Local Operations => Global Behavior

• Nodes ‘sense’ network environment– uncertain, partial information

• Packets directed to a “parent” neighbor– all other neighbors “hear” too

– carry additional organizational information

• Each nodes builds estimate of neighborhood– adjusted with every packet and with time

• Interactively selects parent

• Routes traffic upward Collectively they build and maintain a stable spanning tree

takes energy to maintain structure

node # depth

child? parent?

% link

goodness

17 1 yes 90 .7

6 3 yes 75 .6

...

Predictable global behavior built fromlocal operations on uncertain data


Wireless “Connectivity”

• Controlled study on 13x13 array of Rene nodes

• Single transmitter• Record fraction of

packets received at each node

• Many packets at each of several transmit power levels

complex fall-off over substantial range

‘range’ defined by CEP

Contours of probability of reception from center node for range of transmit power strengths

with Ganesan, Woo, Krishnamacheti, Estrin


Alternative Perspective

• Develop algorithms that are built fundamentally upon a probabilistic connectivity structure

– Embrace noise, rather than fight it

– the network is really another sensor (and actuator)

• Utilize simple, local rules, rather than complex protocols

• Challenge: obtain predictable global behavior

• Challenge: interfaces for imperfect operation


Simple Epidemic Broadcast Schema

Local Rule:if (new mcast) then

take local actionretransmit modified request

• Should forms roughly breadth-first spanning tree• Examples: Network wakeup, command propagation

– Build spanning tree» record parent

• Naturally adapts to available connectivity• Minimal state and protocol overhead

=> surprising complexity in this simple mechanism


Network Discovery: Radio Cells


Network Discovery


Behavior at Scale

• Variations in connectivity

• With many nodes, likely that one far away will hear

• Long links tend to be asymmetric

• Structure dominated by contention


Final Tree


Power Laws ?

• Most nodes have very small degree (ave = .92)• Some have degree = 15% of the population• Few large clusters account for most of the edges

1

10

100

1000

1 10 100

Cluster Size (1 + # children)

Co

un

t

1

10

100

1000

1 10 100

Cluster SizeL

ink

s


Open Territory => Many Children

• Example: Level 1



• Example: Level 2 – variation in distance



• Example: Level 3 – long links


Importance of Asymmetric Links

• Asymmetric Link: – >65% successful reception in one direction – <25% successful reception in the other direction

• 10%-25% of links are asymmetric• Many long links are asymmetric

– in large field it is likely that someone far away can hear you– what does this mean for protocol design?


Collisions are primary factor

• Nodes out of range may have overlapping cells– hidden terminal effect

• Collisions => these nodes hear neither ‘parent’– become stragglers

• As the tree propagates – folds back on itself

– rebounds from the edge

– picking up these stragglers.

• Seen in many experiments

• Mathematically complex because behavior is not independent beyond singe cell


Probabilistic Connectivity Model

• Radio signal fades with distance in complex manner depending on environment• Radio receiver has complex behavior to extract signal• What matters to algorithms is whether packets are delivered or not work directly with probabilistic communication model but which one?

• Calculate comm. rates for numerous transmit/receive pairs at range of distances


Fall-off of Probability of Comm.

Low Power High Power


Drive Simulation from Empirical Stochastic behavior

clear silenttransition


Example Cell Coverage from Model

feet

feet


Reception Model with Collisions

• Second experiment with two nodes sending at once, record which nodes hear which one

– follows P success closely

• When does a second sender collide?– Clear comm. region => YES

– silent region => NO

– transition region => collides if would have communicated

• Reception Model– Assume pij is the probability of success (i->j), Probability for B to

receive A’s message

= pab* i=collider(1-pib)


Common Special Case: Data Gathering

• Collection of nodes take periodic samples• Stream data towards a root node

• Root announces interest– depth = 0

• Nodes listen to neighbors• When hear neighbor with smaller depth

– start transmitting data to good neighbor with smallest depth– set own depth to one greater and include with data

• Data transmission continuously reinforces / adjusts routes


Use Case Assumptions

• Application– N-to-1 all data, no aggregation– Each node generates small packets at regular interval

» Appln phase shift on collision– Routed to a specific node, e.g, base station or root of request– data rate below saturation– 7x7 grid, 10 ft spacing => several hops

• Underlying– link-level acknowledgement (may be used for retransmission)– CSMA with simple channel sensing, fixed backoff, initial random delay– Receivers always attentive

» may use sampled listening

• Broadcast: choose parent = first contact• Shortest Path: choose parent = closer (which one)• Min path loss: choose parent = next step on min loss path


Max-Path-Reliability routing

• each node maintains estimate of loss rate over entire path to root

• select nbr on the minimum loss path as parent– Pi[to root] through j = P[link i,j] * Pj[to root]

– assuming loss rate along path is independent of how packets enter the path

• Subtleties– estimating link rates

– transient error in link rate may lead to cycles

– rate of updates

– stability vs responsiveness

– congestion

– warm-up phase

95

95

9565

70

99

60

P=0.86


Broadcast

• overall success rate: 18.9%


SP50 Path reliability



MPR path reliability



Routing Distance Distribution

SD50MPR


Distribution of success rates

MPR SP50


Observations

• Better success rate with more, better hops

• Partial information causes temporary loss– cycles introduced when link error estimates are poor

– tracks std. dev. of link error rates

– if difference between candidates not statistically significant, choose based on hop count

– especially important during warm-up phase

» also after topology changes

• Shortest Path will tend to use most marginal neighbors

– although range is highly variable, for particular pairs of nodes at particular distances connectivity is very bimodal


SP75 Path Reliability

• overall success rate: 52.7% (vs 54.7% for MPR)


Multihop Path-Rate Contours

Min Path Loss Min Hop (75% nbr)


Hop-by-hop retransmission

• Decent neighbors become good neighbors if you are willing to chat for a moment

• may choose parent that is on “path of least transmissions”

– expected number of transmissions = 1 / P(success)

• Distributed computation is estimate of SUM of retransmissions

• A few retransmissions make large difference in success rate


Overall Results

Success Rate to BS

# Retrans / App packet

Total Msg. Overhead/ pack. Recv. By BS

Ave.

Hop

Count

MRP 83.94% 1.03 8.36 6.12

Shortest Path

85.79% 0.82 7.13 6.18

Broadcast 42.96% 5.76 13.13 4.92


Fairness

MRPSP75

Broadcast


Stability

• How often does the routing algorithm change?

• # of parent changes per unit time

MRPSP75


Stability (Broadcast)


Key Building Block: Link estimator

• Nodes assess quality of link from packets they hear

– directed to them or snoop

– sequence number => infer losses between packets

• For each new packet (or empty window) revise estimate of link probability

• classic EWMA– P i+1 = P i + (1-) X i , where X i = 0,1 if loss, success


Link Probability Changes Dynamically


Revisit Classic Estimator

• Want estimator that is responsive to change, but stable with low error

• Candidates– EWMA stable, agile, flip-flop

– Moving Average

– ...

• Best ended up being the own that we eyeballed– EWMA cascaded with average over a time window


Estimator driven from prob. generator


Empirical Trace


Bottom Line

• Can only estimate link rate to within ~10%

• Takes about 100 packets to settle

• Design distributed algorithms with this kind of approximation of an inherently noisy world


So how about that demo?


Higher Level Network Services

• Time Synchronization– many nodes with drift and offset variation

– classic:

» pairwise timesynch limited to half the variance in RTT

» NTP hierarchy to establish rough global time

– local timesynch

» deep integration into network stack (Hill)

» multiple receivers, not send-rcv (Ellison, Girod)

– global propagation: huge open distributed algorithm question

• Ranging & Localization

• Naming

• Data distribution, Aggregation, Storage

• We’ve just begun to tackle the full problem


Conclusion

• The networking part of the “future sensor networks” very different the power-hungry, sparse IP world

– severe constraints, noisy, dynamic, partial information

– Low power means you hardly ever listen

– self-organized and inherently distributed

– opportunity to work across layers of abstraction

• Presents a rich set of open problems– Probabilistic connectivity model is usable and powerful

– On-line distributed approximations to global optimization

– a whole new internet to invent

• Applications define valid assumption sets for network design

– guide the front of progress in this huge space


Engineering Perspective

• Low-hanging fruit– enough base stations that only need single hop

• Mid-range fruit– stationary, well-powered multi-hop backbone

• Research centroid– unstructured, ad hoc, multihop routing in energy starved

environment

Documents

Tiny Networking