Adopting Ideas from Interplanetary Networking for Sensor Network Applications

11/12/04 CENS Seminar 1

Adopting Ideas from Interplanetary Networking for Sensor Network Applications

Andrew Parker, UCLA

Scott Burleigh, JPL

Richard Guy, UCLA

Deborah Estrin, UCLA


Interplanetary Network Vision

• Picture of sensor networks in space

QuickTime™ and aTIFF (LZW) decompressor

are needed to see this picture.

* V. Cerf, InterPlanetary Internet, 2004


Why We Care


Outline

• Mexico Seismic Array Deployment: Set Up• Interplanetary Networking: Set Up & Relevance• Delay Tolerant Networking (DTN): Architecture• DTN vs. Email• Mexico Deployment: DTN Approach• Modular DTN Architecture for Sensor Networks


Mexico Deployment

• People: Paul Davis, Deborah Estrin, Richard Guy, Martin Lukac, John Wallace, Monica Kohler, Ramesh Govindan, Igor Stubailo, Allen Husker, Katie Mika, John Propst, Sam Irvine, Jeremy Elson, et al.

• Seismic array of 50 nodes• 5 km apart• Spanning 250 km• Through jungle, mountain,

and urban environments• Targeting end of Q1 2005


Data Story

• Every bit is important. Can’t lose data.

• 24 bits / sample, 100 samples / sec, 3 channels = 900 bytes/sec, 3.24 MB/hour, 1.2-1.8 MB/s compressed, per Node

• 1 GB Flash will hold 20 days of compressed data

• Conventional methods of retrieval

– Satellite uplink (expensive)

– Data mule (graduate student)

• Is there a better way?

CENS Data CommunicationsController

Guralp Seismometer

Q330 24 bit Digitizer

Stargate


A Better Way: Networking the Array

• Goal: Multihop data from the edge into Mexico City• Mexico deployment is different from typical sensor network

applications– Every bit of data is important

– Radio connectivity is directional • Due to distance, power, and regulations, we’re using directional

antennas

– Routing is over a linear topology

– Radio communication is not the greatest power consumer


Braided String

• Introduces some path redundancy• 802.11b

– 200 mw radio (SMC PCMCIA card)

• Yagi antenna– 20 dbi– 30 degree spread– Tested with 2-way splitter– 5 - 10 km in LA area– 4 Mb/s - 150 kb/s

• Parabolic antenna– 24 dbi– 5 - 10 degree spread– 4 way splitter– Tested to ~ 25 km– Closer to 1 Mb/s


Routing is Hard to Do Right

• Possible approach: Static Routes - topology is linear and nodes are immobile

• Problem: Still as brittle as a single string -- can’t back track around breaks

• Possible approach: Use AODV, DSR, Roofnet, etc.• Problem: Flapping links will make establishing end-to-end

routes nearly impossible

• What to do, what to do…?


Outline



Interplanetary Networking: Motes in Space

• Huge transmission delays– Several seconds to

the Moon

– 5 - 20 minutes to Mars

– 1 hour to Jupiter

– 7 hours to Pluto

• Lossy links• Long disconnects

– Sometimes predictable


Mars Communication

http://www.astrosurf.com/lombry/qsl-mars-communication3.htm


Mars Connectivity

QuickTime™ and aCinepak decompressor



Oops!


Properties of Interplanetary Networking

•End to end paths unreliable•Interactive / chatty protocols break•Need to communicate across varying network technologies, including non-IP networks

•Links are lossy and high delay–Very high round trip times,

–Even for single hop

•Links are often disrupted–Node mobility

–Node powered down

•Asymmetric links•Asymmetric node capabilities


Delay Tolerant Network Architecture

• Messaging service (Like Email) – Insulates applications from

network behavior

– Asynchronous, deferred transmission

– Non-interactive end-to-end

You’veGotDTN!


Delay Tolerant Networking

• Bundles are Basic Unit of Transport– Written as files onto persistent store

– Resilient across server / node restarts

• Custody Transfer of Bundles– Represents a QOS agreement: custodian tries real hard to transfer

custody and not delete until this has happened

– Conceptually moves the “end” of an end to end transaction

– Custody is asserted, rather than given.

• Overlay routing of Bundles among Custodians– Custodians may be separated by intermediate nodes.


Delay Tolerant, Not Run-Over-by-a-Bus Tolerant

• Bad things can still happen with Custody Transfer• Brings up the question of trust. When should a node trust

another more than it itself? • Risk vs. Resource trade-off

VS


Bundle Agents and Clients

• Client nodes register with Bundle Agents to send and receive Bundles on their behalf

– Email analogy: Pop or IMAP client connecting to a server– Based on the destination name, delivery is made to the corresponding

Bundle Agent.– At this point it’s considered to be done. Again, similar to Email.

• Naming scheme allows for late binding and separation of names from nodes

– Determination of which node(s) receive the message may be deferred

• Connects different domains via gateways– Late binding names allow different domains to communicate (through

gateways). – Connects loosely coupled “Internets”– Official DTN specification uses Tuples:

• (Interdomain Label, Opaque Intradomain Label)


Outline



DTN vs. Email

DTN? Email?What’s the difference?

Professor Culler


DTN vs. Email

Cool

Professor Culler

Biggist Difference: DTN Overlay

DTN is able to make progress towards the destination, even when no contemporaneous route exists


Outline



Mexico Deployment: Routing

• DTN has the “Tuple” address: (Interdomain, Intradomain)– Doesn’t really apply to Mexico

– Single domain

• Hybrid Routing Approach– You know where you want to get to, but not how

– Overlay static on top of mesh routing• Using Roofnet

– link-quality aware

– minmizes number of transmissions

– Use mesh routing to tell you the next hop towards the furthest reachable downstream candidate


Hybrid Examples

• Normal case• Simple break• Partition• Reconnect


Hybrid Examples



Hybrid Examples



Hybrid Examples



Hybrid Examples



Mexico Deployment: Handling Data

• DTN suggests exchanging Bundles (Files)• Divide data into one-hour segments• Augment with meta-data

– To, From, Data, Size, etc.

• 1.2 - 1.8 MB compressed• Store as files on disk (even on intermediate hops)• Easy to recover from server and node restarts• Human manageable


Node Software Architecture

BundleBundle

Bundle

Outbox

Duiker

Bundle Forwarder

Bundle Receiver

RoutingDataBundle Sender

Seismic Data(Q330)Upstream

Nodes

DownstreamNode

BundleBundle

Bundle

Inbox

Data FileData File

Data File

Drafts

CENS NODE


Managing Space

• Nodes are equipped with 2 - 4 GB of storage: 40 - 80 days of data

• A Bundle is deleted when ACK is received from sink for locally generated data– ACK is just another bundle

– Application-level ACK

• When space gets low– Node refuses to accept transient data (“route around me”). Better

than accept and drop.

– Delete transient data when space gets low

• No intermediate custody transfer• This policy delays the deletion of original data the longest


Transmission Priority

• Transmission priority affects performance– Want to avoid starvation

• Priority can be based on:– Age

– Transient vs. Local

– Number of hops traversed

– Number of times sent

– Some other measure of how hard it was to get this far


What about Off the Shelf DTN?

• There is an official Bundle specification

• There are 1 1/2 reference implementations of the Bundle specification

• Why not use it?

• Because…– Only reason why Mars worked, and only way Mexico will work, is because

scenario specific information was used. You MUST do this.– It’s impossible to do a good job in all scenarios– The more generic you try to be, the more it looks like flooding.– The more options and switches you support, the more brittle and complex it

becomes• See Sendmail configuration file

• How do we remain flexible and customizable, yet stay SIMPLE?


A Modular DTN Software Architecture

• Like Click

• Modular software IP router

• Easy to plug modules together

• Easy to create new modules

• Results in an optimized router



// Declare three elements…src :: FromDevice(eth0);ctr :: Counter;sink :: Discard;// .. Connect them togethersrc -> ctr;ctr -> sink;


Click’s Elegance

• Click modules consume and produce the same data structure

– IP packets

• Same is nice and simple. Easy to compose modules if they input/output the same thing

• IP is nice too (as opposed to something obscure or too generic)

– Strong foundation in specifications

– Many compliant implementations as examples

– People are already familiar with certain features and behaviors of IP routers




Click and DTN - High Level

• IP packets• IP specifications• Lots of existing IP routers• Lots of specific IP

features/behavior to choose from

• Bundles• Bundle specifications

– Experimental and evolving

• Very few Bundle servers• Lots of Bundle features and behavior

to choose from– In this area, Bundles have a richer

set of potential behavior than IP

Click DTN


Introducing this Approach to Sensor Networks

• EmStar– Component based framework for sensor network applications (on Linux)– Lots of services and applications already existing– Simulation / Emulation / Deployed Modes– Heterogeneous applications (Stargates / Motes)– Debugging, monitoring, and visualization– Growing user-base

• UCLA, USC, MIT, Umass Amherst, Ohio State, etc.• Intel Research, Xerox PARC, Microsoft Research

Emulation Array

Node001 …

Emulation Mode

HostMote Protocol

Tra

nsce

iver

Mot

e

Tra

nsce

iver

Mot

e

Tra

nsce

iver

Mot

e

Tra

nsce

iver

Mot

e

MN MN MN MN

Node002

Node003

NodeN

Simulation Frameworkwith real RF channels

Visualization Tools

Client Server

kfusd.o

/dev/fusd/dev/servicename

Kernel

User

Robust multi-process,microkernel architecture


Relationships Between EmStar Modules Unclear

How to bring Click’s readability and ease-of-use to EmStar?

include link/link.run &link_udp(udp0); &link_linkstats(udp0,ls0,show="leds:core"); &link_neighbors(ls0,show="hide"); process mdiff { type = once; noclean; cmd = "devel/microdiff/mdiff --uses ls0"; waitfor = ls0; }

process mdiff_test { waitfor = mdiff; type = once; noclean; cmd = "devel/microdiff/mdiff_test"; } process mdiff_filter { waitfor = mdiff; type = once; noclean; cmd = "devel/microdiff/mdiff_filter"; }

Example EmRun Configuration File


Bolt: Architecture Description Language for EmStar

• Makes relationships between modules of an EmStar application explicit

• Began as a class project with Eddie Kohler and Todd Millstein last spring

• Implemented in CIL and Ocaml


udp0 = udp() ;ls0 = linkstatsd();nd = neighbord();mdiff = Mdiff();

mdiff_simple_app [packet_dev] <-> [app_packet_opts] mdiff;mdiff_simple_filter [packet_dev] <-> [filter_packet_opts] mdiff;

mdiff [lu_opts] <----------------------> [lp_opts] ls0 [lu_opts] <-> [opts] udp0;mdiff [sc_opts] <- [s_opts] nd [ls_opts] <-- [s_opts] ls0; nd [lu_opts] <-> [lp_opts] ls0;

Bolt Config.: White Board to Emacs Buffer

Instantiating componentsData FlowConnecting component devices


BUT WAIT, THERE’S MORE!…


Click Modules vs. EmStar Modules

• Click modules are C++ classes• EmStar modules are separate processes• Click modules communicate via function calls• EmStar modules communicate by passing bits or text via

device files• Since Click modules are compiled, they benefit from basic

compiler checks and optimizations. Click itself does higher level checks and optimizations

• Wouldn’t it be nice if EmStar modules enjoyed similar benefits?


Bolt: Compiler-like Analysis and Optimization for EmStar

• It Slices! It Dices!

• It does static analysis and optimizations not otherwise possible across EmStar modules

• No user changes or annotation required of EmStar code– Though it could help

• Bolt statically infers type-safety violations between two functions in different processes (EmStar modules)

– Despite type-obscuring function calls and heavy use of function pointers

• Bolt uses subgraph isomorphism to identify and swap out combinations of modules with more efficient combinations that perform the same function

• Analysis not completely sound nor complete, but hard given the circumstances


Implementing DTN Using Bolt

• DTN looks like any other set of EmStar modules

• What exactly are the EmStar modules that make up the DTN suite?– Click has benefited from the existence of numerous IP router

implementations– DTN must build up experience from real deployments (Mexico and others)– Identify and extract reusable DTN related components, for example:

• Managing a large number of concurrent file transfers between a pair of nodes

• Despite large delay and disconnections

• Recover across restarts

• LTP (S. Burleigh) & File Mover (A. Parker)

– Goal is not necessarily to be able to build a complete DTN solution for your particular situation

– Rather, to reuse the ones that make sense, and build from there


Outline



References

• DTN: http://www.dtnrg.org• IPN: http://www.ipnsig.org• Click: http://www.pdos.lcs.mit.edu/click• EmStar: http://cvs.cens.ucla.edu/emstar• Bolt: http://lecs.cs.ucla.edu/~adparker/Bolt

http://www.dtnrg.org/

http://www.pdos.lcs.mit.edu/click

http://lecs.cs.ucla.edu/~adparker/Bolt

Documents

Adopting Ideas from Interplanetary Networking for Sensor Network Applications