Embed Size (px)
Citation preview
Mote Herding for Tiered Wireless Sensor Networks
Thanos Stathopoulos†
Lewis Girod†
John Heidemann‡
Deborah Estrin†
Center for Embedded Networked Sensing† UCLA, Department of Computer Science
‡ USC, Information Sciences Institute
{[email protected], [email protected], [email protected], [email protected]}
AbstractWe proposeMote Herding, a new system architecture
for large scale, heterogeneous sensor networks. Moteherding uses a mix of many 8-bit sensor nodes (motes)and fewer but more powerful 32-bit sensor nodes (mi-croservers). Mote herding groups motes intoflocksthatare connected via a multihop network to a microserveracting as ashepherd. Shepherds exploit their greatercommunications and compute power to form an overlaynetwork, with many flocks joining to form aherd. Bykeeping each flock small and utilizing several shepherds,the herd can support many nodes with better latency, re-liability, and energy efficiency than homogeneous archi-tectures. Using the Mote Herding abstractions, we haveimplemented a set of services that run across both plat-forms, namely a mote routing service, a data reliabilityservice and a resource discovery service that is based onthree subservices. We evaluate the performance of ourservices using simulations and emulations of both sam-ple scenarios and a habitat monitoring application. Ourresults show that the mote routing service is able to main-tain better than than 99% connectivity at high networkdensities, while incurring 60% lower transmission over-head than two other routing protocols. We also show thatthe data reliability service is able to deliver 100% of thedata even when more than 30% of the network exhibitsfailures. Finally, we show that the habitat monitoring ap-plication that uses Mote Herding is able to deliver all ofthe data with low control overhead and latency.
1 Introduction
The past year has seen the realization of several success-ful sensor network deployments [26, 19, 2, 23]. This de-ployment experience is moving the field into a more ma-ture state, with small and medium-sized networks run-ning sampling applications. Pushing sensor networksbeyond simple “sample and return” applications will in-troduce new engineering challenges, particularly regard-
ing scaling to larger numbers of nodes, more and moredistant communication, and use of more sophisticatedsensor processing algorithms inside the network.
Currently, wireless sensor network devices fall intotwo main categories. The first category includes tiny,inexpensive, resource-constrained nodes that operate forlong periods of time on battery power. The most widelyused device in this class is the Crossbow Mote [14] basedon the 8-bit AVR microcontroller. The second categoryincludes small 32-bit nodes that host sophisticated pe-ripherals and megabytes of RAM and flash and typi-cally either run duty-cycled or connected to large energysources. The XScale-based Intel Stargate [16] platformis a popular representative of this category. The type ofdevice used in a sensor network is paramount in definingthe capabilities and limitations of the entire system.
However, neither class is suitable forall applications.In general, low-cost platforms are ideal for applica-tions that can operate with that level of functionality,and suited for highly energy-optimized, vigilant applica-tions. But there are also applications that require morecomputation or storage and can justify the additionalcosts [4]. Moreover, many applications will require amixture of both in order to combine densely deployedlower-end sensors with more sparsely deployed higher-end nodes [2]. It is for those reasons that the sensor net-work community has been increasingly exploringtieredarchitectureswhere the system is composed of a mixtureof platforms with different costs, capabilities and energybudgets.
While adding more powerful hardware increases thecapability of the network, there are currently few guide-lines about how to structure software to easily take ad-vantage of a heterogeneous sensor network.
Mote Herding is a system architecture and a set ofservices that targetslarge scale, heterogeneouswirelesssensor networks. Its basic network abstraction and build-ing block is theflock. The flock is a collection of motesthat connect via a multi-hop routing tree to a microserver
called theshepherd. The microservers communicateamong themselves with a separate, higher-capacity net-work over which they can exchange information morerapidly and more reliably than motes. The collection offlocks and corresponding shepherds forms theherd. Theend result is atieredsystem, that can disseminate data ef-ficiently through an overlay network and multiple trees.
Mote Herding is based on the principle that movingcomplexity from the mote network to the microservernetwork aids development of more reliable and efficientservices. Mote Herding accomplishes this by leveragingthe CPU and storage resources of microservers to im-prove the computational capability, decision-making andlong-term reliability of the deployed network. To demon-strate this we have developed several Mote Herdingser-vicesbased on this principle. We consider two classesof services:flock services run on a single flock and thecorresponding shepherd, whileherdservices span the en-tire herd and use flock services to interface to the motenetwork.
The main contribution of this paper is to introduceMote Herding as a new architecture for large, tiered,wireless sensor networks. We present the design prin-ciples behind this architecture (Section 2), and demon-strate the effectiveness of these principles by design-ing communication services tailored to this architecture(Section 3) and building higher-level services on thisfoundation (Section 4). We then evaluate the benefitsof tiered architectures (Section 5). Our results showthat our routing service is able to maintain higher than99% connectivity in medium or high network densitieswhile incurring a low overhead. In addition, our reli-ability service is able to deliver100% of the data withlow latency even in the presence of considerable link fail-ures. Throughout the paper we present related work as itarises.
2 Mote Herding Architecture
The basic network abstraction of the Mote Herding archi-tecture is theflock. The flock is a collection of motes thatcan communicate (with the routing protocol we describein Section 3.1) to a microserver, called the flock’sshep-herd. Shepherds use standard IP-based ad hoc routingover an 802.11 network to run areliable state replica-tion protocol (described in Section 3.2) that allows themto easily share changing information like flock member-ship. Theherd is the collection offlocksandshepherdsthat forms the entire mote/microserver network. Figure 1illustrates the components of the architecture.
Each mote has a one-to-one relationship with its shep-herd, and so is part of exactly one flock at a time. Theshepherd is responsible for its own flock and is alsoau-thoritative for it: for example, queries from other mi-
Flock Shepherd
Herd
Flock ShepherdShepherd
Herd
Figure 1:The Mote Herding architecture and its components,theflock, theshepherdand theherd
croservers about motes in a shepherd’s flock are an-swered authoritatively by the shepherd. This howeverdoes not preclude other microservers in the networkkeeping cached copies of another shepherd’s list. In thataspect, the design is similar to DNS.
To simplify on-mote routing, intra-flock routing istree-based, with each mote knowing only how to reachthe shepherd of its flock . In addition, inter-flock com-munication can only take place in theshepherdnetwork.In this architecture, arbitrary mote-to-mote routing istherefore only possible through the herd-level overlaynetwork. Compared to the intra-flock network, herd-level (inter-flock) routing is mesh-based, treating all mi-croservers as peers.
Based on the above, Mote Herding can be thought ofas alocally centralized/globally distributedarchitecture.Each flock has centralized properties, since data fusionand flock control happens on the shepherd, while the net-work of shepherdsis a distributed system that shares in-formation about each flock, delegates tasks to the appro-priate shepherds and makes collaborative decisions.
While there have been several specific approaches toexploit the capabilities of tiered networks [24, 15], devel-opment thus far has been difficult because each approachhas been hand-crafted. Mote Herding begins to exploreguidelines to clarify how services should be developedin tiered networks, as some other concurrent projects arealso doing [12]. Mote herding preserves the peer-natureof original sensor networks, but only at the microservertier, adopting a centralized approach to simplify the motelevel.
2.1 Design Principles
Mote Herding revolves around the following primary de-sign principle:
Shift system complexity from motes to microservers forthose operations that require distributed decision makingon the motes.
2
A B
3
21
A B
3
21
A B
3
21
A B
3
21
a) b)
Figure 2: A sink selection scenario that illustrates the dif-ference between mote-based and microserver-based distributeddecision making.
This design principle yields several corrolaries:1. Centralize decision making on a microserver
thereby improving system performance as decisionscan be made based on a more complete data set.
2. Reduce volatile state required for system function-ality on motes.
3. Utilize computational and communication re-sources on microservers along with their peer natureto support more complex functionality.
4. Program a significant part of the system in a familiarand resource-rich environment.
In a collaborative distributed system, nodes make de-cisions based on their local information as well as infor-mation shared by other nodes. In the majority of cases,the accuracyandcorrectnessof the decision is directlyproportional to the amount of information available tothe node, both spatially (i.e., reports from as many nodesas possible) and temporally (i.e., history of past reports).The resource constraints of the 8-bit platform howeverplace a limit on the spatial and temporal fidelity of thedata. However, the limited storage and network capacityavailable to an 8-bit mote platform constraints their spa-tial and temporal views. As a result, decisions made ona mote often must be based on limited information, evenassuming complete message reception.
On the other hand, the 32-bit platform has amplestorage and computational power to support collabora-tive decision-making algorithms and more complex al-gorithms than are possible on motes. They also havesufficient network bandwidth to operate in a distributedmanner. This observations suggests that microserverscan serve as “concentrators” or “fusion points”, takinginput from multiple motes in a flock and combining thisinformation to provide better decisions than an individ-ual mote can make alone. Such an operation requires thatmotes send data back to their microserver. However, thisdoes not apply to all forms of data; only data required fordecision-making needs to be sent. Sensor data islocal toa node and does not require knowledge of or communi-cation with other nodes. In contrast, neighbor estimation,path estimation and best sink selection are operations thatneed data from other nodes and as such incur a commu-nication cost as well as a storage cost.
Figure 2 presents an example that illustrates the differ-
Mote Routing
MicroserverState Replication
Data Reliability
RegistrationProtocol
ResourceDiscovery
Data Replication
Tasking
Sampler
Herd Services
Flock Services
Herd Foundation Service
Flock Foundation Service
Mote Routing
MicroserverState Replication
Data Reliability
RegistrationProtocol
ResourceDiscovery
Data Replication
Tasking
Sampler
Herd Services
Flock Services
Herd Foundation Service
Flock Foundation Service
Figure 3:The Mote Herding Service Classes and their corre-sponding foundation services.
ences between mote-based and microserver-based deci-sion making. Mote3 wishes to attach to the best avail-able sink, based on some well-defined selection metric.In (a), the microservers send beacons (dashed arrows)that get propagated to the mote network. The mote re-ceives the beacons from both microservers and selectsthe best one to reply to. In (b), the mote sends a joinbeacon (dashed arrows) that eventually reaches the mi-croservers. The microservers compare metrics and the“winner” replies to the mote. In (a), the memory andbandwidth requirements on mote3 scale linearly withthe number of microservers. In (b), memory and band-width requirements areO(1). By delegating decisionsto the microservers, we can therefore reduce the motes’state and bandwidth requirements and thus improve thenetwork densityscaling properties.
The mote network bandwidth still places a limit on theamount of data that can be sent to a single microserver.As networks grow in numbers of nodes, at some pointsending all relevant data to a central point becomes im-possible. Ifmultiple microservers exist, each gatheringsystem data for its own flock, the microservers can thenexploit their high-bandwidth network tosharethat infor-mation among themselves. By replacing a flat networkwith two radio tiers it becomes possible for each mi-croserver to maintain a view of the entire network whilenot overburdening individual motes. Adding multiplemicroservers to the system therefore has a dual effect: itreduces theeffectivediameter of the mote network whilegiving each microserver aglobal view.
2.2 Mote Herding Services
Building on the concepts offlocksandherds, Mote Herd-ing services are split into flock- and herd-wide services.
Flock servicesare services that operate on a flock ofmotes under supervision of a single shepherd. Those ser-vices are “hybrid” in the sense that they reside on bothmotes and microservers. Flock services adhere to the de-sign principles outlined above: motes deliver data to theshepherd; the shepherd makes decisions based on thatdata and then forwards those results back to each mote.These services emphasize themaster-slaverelationship
3
that exists between flock motes and their shepherd. Flockservices can use other services in the same flock (weshow example interactions in Sections 4.1 and 4.2.1), butthey do not interact with motes in other flocks.
Herd servicesare services that operate on the en-tire network, building on flock services to provide morecomplete applications. Unlike flock services, herd ser-vices run only on the higher-performance microservernetwork. If they need to interact with motes, they do sowith an appropriate flock service. Herd services treat allmicroservers as peers, allowing distributed and collabo-rative decision making based on both flock services andother herd services.
The commonality across all services is distributed de-cision making. Multi-hop communication and informa-tion sharing is therefore the foundation of other services.Thesefoundationservices for communication are sim-ple multi-hop routing inside flocks, and a state replica-tion service calledStateSyncat the herd level. We ex-plore these foundation services in the next section thenpresent several higher-level services built on top of thisfoundation in Section 4. Figure 3 shows the relationshipbetween foundation and composite services at the flockand herd levels.
3 Foundation Services
Mote Herding foundation services provide the essentialfunctionality on which higher-layer services build upon.These services are bidirectional mote-to-microserverrouting at the flock level, and reliable state replicationat the herd level.
3.1 Flock Foundation Service: Mote Rout-ing
Flock services depend on communication between motesand their shepherd. Motes pass data, possibly over mul-tiple hops, to their shepherd, while the shepherd dis-patches control messages to motes. This section exploresour design of centralized, bidirectional routing within aflock.
Tree-based routing is widely used in sensor networks,where all nodes construct a multi-hop routing tree to acentralized root (or gateway or sink). In Mote Herdingwe construct many such trees, one per flock, between themotes of the flock and their shepherd.
MintRoute [22] and Directed Diffusion [13] are tworouting protocols that are often used in sensor networks.TheMultihop routing protocol which is part of the ESSdeployment at James Reserve [26] combines a simpli-fied Diffusion and MintRoute. Each of these protocolsuses adistance-vector-like routing algorithm to deter-mine routes back to the sink, and employslink estimation
andneighbor tablesto dynamically discover the topol-ogy of the wireless network. With distance vector rout-ing algorithms, each node independently selects a nexthop from its neighbors that reduces its distance towardsthe sink. Some protocols use different different metrics(such as ETX [6]) to define the “best” next hop, possiblybased on link quality, available energy, or other charac-teristics.
Recent experiences with the ESS deployment haveuncovered several underlying issues that stem from theaforementioned properties of those routing algorithmscombined with the resource constraints of the mote plat-form. Those issues are: (a) neighbor-table overflow, (b)count-to-infinity and (c) routing loops. We address eachof these issues in more detail.
To populate its neighbor table, the mote needs to learnabout its neighbors. The state requirements for the neigh-bor table are thereforeO(n), wheren is the number ofneighbors that a mote can have. Because of resource con-straints, motes are typically configured with relativelysmall, statically sized neighbor tables. In dense wire-less sensor networks, the number of potential neighborscan exceed the available storage capacity allocated forthe neighbor table [22]. In that case, the designer ofthe routing protocol must implement aneighbor evictionpolicy [22]. In doing so however, the selection of a parentis now based on asubsetof the actual neighbor list, thusartificially limiting the choices (and paths) a mote canhave. Moreover, eviction/insertion policies suffer fromthrashing, where a mote is inserted in the table only tobe removed in the next iteration. This can lead to routinginstabilities if this mote was indeed the next hop towardsthe sink.
The distance-vector protocol is known to suffer fromthe “count-to-infinity” problem. This can adversely af-fect the performance of routing by creatingloops. As aresult, special care (like poison-reverse or split-horizon)must be taken to avoid loops or repair them if theyform. In MintRoute, motes monitor forwarding trafficand snoop on the parent address in each neighbor’s mes-sages in order to identify neighboring children and thusdisqualify them from the parent selection process. Nev-ertheless, those mechanisms are not infallible and loopscan still form (especially n-hop loops). In particular,in one of our MintRoute simulations, we experienced asystem-wide count-to-infinity problem that resulted in allthe motes in the network having routing loops.
The aforementioned issues are examples of distributeddecision-making on the motes that is based onincom-pleteand potentially,stale, data.
In mote herding we use CentRoute to communicatebetween motes and their shepherd. Following the princi-ples layed out in Sections 2.1 and 2.2, CentRoute shiftsall decision making to the shepherd. CentRoute ad-
4
Shepherd
21
3
a)
Shepherd
21
3
b)
Shepherd
21
3
c)
Shepherd
21
3
d)
Shepherd
21
3
a)
Shepherd
21
3
Shepherd
21
3
a)
Shepherd
21
3
b)
Shepherd
21
3
Shepherd
21
3
b)
Shepherd
21
3
c)
Shepherd
21
3
Shepherd
21
3
c)
Shepherd
21
3
d)
Shepherd
21
3
Shepherd
21
3
d)
Figure 4:A CentRoute tree forming (join) operation.
dresses the main problems described above: since it runson a microserver with megabytes of memory, neighbortable size is not a limitation. Since all path selection androuting decisions are made centrally, different motes willnot come to different decisions that can lead to inconsis-tencies. In Section 5.1.1 we conduct a number of sim-ulations to show that these approaches in CentRoute aresuccessful at providing high network connectivity and re-ducing the number of loops.
3.1.1 Routing tree formulation
Following the principles outlined in section 2.1 we de-signed CentRoute to be a dynamic single-shepherd rout-ing protocol based on source routing, with all routing de-cisions made on a centralized point, the shepherd.
In order to join a tree and thus be part of a flock, eachmote periodically broadcasts join request messages. If amote that is already part of a tree receives this message, itforwardsit towards its shepherd, via its parent, by gener-ating a unicast packet. In addition, the forwarding moteadds its own address to the packet, thus recording thepath the packet is taking towards the sink. This is essen-tially a similar approach to Source Routing [17]. Uponreception of a request message, either by direct broad-cast or via a forwarded packet, the microserver will gen-erate a join reply packet and unicast it towards the re-quester, by reversing the packet’s received path and us-ing source routing. In addition, the microserver storesthis path information to use whenever it needs to send aunicast packet to that mote. When the mote receives thejoin reply message, it becomes attached to the replyingmicroserver—its shepherd (we discuss multi-shepherdresolution in Section 3.1.4). It also sets the last motethat forwarded the join reply as itsparent. Finally, sincethe mote is now part of a flock, it stops generating furtherjoin request messages.
The join operation in CentRoute happens inphases,since, in order to forward a join request packet, a mote
needs to be attached to a tree. When the network startsup, no motes (besides those directly attached to mi-croservers) are parts of a tree. Initially, only the motesthat are one hop away from a microserver will be withinrange of an attached node (in this case, the microserver).In the next round, the motes that are two hops away willbe able to attach, and so on, until the entire network be-comes connected. Figure 4 shows an example tree form-ing operation. In (a), all motes send a join request, butonly motes1 and2 are within range of an existing tree(the shepherd itself). In (b), the shepherd sends join re-ply messages (dashed arrows) to motes1 and2, therebygrafting them to the tree. In (c), mote3 sends anotherjoin request. This time, motes1 and2 which are now partof the tree forward the join request towards the shepherd.In (d), the shepherd grafts node3, via the path that goesthrough mote1. Mote3 also sets its parent to be node 1,so any subsequent upstream traffic will be sent to it.
The centralized nature of the tree formulation in con-junction with source routing and the absence ofanymotedecisions concerning path selection results in a consid-erable reduction in the number of transient loops (Sec-tion 5.1.1). In particular, routing state is always set up byconstructing a path from the shepherd to the mote via asource routed packet. This packet trails behind it a loopfree path to the shepherd so any old state in the networkthat somehow happened to meet this path would there-fore have a loop free path to the shepherd as well. As aresult, no route is ever added that can have a loop, and no“old” route that is disrupted can result in a loop becauseall it can do is join a loop free route, or not work at all.A formal analysis of the loop performance of CentRouteis part of future work.
3.1.2 Routing Metric and Link Quality Estimation
Since path selection is done on a centralized point inCentRoute, the motes need to provide the shepherd withsufficient information for that decision. When a mote re-ceives a request, it will include alink quality estimatewhen forwarding the request towards the shepherd.
In the example of Figure 4, motes 1 and 2 will writetheir estimate of link quality to mote 3 into the requestbefore passing the join request towards the shepherd.These estimates computed at join time enable the shep-herd to annotate a tree with link quality estimates alongeach edge, and thus to select paths using ETX [6] as arouting metric.
Motes in CentRoute use the join request beacons to getlink quality estimates, by having each mote send severalbeacons in quick succession. In order to reduce controloverhead, receiving motes that are part of the tree onlyforward anaggregatejoin request that contains the thetotal number of broadcast requests received by that mote
5
in that period. Using this information, the microserverhas sufficient data to form aninstantaneouslink qualityestimate and subsequently, a path estimate.
An important property of the estimation algorithm isthat link quality estimates need only be kept for as longas the mote is trying to join. This is in contrast withtraditional link estimation schemes where estimates needto be kept at all times and cannot be discarded unless amote is evicted from the neighbor table.
3.1.3 Stability and Maintenance
In order to increase the stability of the network, we de-signed CentRoute so that it doesn’t try to find improvedpaths (in terms of its routing metric) once the originalpaths have been established. As long as the individuallink qualities don’t change radically, the initially selectedpath will not be too far from theoptimal path, at anypoint in time. As a result, the protocol can form morestable topologies, while sacrificing optimality at somefuture point in time. This process also has an “anneal-ing” effect towards stability, because temporally stablelinks, once discovered, are likely to remain selected.
Even though CentRoute does not attempt to find a bet-ter path when one has been established, it still needs toensure that the existing path isvalid, i.e., all links fromthe mote to its shepherd are still viable. CentRoute useslink-layer ACKs and packet retransmissions to deducelink failure. Once the link fails, the mote at the point offailure considers itself detached from the tree. It then in-vokes thejoin mechanism, described in 3.1.1, by startingto transmit join request messages. If a mote receives ajoin request from its parent, it immediately detaches it-self from the tree and also invokes thejoin mechanism.Since the repair mechanism is in effect the same as thejoin mechanism, it inherits its loop prevention properties.
In order to reduce the control overhead when the net-work is quiescent CentRoute is anon-demandprotocol.As a result, CentRoute does not require periodic controlmessages to maintain its paths; instead it discovers andrepairs broken paths on demand, when triggered by datapackets. This approach trades off reduced control over-head for potentially higher path repair time.
3.1.4 Dynamic shepherd selection
To increase the robustness of the network, CentRoutewas designed to supportdynamic shepherd selection.Since the mote assignment to flocks and shepherds isnot pre-configured, a mote can associate with a differentshepherd and flock if its current shepherd becomes un-available, such as due to microserver failure. We choosenot to allow motes to associate with multiple shepherdsin keeping with our goal of minimizing mote-side com-plexity (Section 2.1).
When multiple microservers exist in the network,a mote join request can be received by several mi-croservers, as shown in Figure 2. Since mote-to-shepherd assignment is one-to-one, the “competing” mi-croservers need to select which flock the mote shouldjoin. As each microserver already knows the best pathto the mote from its own flock, the microservers needonly compare the values of their local path metrics forthe mote in question. Essentially, this is the same op-eration as path selection on a single microserver, withinformation about alternative paths through other flocksprovided by other microservers.
3.2 Herd Foundation Service: MicroserverState Replication
Herd services run on the microserver network, joiningall microservers into a common distributed system. Tosupport distributed algorithms that depend on informa-tion from multiple flocks,microserver state replicationis provided as the foundation of herd services.
Mote Herding uses a protocol called StateSync [10] asits foundation service for reliable state replication acrossthe herd. StateSync is a proactive publish-subscribesystem that strives to provide low-latency updates topublished data over multiple hops with low overhead.StateSync is a reliable protocol with a probabilistic la-tency bound that publishes a node’s state over multiplehops by proactively sending differences. Through theStateSync API, a node publishes a table of data, andthe StateSync system automatically computes the diff re-quired to update the subscribers. StateSync is designedto support applications that need reliable publication oftheir current state and timely propagation of updates, forcases where the subscriber needs access to the data ata higher rate than the rate of change of the data values.ISIS also provided reliable shared data, but targeted pri-marily at LANs [3].
StateSync’s proactive publishing model does not fitall applications. Those that have a high ratio of dataelements and updates to data access require a cachedrequest-reply protocol such as Hood [21]. This type ofservice is important future work for Mote Herding.
4 Flock and Herd Services
Using the foundation services we can now build compos-ite flock and herd services. In addition to the foundationservices for flock and herd communication, Mote Herd-ing currently provides two flock services, mote data re-liability and a mote registration protocol and three herdservices, resource discovery, resource caching and querydissemination. We have considered a number of possi-ble additional services, including a global tasking service
6
that distributes tasks to each mote in the network, a datareplication service that replicates flock data at each mi-croserver to improve robustness and a system monitoringservice that continually monitors the entire network. Im-plementation of these additional services is future work.
4.1 Mote Data Reliability
In mote-based Wireless Sensor Networks, the wirelesscommunication is known to show variable and oftenlarge loss rates [5, 25, 22]. As such, application design-ers need to take packet loss as a given and design appli-cations to be loss-tolerant. An important class of appli-cations require high reliability and would benefit from adata reliabilityservice.
DataRelis the Mote Herding data reliability protocol,a flock service that strives to provide guaranteed data de-livery in a lightweight manner. As a result, we designedDataRel to be a window protocol that uses end-to-endACKs and source-based packet buffering, in a mannersimilar to TCP. DataRel can use end-to-end ACKs effi-ciently, since the CentRoute service allows for unicastpacket transmissions from the shepherd to any mote inthe flock. In addition to end-to-end ACKs, DataRel uti-lizes link-layer ACKs and hop-by-hop retransmissions toimprove the probability of successful delivery, contraryto PSFQ [20] and similar to RMST [18]. RMST operatessolely on 32-bit nodes over Directed Diffusion with allnodes being peers while DataRel is a mote-to-shepherdreliability protocol. PSFQ is a mote-based hop-by-hopprotocol that uses NACKs for error recovery and a con-trolled flooding algorithm for data distribution.
The mote’s shepherd is not necessarily thefinal desti-nation for a data packet. Once the packet reliably reachesthe shepherd, it can be forwarded to its final destina-tion using a conventional IP reliability service, like TCP.However, for the purposes of the DataRel protocol—andsince DataRel is a flock service—we consider the end-points to be the source mote and its shepherd. This ap-proach, which is similar to Split-TCP [1], is based onthe realization that, like satellite and terrestrial networks,the mote network and the microserver network have verydifferent communication properties, especially in termsof bandwidth, latency and bit error rate. Alternatively,one could view DataRel as an example of delay-tolerantnetworking [8], where the flock and the herd are separateDTN regions.
4.2 Resource Discovery
Our first composite service at the herd level isresourcediscovery. Resource discovery is essential in sensor net-works where nodes have different capabilities, such asdifferent sensors or actuators, or where applications wish
CentRoute
StateSync
Data Reliability
RegistrationProtocol
ResourceDiscovery
ResourceCaching
Query Resolution
CentRoute
StateSync
Data Reliability
RegistrationProtocol
ResourceDiscovery
ResourceCaching
Query Resolution
Figure 5: The Resource Discovery service connection dia-gram.
to select subsets of the network based on other factors,such as location or coverage area.
Common approaches to resource discovery include:direct queries on demand, in-network caching or cachingat a central site. The one-phase pull variant of DirectedDiffusion [13] is an example of a direct query approach,while the Ninja Service Discovery Service [7] is an ex-ample of an in-network caching approach. The cachingapproaches in general provide faster query resolutionspeed, at the expense of additional complexity and over-head for handling cache misses or “stale” data.
Much like routing, the main factor that influences thechoice between on-demand and a priori approaches is theamount of temporal variation in the data. Sensor datais tied to physical phenomena and as such can changerapidly or slowly, depending on the physical process be-ing measured. For example, an acoustic signal changesmuch more rapidly than a temperature value. In additionto sensor data, link quality, instantaneous queue size andreceived signal strength indicator are other examples offrequently varying data. On the other hand, hardware andsoftware capabilities of a platform constitute informationthat changes slowly over time. For example, the actualsensors installed on a mote (as opposed to their values)or the mote’s radio constitute information that character-izes the mote and does not change often. In terms ofoverhead, an on-demand polling protocol is more appro-priate for high temporally varying data, while an a pri-ori caching approach is better suited for low temporallyvarying data as information can remain “fresh” for largerperiods of time.
We have implemented ResDisc using the mote herdingprinciples. Since mote resources for the most part ex-hibit low temporal variation, we adopted the in-networkcaching approach. The design of ResDisc is based onthree sub-services. Theregistration protocolflock ser-vice provides resource data from the motes in the flock.The resource cachingherd service is responsible for ex-
7
porting local resource data provided by the registrationprotocol as well as maintaining its cache of resource datafrom other shepherds. Finally, thequery resolutionherdservice is responsible for replying to resource discoveryqueries. Figure 5 presents the Resource Discovery ser-vice diagram.
In the following sections, we describe the registra-tion protocol and resource caching service in more detail.The query resolution service has not been completely im-plemented yet and as such we defer its discussion to fu-ture work.
4.2.1 Registration Protocol
The goal of the registration protocol is to “register” re-source information of a mote with its shepherd, as wellas keep the shepherd updated if this information changes.Consequently, the registration protocol is aflock service.
To reduce communication overhead as well as improvethe update time, we designed the registration protocolto be event-based as opposed to using a periodic refreshmechanism. As a result, the protocol is invoked automat-ically when one of the following events occurs: the motejoins a shepherd’s tree, a registered resource changesvalue or when instructed by an application. A further op-timization that has not been implemented yet includes re-sponding to direct queries from the shepherd. To ensurecorrect ordering of events as well as resolve potential am-biguities, we used a simple sequence number scheme inwhich a new sequence number is generated when a newevent occurs.
The absence of a periodic refresh mechanism intro-duces a reliability issue, as registration messages, if lost,will not be repeated. As a result, the registration protocoluses the DataRel mote data reliability service introducedin section 4.1 to guarantee reception of its messages bythe shepherd. Finally, to allow for flexibility in the regis-tration data, we chose to use a variable-size type-length-value (TLV) format for the actual data transfer.
4.2.2 Resource Caching
One of the goals of the resource discovery service is theability to reply to any query as quickly as possible, byhaving all available information on each microserver. Toaccomplish this, we designed the resource caching ser-vice, a herd service that supplements the registration pro-tocol flock service.
Resource caching is a simple service that is respon-sible for exporting resource data obtained through theregistration protocol to other shepherds. At the sametime, it receives similar data from the remote shepherdsand caches it. To accomplish this functionality as wellas ensure consistency among caches throughout the mi-croserver network, resource caching uses the state repli-
0
20
40
60
80
100
120
4 8 12 16 20 24 28 32
Net
wor
k co
nnec
tivity
(%
)
Mote neighbor density (number of motes)
CentRouteMintRoute
Multihop
Figure 6: Network connectivity percentage as a function ofmote neighbor density for CentRoute, MintRoute and Multi-hop.
cation herd foundation service. In each shepherd, re-source caching maintains a table containing all motes ithas data for (potentially all motes in the network). Thistable can then be used as an input to the query resolutionservice.
5 Evaluation
In this section we evaluate the three Mote Herding ser-vices that have been fully implemented, namely routingand data reliability for flocks, and resource discovery forherds. We also use a habitat monitoring application toevaluate a set of Mote Herding services working in col-laboration. We run our experiments using the EmStarframework [9]. The flock services are evaluated usingthe EmTOS [11] emulation module of EmStar, which al-lows development of fully functional NesC applicationsin the resource-rich environment of a 32-bit platform.
5.1 Mote Routing
Our routing experiments focus on three important as-pects of any routing algorithm, specifically network con-nectivity, loop probability and control overhead. In allour routing experiments, the experimental topology con-sisted of a 100-mote simulated network, arranged in a10-by-10 grid with uniform 10m spacing between motes.We then place a single sink, in the middle of the bottomrow of the grid at coordinates (50m,0m). To study vari-able network densities we keep this topology fixed andvary the simulated radio range. In all our routing ex-periments, we compare the performance of CentRouteto MintRoute and Multihop. The neighbor table sizeon both MintRoute and Multihop was set at their defaultvalue of16. We vary network density, so at high densitieswe expect the number of neighbors to exceed this fixedrouting table size. The periodic beacon rate for Mint-Route and Multihop was set to30 seconds. Since Cent-Route is an on-demand protocol, it requires data trans-
8
missions to perform route maintenance, as seen in Sec-tion 3.1.3. Therefore, the CentRoute experiments con-tained data transmissions, at a rate of one packet every30 seconds per mote. The maximum number of trans-mission retries per packet, used in the CentRoute repairmechanism was set to5.
5.1.1 Network Connectivity
First we look at how well each routing protocol main-tains connectivity to the sink. We study how connectiv-ity varies as we change the network density by increasingthe simulated radio range.
For the purpose of our experiments, we definemoteneighbor densityto be the number of one-hop neigh-bors of a given mote that have a link quality of70% orhigher. Since we use a grid topology with even spacing,the neighbor density increases in steps of4. We note thatthe actual number of motes that a single mote can hearcan be considerably higher than its neighbor density [22]and that property is reflected by our simulated channelmodel.
In order to measure network connectivity, we periodi-cally trace a path from each mote in the network towardsthe sink. If a path is found, we consider the mote to beconnected for that particular time sample. At each timesample, network connectivity is the sum of all the motesthat have a path to the sink. The duration of all our rout-ing experiments was3 hours and our sampling periodwas10 seconds. The first5 minutes of each experimentwere discarded, so as to allow each routing algorithm toreach stable state. In this and all following experiments,error bars in experimental results are95% confidence in-tervals.
Since CentRoute does not contain any neighbor tablesor any other per-neighbor state that is limited to a maxi-mum number of motes, we expect it to perform better asmote neighbor density increases. By contrast, we expectMintRoute and Multihop to exhibit loops and blackholesas density increases, for the reasons given in Section 3.1.
From Figure 6, we note that with CentRoute the net-work is almost always connected, regardless of moteneighbor density. In fact, the performance of CentRouteactually improvesas density increases, rising from90%at mean density of4 neighbors to99.9% when nodeshave 12 or more neighbors. In contrast, MintRoutestarts at97% connectivity then reaches99.9% in mediumdensities, but connectivity falls off as density exceedsthe static routing table size (above 16 neighbors). Thisdegradation is because, when neighborhood size exceedsthe routing table size, there are several cases in bothMintRoute and Multihop when mote A considers moteB a neighbor but not vice versa. This disparity is dueto table thrashing—mote A has mote B in its table, but
Neighbor CentRoute MintRoute Multihopdensity loop prob. loop prob. loop prob.(motes) (%) (%) (%)
4 0 0.47 2.118 0 0.01 1.9212 0 0.02 1.8816 0 0.03 2.4820 0 0.03 3.6524 0 0.04 4.828 0 0.04 3.1232 0 0.02 7.27
Table 1:Average loop occurrence probability as a function ofdensity for CentRoute, MintRoute and Multihop.
mote B has evicted mote A. As a result, a mote is unableto find a parent and stays disconnected for large periodsof time. This problem could possibly be solved by ex-changing neighbor evictions at the cost of even greatercomplexity in the mote network.
Multihop exhibited its best performance in mediumdensities (8–24 neighbors/mote). At high densities (24–32 neighbors/mote), Multihop performance is hamperedby a high probability of aloop occurrence in addition tothe table thrashing problem. At 4 neighbors/mote, Multi-hop could not connect the network because of its routingmetric, an 8-bit value that decreases exponentially (basedon link quality) as the path length increases. In low den-sities, when path lengths are considerable, the routingmetric reaches its minimum value of0 after a few hopsand as such no more motes can be added to the path. Ineffect the current Multihop implementation has an arbi-trary maximum path length cutoff point.
The initial decreased performance of CentRoute is dueto its centralized nature in conjunction with the tree for-mulation and repair mechanism. A path in CentRoutewill be invalidated ifanynode in the path fails to trans-mit a packet to its parent, after a certain number of re-tries (currently set at5 retries per attempt). Therefore theprobability of a path invalidation is directly proportionalto the path length, which in turn is inversely proportionalto neighbor density.We could reduce this problem by in-creasing the number of retries at low densities, or afterrepeated failures.
To understand the relative effect of loops on networkconnectivity, we explicitly measured the number of loopsthat occurred for each protocol. The CentRoute protocol,due to its centralized shepherd-based path constructionand source routing is expected to have very few, if any,loops (Section 3.1.1). The distance-vector algorithms areexpected to have more loops than CentRoute. Multihop,in particular, that has no loop-prevention mechanism isexpected to have the worst performance.
Table 1 shows the percentage of loops encounteredas a function of density for the three routing protocols.CentRoute did not exhibit any loops that were detectable
9
0
1
2
3
4
5
6
32 28 24 20 16 12 8 4
Ove
rhea
d (b
ytes
/sec
/mot
e)
Mote Neighbor density (number of motes)
CentRouteMintRoute
Multihop
Figure 7:Per-mote byte overhead as a function of mote neigh-bor density for CentRoute, MintRoute and Multihop.
Usage RAM ROM RAM cost(bytes) per neighbor
CentRoute 1274 17184 0MintRoute 1689 12588 18Multihop 1560 17292 19
Table 2:Mote RAM and ROM usage in bytes for CentRoute,MintRoute and Multihop.
with our 5-second sampling period. A proof that Cent-Route is loop free is an area of active work. MintRouteloop probability was highest in the low density case butwas an order of magnitude less in all other case. Webelieve that this disparity is due ton-hop loopsin con-junction with the particular geometric properties of thegrid topology. MintRoute does not allow a mote to se-lect any of its neighboring child motes as a parent, thuspreventing 1-hop loops. N-hop loops cannot be detectedhowever. At very low densities, the limited number ofalternative paths in the network combined with the largerpath lengths and the geometric properties of a grid topol-ogy result in n-hop loops occurring more often. Finally,Multihop which lacks a loop-prevention mechanism andalso uses a non-monotonic routing metric has the highestprobability of a loop occurrence.
5.1.2 Control Overhead
The control overhead of the routing algorithms is thetransmission overhead incurred by their operation as wellas their memory usage on the mote. The transmissionoverhead of MintRoute and Multihop is due to their pe-riodic neighbor beacons and route advertisement mes-sages. The transmission overhead of CentRoute containsjoin requests as well as join forward and join reply mes-sages. In this section we conduct experiments to verifythat CentRoute control overhead is similar to overheadof alternatives.
CentRoute repairs can be quite expensive especiallyif the paths are long and the failure happens close tothe sink, as a considerable number of motes will be af-
fected. In contrast, the distance-vector protocols dolo-cal repairs which are not as expensive. As a result, weexpect the CentRoute overhead to be high when the net-work is sparse and get progressively lower as the densityincreases.
Figure 7 depicts the average per-mote transmissionoverhead in bytes per second for the three routing pro-tocols as a function of mote neighbor density. In thelowest density, CentRoute has the highest cost of all thethree protocols. This is due mainly to the repair mecha-nism’s inability to find good paths, as the available selec-tions are limited. As the network progressively becomesmore dense, CentRoute can find alternate paths to thesink that are more stable than the initial selection. Therepair mechanism does not need to be invoked as oftenand transmission overhead is reduced considerably.
MintRoute and Multihop, on the other hand, initiallystart with a low transmission overhead but grow greatlyas the neighbor density increases. Since neighbor tablesare transmitted by each mote as part of the link estimator,higher densities result in a higher transmission overhead,up to a saturation point that is based on the maximumsize of the table. The overhead reduction for MintRouteand Multihop in very high and very low densities is dueto their route advertisements, which are only transmit-ted when a mote is connected. As shown in Figure 6,network connectivity for MintRoute and Multihop is re-duced at very low and very high densities and as suchless route advertisements are transmitted.
A second kind of overhead is memory usage of eachalgorithm. Table 2 shows the RAM and ROM foot-print of each of the three routing algorithms, using theTinyOS default message length (29 bytes payload). Wenote that CentRoute has the lowest RAM usage whichdoesn’t increase with the neighborhood size. In contrast,MintRoute and Multihop have comparable RAM usageand per-neighbor costs. The per-neighbor cost places aconsiderable limitation in the operation of the distance-vector alrogithms. Increasing the neighbor table from itsdefault size of16 to 50 so as to support larger numbersof neighbors incurs an additional RAM cost of about680bytes, while the size of the table itself is almost1000bytes, i.e.,25% of the total RAM of the Mica2 mote.The real advantage of CentRoute here is that it shifts theburden of variable size data structures like the routing ta-ble to resource-rich shepherds, while the other protocolsmust allocate routing tables statically sized to accommo-date worst-case neighborhood sizes.
Based on all our routing experiments we feel thatthe CentRoute foundation service has achieved its de-sign goals. By centralizing the routing decisions on themicroserver and removing per-neighbor state on motesCentRoute manages to scale well with increasing neigh-bor density as well as avoid loops while at the same time
10
0
20
40
60
80
100
0 5 10 15 20 25 30 35 40 45
Pac
ket d
eliv
ery
(%)
Mote radio failure rate (%)
DataRelmDTN
Figure 8:Average packet delivery ratio as a function of moteradio failure rate for DataRel and mDTN.
incurring a low overhead. However, the centralized prop-erty of CentRoute is not well suited for sparse networkswith long path lengths. The reason is due to the overheadassociated with transmitting all the control data back toa single point in addition to the expensive repair mecha-nism is invoked more frequently.
5.2 Data Reliability
Our reliability experiments focus on the performance oftwo mote data reliability protocols: the DataRel flockservice introduced in section 4.1 and the mDTN relia-bility protocol used in the ESS deployment [26]. a hop-by-hop protocol calledmDTN that is based on the prin-ciples of Delay Tolerant Networking[8]. MDTN useslink-layer ACKs and hop-by-hop retransmissions to reli-ably deliver the data. If next-hop delivery fails, mDTNstores the packet in non-volatile storage (the mote’s flashmemory) for later retransmission. DataRel ran over theCentRoute service while mDTN ran over the Multihoprouting protocol. For our experiments, we again usedthe 100-mote grid topology plus a single sink in the mid-dle of the bottom row. However, we now fixed the moteneighbor density to12 and instead changed themote ra-dio failure probability. We used a simple failure model,in which every mote has a given probabilityk for its ra-dio link to fail. The failure generator runs periodicallyand applies the failure probability to every mote in thenetwork. In the next iteration, the failure generator “re-vives” failed motes before applying the failure probabil-ity again. The result is that at any point in timek% ofall the motes in the network will be unable to receive ortransmit any packets.
In all our data reliability experiments we used a simpleperiodic data generating application. The data transmis-sion rate was set to one packet every30 seconds, whilethe fault generator ran every5 minutes. Our experimentslasted until all motes had transmitted200 packets each,or until a 6-hour limit was reached. The DataRel win-dow size was set to1, so an ACK was sent by the sink
0
20
40
60
80
100
120
140
160
0 10 20 30 40
Add
ition
al tr
ansm
issi
on o
verh
ead
(%)
Mote radio failure rate (%)
DataRelmDTN
Figure 9: Per-mote additional transmission overhead as afunction of mote radio failure rate for DataRel and mDTN.
across the flock for each packet received. Maximumhop-by-hop retransmissions for each protocol was set to5. During the course of our experiments we found thatthe default value of the mDTN retransmission timer wasset too high, resulting in very high latencies and higherloss rates. As a result, we changed the retransmissiontimer period of mDTN from10 minutes to10 seconds,the same as DataRel’s retransmission timer.
5.2.1 Packet Delivery Ratio
In order to measure the delivery ratio, we annotate eachdata packet with a sequential sequence number We theninspect the sequence number space for each mote at thereceiver and discard duplicates. The remaining packets,divided by the total number of original packets gener-ated by each mote is the per-mote packet delivery ratio.Finally, we compute the mean and 95% confidence inter-vals by averaging all per-mote results.
Figure 8 shows the average packet delivery ratio as afunction of mote radio failure rate for the two protocols.When no failures are present mDTN delivers on aver-age96% of the data while DataRel delivers100% Thesituation changes however as the mote radio failure rateincreases. DataRel is able to keep100% delivery ratefor a failure rate of up to30%, dropping to an averageof 97.2% for a failure rate of40%. At the same time, theperformance of mDTN degrades considerably, especiallyat high failure rates. The performance degradation is dueto the nature and implementation of mDTN. Specifically,mDTN uses link-layer ACKs to decide whether a packethas been successfully delivered to the next hop; if not, itstores it. Link-layer ACKs however carry no informationabout the packet’s fate inhigher layers of the stack. InmDTN, a packet can be successfully delivered to a motebut subsequentlydropped. This is mostly due to a re-ceive buffer overflow as a result of network congestionbut can also be caused by more complex failure modes.A higher-layer ACKing scheme would solve this issue,at the expense of more transmission overhead. DataRel,
11
0
50
100
150
200
250
300
350
400
0 10 20 30 40
Per
-mot
e st
orag
e bu
ffers
use
d (a
vera
ge)
Mote radio failure rate
DataRelmDTN
Figure 10:Per-mote average storage buffer usage as a functionof mote radio failure rate for DataRel and mDTN.
being an end-to-end protocol, does not suffer from suchissues.
5.2.2 Control Overhead
The control overhead of the reliability protocols has twoaspects: transmission overhead and buffer overhead. Thetransmission overhead is the number ofextra transmis-sions incurred by the protocol. If a packet needs to travelN hops to reach its destination then any additional trans-missions overN constitute the transmission overhead.For mDTN the transmission overhead is related to hop-by-hop retransmissions at each mote. The DataRel trans-mission overhead includes hop-by-hop retransmissions(via CentRoute), as well as packet retransmission at thesource and ACK transmissions and retransmissions. Thebuffer overhead is the amount of buffer space requiredat each mote to store unacknowledged packets. SinceDataRel is an end-to-end protocol, we expect its trans-mission overhead to be considerably higher than mDTN.At the same time, since mDTN can potentially storepackets on every mote along a path, we expect it to havea higher buffer usage.
Figure 9 shows the per-mote additional transmissionoverhead as a function of mote radio failure rate for thetwo reliability protocols. DataRel overhead is alwaysabove100%. This is due to the end-to-end ACKs whicheven in the absence of retransmissions incur a100%overhead for an ACK window of1, as each packet needsto be ACKed. A larger ACK window on the senderwould reduce the transmission overhead of DataRel atthe expense of more buffer usage. As expected, mDTNis consistently less expensive than DataRel although thedifference is less pronounced at higher failure rates. Thisis mostly due to the nature of the underlying Multihoprouting protocol. When a mote radio fails, it takes a cer-tain amount of time for Multihop to decide that this moteis no longer a valid parent. During this time, mDTNwill continue to attempt to transmit data over that mote.When the failure rate is high this can lead to a large num-
Usage (bytes) RAM ROMDataRel 100 6952mDTN 338 13278
Table 3:Mote RAM and ROM usage in bytes for DataRel andmDTN.
Latency (sec) Mean 95% Quart. MaxRegistration Protocol 0.61 5.18 6.97
Resource Caching 0.12 0.32 0.41
Table 4:Resource update latency for the registration protocoland resource caching parts of the Resource Discovery service.
ber of unnecessary retransmissions. In contrast, Cent-Route path failure notification is very quick and as suchDataRel does not suffer from this problem.
Figure 10 shows the average buffer usage required byeach protocol as a function of mote radio failure rate.As expected, DataRel uses less buffers than mDTN sinceit only buffers a packet at theoriginal sender. MDTNon the other hand can potentially buffer a packet ateveryintermediate hop as the packet makes its way towards thefinal destination. This case is especially probable at highfailure rates.
Buffering at intermediate nodes has another effect onoverhead: it requires additional logic on the receptiondata path as a mote needs to determine how to treat eachindividual packet. In addition, mDTN includes supportfor multiple sinks which further complicate its imple-mentation.
Table 3 presents the RAM and ROM footprint of eachprotocol. Due to the aforementioned reasons, mDTN ismore expensive than DataRel in terms of both RAM andROM. The ROM cost in particular underlines the relativesimplicity of DataRel compared to mDTN.
Our reliability experiments reinforce our design deci-sion on shifting complexity from the motes to the mi-croservers. DataRel, a simple end-to-end protocol candeliver 100% of the data even at high failure rates asopposed to the more complex mDTN. The difference isprimarily due to the end-to-end ACKs made possible byusing the CentRoute flock foundation service. However,as expected for an end-to-end protocol, DataRel incursa substantial transmission overhead. Nevertheless, theoverhead can be reduced by using larger ACK windows,at the expense of more storage buffer usage.
5.3 Resource Discovery
Our resource discovery experiments focus on the timerequired for updates to propagate to the entire network.This knowledge can be then used to provide an upperbound on the rate that a client can query the service. Forthis experiment, we used our experimental mote testbedconsisting of40 Mica2 motes placed at the ceiling of ourbuilding’s 3rd floor, as shown in Figure 11. We used
12
a simulated channel for our microserver network con-sisting of 3 shepherds and assumed that all shepherdsare within range. To simulate a resource update, wechanged the “sensor type” value on each mote thus caus-ing the registration protocol to propagate the update, asdiscussed in Section 4.2.1.
Our latency measurements consisted of two parts: thefirst part measured the time required for the registra-tion protocol to update the resource value on the localshepherd. The second part measured theadditional timerequired for the resource caching service to update allthe shepherds in the network with the new data obtainedfrom the registration protocol. The second measurementwas complete when all three shepherds had converged onthe same value.
Table 4 shows the mean, 95% quartile and maximumlatency of an update, in seconds, for both the registra-tion protocol and the resource caching service. Since theregistration protocol operates on the mote network, it in-curs a higher latency than the resource caching servicewhich utilizes the 802.11 network. In particular, regis-tration protocol latency is dependent on the mote’s pathlength to the sink—with each hop adding approximately100 milliseconds of propagation delay—as well as thequality of the path. A low-quality or broken path canresult in a DataRel retransmission or even a CentRoutepath repair thus having an negative effect on latency. Theresource caching service does not suffer from those is-sues although we expect its latency performance to de-grade as we increase the hop diameter of the microservernetwork, based on experiments presented in [10].
Based on our results, the mean time required for an up-date originating from a mote to reach all the shepherds is0.73 seconds, while the maximum is7.31 seconds. Inthe future, we plan to do more extensive latency mea-surements on our Resource Discovery service, in orderto establish a more accurate upper bound as well as com-pare our design choices with the alternatives presented inSection 4.2.
5.4 A complete application: Habitat Moni-toring
Our final set of experiments involved implementing asimple multi-sinkhabitat monitoringapplication usingMote Herding and comparing its performance with asimilar application. In our application, motes periodi-cally send data back to a single or multiple microserversusing DataRel on top of CentRoute. We measured theapplication’s performance with data delivery ratio, totalnumber of transmissions and packet latency as a func-tion of the number ofsinksin the network and comparedit with the ESS application, which uses mDTN on top ofMultihop. Both applications were tested using our exper-
Figure 11: The experimental testbed used in our experiments.Nodes with darkened boxes were the shepherds/sinks.
Num. Delivery Total transm. Latencyof Ratio (%) (packets) (sec)
sinks MH ESS MH ESS MH ESS1 (16) 100 84.24 1501.5 3089.4 2.75 12.27
2 (3,33) 100 90.30 710.28 2493.7 0.98 1.723 (all) 100 94.06 588.77 2423.9 0.24 0.63
Table 5:Delivery ratio, total number of transmissions and la-tency results for the Mote Herding (MH) and ESS versions of ahabitat monitoring application as a function of the total numberof sinks in the system.
imental testbed.Table 5 depicts the results of our experiments. The
Mote Herding application outperformed the ESS appli-cation in terms of delivery ratio, total number of trans-missions and latency. We note that increasing the numberof sinks has a positive effect on the performance of bothapplications, especially in terms of reducing the num-ber of transmissions and packet latency. A larger num-ber of sinks reduces the effective mote network diameter,thereby leading to shorter paths.
Based on our encouraging initial testbed results, in thefuture we plan to deploy our Mote Herding-enabled habi-tat monitoring application in the field and compare itsperformance to that of real-world applications.
6 Conclusions
In this paper we presented Mote Herding, a new systemarchitecture that targets large scale heterogeneous sen-sor networks, comprised of 8-bit motes as well as 32-bitmicroserver nodes. Mote Herding is based on two keyabstractions: theflockwhich is a collection of motes con-nected to a single microserver called the shepherd and theherd which is comprised of the collection of flocks andtheir shepherds. Based on those abstractions, Mote Herd-ing introduces two classes of services. Flock services are
13
services that run on a single flock and its shepherd. Herdservices operate on the entire network but are locatedsolely on the microserver network and interface to themote network via corresponding flock services. A foun-dation service is used as a building block for all otherservices in its class.
Using the Mote Herding abstractions we implementedseveral services, including centralized intra-flock rout-ing, a mote data reliability flock service and a resourcediscovery herd service based on three subservices. Weevaluated the performance of our services using simu-lations and emulations of both sample scenarios and ahabitat monitoring application and compared them withother protocols and services. Our results show that ourcentralized mote routing service, CentRoute, is able tomaintain better than 99% connectivity at high networkdensities with 60% less overhead than two other routingprotocols. Our data reliability service was able to main-tain 100% reliability in up to 30% link failure rates. Fi-nally, our emulated habitat monitoring application basedon Mote Herding was able to consistently outperform asimilar habitat monitoring application in terms of datadelivery, transmission overhead and latency.
References
[1] H. Balakrishnan, V. N. Padmanabhan, S. Seshan, andR. H. Katz. A comparison of mechanisms for improvingTCP performance over wireless links. volume 5, pages756–769, 1997.
[2] M. Batalin, G. S. Sukhatme, Y. Yu, M. H. Rahimi, G. Pot-tie, W. Kaiser, and D. Estrin. Call and response: Experi-ments in sampling the environment. InACM SenSys 2004.
[3] K. Birman and R. Cooper. The isis project: Real experi-ence with a fault tolerant programming system.OperatingSystems Review, pages 103–107, Apr. 1991.
[4] A. Cerpa, J. Elson, D. Estrin, L. Girod, M. Hamilton, andJ. Zhao. Habitat monitoring: Application driver for wire-less communications technology. InSIGCOMM Wksp. onComm. in Latin America and the Carribean, Costa Rica,Apr. 2001.
[5] A. Cerpa, J. L. Wong, M. Potkonjak, and D. Estrin. Sta-tistical model of lossy links in wireless sensor networks.In IPSN 2005.
[6] D. S. J. D. Couto, D. Aguayo, J. Bicket, and R. Morris. Ahigh-throughput path metric for multi-hop wireless rout-ing. In Mobicom 2003. ACM, 2003.
[7] S. E. Czerwinski, B. Y. Zhao, T. D. Hodes, A. D. Joseph,and R. H. Katz. An architecture for a secure service dis-covery service. InMobile Computing and Networking,pages 24–35, 1999.
[8] K. Fall. A delay tolerant network architecture for chal-lenged internets. InProceedings of SIGCOMM 2003.
[9] L. Girod, J. Elson, A. Cerpa, T. Stathopoulos, N. Ra-manathan, and D. Estrin. Emstar: a software environment
for developing and deploying wireless sensor networks.In USENIX Tech. Conf. 2004.
[10] L. Girod, M. Lukac, A. Parker, T. Stathopoulos, J. Tseng,H. Wang, D. Estrin, R. Guy, and E. Kohler. A reliablemulticast mechanism for sensor network applications. inCENS Technical Report 48.
[11] L. Girod, T. Stathopoulos, N. Ramanathan, J. Elson,D. Estrin, E. Osterweil, and T. Schoellhammer. Tools fordeployment and simulation of heterogeneous sensor net-works. InProceedings of SenSys 2004, November 2004.
[12] R. Govindan, E. Kohler, D. Estrin, F. Bian, K. Chin-talapudi, O. Gnawali, R. Gummadi, S. Rangwala, andT. Stathopoulos. Tenet: an architecture for tiered embed-ded networks. InCENS Technical Report No. 53.
[13] J. Heidemann, F. Silva, C. Intanagonwiwat, R. Govindan,D. Estrin, and D. Ganesan. Building efficient wirelesssensor networks with low-level naming. InSOSP. ACM,2001.
[14] J. Hill, R. Szewczyk, A. Woo, S. Hollar, D. Culler, andK. Pister. System architecture directions for networkedsensors. InASPLOS-IX, 2000.
[15] W. Hu, V. N. Tran, N. Bulusu, C. tung Chou, S. Jha, andA. Taylor. The design and evaluation of a hybrid sensornetwork for cane-toad monitoring. InIPSN 2005.
[16] Intel. Intel stargate, http://platformx.sourceforge.net.
[17] D. B. Johnson and D. A. Maltz. Dynamic source routingin ad hoc wireless networks. In Imielinski and Korth, ed-itors,Mobile Computing, volume 353. Kluwer AcademicPublishers, 1996.
[18] F. Stann and J. Heidemann. RMST: Reliable Data Trans-port in Sensor Networks. InProc. of the First Intl. Wkshpon Sensor Net Protocols and Appl.IEEE, 2003.
[19] R. Szewczyk, A. Mainwaring, J. Polastre, J. Anderson,and D. Culler. An analysis of a large scale habitat moni-toring application. InSenSys 2004.
[20] C. Wan and A. Campbell. PSFQ: A Reliable TransportProtocol For Wireless Sensor Networks. InWSNA 2002.
[21] K. Whitehouse, C. Sharp, E. Brewer, and D. Culler.Hood: a neighborhood abstraction for sensor networks.In MobiSYS ’04, pages 99–110. ACM Press, 2004.
[22] A. Woo, T. Tong, and D. Culler. Taming the underly-ing challenges of reliable multihop routing in sensor net-works. InSensys 2003.
[23] N. Xu, S. Rangwala, K. K. Chintalapudi, D. Ganesan,A. Broad, R. Govindan, and D. Estrin. A wireless sen-sor network for structural monitoring. InACM SenSys2004.
[24] M. Yarvis, N. Kushalnagar, H. Singh, A. Rangarajan,Y. Liu, and S. Singh. Exploiting heterogeneity in sensornetworks. InIEEE Infocom 2005.
[25] J. Zhao and R. Govindan. Understanding packet deliveryperformance in dense wireless sensor networks. InSensys2003.
[26] Extensible sensing system: An advanced network de-sign for microclimate sensing.http://www.cens.ucla.edu.
14
