Review Article Experimental Research Testbeds for …downloads.hindawi.com/journals/ijdsn/2015/630210.pdfthe demanding need for their experimental veri cation and evaluation, researchers

Review ArticleExperimental Research Testbeds for Large-Scale WSNs:A Survey from the Architectural Perspective

Hiecheol Kim, Won-Kee Hong, Joonhyuk Yoo, and Seong-eun Yoo

School of Computer and Communication Engineering, Daegu University, Gyeongsan 712-714, Republic of Korea

Correspondence should be addressed to Seong-eun Yoo; [email protected]

Received 24 May 2014; Revised 18 August 2014; Accepted 5 September 2014

Academic Editor: Poh Kit Chong

Copyright © 2015 Hiecheol Kim et al. This is an open access article distributed under the Creative Commons Attribution License,which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Wireless sensor networks (WSNs) have a significant potential in diverse applications. In contrast to WSNs in a small-scale setting,the real-world adoption of large-scale WSNs is quite slow particularly due to the lack of robustness of protocols at all levels. Uponthe demanding need for their experimental verification and evaluation, researchers have developed numerous WSN testbeds.While each individual WSN testbed contributes to the progress with its own unique innovation, still a missing element is ananalysis on the overall system architecture and methodologies that can lead to systematic advances. This paper seeks to providea framework to reason about the evolving WSN testbeds from the architectural perspective. We define three core requirements forWSN testbeds, which are scalability, flexibility, and efficiency. Then, we establish a taxonomy of WSN testbeds that represents thearchitectural design space by a hierarchy of design domains and associated design approaches.Through a comprehensive literaturesurvey of existing prominent WSN testbeds, we examine their best practices for each design approach in our taxonomy. Finally, wequalitatively evaluateWSN testbeds for their responsiveness to the aforementioned core requirements by assessing the influence byeach design approach on the core requirements and suggest future directions of research.

1. Introduction

A typical wireless sensor network (WSN) consists of a largenumber of small-sized, battery-powered nodes with wirelesscommunications and various sensing capabilities [1].They arelimited in battery energy, CPU power, and communicationcapability. WSNs attract a great deal of research attentionin the past decade [1–7], and in recent years we have seenan increase in prototype WSN deployments to evaluate thefeasibility, performance, and cost-effectiveness in a variety ofapplications, including environment monitoring, infrastruc-ture health diagnosis, and unmanned surveillance [8].

Despite the increase in prototype WSN deployments, theadoption of WSN applications in our real world does notkeep the pace. This is phenomenal for large-scaled WSNapplications. Of many reasons, the most critical one is thelack of reliability in wireless communication due to unstableprotocol behaviors that do not appear in the small-scalesetting. For this reason, the complete verification of perfor-mance as well as behavior is essential especially for large-scale WSN applications. Traditionally, protocol designers

resort to analytic models or simulation to verify and eval-uate their proposals [9]. While these methods are relativelysimple and convenient, they are inherently limited in theaccuracy of evaluation results, as addressed in a number ofliteratures [10, 11]. To overcome the limitation, especially inthe implementation time, comprehensive experiments on arealistic environment are essential. WSN testbeds have longbeen regarded as a viable option for full-blown physicalexperiments. Over the years, a great deal of research efforthas made considerable progress in building general-purposeWSN testbeds mostly from the research community [12–15].Aside from these efforts, a large body of works has beendevoted to addressing specific issues associated with WSNtestbeds such as WSN testbed management [16] and healthmonitoring [17].

In spite of such decade-long research efforts, researchesonWSN testbeds are still ongoing for several reasons. Firstly,the user population of WSN testbeds becomes increas-ingly diverse, requiring a more distinctive set of featuresappropriate for new user groups. For example, beyond theacademic researchers, users from industries keep increasing

Hindawi Publishing CorporationInternational Journal of Distributed Sensor NetworksVolume 2015, Article ID 630210, 18 pageshttp://dx.doi.org/10.1155/2015/630210

2 International Journal of Distributed Sensor Networks

[8]. A convenient experimental environment supporting real-time interactions is especially precious for industry users astheir works on WSN testbeds are centered largely on codedebugging for error-free implementation ofWSNnetworkingstacks. Secondly, large-scale WSNs play a key role in IoT(Internet of things) and CPS (cyber physical system) whichbecome increasingly active research areas. Indeed, severalprominent testbeds in those fields include a WSN testbedfacility as one of their fundamental building blocks [18–20].

WSN testbed researches over years have contributed tothe progress of WSN testbeds with their unique innovations.For several reasons, the overall system architecture andmethodologies that can lead to systematic advances of WSNtestbeds still remain unexplored. Firstly, a well-acceptedagreement about WSN testbed architectures does not yetexist because WSN applications are inherently diverse, eachwith varying requirements. Secondly, decent use cases thatcan suggest architectural design directions are not availableyet. Finally, modest criteria to compare and contrast WSNtestbeds are not well established since each individual WSNtestbed is based on its own ad hoc design sometimes with anemphasis on a set of specialized features.

Associated with the analysis on overall system archi-tectures and methodologies, several research surveys inliterature describe a number of prominent WSN testbeds,highlighting their key features [12–15]. Clearly, they pro-vide a perspective to compare and contrast existing WSNtestbeds in terms of physical form-factors, such as hardwareinfrastructure, software tools, and network size. To the bestof our knowledge, they are not sufficient enough to helpcomprehensively understand an architectural perspective ofWSN testbeds; a well-established framework to help analyzearchitectural features ofWSN testbeds is lacking due partly totheir focus on specific application domains [12, 13] or on thefunctional description of WSN testbeds [14, 15].

Our paper, building upon the contributions made by ear-lier research surveys and complementing their missing part,aims at establishing an architectural framework for analyzingdesign approaches of the existing WSN testbeds from thearchitectural viewpoint.We first identify a set of core require-ments forWSN testbeds by examining pragmatic experimentscenarios. Subsequently, we establish a taxonomy for WSNtestbeds that exposes the design space of a WSN testbedfrom the architectural viewpoint. Based on the taxonomy,we categorize existing prominentWSN testbeds and examinefor each category their designs to get a deep understandingof their approaches and best practices. Based on this under-standing, we qualitatively evaluate existing WSN testbedswith respect to the aforementioned core requirements.Finally, we discuss current trends surroundingWSN testbedsand their implications to future WSN testbed architectures.

The rest of the paper is organized as follows. Section 2introduces the set of core requirements forWSN testbeds thatwe identify for pragmatic large-scale WSN experiments. InSection 3, we illustrate a taxonomy for WSN testbeds whichexposes the architectural design space of WSN testbeds,thereby enabling classifying WSN testbeds from the archi-tecture perspective. In Section 4, based on our taxonomy,

we provide a comprehensive survey on design practices ofexisting prominent WSN testbeds. In Section 5, we evaluateWSN testbeds with respect to the core requirements iden-tified in Section 2. Finally, in Section 6, concluding remarksand future researches are offered.

2. Requirements for WSN Testbeds

Recent advancedWSN testbeds are intended for public use byexternal researchers and developers from various applicationdomains. If a WSN testbed is to be useful for them, it hasto address numerous requirements. This section discussesimportant requirements for general-purpose WSN testbeds.In Section 2.1, we introduce a generic organization of WSNtestbeds and its associated terminology in order to set thecontext of the discussion. In Section 2.2, we illustrate the setof core requirements that we identify from the pragmaticperspective.

2.1. Generic WSN Testbed Organization. Apparently, it is noteasy to bring a typical organization for WSN testbeds sinceeach one is built with its own unique design to address aset of specific requirements of interest. To frame a contextfor discussion, we thus identify the set of generic buildingblocks of a WSN testbed through a close examination of theexperiment lifecycle. In general, an experiment on a WSNtestbed consists of a series of steps, crudely experiment setup,experiment execution, and result analysis. Following thesesteps, users need to interact with the WSN testbed facility.The first essential interaction is to reserve a set of nodes froma WSN testbed. Subsequently, users have to load embeddedsoftware into those reserved nodes, that is, reprogramming.Another essential interaction is to monitor the experimentstate. For evaluating distributed sensing applications, contin-uous data logging from each individual node is essential. Thelog data enables users to obtain full insight into the operationof the WSN application and further allows debugging oranalyzing the behavior or the performance as well. Finally,users are essentially engaged in sporadic control activitiessuch as starting nodes for initiating an experiment execution.

The aforementioned essential user interactions exposethe least set of building blocks that a WSN testbed hasto be equipped with. The first one is a dedicated networkinfrastructure, called usually back-channel, for handlingmanagement traffic between nodes and management com-puters which are produced for user interactions includingmonitoring, control, and reprogramming. Using the nativeradio channel of the WSN for the experimental traffic canpotentially disturb the behaviors of WSN communication,eventually distorting evaluation results. The second one isa computing infrastructure such as repositories for log dataand servers for hosting management services. In this paper,we call the computing infrastructure a testbed managementserver or a testbed server in short.

Figure 1 depicts the generic view on the organizationof WSN testbeds based on the above discussion. A WSNtestbed consists of three parts, a WSN field, a back-channelinfrastructure, and a testbed server.

International Journal of Distributed Sensor Networks 3

Internet

Testbedserver

Back-channelinfrastructure

WSN field

User UserTe

stbed

Logdata

· · ·

Figure 1: Generic building blocks of a WSN testbed.

2.1.1.WSNField. AWSNfield is a physicalWSN to hostWSNapplications under experiments. The principal element of aWSN field is a collection of sensor nodes. From now on, asensor node will be called usually a node in short. Across theexistingWSN testbeds,WSNfields exhibit several differences.The first difference is the node hardware. A number of WSNtestbeds employ the sensor node platforms commonly usedby theWSN research community. For instance, Telos [21] andTmote series [22] belong to them. In several WSN testbeds,custom-designed nodes are used for their WSN testbeds.These nodes usually have additional features associated withWSN testbeds such as back-channel communication inter-face and power profiling hardware. Other differences include,but are not limited to, an apparatus for supporting mobilityand power provision schemes.

2.1.2. Back-Channel. A back-channel is the network to carrythe management traffic. Typically, sensor nodes do not sup-port a back-channel functionality and thus the infrastructuralsupport is provided independently of the WSN field. Acrossthe existing WSN testbeds, designs of their back-channelsexhibit a large variance in terms of communication methodsand hardware organizations. Some WSN testbeds use on-board wired communication ports such as RS-232c and USBport. Others often adopt broad-band networks with someadditional network devices such as back-channel gateways.In implementing a back-channel, eachWSN testbedmakes itsown trade-off with respect to such criteria as implementationcost, bandwidth, and scalability.

2.1.3. Testbed Server. A testbed server is a computing facilityfor experimentation and administration services, respec-tively, for experimenters and administrators. A testbed serverconsists of both hardware and software. A testbed serverexposes the capability of aWSN testbed to the outsides and isthus regarded as an access point for users. Services providedby a testbed server will be called testbed services, and the

software, which implements those services, is called testbedsoftware. Sometimes, testbed software is disintegrated into aserver part and a client part as in the general client/servermodel. In this case, they are, respectively, called server-side software and client-side software. For users to accesstestbed services, the web interface is used in many WSNtestbeds, while custom-tailored or publicly distributed client-side software is used in several other WSN testbeds.

2.2. Requirements. The embedded and distributed nature ofWSNs, potentially large-scaleWSNs, and diverseWSN appli-cation domains render the design of a versatile WSN testbedinherently difficult. The design should reflect perspectivesof diverse entities including the WSN testbed infrastructure,WSN applications, and WSN testbed users. With this inmind, we select three core requirements for WSN testbeds:scalability, flexibility, and efficiency.

2.2.1. Scalability. This means the capability to increase thenumber of nodes in the WSN testbed without restrictingthe robustness in experiments. One of important goals forbuilding aWSN testbed is to evaluate large-scale WSN appli-cations since the evaluation of small-scale WSN applicationscan be undertaken just with some ad hoc apparatus. A typicalWSN application potentially is prone to clash when runningon tens or hundreds of nodes, even though it is sufficientlyverified in a small-scale setting. For instance, unexpectednetwork behaviors may appear due to increased topologychanges. Software bugs that are not easily discovered in asmall-scale setting such as a bounded-buffer overflow mayoccur. Scalability is thus important for putting scenarios oflarge-scale real-world deployments into experimentation.

2.2.2. Flexibility. This means the capability to allow usersto conduct realistic experiments for a wide range of WSNapplications. Considering the high deployment cost of aWSN testbed, the flexibility is essential in the WSN testbeddesign since it ultimately determines the extent of theirutilization. For higher flexibility, the design of WSN testbedshas to effectively deal with the high degree of hardware andenvironmental diversity across the large spectrum of WSNapplications. As for the hardware diversity, it is desired toflexibly replace nodes or at least radio and sensormodules in aWSN testbed appropriate to the hardware requirement of thetarget application. On the other hand, as for the environmentdiversity, a WSN testbed should allow users to flexibly setup experimental environments appropriate to their WSNapplications. Environmental factors subject to considerationsinclude RF characteristics, such as radio propagation rangeand interference, and node dynamics like physical nodefailure and node mobility.

2.2.3. Efficiency. This means the capability to enable usersto conduct experiments effectively in terms of time andlabor. Ordinary users do not have deep understanding onthe WSN testbed itself. Moreover, as previously discussed,a typical WSN experiment is inherently complex due tothe embedded and distributed nature of WSNs; it requires


WSN testbed

Nat

ive

Grid

Sing

ular

Clus

tere

d

Fede

rate

d

WSN fieldstructure

Nat

ive

Broa

dban

d/on

-boa

rd B

GW

Broa

dban

d//e

xter

nal B

GW

Hyb

rid

Back-channelorganization

Mon

olith

ic

Tigh

tlyco

uple

d

Loos

elyco

uple

d

Servicearchitecture

WSNlayout

Des

ign

appr

oach

Des

ign

dom

ain

Figure 2: Taxonomy of WSN testbeds.

Testbedserver

Native back-channel

· · ·

(a) Native-BC

Testbedserver

Broadband

BGW BGW

back-channel

· · ·

(b) Broadband-BC

Testbedserver

BGW

Broadbandback-channel

Native back-channel

Native back-channel

BGW· · ·

· · ·· · ·

(c) Hybrid-BC

Figure 3: Design approaches in the back-channel organization domain.

embedded development, reprogramming, monitoring, andcontrol, result analysis over numerous data streams from eachindividual node, and so on. Efficiency is thus critical for users.Unless the efficiency issue is sufficiently addressed, even aWSN testbed built on a cutting-edge technology may remainunderutilized. To enhance the efficiency, an integrated set oftools and services should be provided over the lifecycle of theexperimentation.

3. Taxonomy

The large spectrum of WSN applications results in numer-ous WSN testbeds distinctive in many design features. Forsystematic advances of WSN testbeds, it is very importantto be able to sort out these features and then characterizetheir designs coherently from an architectural perspective.To this end, we propose a taxonomy. In building it, we firstexplore a set of design domains at the level of architecturalbuilding blocks. For each design domain, we identify a set ofdisjoint design approaches in its design space. Figure 2 showsour taxonomy that consists of four design domains: back-channel organization, service architecture,WSNfield structure,and WSN layout. In the figure, design approaches are alsolisted, respectively, for each design domain. In this section,our taxonomy is explained with each core design domain ina separate subsection.

3.1. Back-Channel Organization. Theback-channel organiza-tion of a WSN testbed, which is the first design domain inour taxonomy, is important because it affects many aspects ofthe WSN testbed which include cost-effectiveness, physicalform factor, and scalability. A number of approaches havebeen employed in the design of a back-channel. We classifytheir organizations into three: native-BC (back-channel),broadband-BC, and hybrid-BC, as shown in Figure 3.

3.1.1. Native-BC. A back-channel is implemented by usingon-board serial communication. Except for a few WSNtestbeds using RS-232c, the USB communication is typicallyused inWSN testbeds. Each serial port is wired directly to thetestbed server. In case of USB, indirect wiring via USB hubsis used as well.

3.1.2. Broadband-BC. Broadband networks, mostly eitherEthernet or WiFi, are used to implement back-channels.Each node in the WSN testbed is equipped with a dedicateddevice for protocol conversion. While the device is calleddifferently such as a sensor host [23], a super node [24], orjust a gateway [25], we call it back-channel gateway (BGW)as it provides the protocol conversion between on-boardserial communication, which are mostly RS232c, and thebroadband network. According to the way to implement the


Testbedsoftware

(a) Monolithic

Internet

Client-side

(thin)

Server-sidesoftware

software· · ·

(b) Tightly coupled

Internet

(extensible)

Server-sidesoftware

Client-sidesoftware· · ·

(c) Loosely coupled

Figure 4: Design approaches in the service architecture domain.

back-channel gateway, we divide this broadband-BC furtherinto two approaches. One is to use an adaptor module tobe piggy-backed on the node and the other is to use anexternal controller which is usually a single board computer.They are called, respectively, broadband-BC with an on-board BGW and broadband-BC with an external BGW, beingseparately dealt in the taxonomy due to their distinctiveinfluence on the overall WSN testbed architecture. As itis named, the key advantage of the broadband-BC is toget higher bandwidth over the native-BC. Moreover, thisapproach enables exploiting standard networking equipment,facilitating the implementation of a back-channel.

3.1.3. Hybrid-BC. A combination of a native-BC and abroadband-BC is used to implement a back-channel.The keymotivation is to reduce the number of back-channel gatewaysby dividing nodes in theWSNfield into a set of groups, whereeach group is managed by a single back-channel gateway.Hence, the network between the nodes in a group and theirback-channel gateway is in a form of the native-BC, whilethe network between back-channel gateways and the testbedserver is in a formof the broadband-BC, as shown in Figure 3.

3.2. Service Architecture. Over the hardware resources of aWSN testbed, a set of services is to be provided for users whoare experimenters, administrators, or technical managers.The organization for these services, which we will call servicearchitecture, is important in the WSN testbed design. Forinstance, it affects the freedom to customize the client partof testbed software. Hence, we select the service architectureas the second design domain in our taxonomy. Associatedwith the service architecture, we identify three disjoint designapproaches: monolithic, tightly-coupled, and loosely-coupled,as shown in Figure 4.

3.2.1. Monolithic. This architecture assumes the hardwareinfrastructure where a single computer serves as both a clientdesktop and a testbed server. The testbed software is notdisintegrated into a client part and a server part and thus isimplemented monolithically into a single piece.

3.2.2. Tightly Coupled. This service architecture assumes aWSN testbed that is tiered into three layers: clients, a testbedserver, and a WSN field. A client desktop computer anda testbed server host client-side software and server-sidesoftware, respectively. These are tightly coupled such that themission of client-side software is solely to deliver the servicesprovided by server-side software to users; client-side softwareis not permitted to provide its own proprietary services.In this service architecture, the whole testbed services areusually implemented centrally as server-side software, andclient-side software provides only the user interface to accessthose services.

3.2.3. Loosely Coupled. Different from the aforementionedtightly coupled one, this service architecture allows client-side software to provide its own proprietary services inde-pendent of server-side software. In other words, client-sidesoftware not only serves the delivery mission, but also canbe extended to provide a set of functionalities demandedby experimenters. In this architecture, client-side software isoften allowed to directly interact with nodes in theWSNfield,as depicted by a line in Figure 4(c).

3.3. WSN Field Structure. Across the WSN testbeds, WSNfields are different not only in their node hardware, but alsoin their structures. As the WSN field structure, in particular,significantly influences various important aspects includingheterogeneity and scalability, we select it as the third designdomain in our taxonomy. In exploring its design approaches,we introduce a term: WSN cluster. A WSN cluster is asection of a WSN field in which communication betweennodes is made only via the in-band radio hops; all thenodes reachable only via any gateway are excluded. A WSNtestbed can be built with either a single WSN cluster ormultiple WSN clusters. A multiclustered WSN testbed issometimes constructed by federating multiple independentWSN testbeds. Consequently, we divide the design spaceof the WSN field structure into three design approaches:singular, clustered, and federated.


Internet

User User

Testbedserver

· · ·

(a) Singular

Internet

User User

Testbedserver

· · ·

· · ·

(b) Clustered

Internet

User User

Federationmanager

Testbedserver

Testbedserver

· · ·

· · ·

(c) Federated

Figure 5: Design approaches in the WSN field structure domain.

3.3.1. Singular. A singular WSN testbed refers to a WSNtestbed with a single WSN cluster. This is the most pop-ular since experiments to evaluate the scalability of WSNnetworking protocols, such as routing protocols, are usuallyconcerned with the number of nodes at theWSN cluster levelrather than at the entire WSN field level (Figure 5).

3.3.2. Clustered. A clustered WSN testbed refers to a WSNtestbed composed of more than one WSN cluster. WSNclusters are connected with gateways and spread over awide geographical area. Generally, the cluster boundary istransparent to users, and thus all the nodes over the WSNclusters of aWSN testbed are collectively managed by a singlemanagement entity.

3.3.3. Federated. A federated WSN testbed refers to a WSNtestbed constructed by combining multiple independentWSN testbeds. Although each member WSN testbed isremotely located and independently administrated, the fed-eration provides a common view to users in terms of theadministration policy and the way to utilize each individualmember WSN testbed. The federation concept is not new. Itis conceived by pioneering research initiatives such as GENI[26] and FIRE [27] to construct a large-scale heterogeneousexperimental infrastructure for the future Internet.

3.4. WSN Layout. In deploying a WSN testbed, the way toplace WSN nodes determines physical form factors such asphysical distance between nodes. These factors may affectseveral important aspects of the WSN testbed includingdeployment and maintenance cost and the possibility tocontrol node density within a radio propagation range. Incurrent best practices, options for theWSN layout are dividedinto two approaches, namely, native and grid.

3.4.1. Native. This layout places WSN nodes in a similarway to general WSN deployments. WSN nodes are separatedsubject to ordinary radio propagation ranges. Typically,distance between nodes ranges from 10 to 30 meters.

3.4.2. Grid. This layout places WSN nodes in a relocatablefixture in the form of a two- or three-dimensional grid.Typically, distance between nodes ranges from a few tencentimeters to a few meters.

4. Designs

Based on our taxonomy discussed in the previous section,this section provides a survey of existing prominent WSNtestbeds. In Section 4.1, we classify WSN testbeds accordingto our taxonomy. In subsequent subsections, we present adetailed examination of their designs, respectively, for eachdesign domain in our taxonomy.

4.1. Classification. The scope of WSN testbeds in our reviewis devoted to general-purpose research WSN testbeds withfocus on protocol-centric experiments in a large-scale setting.Consequently, numerous WSN testbeds are excluded fromour review. Firstly, research WSN testbeds without back-channel support are excluded since the absence of back-channel potentially causes the degradation of fidelity in theirevaluation results. Secondly, trial WSN testbeds are excluded.Here, a trial WSN testbed refers to a WSN testbed thatis deployed in a real geographical area for evaluating thefeasibility of aWSN application in a particular domain beforeits full-blown real deployment. TrialWSN testbeds are usuallyspecialized to a set of domain-specific requirements; there-fore, it is not adequate to evaluate them from the general-purpose viewpoint. Finally, IoT-centric WSN testbeds arealso excluded. A heterogeneous mix of IoT devices, such asRFID and domain sensors, is considered out of the scopeof this paper. Nevertheless, among the aforementioned WSNtestbeds, a number of them embody prominent features. Forinstance, SmartSantander [20] provides a city-scale exper-imental facility with 20,000 sensors in support of typicalapplications and services for a smart city, allowing horizontaland vertical federation.

Table 1 provides a featured summary of the researchWSNtestbeds examined in this paper. We try to select a compre-hensive collection of prominent research WSN testbeds tothe best of our knowledge, but we must acknowledge that


Table 1: Featured summary of WSN testbeds under examination.

WSN testbed Nodes/deployment Features

ASSERT [28] 40 nodes including 25 Mica2, 1laboratory office

Flexible control of radio environment based on specialized hardware,controllable network topology, supporting multiple nodes based on

the UUT concept, and Linux-based specialized controller forback-channel

CPN [29] 92 nodes (TelosB, IRIS, andMICA), over different buildings

Back-channel based on UTP-to-USB, persistent device naming, andpowerful visualization tool support

EasyTest [30] 16 EZ521 nodes, 1 laboratoryoffice

Multiradio support based on custom-designed nodes and support802.11 b/g experiments based on EZ271 nodes

Emulab [31, 32] 31 nodes (25 Micaz, 6 mobile), 1laboratory office

First WSN testbed with remote access capability, support for mobilitywith Garcia robot, and MIB600CA-based back-channel

Indriya [33] 127 TelosB nodes, 3Ddeployment over 3 floors

Active USB infrastructure for back-channel and adopting the Motelabmanagement software

Kansei [34] 210 XSM and Tmote sky node, 1laboratory office

One of pioneering WSN testbeds, mixed-simulation support, mobilitysupport using robots, event injection capability, and deployed on grid

fixtures

KanseiGenie [35] 576 nodes, 1 laboratory officeFederated WSN testbed based on GENI environment, deployed with

Kansei and NetEye, and high node heterogeneity (96 XSM, 384TelosB, and 96 iMote2)

KonTest [36] 60 TelosB nodes, 6 office rooms One control PC per room directly connected with nodes with USBback-channel

LabVIEW [37] 23 MicaZ and 8 cricket nodes,laboratory office

PC-based infrastructure, real-time visualization tools based onpopular LabVIEW programming environment

Motelab [38] 190 TelosB notes, 3 floors of alaboratory building

First WSN testbed with web-based services, in situ powermeasurement, usage of Stargate Netbrige for back-channel, and

deployed on grid fixtures

NetEye [39] 130 TelosB nodes, 1 laboratoryoffice

Static 3db attenuator to each antenna for different power levels, usageof laptop computers for back-channel

Sensei-uu [23] Mostly TelosB nodesSupporting smart phone and Lego mobile node, supporting mobility,usage of WL-500G premium broadband routers for back-channel

gateways

SensLAB [40] 1024 WSN430 nodes, 4 differentsites in France

MAC heterogeneity (521 with 802.15.4 MAC, 512 with free MAClayer), radio monitoring based on control nodes, energy

measurement, repeatable mobility via artificial train, and VPNconnectivity between sites

IoT-LAB [41] 2700 nodes, 6 sites in France

Federated architecture with OAR-based reservation support, asuccessor project of SensLab, highly heterogeneous nodes (MSP430,

STM32, Cortex A8-based nodes), and multiple OS support(FreeRTOS, Contiki, Riot, TinyOS, and OpenWSN Linux)

SignetLab [42] 48 eyesIFx-v2 nodes, 1 laboratoryoffice

Back-channel based on the active USB infrastructure, services basedon a single control PC, and Java-based client-side software

TWIST [24]204 nodes (102 TelosB, 102

eyesIFX), 3 floors in an officebuilding

Flat, hierarchical, and segmented architecture, a clustered WSNtestbed with 3 sections, and usage pf 40 NSLU2-based super nodes for

the back-channel

WSNTB [43] 34 nodes (Octopus I II), 2laboratory offices

Specialized hardware to support power consumption and timingmeasurements and distributed management

WeaverLab [44] 80 WSN120 nodes, 1 laboratoryoffice

Node heterogeneity based on modular node design, a dense grid of1.5m × 2m, visual user interface with real-time interaction, and

loosely coupled service architecture

WISEBED [25] 750 nodes, indoor and outdoor, 9different locations in Europe

Federated WSN testbed, mixed-simulation support, in situ powermeasurements on some nodes, mobility support using 40 mobile

nodes, high node heterogeneity (200 iSense, 143 TelosB, 108 G-Node,100 MSB-A4, 44 SunSPIT, 60 pacemate, and 24 Tnode)


Kontest, Labview,SignetLab

EasyTest,Emulab

WeaverLabMoteLab,

KanseiIndria, Neteye,

and TWIST

KanseiGenieSensLabIoTLab

Native-BC/on-board BGW

Broadband-BCBroadband-BC/external BGW

Hybrid-BC

Looselycoupled

TightlycoupledSingular

Federated

Clustered WSNTB

WSN fieldstructure

Back-channelorganization

Servicearchitecture

WISEBED

CPN

Assert

Sensei-uu

Monolithic

Figure 6: Classification result of WSN testbeds.

several notable research WSN testbeds are excluded due toour ignorance.

Figure 6 depicts the classification result of the WSNtestbeds according to our taxonomy. For pictorial clarity,only three design domains are depicted. In the figure, theback-channel organization, the WSN field structure, andthe service architecture occupy, respectively, one axis in thethree-dimensional diagram. The result on the remainingdesign domain, that is, theWSN layout, is reported in Table 3.

4.2. Back-Channel Organization Domain

4.2.1. Native-BC Organization. This design approach pro-vides a cost-effective and handy way to implement a back-channel. The use of an on-board external communicationinterface as the back-channel makes any other communica-tion equipment unnecessary, thereby enabling using nodeswithout any modification. On the contrary, the native-BCorganization has inherent limitations associated with bothwire-length and the number of physical connectors. As aresult, this design approach has been used mainly for small-scale WSN testbeds mostly using desktop computers as theirtestbed servers.

In practical designs of the native-BC organization, USB isa preferred option over RS232c. This is because USB is moreextensible than RS232c in terms of both the number of portsand cabling coverage. The maximum length of an ordinarypassive cable is limited to 5 meters. On the other hand, withactive USB cables, the coverage can be extended to 25 metersas daisy-chaining is allowed up to five active cables. In termsof ports, a network based on USB hubs can constitute 127physical ports, each connecting one node [36]. Consequently,a design of the native-BC organization falls on one of threemethods: RS232c, a passive USB network, and an activeUSB network. LabVIEW [37] takes RS232c. Kontest [36] andSignetLab [42] adopt the active USB network method.

4.2.2. Broadband-BC Organization. As discussed earlier, thebroadband-BC organization uses a broadband network toimplement a back-channel. Most of the WSN testbeds inthis design approach have adopted Ethernet or WiFi astheir broadband networks just for taking advantage of thecommercially mature technology. Therefore, designs of thebroadband-BC organization are characterized by the wayto implement the back-channel gateway which must beprovided for each node. Below, we examine the design of eachvariant of this approach.

Broadband-BC with an On-Board BGW Module. A back-channel gateway is implemented as a hardware module,which we call a BGW module, and can be piggy-backeddirectly to a node. Typically, a BGWmodule uses commercialTCP/IP chip-set and provides a protocol conversion capabil-ity such as RS232C-to-TCP/IP or USB-to-WiFi. NumerousWSN testbeds take this design approach including Easitest[30], Emulab [31], WSNTB [43], andWeverLab [44]. In somecases, the module is integrated directly in the node hardware.This is taken especially when a new proprietary node isdeveloped for theirWSN testbeds.The embedded integrationenhances their hardware reliability, and the installation ofthe WSN testbed becomes less troublesome due to fewerhardware pieces. For example, ASSERT [28] and EasiTest[30] develop their own nodes in which, respectively, a 10/100Ethernet module and a IEEE 802.11b/g WiFi module areintegrated inside their hardware.

Broadband-BC with an External BGW Controller. A back-channel gateway is implemented as an external controllerwhich is attached to a node via RS232c. As for the controller,a single-board computer is usually used. For instance, Kansei[34] uses XSS (Extreme Scale Stargate) single-board com-putes, and Motelab [38] uses Stargate NetBridge which issimilar to XSS in its hardware specification.


4.2.3. Hybrid-BC Organization. Themain aim of the hybrid-BC organization is to reduce the number of gateways byeliminating the need for coupling each individual node to adedicated back-channel gateway, while securing the propertyof the large-scaleWSN testbed deployment of the broadband-BC. Each gateway device thus serves more than one node. Ahybrid-BC organization is composed of three parts: native-BC networks, back-channel gateways, and a broadband-BCnetwork. In practice, designs of the hybrid-BC organizationare primarily distinguished by the back-channel gateway.Across the existingWSN testbeds, choices of both the native-BC part and the broadband-BC part are almost the samewith the pure hybrid-BC approach. As for the native-BCorganization, a USB network is used, and Ethernet or WiFiis used for the hybrid-BC part. Indriya [33] and TWIST [24]use an activeUSBnetwork, whileNetEye [39], Sensei-uu [23],and WISEBED [25] use a passive USB network.

Designs of the hybrid-BC organization are always sup-posed to use external BGW controllers for back-channelgateways. TWIST [24] uses NSLU2 single-board computers.In its deployment, 46 wall-powered NSLU2s are installed tomanage 204 WSN testbed nodes. On the contrary, Indriya[33] and NetEye [39] use PCs. In Indria, 6 Mac Mini PCsare used to manage 127 nodes, where each Mac Mini PCis capable of servicing more than 100 USB devices. NetEyeuses 12 Dell Vestro laptops for 130 nodes. As an exceptionalcase, CPN [29] employs a USB-to-UTP converter as a back-channel gateway where each USB-to-UTP converter canserve up to 4 nodes. In this design, Ethernet wires for thebroadband-BC part eliminate the need for active USB cablesor additional hubs as signal repeaters in its native-BC part.

4.3. Service Architecture Domain

4.3.1.Monolithic Service Architecture. This design approach isthe most simplistic among the three design approaches in theservice architecture domain.Their services are usually built inan ad hoc fashion. Further, their management functionalitiesmay not be explicit, leaving the details to developers.The sim-plicity of this service architecture enables a fast prototypingof a working WSN testbed [28, 37]. Also, users can freelybring services suitable for their own experiments since theydo not need to be concerned about the very complex issuesraised when sharing WSN testbeds. For instance, LabVIEW[37] offers real-time visualization ofmonitoring data by usingthe popular LabVIEW programming environment. ASSERT[28] enables user to perform delicate configurations entailedfor each individual experiment such as setting radio topologyand attenuation.

4.3.2. Tightly Coupled Service Architecture. As discussedearlier, in this design approach, the whole testbed servicesfor users are provided by server-side software, and client-side software has a single role to enable users to accessthose services. A number of well-established multiuser WSNtestbeds take this service architecture, as shown in Figure 6.Due to the clear-cut role of client-side software, the designs ofclient-side software over theseWSN testbeds do not show any

significant differences. They usually adopt a web browser toprovide only a thin interface for accessing the remote serviceson the testbed server. However, their design practices ofserver-side software exhibit a large variance in their softwareand hardware infrastructures. Simplistically, several WSNtestbeds are built upon a single server with basicmanagementservices such as accounting, reservation, and data logging [31,33, 38, 39]. On the other hand, several WSN testbeds, whichinclude Kansei [34], KanseiGenie [35], SensLab [40], TWIST[24], and WSNTB [43], employ a cutting-edge infrastructurewith a set of sophisticated services over several dedicatedservers. For instance, SensLAB [40] implements the server-side software based on state-of-the-art standard technologiesincluding an OSGi framework, hypervisor, LDAP directorytree, SQL database, and Apache Tomcat as its servlet con-tainer.

4.3.3. Loosely Coupled Service Architecture. In this designapproach, the role of client-side software is not only toprovide accesses to server-side software, but also to provideproprietary services for users. Instead of web browserstypically used in the tightly coupled architecture, eithercustomization or a third-party implementation of client-sidesoftware can thus be conducted. WSN testbeds adopting thisarchitecture take different emphases and methods in theirimplementations [23, 25, 44]. For instance, WISEBED [25]aims at creating technology independent client-side software.It provides clear-cut service APIs for the implementationof client-side software. All APIs are specified as standardweb services using WSDL documents and a human-readabledocumentation. As server-side software provides a completeset for testbed services, client-side software is usually slimwith only the access capability. On the other hand,WeaverLab[44] is motivated to achieve cost-effectiveness by sliming themanagement infrastructure as well as to provide users withdirect interactions with nodes. Testbed software is split overboth client computers and the testbed server. The client-side software can provide users with experiment-related ser-vices such as reprogramming, monitoring, and data logging.The server-side software is focused on administration andmaintenance functionalities. Decoupling services based onthe clear notion of user services and management servicesoffer an enlarged freedom to evolve individually along its ownpath. As a result, WeaverLab [44] provides a rich set of GUI-based experimental supports within the client-side software,which can be even more specialized on demand from a thirdparty.

4.4. WSN Field Structure Domain

4.4.1. Singular WSN Field. Most of the WSN testbeds takethis WSN field structure, as shown in Figure 6. The WSNcluster in this singular structure is organized mainly withhomogeneous nodes except for several WSN testbeds includ-ing EasiTest [30], TWIST [24], and WSNTB [43]. Comparedwith clustered or federated WSN testbeds, singular WSNtestbeds are thus relatively limited in hardware heterogeneityof WSN testbed nodes. Traditionally, a singular WSN testbed


is managed as a stand-alone facility administrated by a singleinstitution.

4.4.2. Clustered WSN Field. Only a few WSN testbeds takethis WSN field structure [29, 43]. WSN testbeds adoptingthis WSN structure are deployed either on single site or onmultiple sites. As the former case, WSNTB [43] is composedof twoWSN clusters within a single site, where WSN clustersare different from each other in the radio communicationmethod. When the radio severance occurs among discretespaces such as rooms and floors, a clustered WSN fieldstructure is also adopted on a single site. As the latter case,WSN clusters of CPN [29] are deployed, respectively, on aseparate building and are connected via the Internet. Due tothe radio severance between WSN clusters, the scalability inthe context of this paper may remain the same for both cases,even if the size of the WSN testbed increases.

4.4.3. Federated WSN Field. Several WSN testbeds are builtaround the federation concept in pursuit of scalability andheterogeneity. KanseiGenie [35] in Ohio State Universityrefactors the Kansei WSN testbed into a federation versionusing GENI [26]. It consists of a site at the Ohio StateUniversity, which has four different sensor fabric arrays, and asite at Wayne State University, which has two different sensorfabric arrays. SensLAB [40] aims at unifying four discreteheterogeneous WSN testbeds into a single WSN testbed of1,000 nodes. WISEBED [25] demonstrates the virtual WSNtestbed that comprises transparently physical WSN testbedsalong with simulated and emulated elements. Recently, IoT-LAB [41], the successor project of SensLAB, delivers a morerobust federated WSN testbed based on advanced federationsoftware technologies.

An eventual challenge of building a federated WSNtestbed is to provide users with transparent access to mem-ber WSN testbeds which usually comprise heterogeneousresources under multiple independent administrative enti-ties. Because of the challenge, building a federated WSNtestbed entails a supplemental network and a software infras-tructure over the stand-alone WSN testbed facility. Theinfrastructure network of a federated WSN testbed consistsof intrasite networking and intersite networking. Firstly,intrasite networking is quite similar across the federatedWSNtestbeds.They are typically based on VLAN (virtual LAN). Inparticular, WISEBED employs an overlay network in whichthe site gateway encapsulates and relays data between theserial link of the attached nodes and the site server. Sec-ondly, intersite networking for the interconnection betweenmember WSN testbeds exhibits a large variance across thefederatedWSN testbeds. WISEBED and KanseiGenie use thepublic network. Testbed servers of member WSN testbedsare directly connected to the Internet without any specificnetwork facility. In contrast to them, SensLAB interconnectsmember WSN testbeds with VPN (virtual private network)via a dedicated router in each site.

Software infrastructure for federation is focused on thetransparent access to the resources inmemberWSN testbeds;therefore, building a trust system, resource management,

and virtualization of testbed services emerge as centralthemes. Firstly, the trust system provides authenticationand authorization to enable users to access member WSNtestbeds. KanseiGenie and SensLAB use login credentials ontheir portal accounts. WISEBED provides the SNAA (sensornetwork authentication and authorization) service based ona well-known distributed single sign-on system, Shibboleth.Once an account is provided, a user is able to authenticatehimself by sending a web service request to the federatorwhich represents a federated SNAA of several WSN testbedsand offers a secret authentication key in return. In IoT-LAB, all the objects including resources exist in a sharedname space, organized in a hierarchy of authorities andsubauthorities. The authentication mechanism is based on apublic key infrastructure, where each object has a key pairand is associated with a signed certificate.

Secondly, resource management aims at a frameworkfor transparent resource usages across the potentially het-erogeneous resources in multiple member WSN testbeds.The framework shall allow users to be able to browse,discover, and reserve resources in a uniform way in spiteof their heterogeneity. A minimum requirement is thusthat the exposition of resource information to users and itsreservation procedure should be general enough, while spe-cialization is accommodated. KanseiGenie clearly addressesthis requirement by using the ORCA control frameworkwhich is investigated intensively in the GENI project. ORCAuses a resource lease contract abstraction in which resourceproviders advertise or delegate their resources to brokerintermediaries, and users request resources to those brokers.In WISEBED, resources are named using URN (uniformresource names) in which each node in the federation isuniquely identified by the combination of the member siteID, the sensor network ID, and the node ID. Resource spec-ifications about node capabilities and attributes are postedin a user-readable text. A special client-side implementation,called a federator, federates multiple WSN testbeds, exposingtheir features as a single virtual testbed server instance.As a result, the resource reservation service provided byeach individual WSN testbed is aggregated to form a virtualsingle service. SensLAB is based on a homemade portal,using LDAP to store its users and keys. Sharing resourcesamong users is implemented using the OAR, which is aversatile resource manager for large clusters, particularlypopular in the grid community. IoT-LAB takes different stepstoward standard-based implementation. It adopts the slice-based federation architecture (SFA) which is a control planearchitecture by theGENI initiative. Resource specifications inXML describe the set of resources made available by a WSNtestbed. A registry managermanages authorities, users, slices,and resources. A registry is typically provided by an authorityto manage all of these items. Aggregate manager APIs areprovided to browse and reserve resources. An aggregatemanager is typically deployed by an individual WSN testbed.

Finally, the virtualization aims at offering a uniform ser-vice abstraction across thememberWSN testbeds. Generally,its implementation is based on web services. Deployed ontheir testbed servers, they are directly accessible by users.KanseiGenie provides an aggregate of aggregate manager


(AAM), the individual component managers (CM) for eachdevice type, and an ORCA site authority module. Providingan iWSN interface as a Web services interface, WISEBEDimplements generic standard services and offers their APIsthat range from managing federation, such as creatingWISEBED-compliant custom WSN testbed, to conductingexperiments by users. These APIs include SNAA APIs forauthentication and authorization, RS APIs for reservation,WSN APIs for conducting, and finally controller APIs forcollecting data from nodes. In SensLAB, web services, namedexperimentation handler, serve as interaction points withthe nodes in a site. With respect to each individual node,the experimentation handler provides firmware update, theenergy consumption monitoring, and polling.

4.5. WSN Layout Domain

4.5.1. NativeWSNLayout. Most of theWSN testbeds take thisapproach, as shown in Figure 6. In the native WSN layout,someWSN testbeds take a dedicated space, while others sharelaboratory rooms and corridors by installing nodes on theirceiling. This WSN layout inherently requires a large space;when preserving the normal radio transmission power inWSNs, distance between nodes in the layout ranges from10 meters to 30 meters. Indeed, several large-scale WSNtestbeds with over 100 nodes, for example, CPN [29], Indiya[33], and TWIST [24], are typically deployed over severalfloors. As a side effect, the cost of initial deployment andmaintenance usually increases as the network size becomeslarger. As a positive aspect, this approach provides a morerealistic environment in terms of the space itself.

4.5.2. Grid WSN Layout. In this approach, nodes are placedon a relocatable square fixturewith each side of approximatelytwo to five meters. Compared with the native WSN layout,the grid WSN layout requires a quite smaller space, therebybeing quite advantageous to scalability. Moreover, the instal-lation and maintenance cost become relatively manageable.However, a difficulty with this approach is the need to controlradio propagation distance such that an appropriate nodedensity similar to the native WSN layout can be maintained.To this end, a naive RSSI control mechanism is used inMotelab [38] and Kansei [34]. As a more elaborated scheme,NetEye [39] instruments each node with a 3 dB SMA (Sub-Miniature version A) attenuator and an omnidirectional RHseries monopole antenna. As another elaboration, ASSERT[28] implements a radio grid with both FPGA-based special-ized hardware to control signal attenuation and coaxial cablesto connect attenuators. It allows all the signal manipulationspurely in the RF domain such that communication betweennodes does not leak into the environment, thereby enablingexperimenters to inject radio noises into the system andobserve their influences on the system being studied.

5. Evaluation

In this section, we evaluate WSN testbeds listed in Table 1with respect to the core requirements for WSN testbeds

identified in Section 2. In Section 5.1, we describe themethodology used in our evaluation. In Sections 5.2, 5.3, and5.4, we explain the set of criteria for each individual corerequirement, which will be collectively used for assessing thefulfillment level of the core requirement. Influences exertedon each criterion by design approaches in our taxonomy areexplained as well. In Section 5.5, we illustrate the influenceassessment with respect to each criterion. Finally, the evalu-ation result of WSN testbeds obtained by applying the resultof the influence assessment to eachWSN testbed is explainedin Section 5.6.

5.1. Evaluation Methodology. In the evaluation of WSNtestbeds, we employ amethodology that consists of two steps:influence assessment and fulfillment assessment.

5.1.1. Influence Assessment. This is aimed at assessing infer-ences that each design approach in our taxonomy exerts oneach criterion for the core requirements forWSN testbeds. Atfirst, for each core requirement, that is, scalability, flexibility,and efficiency, we sort out a set of the criteria that will becollectively used to assess the fulfillment level of a WSNtestbed for the core requirement. Each individual designapproach in the taxonomy may have either a positive ora negative influence on a criterion. We thus examine andassess how each design approach exerts an influence on eachindividual criterion. Just for reference, the criteria and theresult of the influence assessment are shown in Table 3. Theprocedure for the influence assessment is detailed shortly inSection 5.5.

5.1.2. Fulfillment Assessment. This is aimed at assessing howwell a WSN testbed fulfills each of the core requirements.Given a WSN testbed, two kinds of information are usedin the assessment of fulfillment levels. The first one is theclassification result for the WSN testbed. In Section 3, wealready classified the WSN testbed using our taxonomy. Theclassification result is shown, respectively, in Figure 6 andin Table 3. According to the classification result, the WSNtestbed is featured with a set of four design approaches taken,respectively, from each individual design domain.The secondkind of information is the result of the influence assessmentwhich contains the influence type for each pair of a designapproach and a criterion. The set of influence types obtainedfor the design approaches of the WSN testbed from theinfluence assessment result enables us to assess the fulfillmentlevels of the WSN testbed. This procedure is detailed shortlyin Section 5.6.

5.2. Scalability Criteria. Scalability of WSN testbeds can beinfluenced by many factors from both technical and non-technical area. Among them, we select the three most criticalones, wiring, space, and cost, as the criteria for scalability.

5.2.1. Wiring. The wiring method for implementing a back-channel may restrict the number of nodes in the deployment.For example, as previously discussed, the coverage of activeUSB cables is within a maximum distance of 25 meters. An


active USB-hub infrastructure is limited to the maximum of127 ports.

The wiring method for a WSN testbed is determinedonly by its back-channel organization; the other three designdomains in the taxonomy do not affect the wiring method.Each design approach in the domain of the back-channelorganization embodies its own inherent influential character-istics on the wiring criterion. Firstly, the native-BC approachexerts a negative influence on the wiring criterion since ituses either RS232c or USB in the back-channel. Secondly, thebroadband-BC approach hardly exerts a negative influenceon the criterion because Ethernet or WiFi, which is typicallyemployed in this approach, is subject to neither distance norport exhaustion. Finally, the hybrid-BC approach also hardlyexerts a negative influence on the criterion. Typically, WSNtestbeds with the hybrid-BC adopt USB as their native partand Ethernet or WiFi as their broadband part. Althougha limitation is enforced by USB to the number of nodesper back-channel gateway, the limitation can be avoided byadding more back-channel gateways [24, 29, 33].

5.2.2. Space. Apparently, the space required for the deploy-ment of a WSN testbed may hinder its deployment. Prac-tically, it is difficult to afford a sufficient space due toeither physical or budget limitation. Indeed, numerous WSNtestbeds use corners or ceilings of their laboratories [36, 43].To ensure a large-scale deployment, the space requirementshould be as small as possible; the node density per unitspace should be as high as possible. In this respect, the WSNlayout is the only design domain that may influence the spacecriterion. Assessing the influence of each design approachin the WSN layout domain is very straightforward. Thenative WSN layout approach may exert a negative influencesince it typically requires as much space as in the real-worlddeployment. On the contrary, the influence exerted by thegrid WSN layout approach is negligible due to its far smallerspace requirement.

5.2.3. Cost. Deployment cost serves potentially as a nontech-nical barrier to test scalability.Of the four design domains, theback-channel organization can potentially exert influences onthe deployment cost as it directly affects the cost per node.Of the four back-channel organizations, the broadband-BCwith an external BGW controller can exert a substantiallynegative influence on the cost criterionmainly due to its needof a back-channel gateway per node. This is well addressedin the design of Indriya [33] which reports a comprehensivecost analysis between the broadband-BC and the hybrid-BCapproach. It is very straightforward that the other approachesdo not exert any substantially negative influence on the costcriterion.

5.3. Flexibility Criteria. Given a large body of WSN appli-cations, flexibility of WSN testbeds can be influenced bymany factors from both technical and nontechnical areas.Among them, we select the two most critical ones, variablenode support and radio environment control, as the criteria forflexibility.

5.3.1. Variable Node Hardware. The variable node hardwareis essential for flexibility. Three typical options exist forsupporting the hardware variability. The first option is touse a heterogeneous mix of node platforms in the WSNfield. For instance, TWIST [24] takes this option.The secondoption is to develop new nodes with hardware variabilityin mind. Several projects with stable funding sources takethis option [28, 30, 43, 44]. Some of them take a modulardesign with a broad set of daughter modules [44], whilethe others integrate multiple functions in a single hardware[28, 30, 43]. Finally, the third option is to pursue federation[23, 25, 35, 40]. Federation is viewed as a macroscopic moveto the hardware variability. For instance, WISEBED [25]introduces heterogeneity by using different classes of nodesfrom an embedded 8-bit microcontroller platform to a fullyequipped PC.

In the design space, only a single design domain, theWSN field structure, may influence this criterion of thevariable node hardware. Each design approach in this domainexerts its own inherent influence on the criterion. Firstly, asingular WSN field may exert a modestly negative influenceon the criterion as it has to resort to the aforementioned firsttwo options for hardware variability. These options do notguarantee a sufficient level of hardware variability. In case ofthe first option, under a heterogeneous mix of nodes, it israther complex and even impractical to prepare and conductexperiments by remote users. The second option can providethe modest degree of hardware variability; however, it wouldnever be sufficient in the presence of the large spectrum ofWSNapplications. Secondly, the clusteredWSNfield presentsa more enlarged opportunity for hardware variability thanthe singular WSN field since it allows a set of heterogeneousWSN clusters. Finally, the federation is clearly themost viableoption for hardware variability as a rich set of variable nodescan be provided from the member WSN testbeds.

5.3.2. Radio Environment. Capability to control the radioenvironment is essential for flexibility. It allows experimentsto set up radio environment realistically for each uniquetarget environment. Among others, radio inference and radiocongestion are relatively more critical issues in the radioenvironment. Radio inference is usually controlled by noiseinjection, where external devices not participating in a WSN%emit radio packet on the WSN radio band in order totamper the WSN transmissions [34]. The control of radiocongestion is typically based on the adjustment of radiotransmission distance through transmission power control.Assuming the constant network workload per node, givena radio transmission distance, the radio congestion dependson the spatial density of nodes, whereas, given a constantspatial density, it depends on the radio transmission distance.As each node in a WSN testbed is rigidly placed in afixed location, the control of the radio transmission distanceprovides a way to adjust radio congestion. In the architecturaldesign space, only the WSN layout domain can potentiallyinfluence the radio environment control. In the WSN layoutdomain, the grid WSN layout approach is advantageous forradio environment control due to its compact deployment.


On the contrary, the extent of radio environment control ispotentially restricted in the native WSN layout.

5.4. Efficiency Criteria. The efficiency of a WSN testbed canbe greatly influenced by the experimental tools that theWSNtestbed provides for experimenters. Over the experimen-tation lifecycle, a set of experimental tools is required tosupport experimental activities of experimenters. To namea few, it includes tools for reserving resources and parallelreprogramming in the experiment setup stage and tools formonitoring the progress of the experiment in the experimentexecution stage. Across WSN testbeds, the provision of thesetools varies depending on the investment to the implemen-tation. However, there is a set of ingredients that renderthose tools versatile enough to fulfill the efficiency. Amongthem, we select the two most essential ingredients, resourcetransparency and real-time interactiveness, as the criteria forefficiency.

5.4.1. Resource Transparency. Resource transparency is torender an experimenter to have a local and private viewfor the resources of a shared remote WSN testbed as if theresources are privately owned by the experimenter. Resourcetransparency is required over the whole experimentationcycle. As an example, we examine resource reservationneeded at the experiment setup stage. For a WSN testbedshared by many users, resource reservation becomes quitecomplex due to the need of a large volume of information onthe WSN testbed resource. To name a few, the informationincludes the technical specification of each WSN node andboth the physical and the radio layout of nodes. Therefore,a complete set of tools to browse, discover, request, andreserve resources is needed in order to render users to beable to transparently utilize WSN testbed resources. As anexample, WeaverLab [44] employs a GUI representationwhere users can choose nodes by clicking correspondingicons on a graphical node layout. SensLAB [40] provideslocale information in the form of (𝑥, 𝑦, 𝑧) coordinates foreach WSN testbed node.

The resource transparency criterion is potentially influ-enced by the service architecture domain. It is partly becausethe improvement obtainable by implementation efforts isinfluenced by the architectural aspect of the service archi-tecture. Ideally, a sufficient level of resource transparency isfeasible only for private WSN testbeds based on the mono-lithic service architecture. For a sharedWSN testbed based oneither the tightly coupled or the loosely coupled architecture,significant efforts on both designs and implementations areentailed in order to provide users with an appropriate level ofresource transparency.

5.4.2. Real-Time Interactiveness. Real-time interactiveness isanother essential ingredient for efficiency. Over the experi-mentation lifecycle, experimenters can greatly benefit fromreal-time interactiveness. Firstly, interactive exposition oflog data enable users to capture in a real-time manner theexecution trace of interest. It is quite essential at the debug-ging stage of WSN protocols. Secondly, real-time experiment

monitoring enable users to capture in a real-time fashion theknowledge of any permanent or temporary node breakdowncaused by software crash or hardware malfunction. Suchknowledge is especially valuable for supervising the progressof some critical tasks such as parallel reprogramming overtheWSN nodes. It allows experimenters to have a confidenceon the fidelity of the experiment result. Finally, real-timeinteractive exposition of log data facilitates the developmentof visualization tools to depict the data stream.Allowing usersto capture the temporal variance of performance valuses in areal-time manner, these tools can greatly relieve users fromthe labor entailed when analyzing raw data.

The real-time interactiveness criterion is potentially influ-enced by the design domain of the service architecture. Toprovide real-time interactive interfaces for users, some archi-tectural support is required. It is desired to extend flexibilitywith value-added functionalities such as GUI interfaces forreal-time monitoring. Ideally, the ability for the client-sidesoftware to directly access the back-channel and get fedthe log data stream is also needed. Monolithic and looselycoupled architectures are amenable to such architecturalsupport. In tightly coupled architectures, the implementationof real-time interactiveness tends to be somewhat complexand costly. Unless the server-side software provides servicesto promptly push log data to the client-side software, real-time interactiveness may become an infeasible option. Cus-tomization or extension of value-added functionalities on theclient-side software by a third party may not be allowed aswell.

5.5. Influence Assessment. Based on discussions in the pre-vious subsections, we assess inferences that each designapproach in our taxonomy exerts on each criterion of the corerequirements for WSN testbeds. Table 2 shows the result ofthe influence assessment.

Influences on a criterion by a design approach areclassified into four different types.The first type indicates thatthe design domain, to which the design approach belongs,is not related to the criterion in terms of influence. Thisinfluence type is denoted by UNRELATED with symbolicrepresentation “—” in the table. The second type representsa negative influence that prevents the design approach fromfulfilling the criterion. This influence type is denoted byHARD-BARRIER with symbolic representation “X.” Thethird type represents another negative influence that becomesalso a barrier against the fulfillment of the criterion by thedesign approach, but the barrier can be practically resolved byappropriatemeasures. For example, as for a negative influenceon the cost criterion, an increased expense can resolve thebarrier.This type of influences is denoted by SOFT-BARRIERwith symbolic representation “△.” Finally, the fourth typeindicates any trivial or no influence that poses no barrieragainst the fulfillment of the criterion. This type is denotedby NO-BARRIER with symbolic representation “O.”

While discussions in the previous subsections makethe assessment result in Table 2 self-explanatory, we pointout several entries for elucidation. Firstly, while both thebroadband-BC with an external BGW and the hybrid-BC


Table2:Re

sultof

influ

encesa

ssessm

ent.

Requ

irement

Criteria

Back-chann

elorganizatio

nServicea

rchitecture

WSN

field

structure

WSN

layou

tNative

Broadb

and/on

-board

BGW

Broadb

and/externalBG

WHybrid

Mon

olith

icTightly

coup

led

Loosely

coup

led

Sing

ular

Clustered

Federated

Native

Grid

Scalability

Wiring

XO

OO

——

——

——

——

Space

——

——

——

——

——

△O

Cost

OO

△O

——

——

——

——

Flexibility

Varia

bleH

/Wsupp

ort

——

——

——

—△

OO

——

Radioenvironm

entcon

trol

——

——

——

——

——

△O

Efficiency

Resource

transparency

——

——

O△

△—

——

——

Real-timeinteractiv

eness

——

——

O△

O—

——

——

“—”:un

related,“X”

:hard-barrier,“△

”:soft-barrier,and“O

”:no

-barrie

r.


Table3:Re

sultof

fulfillm

entassessm

entfor

WSN

testb

eds.

WSN

testb

edBa

ck-chann

elarchitecture

Servicea

rchitecture

WSN

field

structure

WSN

layou

tFu

lfillm

entlevel

Native

Broadb

and/on

-board

BGW

Broadb

and/externalBG

WHybrid

Mon

olith

icTightly

coup

led

Loosely

coup

led

Sing

ular

Clustered

Federated

Native

Grid

Scalability

Flexibility

Efficiency

ASSER

To

oo

o3

23

CPN

oo

oo

22

1Ea

syTest

oo

oo

21

1Em

ulab

oo

oo

21

1Indriya

oo

oo

21

1Ka

nsei

oo

oo

22

1Ka

nseiGenie

oo

oo

23

1Ko

nTest

oo

oo

01

3LabV

IEW

oo

oo

01

3Motelab

oo

oo

22

1NetEy

eso

oo

o3

21

Sensei-uu

oo

oo

21

2SensLab

oo

oo

22

1IotLAb

oo

oo

22

1Sign

etLab

oo

oo

01

3TW

IST

oo

oo

21

1WSN

TBo

oo

o2

21

WeaverLab

oo

oo

32

2WISEB

EDo

oo

o2

22


require external back-channel gateways, only the influenceon the cost criterion by the former approach is assessedas SOFT-BARRIER due to its need for a back-channelgateway per node. Secondly, the influence on the resourcetransparency criterion is typed into SOFT-BARRIER forboth the tightly coupled and the loosely coupled servicearchitecture because they embody inherent complexitiesin supporting resource transparency. Thirdly, the influenceexerted by the tightly coupled architecture on the real-timeinteractiveness criterion is assessed as SOFT-BARRIER. Thereason for the assessment is the difficulty of implementing thereal-time interactive monitoring capability due to both theserver-based centralized data logging and the lack of directinteractions between client-side software and nodes.

The result of influence assessment in Table 2 enables usto get a clear perspective on the influential relation betweendesign domains in the taxonomy and core requirementsfor WSN testbeds. On one hand, it reveals that each corerequirement is influenced by a subset of design domains.Scalability is affected by both the back-channel organizationand the WSN layout structure. Flexibility is affected byboth the WSN structure and the WSN layout. Efficiency isinfluenced only by the service architecture. On the otherhand, it shows that each design domain exerts influencesmostly on a single core requirement. Only the WSN layoutdomain exerts influences on two core requirements, that is,scalability and efficiency.

5.6. Fulfillment Assessment of WSN Testbeds. Based on theresult of influence assessment in Table 2 and the classificationresult for WSN testbeds in the left-hand columns of Table 3,we can evaluate how well each WSN testbed under examina-tion fulfills each of the core requirements for WSN testbeds.

In the fulfillment assessment, we employ a 4-level gradingsystem. The degree of fulfillment is graded into four levelsfrom Level-0 to Level-3. A level value represents how muchpositively a WSN testbed fulfills a core requirement; 0 is theleast and 3 is the best. More precisely, Level-0 indicates thata WSN testbed cannot fulfill a core requirement. On thecontrary, Level-3 indicates that a WSN testbed can fulfill acore requirement without having negative influences on anycriterion. Level-1 or Level-2 indicates that it can fulfill a corerequirement, but modest investments have to be made toresolve the negative influences on a subset of criteria. Level-1and Level-2 are distinguished by the number of the criteriawith negative influences.

Given a WSN testbed, the fulfillment assessment ofeach core requirement is based on the following procedure.Firstly, if there is at least one criterion among the criteriaassociated with a core requirement on which the influencetype is HARD-BARRIER, the fulfillment level for the corerequirement is graded as Level-0. This is applied only toscalability since other core requirements, that is, flexibilityand efficiency, do not have entries with HARD-BARRIER inTable 2. Secondly, unless a core requirement has any criterionassessed as HARD-BARRIER, its fulfillment is graded by thenumber of the criteria that have at least one entry marked asSOFT-BARRIER in the result of the influence assessment. If

the number of such criteria is only one, the fulfillment level isassessed as Level-2.Otherwise, the fulfillment level is assessedas Level-1.

Table 3 shows the result of the fulfillment assessment.Overall, all theWSN testbeds under examination are not idealwithout completely satisfying all the three core requirements.In case of scalability, three WSN testbeds have criticaldrawbacks with Level-0, while only two WSN testbeds getthe highest fulfillment level. The rest of the WSN testbedsmodestly fulfill the scalability with fulfillment Level-1 orLevel-2. In contrast to the scalability, no single WSN testbedfails to fulfill either flexibility or efficiency; there is not aWSNtestbed with Level-0, respectively, for each core requirement.This is an expectable result because both requirements aredependent more on either software or hardware implemen-tation than on system architectures. Still, their fulfillmentlevels exhibit variance across theWSN testbeds, ranging fromLevel-1 to Level-3. As the assessment result of each WSNtestbed is self-explanatory from the result of the influenceassessment in Table 2, we do not expose discussions on eachindividual WSN testbed.

It is worth to note that the result of the fulfillmentassessment for each WSN testbed in Table 2 reflects solelythe architectural perspective of the WSN testbed subject toour taxonomy. Except Level-0, each value in the fulfillmentassessment represents the lower bound in practice rather thanthe upper bound in practice. As noted earlier, a WSN testbedcan completely overcome negative influences of the SOFT-BARRIER type which degrade the fulfillment level to eitherLevel-1 or Level-2. Apparently, large-scaleWSN testbeds builton a large volume of elaboration such as SensLAB [40]possess higher fulfillment levels in practice. Nevertheless,architectural merits and demerits of design approaches iden-tified by our fulfillment assessment are clearly useful forsystematic improvements of the current WSN testbeds.

From the architectural perspective, the evaluation resultof current state-of-the-art WSN testbeds along with dis-cussions in Sections 3 and 4 elucidates several importantpoints. Firstly, efficiency of user experimentation is to bemore valued in theWSN testbed design.This is a prerequisiteto divert the current situation that fairly many well-knownWSN testbeds are retired, stay dormant, or remain under-utilized. Secondly, strategic consideration should be devotedto pragmatic aspects. Currently, there would be no serioustechnological barriers against securing the scalability ofWSNtestbeds. It is rather probable that nontechnological factorssuch as deployment cost and space become a huddle againstthe scalability. In addition, we need to deviate from theconfusion that federation would contribute to the scalability.Different from general network disciplines such as the futureInternet, federation in the context of WSN testbeds con-tributes primarily to hardware variability.Thedevelopment ofa full-blown federatedWSN testbed is thus to be preceded bya comprehensive analysis on the trade-off between the utilityvalue and the financial investment. Finally, more elaborationis required for some promising design approaches. To namea few, the grid WSN layout with delicate radio controlis precious for scalability and flexibility but still remainsin immaturity. The loosely coupled service architecture is


promising for efficiency but is not sufficiently explored.Design alternatives based on well-formed combinations ofsuch design approaches would provide insight into solutionsto current open problems such as narrowing the potential gabbetween experimental environments of WSN testbeds andreal deployment environments to a reasonable level.

6. Conclusion

In a large-scale setting, appropriate WSN testbeds are essen-tial for both researchers in academia and developers inindustries for facilitating experiments of WSN protocols atall levels. Despite the decade-long WSN testbed research,active research and development are still ongoing particularlyas a part of the current large-scale IoT research. This paperseeks to provide a framework to reason about the evolvinglarge-scale WSN testbeds from a pragmatic and architecturalperspective. A novel architectural taxonomy for general-purposeWSN testbeds proposed in this paper proves effectivein examining and understanding architectural features ofexisting WSN testbeds. Moreover, it serves as an elegantmeans to qualitatively evaluate WSN testbeds for theirresponsiveness to a core set of requirements in the large-scale context. Architectural merits and demerits of designapproaches identified by our evaluation for current WSNtestbeds would clearly pave a way for systematic advances incurrent as well as futureWSN testbeds. As future research, weplan to extend the current taxonomy by incorporating ver-tical specialized functionalities, such as mobility and healthmonitoring, and to elaborate the evaluation framework byapplying a distinctive weight-based ranking to the set ofevaluation criteria.

Conflict of Interests

The authors declare that there is no conflict of interestsregarding the publication of this paper.

Acknowledgments

This research was supported by the IT R&D program ofMOTIE/KEIT. [10041145, Self-Organized Software platform(SoSp) for Welfare Devices].

References

[1] I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci,“Wireless sensor networks: a survey,” Computer Networks, vol.38, no. 4, pp. 393–422, 2002.

[2] I. Demirkol, C. Ersoy, and F. Alagoz, “MAC protocols forwireless sensor networks: a survey,” IEEE CommunicationsMagazine, vol. 44, no. 4, pp. 115–121, 2006.

[3] Q. Wang, Y. Zhu, and L. Cheng, “Reprogramming wireless sen-sor networks: challenges and approaches,” IEEE Network Mag-azine, vol. 20, no. 3, pp. 48–55, 2006.

[4] N. A. Pantazis and D. D. Vergados, “A survey on power controlissues in wireless sensor networks,” IEEE CommunicationsSurveys & Tutorials, vol. 9, no. 4, pp. 86–107, 2007.

[5] R. Sugihara and R. K. Gupta, “Programming models for sensornetworks: a survey,”ACMTransactions on Sensor Networks, vol.4, no. 2, article 8, 2008.

[6] Y. Zhou, Y. Fang, and Y. Zhang, “Securing wireless sensor net-works: a survey,” IEEECommunications Surveys&Tutorials, vol.10, no. 3, pp. 6–28, 2008.

[7] W. Dong, C. Chen, X. Liu, and J. Bu, “Providing OS supportfor wireless sensor networks: challenges and approaches,” IEEECommunications Surveys & Tutorials, vol. 12, no. 4, pp. 519–530,2010.

[8] M. Garcia, D. Bri, S. Sendra, and J. Lloret, “Practical deploy-ments of wireless sensor networks: a survey,” InternationalJournal on Advances in Networks and Services, vol. 3, no. 1-2, pp.170–185, 2010.

[9] B. Musznicki and P. Zwierzykowski, “Survey of simulators forwireless sensor networks,” International Journal of Grid andDistributed Computing, vol. 5, no. 3, pp. 23–50, 2012.

[10] U. Colesanti, C. Crociani, and A. Vitaletti, “On the accuracy ofOMNeT++ in the wireless sensornet works domain: simulationvs. testbed,” in Proceedings of the 4th ACM Workshop on Per-formance Evaluation of Wireless Ad Hoc, Sensor, and UbiquitousNetworks, pp. 25–31, 2007.

[11] K. Pawlikowski, H.-D. J. Jeong, and J.-S. R. Lee, “On credibilityof simulation studies of telecommunication networks,” IEEECommunications Magazine, vol. 40, no. 1, pp. 132–139, 2002.

[12] M. O. Farooq and T. Kunz, “Wireless sensor networks testbedsand state-of-the-art multimedia sensor nodes,” Applied Mathe-matics and Information Sciences, vol. 8, no. 3, pp. 935–940, 2014.

[13] A. Gluhak, S. Krco, M. Nati, D. Pfisterer, N. Mitton, andT. Razafindralambo, “A survey on facilities for experimentalinternet of things research,” IEEE Communications Magazine,vol. 49, no. 11, pp. 58–67, 2011.

[14] L. Steyn and G. Hancke, “A survey of wireless sensor networktestbeds,” inProceedings of the IEEEAFRICON, pp. 1–6, Septem-ber 2011.

[15] A. Dwivedi and O. Vyas, “An exploratory study of experimentaltools forwireless sensor networks,”Wireless SensorNetwork, vol.3, no. 7, pp. 215–240, 2011.

[16] G. Wagenknecht, M. Anwander, T. Braun et al., “TARWIS: amanagement architecture for heterogeneous wireless sensornetworks,” in Proceedings of the 6th International Conference onWired/Wireless Internet Communications, pp. 177–188, 2008.

[17] S. Bapat, W. Leal, T. Kwon, P. Wei, and A. Arora, “Chowkidar:a health monitor for wireless sensor Network testbeds,” inProceedings of the 3rd International Conference on Testbeds andResearch Infrastructures for the Development of Networks andCommunities (TridentCom ’07), pp. 1–10, Lake BuenaVista, FLa,USA, May 2007.

[18] J. Kim, J. Lee, and J. Yun, “M2Mservice platforms: survey, issues,and enabling technologies,” IEEE Communications Surveys &Tutorials, vol. 16, no. 1, pp. 61–76, 2014.

[19] C. Siaterlis, B. Genge, and M. Hohenadel, “EPIC: a testbedfor scientifically rigorous cyber-physical security experimenta-tion,” IEEE Transactions on Emerging Topics in Computing, vol.1, no. 2, pp. 319–330, 2013.

[20] J. Bernat, “SmartSantander : the path towards the smart cityvision,” in Proceedings of the 1st ETSI M2M European Telecom-munications Standards Institute Machine to Machine Workshop,Sophia Antipolis, France, October 2010.


[21] J. Polastre, R. Szewczyk, and D. Culler, “Telos: enabling ultra-low power wireless research,” in Proceedings of the 4th Interna-tional Symposium on Information Processing in Sensor Networks(IPSN ’05), pp. 364–369, April 2005.

[22] Tmote SkyData Sheet, pp. 366–379,Moteiv, San Francisco, Calif,USA, 2006.

[23] O. Rensfelt, F. Hermans, P. Gunningberg, and L.-A. Larzon,“Repeatable experiments with mobile nodes in a relocatableWSN testbed,” in Proceedings of the 6th IEEE International Con-ference on Distributed Computing in Sensor Systems Workshops(DCOSSW ’10), pp. 1–6, Santa Barbara, Calif, USA, June 2010.

[24] V. Handziski, A. Kopke, A. Willig, and A. Wolisz, “TWIST: ascalable and reconfigurable testbed for wireless indoor experi-ments with sensor networks,” in Proceedings of the InternationalSymposium on Mobile Ad Hoc Networking and Computing, pp.63–70, May 2006.

[25] I. Chatzigiannakis, S. Fischer, C. Koninis, G. Mylonas, and D.Pfisterer, “WISEBED: An open large-scale wireless sensor net-work testbed,” in Sensor Applications, Experimentation, andLogistics, pp. 68–87, Springer, Heidelberg, Germany, 2010.

[26] C. Elliott and A. Falk, “An update on the geni project,” ACMSIGCOMMComputer Communication Review, vol. 39, no. 3, pp.28–34, 2009.

[27] A. Gavras, A. Karila, S. Fdida et al., “Future internet researchand experimentation: the FIRE initiative,” ACM SIGCOMMComputer CommunicationReview, vol. 37, no. 3, pp. 89–92, 2007.

[28] E. Nourbakhsh, J. Dix, P. Johnson et al., “ASSERT: a wireless net-working testbed,” in Proceedings of 7th International Conferenceon Testbeds and Research Infrastructure for the Development ofNetworks and Communities, pp. 209–218, 2011.

[29] J. Kowalczuk, M. C. Vuran, and L. C. Perez, “A dual-networktestbed for wireless sensor applications,” in Proceedings ofthe 54th Annual IEEE Global Telecommunications Conference:“Energizing Global Communications” (GLOBECOM ’11), pp. 1–5, December 2011.

[30] Z. Zhao, G.-H. Yang, Q. Liu, V. O. K. Li, and L. Cui, “EasiTest:a multi-radio testbed for heterogeneous wireless sensor net-works,” in Proceedings of the IET International Conference onWireless Sensor Network (IET-WSN '10), pp. 104–108, November2010.

[31] D. Johnson, D. Flickinger, T. Stack et al., “Emulab’s wirelesssensor net testbed: truemobility, location precision, and remoteaccess,” in Proceedings of 3rd International Conference onEmbedded Networked Sensor Systems, p. 306, 2005.

[32] D. Johnson, T. Stack, R. Fish et al., “Mobile emulab: a roboticwireless and sensor network testbed,” in Proceedings of the 25thIEEE International Conference on Computer Communications(INFOCOM ’06), pp. 1–12, April 2006.

[33] M. Doddavenkatappa, M. Chan, and A. Ananda, “Indriya: alow-cost, 3D wireless sensor network testbed,” in Proceedingsof 7th International Conference on Testbeds and Research Infras-tructure and Development of Networks and Communities, pp.302–316, 2011.

[34] E. Ertin, A. Arora, R. Ramnath et al., “Kansei: a testbed forsensing at scale,” in Proceedings of the International Symposiumon Mobile Ad Hoc Networking and Computing, pp. 399–406,2006.

[35] M. Sridharan, W. Zeng, W. Leal et al., “KanseiGenie: softwareinfrastructure for resource management and programmabilityof wireless sensor network fabrics,” in Next Generation InternetArchitectures and Protocols, pp. 275–300, Cambridge UniversityPress, 2010.

[36] K. Iwanicki, A. Gaba, and M. Steen, “KonTest: a wireless sensornetwork testbed at Vrije Universiteit Amsterdam,” Tech. Rep.IR-CS-045, Vrije Universiteit Amsterdam, Amsterdam, TheNetherlands, August 2008.

[37] P. Ballal, V. Giordano, P. Dang, S. Gorthi, J. Mireles Jr., and F.Lewis, “A LabVIEW based test-bed with off-the-shelf compo-nents for research in mobile sensor networks,” in Proceedings ofthe IEEE International Conference on Control Applications, pp.112–118, October 2006.

[38] G. Werner-Allen, P. Swieskowski, and M. Welsh, “MoteLab:a wireless sensor network testbed,” in Proceedings of the 4thInternational Symposium on Information Processing in SensorNetworks (IPSN ’05), pp. 483–488, April 2005.

[39] X. Ju, H. Zhang, and D. Sakamuri, “NetEye: a user-centeredwireless sensor network testbed for high-fidelity, robust exper-imentation,” International Journal of Communication Systems,vol. 25, no. 9, pp. 1213–1229, 2012.

[40] C. des Roziers, G. Chelius, T. Ducrocq et al., “Using SensLABas a first class scientific tool for large scale wireless sensornetwork experiments,” in NETWORKING 2011, vol. 6640 ofLecture Notes in Computer Science, pp. 147–159, Springer, Berlin,Germany, 2011.

[41] G. Z. Papadopoulos, J. Beaudaux, A. Gallais, T. Noel, and G.Schreiner, “Adding value toWSN simulation using the IoT-LABexperimental platform,” in Proceedings of 9th IEEE InternationalConference on Wireless and Mobile Computing, Networking andCommunications, pp. 485–490, October 2013.

[42] R. Crepaldi, A. Harris, A. Scarpa, A. Zanella, and M. Zorzi,“SignetLab: deployable sensor network testbed and manage-ment tool,” in Proceedings of the 4th International Conference onEmbedded Networked Sensor Systems (SenSys ’06), pp. 375–376,Boulder, Colo, USA, November 2006.

[43] J.-P. Sheu, C.-J. Chang, C.-Y. Sun, and W.-K. Hu, “WSNTB: atestbed for heterogeneous wireless sensor networks,” in Pro-ceedings of the 1st IEEE International Conference on Ubi-MediaComputing and Workshops (U-Media ’08), pp. 338–343, August2008.

[44] H. Kim, “WeaverLab : an open large-scale wireless sensor net-work testbed,” Tech. Rep. TR-2014-05, Ubiquitous InformationTechnology Research Center, Daegu University, Gyeongsan,Republic of Korea, 2014.

International Journal of

AerospaceEngineeringHindawi Publishing Corporationhttp://www.hindawi.com Volume 2014

RoboticsJournal of

Hindawi Publishing Corporationhttp://www.hindawi.com Volume 2014


Active and Passive Electronic Components

Control Scienceand Engineering

Journal of



RotatingMachinery


Hindawi Publishing Corporation http://www.hindawi.com

Journal ofEngineeringVolume 2014

Submit your manuscripts athttp://www.hindawi.com

VLSI Design



Shock and Vibration


Civil EngineeringAdvances in

Acoustics and VibrationAdvances in



Electrical and Computer Engineering

Journal of

Advances inOptoElectronics

Hindawi Publishing Corporation http://www.hindawi.com

Volume 2014

The Scientific World JournalHindawi Publishing Corporation http://www.hindawi.com Volume 2014

SensorsJournal of


Modelling & Simulation in EngineeringHindawi Publishing Corporation http://www.hindawi.com Volume 2014


Chemical EngineeringInternational Journal of Antennas and

Propagation




Navigation and Observation



DistributedSensor Networks


Documents

Review Article Experimental Research Testbeds for …downloads.hindawi.com/journals/ijdsn/2015/630210.pdfthe demanding need for their experimental veri cation and evaluation, researchers