Assessing the IEEE 802.11e QoS effectiveness in multi-hop indoor scenarios

Ad Hoc Networks 10 (2012) 186–198

Contents lists available at ScienceDirect

Ad Hoc Networks

journal homepage: www.elsevier .com/locate /adhoc

Assessing the IEEE 802.11e QoS effectiveness in multi-hop indoor scenarios

Alvaro Torres, Carlos T. Calafate *, Juan-Carlos Cano, Pietro ManzoniDepartment of Computer Engineering, Universidad Politécnica de Valencia, Camino de Vera, S/N, 46022 Valencia, Spain

a r t i c l e i n f o a b s t r a c t

Article history:Available online 21 July 2010

Keywords:IEEE 802.11eMulti-hop ad hoc networkIndoor testbedns-2 Simulator

1570-8705/$ - see front matter � 2010 Elsevier B.Vdoi:10.1016/j.adhoc.2010.07.011

* Corresponding author. Fax: +34 963877579.E-mail addresses: [email protected] (A. To

upv.es (C.T. Calafate), [email protected] (J.-C. Canupv.es (P. Manzoni).

1 Note: A preliminary version of this paper entitIEEE 802.11e testbed to validate simulation resulpublication in the 34th Annual IEEE ConferenceNetworks (LCN 2009).

The IEEE 802.11e technology is receiving much interest due to the enhancements offered towireless local area networks in terms of QoS. Other application fields for this technologyare wireless ad hoc networks, wireless mesh networks, and vehicular ad hoc networks.In the literature, most of the research works available focusing on the IEEE 802.11e tech-nology offer simulation results alone, being hard to find empirical results of implementa-tions that prove its effectiveness in realistic scenarios. Additionally, we consider thatstudies of IEEE 802.11e based on simulation platforms have not been thoroughly validatedusing real-life results. In this work we analyze the performance of the IEEE 802.11e tech-nology in real multi-hop ad hoc networks. With this purpose we first we devise a set ofexperiments where we compare the results obtained on a small testbed to those fromthe ns-2 simulation platform. A significant consistency in terms of overall trends is found,although remarkable differences can be appreciated in terms of both delay and throughputresults. Afterward we proceed with a full deployment of IEEE 802.11e enabled stationsthroughout the floor of an university building, performing several experiments using bothstatic and dynamic routing. Experimental results show that QoS can be reasonably sus-tained for both voice and video traffic in multi-hop ad hoc networks, although dynamicrouting protocols can hinder performance by provoking frequent on–off connectivityproblems.

� 2010 Elsevier B.V. All rights reserved.

1. Introduction1

Wireless ad hoc networks are packet radio networkscomposed by independent and heterogeneous stations thatcooperate in routing and packet forwarding tasks, con-forming a dynamic multi-hop network.

Interest in these networks has been growing in the lastfew years since they offer an adequate solution for militarydeployment, disaster relief scenarios, ubiquitous comput-ing environments, as well as to extend network coverage

. All rights reserved.

rres), [email protected]), pmanzoni@disca.

led ‘‘Deploying a realts” was accepted for

on Local Computer

to provide Internet access to members of large dispersecommunities (e.g. university campus, rural areas, etc.)

The proliferation of devices with multimedia and wire-less networking capabilities pave the way towards ubiqui-tous audiovisual communication among peers. Thus, anincreasing interest in supporting QoS in ad hoc networkshas arisen. To meet this need, the IEEE 802.11e [1] workinggroup has enhanced the IEEE 802.11 standard [2] toprovide QoS at the MAC layer. MAC QoS is fundamentalin contention-based wireless networks to achieve trafficdifferentiation in terms of both throughput and delay. Inwireless ad hoc networks this requirement gains furthersignificance since the complex radio interference issuesassociated with multi-hop channel access usually causeperformance to be poor.

In research works such as [3,4] authors offer an analysisof the IEEE 802.11e standard in environments where sta-tions communicate with an access point. Other works such

http://dx.doi.org/10.1016/j.adhoc.2010.07.011

mailto:[email protected]

mailto:calafate@disca. upv.es

mailto:calafate@disca. upv.es

mailto:[email protected]

mailto:pmanzoni@disca. upv.es

mailto:pmanzoni@disca. upv.es

http://dx.doi.org/10.1016/j.adhoc.2010.07.011

http://www.sciencedirect.com/science/journal/15708705

http://www.elsevier.com/locate/adhoc

A. Torres et al. / Ad Hoc Networks 10 (2012) 186–198 187

as [5,6] study the performance of the IEEE 802.11e technol-ogy in infrastructureless ad hoc networks, both static andmobile. The results presented in works such as the ones re-ferred above are based on simulation or analytical models.While such an approach is interesting for a first assessmentof the performance and viability of QoS solutions, theytend to be optimistic since they obviate several problemsassociated with real-life implementations, such as hard-ware limitations, background noise and interferences, soft-ware restrictions and, last but not the least, coding bugs inthe wireless cards’ driver.

In this paper we setup an IEEE 802.11e enabled testbedusing native Linux drivers that allows us to assess the per-formance of this technology in multi-hop ad hoc networkenvironments. We first compare the values obtained inthe testbed against simulation results obtained with thens-2 simulator. Experiments evidence some significantperformance differences in terms of throughput and delay.Afterward we deploy a real scenario in the floor of a uni-versity building, and proceed to evaluate QoS effectivenessin a realistic test case.

The rest of this paper is organized as follows: in the nextsection we refer to some related works regarding IEEE802.11e testbeds. In Section 4 we describe the characteris-tics of our testbed in terms of both hardware and softwareconfigurations. In Section 3 we briefly introduce the IEEE802.11e technology. A result comparison of testbed experi-ments vs. ns-2 simulation is then presented in Section 5. Sec-tion 6 presents a performance evaluation of IEEE 802.11e in adistributed indoor environment characterized by multiplehops and significant physical barriers using both static anddynamic routing. Finally, Section 7 concludes the paper.

2. Related works

In the literature we can find a few recent works wherethe authors setup a testbed to evaluate the performance ofthe IEEE 802.11e technology. Ng et al. [7] setup a testbed todemonstrate that the 802.11e parameters CWmin, TXOPand AIFS work largely in line with analytic and simulationpredictions. Additionally, they use the testbed to demon-strate some known problems with TCP’s performancecaused by cross-layer interaction between the TCP conges-tion control algorithm and the MAC layer CSMA/CA con-tention mechanism, as well a possible solution to thatproblem.

Dangereld et al. [8] have studied the delay in the con-text of protecting a voice call competing against data traf-fic. In particular, they present experimental measurementswhich demonstrate that the use of 802.11e allows to suc-cessfully deliver quality of service to voice traffic in amixed voice and data environment.

Banchs et al. [9] focus on applications of the EnhancedDistributed Channel Access (EDCA) proposed by the IEEE802.11e standard. Experimental results show that EDCAcan effectively be used for traffic engineering purposes.However, the goal of providing service guarantees withEDCA is shown to be more challenging.

Siris and Stamatakis [10] investigate the optimal selec-tion of minimum contention window values to achieve

proportional fairness in a multirate IEEE 802.11e test-bed. The test-bed experiments show that, when stationshave different transmission rates and the same weight,proportional fairness achieves higher performance thanthe time-based fairness approaches in terms of both aggre-gate utility and throughput.

All of the aforementioned testbeds were configured forthe infrastructure mode alone, using Atheros wirelesscards (either PCI or PCMCIA) and the MadWifi driver [11]for testing. Notice that this driver does not fully supportthe IBSS mode of operation, which has hindered research-ers from testing the benefits of the IEEE 802.11e technol-ogy in ad hoc network environments. Additionally, thedriver does not support USB devices nor most of the wire-less cards integrated in laptops.

To the best of our knowledge, our work is pioneer interms of performance assessment of the IEEE 802.11e tech-nology in a multi-hop ad hoc network environment. More-over, we rely on enhanced versions of standard wirelessdrivers for the Linux kernel that support state-of-the-artUSB and integrated wireless cards, which favors a wide-spread adoption of this technology among both research-ers and regular users.

3. Overview of IEEE 802.11e

The IEEE 802.11 standard [2] is a technology whose pur-pose is to provide wireless access to local area networks.Stations using this technology access the wireless mediumusing either the Point Coordination Function (PCF) or theDistributed Coordination Function (DCF). In particular,the Distributed Coordination Function (DCF) uses a lis-ten-before-talk scheme named carrier sense multiple access(CSMA) with collision avoidance (CA). It is used by stationsin a BSS during the CP and also by stations in an IBSS oper-ating in ad hoc mode. Although the CSMA/CA mechanismshows good adaptation to different numbers of transmit-ters, it offers no mechanisms to perform traffic differentia-tion, making QoS support practically unfeasible. The IEEE802.11e working group was created to add QoS supportto the original IEEE 802.11 standard, and in 2005 a newinternational standard [1] was released.

The IEEE 802.11e working group [1] was created to addQoS support to the original IEEE 802.11 standard, and in2005 a new international standard [1] was released. Thisstandard introduces the hybrid coordination function(HCF) which defines two new medium access mechanismsto replace PCF and DCF. These are the HCF controlled chan-nel access (HCCA) and the Enhanced Distributed Channel Ac-cess (EDCA). Concerning 802.11e enabled stations formingan ad hoc network, these must implement the EDCA algo-rithm. The 802.11e QoS support is achieved through theintroduction of different access categories (ACs), and theirassociated backoff entities.

Contrarily to the legacy IEEE 802.11 stations, where allpackets have the same priority and are assigned to a singlebackoff entity, IEEE 802.11e stations have four backoffentities (one for each AC) so that packets are sorted accord-ing to their priority. The different access categories avail-able in IEEE 802.11e stations are: voice (AC_VO), video(AC_VI), best effort (AC_BE) and background (AC_BK).

188 A. Torres et al. / Ad Hoc Networks 10 (2012) 186–198

Each backoff entity has an independent packet queueassigned to it, as well as a different parameter set. In IEEE802.11 legacy stations, this parameter set was fixed, and sothe inter-frame space was set to DIFS and the CWmin andCWmax values were set to 15 and 1023, respectively (forIEEE 802.11a/g). With IEEE 802.11e the inter-frame spaceis arbitrary and depends on the access category itself (AIF-S[AC]). We also have AC-dependent minimum and maxi-mum values for the contention window (CWmin[AC] andCWmax[AC]). Additionally, IEEE 802.11e introduces animportant new feature referred to as transmission oppor-tunity (TXOP). A TXOP is defined by a start time and aduration; during this time interval a station can delivermultiple frames consecutively without contention withother stations. This mechanism, also known as conten-tion-free bursting (CFB), increases global throughputthrough a higher channel occupation. An EDCA–TXOP (incontrast to an HCCA–TXOP) is limited by the value ofTXOPLimit, which is a parameter defined for the entireQBSS and that also depends on the AC (TXOPLimit[AC]).

Table 1 presents the default MAC parameter values forthe different ACs. Notice that smaller values for the AIFSN,CWmin and CWmax parameters result in a higher prioritywhen accessing the channel; relative to the TXOPLimit,higher values result in larger shares of capacity and, there-fore, higher priority.

The relation between AIFS[AC] and AIFSN[AC], is thefollowing:

AIFS½AC� ¼ SIFSþ AIFSN½AC� � aSlotTime; ð1Þ

where SIFS is the shortest inter-frame space possible andaSlotTime is the duration of a slot. As defined by the stan-dard, the AIFSN[AC] parameter must never be less than 2to avoid interference with normal AP operation.

For applications to take advantage of the IEEE 802.11etechnology, datagrams should have their IP Type of Service(TOS) header field set according to the desired user prior-ity. When delivered to an IEEE 802.11e enabled wirelesscard driver, those datagrams will be handled according tothe priority defined, as explained in the IEEE 802.11estandard.

4. Testbed setup

In this section we describe the hardware and the soft-ware configuration details for the real testbed being de-ployed. We also describe in detail how throughput anddelay measurements are performed. The testbed setupinformation provided in this section applies to the experi-mental results presented in Sections 5 and 6.

Table 1IEEE 802.11e MAC parameter values for a IEEE 802.11a/g radio.

Access category AIFSN CWmin CWmax TXOPLimit (ms)

AC_BK 7 15 1023 0AC_BE 3 15 1023 0AC_VI 2 7 15 3.008AC_VO 2 3 7 1.504

4.1. Hardware and software details

To create an IEEE 802.11e enabled multi-hop ad hocnetwork we relied on 11 computers, where eight of themwere low-cost Asus EeePC 901 netbooks, one was a highperformance Lenovo laptop, and the remaining two werefully equipped desktop PCs.

In terms of wireless cards, we opted for using IEEE802.11e enabled cards using the Ralink chipset [12]. Ourdecision was based on the availability of the source codefor the Ralink wireless card drivers for Linux, a basicrequirement to allow performing the required enhance-ments. In particular, on the Asus netbooks we used theirintegrated wireless card (Ralink RT 2860 chipset), and onthe remaining terminals we used either D-Link DWA-140or Linksys WUSB600N-EU wireless USB cards, both basedon the Ralink RT 2870 chipset.

All the terminals involved in the testbed had a GNU/Li-nux operating system installed, either version 2.6.27 (net-books and laptop) or 2.6.24 (desktop). The Ralink wirelesscard drivers used were version 1.7.0.0 for the RT2860 chip-set, and version 1.3.1.0 for the RT2870 chipset.2

The main problem encountered when attempting to de-ploy an infrastructureless wireless network using theaforementioned hardware was the lack of support for IEEE802.11e in the ad hoc mode of operation. Such criticalsetback evidences the lack of interest shown by hardwaremanufacturers in this issue, requiring significant enhance-ments to the original drivers in order to make EDCAchannel access operative in the infrastructureless mode.Additionally, further enhancements were made to add sup-port for the 2.6.27 Linux kernel, to allow fixing the datarate at 54 Mbit/s (a basic requirement for the experimentsof Section 5 to make results comparable with those of ns-2), as well as to correct some detected bugs.3

4.2. Throughput and delay measurements

While simulation traces allow to easily obtain all re-quired metrics, in a real testbed the process is more com-plex. In particular, measuring the throughput requiredenabling the tcpdump capture tool, based on the libpcap li-brary [13], on all traffic destinations (stations Di). The cap-tured traffic was then filtered to discriminate throughputamong the different QoS categories; finally, the values pro-vided by the different destinations were aggregated to ob-tain per-category throughput values.

Concerning delay, the measurement process is signifi-cantly more complex since sender and receiver clocks arenot synchronized. Different delay measurement possibili-ties were discussed by Dangereld et al. [8], and for thistestbed we adopted the solution proposed by Cano et al.[14], where a Fast Ethernet backbone is used to forwardtraffic back to the source, thus allowing to compute delayusing a same clock. To make these measurements straight-forward we developed our own UDP traffic generator. Thistraffic generator includes a time-stamp at the beginning of

2 These drivers are available at: http://www.ralinktech.com/ralink/Home/Support/Linux.html.

3 Drivers are available upon request.

http://www.ralinktech.com/ralink/Home/Support/Linux.html

http://www.ralinktech.com/ralink/Home/Support/Linux.html

0

0.05

0.1

0.15

0.2

0.25

0.3

0 10 20 30 40 50 60 70 80 90 100

End-

to-e

nd D

elay

(s)

Load (Mbit/s)

Fig. 1. Load vs. delay values for the Fast-Ethernet link used as return path.

S1

S2

S3

S4

D1

D2

D3

D4

Fig. 2. Multi-hop ad hoc network scenario used for testing.

Table 2Characteristics of the wireless channelselected (obtained with Wi-Spy 2.4 GHz spec-trum analyzer).

Parameter Value range

Average floor (dBm) �85.94Average peak (dBm) �79.13Duty cycle 11.06


the packet’s payload and, when receiving it again throughthe feedback Ethernet channel, clock differences allow tocompute delay.

To make sure that the Fast Ethernet return path did notaffect delay measurements in a significant manner we cal-culated the additional delay introduced by the wired chan-nel for different load values. The experimental results areshown in Fig. 1.

The maximum load measured on the return channelduring our testbed experiments never surpassed 5 Mbit/s,which means that the additional delay introduced by theFast-Ethernet connection is rather small. Nevertheless,we used these results to obtain unbiased estimations ofall delay values, thus measuring the delay associated withthe wireless counterpart much more precisely.

5. Comparative analysis of simulation and testbedresults

In this section we present a performance assessmentstudy of the IEEE 802.11e technology in a multi-hop adhoc network environment where we compare simulationresults against the results obtained in our testbed.

We have split our analysis into two sets of experiments.In the first set of experiments (Section 5.2) our purpose isto assess the traffic differentiation capabilities of the EDCAchannel access mechanism in the proposed multi-hop adhoc network. In the second set of experiments (Section5.3) our goal is to determine the degree of robustnessand stability that the two highest access categories (videoand voice) experience when varying the amount of best ef-fort and background traffic.

5.1. Scenario definition

Fig. 2 shows the multi-hop ad hoc network scenario wehave selected for our comparative study. The scenario in-cludes four source–destination pairs, as well as three inter-mediate stations, thus conforming a total of 11 stations.Unless stated otherwise, all traffic sources are set to gener-ate the same data rate in all four ACs, and the traffic typechosen is constant bitrate (CBR) with packet size fixed at

512 bytes to fully control the load being injected. In ourexperiments we avoid synchronization among trafficsources to prevent results from being biased towards aworst case scenario; this was achieved by adding a smalljitter to the inter-packet interval.

The simulation results used for comparison where ob-tained using the ns-2 simulator [15]. It was configuredwith the same scenario, and all nodes have an IEEE802.11g radio operating at a fixed data rate of 54 Mbit/s.The distance between stations that are one hop away isof 200 m, and simulation takes place within an area sized1000 � 400 m. Notice that the carrier sensing range is of550 m, meaning that interferences make spatial reuse tobe significantly reduced (in particular, it is null for the cen-ter station).

For our testbed we could not use exactly the same set-tings due to space and radio range restrictions. In fact, tooperate efficiently with IEEE 802.11g cards at a fixed datarate of 54 Mbit/s, stations must be at most a few metersaway. Thus, in our testbed all the stations are within thesame room (separated by about 1 m), and the topologyshown in Fig. 2 was enforced logically through an adequateconfiguration of the firewall using the iptables tool [16].Despite the total channel capacity of the testbed is reducedwith respect to the simulator, we consider that it offers thebest trade-off when assuming basic feasibility require-ments. Nevertheless, when performing result comparisons,we shall take this restriction into consideration.

Due to the presence of other sources of noise in the2.4 GHz band, we used the Wi-Spy spectrum analyzer [17]to aid us in determining the best channel for experimenta-tion. The characteristics of the channel used are shown inTable 2 for the sake of completeness. Those values wereobtained with all the stations participating in the testbedturned off.

We ran the same set of tests using both the wirelesstestbed described and the ns-2 simulator. All experimentsrun for 120 s, being that the first and the last 10 s of eachexperiment were discarded. The chosen performance met-rics are the throughput and the end-to-end delay values.


Each experiment was repeated 20 times, which allowed usto determine, with a degree of confidence of 95%, that themaximum error for the mean is under 5% for all the valuespresented in this section.

With respect to the IEEE 802.11e technology, the sta-tions used in our testbed adopt the EDCA channel accessparameters shown in Table 1, and the contention-freebursting (CFB) mechanism is turned off. The same appliesto virtual stations participating in the ns-2 based simula-tion experiments. Ns-2 support for IEEE 802.11e wasachieved by installing the extensions developed by Wie-tholter and Hoene [18].

5.2. Traffic differentiation analysis

When a network is very lightly loaded QoS mechanismsare both ineffective and unnecessary. However, as we in-crease traffic to moderate/high loads, QoS mechanisms be-come fundamental to discriminate among the differenttraffic categories. Therefore, in the current set of experi-ments, we gradually increase the traffic load by the sameamount for all four MAC categories defined by the IEEE802.11e standard.

Fig. 3 shows the effectiveness of the IEEE 802.11e tech-nology in terms of traffic differentiation at different loads.Comparing the results obtained with the simulator withthose obtained in our testbed allows us to draw severalconclusions: (i) In a real testbed the throughput valuesachieved before reaching the saturation points do not al-ways present a matching with the offered load due to morefrequent losses; this is especially noticeable for the voiceand video traffic categories, and is typically associated withchannel noise and external sources of interference. (ii) Inthe testbed the throughput for all traffic types decays moreabruptly when the saturation point for each access cate-gory is reached; additionally, we also observe that the par-ticular saturation point for each access category is reducedcompared to simulation results. Notice that similar differ-ences have also been found by other authors when com-paring simulation results and analytical models [19],which suggests that physical and MAC implementationsin simulators lack realism. In our case, however, this effectis also related to the lower channel capacity of the testbed

0

0.5

1

1.5

2

2.5

3

0 0.5 1 1.5 2 2.5 3 3.5 4

Thro

ughp

ut p

er A

C (M

bit/s

)

Offered traffic per AC (Mbit/s)

AC_BK (Background)AC_BE (Best effort)

AC_VI (Video)AC_VO (Voice)

Offered traffic per AC

Thro

ughp

ut p

er A

C (M

bits

/s)

Fig. 3. Throughput values achieved via simu

environment due to the overlapping in terms of radio cov-erage and the presence of beacons. (iii) The traffic differen-tiation effectiveness of IEEE 802.11e is partially lost in thetestbed compared to the simulation case, which allows toavoid starvation of best effort and background traffic byreducing QoS traffic (mostly in the video access category).

Fig. 4 shows the results for this same set of experi-ments, but now focusing on the end-to-end delay metricinstead. As expected, the most significant performance is-sues referred above guard a strict relationship with theend-to-end performance. In particular, we should remarkthat: (i) contrarily to the simulation experiments, the delayvalues obtained in the testbed experience a much moresteep growth for low load values (notice that a logarithmicscale is used for the vertical axis) and (ii) when reachingmoderate/high saturation levels, the differentiation capa-bilities of IEEE 802.11e in the testbed experience a signifi-cant loss of effectiveness compared to simulation results.In particular, average delay values for video and voice traf-fic are increased by one order of magnitude; such differ-ence should be considered quite significant and takeninto consideration when designing QoS architectures basedon the IEEE 802.11e technology.

5.3. QoS robustness analysis

We now present the results related to the second set ofexperiments, where the aim is to assess whether the IEEE802.11e technology is able to offer steady performance lev-els to active QoS connections when increasing non-QoStraffic. With this purpose we fix the aggregated load forthe video and voice traffic at 1 and 0.5 Mbit/s, respectively,increasing load levels for the best effort and backgroundtraffic categories alone.

Fig. 5 shows the throughput results obtained using thewireless testbed and the ns-2 simulator. It is noticeablethat, in the testbed, the throughput values for video andvoice traffic are not maintained stable, experiencing packetlosses of up to 11.5% and 11.8%, respectively. This is in clearcontrast with simulation results, where throughput levelsare maintained mostly unaltered for all load values. As asecondary issue, we noticed that the throughput valuesfor best effort traffic under saturation differ greatly,

0

0.5

1

1.5

2

2.5

3

0 0.5 1 1.5 2 2.5 3 3.5 4Offered traffic for Best effort and Background ACs (Mbit/s)

AC_BK (Background)AC_BE (Best Effort)


Offered Traffic per AC (Mbit/s)

lation (left) and in our testbed (right).

0.001

0.01

0.1

1

10

100

0 0.5 1 1.5 2 2.5 3 3.5 4

End-

to-e

nd d

elay

per

AC

(s)



AC_VI (Video)AC_VO (Voice) 0.001

0.01

0.1

1

10

100

0 0.5 1 1.5 2 2.5 3 3.5 4

End-

to-e

nd d

elay

per

AC

(s)




Fig. 4. End-to-end delay values achieved via simulation (left) and in our testbed (right).

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

0 0.5 1 1.5 2 2.5 3 3.5 4

Thro

ughp

ut p

er A

C (M

bit/s

)

Offered traffic for Best effort and Background ACs (Mbit/s)



Offered traffic (AC_BE and AC_BK)

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

0 0.5 1 1.5 2 2.5 3 3.5 4

Thro

ughp

ut p

er A

C (M

bits

/s)




Offered Traffic per AC (Mbit/s)

Fig. 5. Throughput stability for voice and video traffic when increasing best effort and background traffic. Data from simulation (left) and from the testbed(right).


leveling out at 2.4 Mbit/s in the ns-2 simulator (not visibledue to scale), while in the testbed they level-out at about0.9 Mbit/s. The observed differences, are mainly due tothe lower overall channel capacity of the testbed environ-ment, as explained earlier.

In terms of end-to-end delay, the results presented inFig. 6 guard some resemblance with the ones of Fig. 4,and again evidence the difficulties at maintaining reduceddelay values for video and voice traffic in a real testbed.The most noticeable difference is detected in the testbedresults at load values close to 0.7 Mbit/s, where a sharp in-crease in the delay values of all traffic categories occurs. Inparticular, video and voice traffic experience an increase ofabout two orders of magnitude compared with simulationresults, while the increase for both best effort and back-ground traffic is limited to one order of magnitude. Thus,the high degree of delay stability obtained via simulationis lost. In terms of absolute values, the maximum delay val-ues achieved in the testbed for video traffic (up to 460 ms)can be considered acceptable for real-time videoconferenc-ing applications, though close to the upper limit. Concern-ing voice sources, the delay values of up to 380 ms are

considered prohibitively high for the support of VoIPtraffic.

6. Evaluating IEEE 802.11e in a realistic indoor scenario

The results presented in the previous section wheremade with all terminals concentrated in a same room. Thisscenario allowed us to use IEEE 802.11g setting the datarate at 54 Mbit/s as intended, thus achieving the highestpossible degree of similitude with the simulation settings.

In this section we drop the constraints set in the previ-ous section, allowing to take full advantage of the wirelesscards by (i) enabling the IEEE 802.11n mode, althoughmaintaining channel width at 20 MHz and taking frequen-cies from the 2.4 GHz band alone and (ii) enabling theauto-rate adjustment mode, used by default in 802.11,whereby the network interface will automatically pickthe best data rate possible according to Signal-to-Noise Ra-tio (SNR) conditions.

The objective of the experiments was twofold: on theone hand we wanted to assess the performance differences

0.001

0.01

0.1

1

10

0 0.5 1 1.5 2 2.5 3 3.5 4

End-

to-e

nd d

elay

per

AC

(s)




0.001

0.01

0.1

1

10

0 0.5 1 1.5 2 2.5 3 3.5 4

End-

to-e

nd d

elay

per

AC

(s)




Fig. 6. End-to-end delay stability for voice and video traffic when increasing best effort and background traffic. Data from simulation (left) and from thetestbed (right).


experienced in a realistic scenario where greater distancesbetween nodes, along with physical barriers, caused theSNR to be quite lower than the overlapped coverage sce-nario tested before (same room). On the other hand, wewanted to compare static against dynamic routing to eval-uate how route selection and route variability impactedQoS performance.

6.1. Node deployment and configuration issues

For our tests we deployed eight Asus EEEPC laptopsthroughout the floor of an university building, enforcingmulti-hop connectivity in a natural manner due to distanceand the presence of brick walls, wooden doors, doubleglass windows and metal filing cabinets in each room.

Fig. 7 shows a detailed map of the third (and last) floorof our School of Informatics at the UPV in Spain. Dark dotstagged with letters represent mobile terminals, and thethick lines connecting them represent the wireless links;only ‘‘good” links, that is, links which are sustained in time,are represented. As expected, physical obstacles become aserious hindrance to signal propagation in the 2.4 GHzband; in particular, notice that link C–G, which mainly tra-verses open space through a corridor, is the longest andalso the weakest link.

Fig. 7. Map of the building floor where tests where made and node deploy

For the experiments that follow we adopted a strategysimilar to the previous section, whereby a wired networkis used for station management and control purposes.The wired network is Fast-Ethernet based, and connectsstations with a control application running on a remotePC through SSH.

With respect to traffic generation, in our tests we willevaluate the QoS conditions experienced by a bidirectionalvideoconferencing session where synthetic video and voicetraffic are generated as independent UDP flows so as totake advantage of the video and voice MAC access catego-ries of IEEE 802.11e. The video data rate is set to 1 Mbit/s ineach direction, while voice data rate is set to 15 kbit/s ineach direction.

In addition to QoS traffic, we also start an FTP/TCP flowbetween both endpoints involved in the videoconferenceas best effort traffic. The purpose of background traffic istwofold: (i) to increase the degree of congestion to matcha more realistic situation and (ii) to take advantage of TCP’sbandwidth greediness to have an estimate of the residualbandwidth available for best effort traffic on the end-to-end connection.

When static routing is used we fill-in the routing tablesof all nodes according to the shortest path criteria based onthe topology shown in Fig. 7. Dynamic routing tests use the

ment details. Approximate floor dimensions (maximum): 60 � 40 m.

Table 3Configuration details for the OLSR routingdaemon.

Parameter Value

HelloInterval 2.0 sHelloValidityTime 6.0 sTcInterval 3.5 sTcValidityTime 10.0 sTosValue 16


well-known OLSR protocol [20] instead. The configurationdetails for this protocol are presented in Table 3. For de-tailed information about the meaning of each parameterplease refer to the RFC [20].

All the results presented in the sections that follow areaverage values where we take the results from 10 experi-ments and discard the two best and the two worst ones,retrieving the mean for the remaining six. Each of ourexperiments lasts for about 120 s.

6.2. Performance results

We now present the performance results obtained inour indoor test scenario. For UDP traffic (video and voice)

20

100

250 500

1000 1800

AB BC CD CG DE EF FG GH

UD

P D

elay

(ms)

Link Identifier

VoiceVideo

Fig. 9. Video and voice end-to-end delay values for all source–destinatio

0

20

40

60

80

100

AB BC CD CG DE EF FG GH 0

5

10

15

20

25

30

35

40

UD

P Pa

cket

Del

iver

y R

atio

(%)

TCP

Thro

ughp

ut (M

bits

)

Link Identifier

VoiceVideo

TCP Traffic

Fig. 8. UDP packet delivery ratio (video and voice) and TCP throughput (bestdynamic (right) routing.

we obtain the packet delivery ratio and the end-to-end de-lay, while for TCP traffic we obtain the throughput.

Fig. 8 shows the results achieved at one hop. We can seethat C–G is the weakest link due to the greater distance be-tween stations. On the contrary, G–H and B–C are the linksachieving the best performance. This can be seen from bothcharts by noticing that the UDP packet delivery ratio isnearly optimum, while TCP traffic also achieves an impor-tant share of bandwidth (up to 37 Mbit/s for the B–C link).Notice that, according to the IEEE 802.11 standard [2], allunicast packets benefit from up to 4 or 7 retransmissionsfor packet sizes below and greater than the RTS/CTSthreshold, respectively. Thus, although only moderate lossrates are experienced, they indicate that most transmis-sions in the C–G link are failing.

By comparing static against dynamic routing we canappreciate significant differences for links A–B, C–G andD–E, especially in terms of video traffic. Since the deliveryratio for voice is sustained, the losses experienced cannotbe related to route failure, being instead related to channelcontention caused by the increase of routing-relatedcontrol traffic. Moreover, since most OLSR packets arebroadcasted, the IEEE 802.11 standard indicates that thetransmission rate must be set to the minimum allowed(either 1 or 6 Mbit/s for 802.11g, depending on the PHY

20

100

250 500

1000 1800

AB BC CD CG DE EF FG GH

UD

P D

elay

(ms)

Link Identifier

VoiceVideo

n pairs at one hop using static (left) and dynamic (right) routing.

0

20

40

60

80

100

AB BC CD CG DE EF FG GH 0

5

10

15

20

25

30

35

40

UD

P Pa

cket

Del

iver

y R

atio

(%)

TCP

Thro

ughp

ut (M

bits

)Link Identifier

VoiceVideo

TCP Traffic

effort) for all source–destination pairs at one hop using static (left) and

0

20

40

60

80

100

120

AC BD BG CE CF CH DF DG EG FH 0

1

2

3

4

5

6U

DP

Pack

et D

eliv

ery

Rat

io (%

)

TCP

Thro

ughp

ut (M

bits

)

Connection Identifier

VoiceVideo

TCP Traffic

0

20

40

60

80

100

AC BD BG CE CF CH DF DG EG FH 0

1

2

3

4

5

6

UD

P Pa

cket

Del

iver

y R

atio

(%)

TCP

Thro

ughp

ut (M

bits

)


VoiceVideo

TCP Traffic

Fig. 10. UDP packet delivery ratio (video and voice) and TCP throughput (best effort) for all source–destination pairs at two hops using static (left) anddynamic (right) routing.


operation mode). Therefore, although the routing overheadis small, the total transmission time for broadcasted pack-ets will be quite high, thus affecting other overlapping datatransmissions for a longer period.

Delay results (see Fig. 9) evidence the effectiveness of802.11e at serving voice traffic with lower delay in bothscenarios. This is mostly due to the lower values adoptedfor inter-frame space and the contention window (bothCWmin and CWmax) for this access category. Also, we canappreciate an inverse proportionality between TCPthroughput and delay, which is expected since higherchannel saturation is usually associated with higher de-lays. The only exception detected occurs for the most prob-lematic link (C–G), where good delay values are detecteddespite the higher loss rate. An in-depth analysis has re-vealed that this was due mostly to loss of network cardsynchronization between terminals C and G during opera-tion, which caused intermittent connectivity problems.When the link was fully operational, though, the perfor-mance was good.

The results for the two-hop experiments are shown inFig. 10. For the static routing scenario we observe thatthe connections associated with endpoints BG, CF, CH,

10

50 100 250 500

1000 2000 4000

10000

AC BD BG CE CF CH DF DG EG FH

UD

P D

elay

(ms)


VoiceVideo


and DG experience a worse performance than the rest.This occurs because all of them make use of the C–G linkto reach the destination, which was shown to offer theworst performance in the previous section. The remainingconnections experience a reasonably good performance,although increased channel contention now provokes morelosses. The most representative situation is observed forthe AC connection, where QoS traffic suffers from lossesbetween 1% and 2%, although the amount of bandwidthavailable is enough to achieve a throughput of about5.3 Mbit/s.

When analyzing the dynamic routing scenarios we canobserve that, again, performance is degraded comparedto the static routing case. We must take into considerationthat now OLSR is dynamically updating routes, being pos-sible that the route used at any given test differs from thestatic routing test. By analyzing Fig. 10 two situations aredetected: in most cases we observe a trend similar to thestatic routing scenario, with the difference that perfor-mance is slightly inferior for voice traffic, and significantlyinferior for video traffic. However, there are a few cases(connections DF and EG) where the routing protocol used(OSLR) is not able to sustain the route during the entire test

10

50 100 250 500

1000 2000 4000

10000

AC BD BG CE CF CH DF DG EG FH

UD

P D

elay

(ms)


VoiceVideo

n pairs at two hops using static (left) and dynamic (right) routing.

0

20

40

60

80

100

AD AG BE BF BH DH EH 0

0.5

1

1.5

2

2.5

UD

P Pa

cket

Del

iver

y R

atio

(%)

TCP

Thro

ughp

ut (M

bits

)


VoiceVideo

TCP Traffic

0

20

40

60

80

100

AD AG BE BF BH DH EH 0

0.5

1

1.5

2

2.5

UD

P Pa

cket

Del

iver

y R

atio

(%)

TCP

Thro

ughp

ut (M

bits

)


VoiceVideo

TCP Traffic

Fig. 12. UDP packet delivery ratio (video and voice) and TCP throughput (best effort) for all source–destination pairs at three hops using static (left) anddynamic (right) routing.


period, thus causing the performance to be much worsethan for the static routing case.

Two-hop delay values (see Fig. 11) show that, similarlyto the one-hop results, dynamic routing will often produceresults that are much worse that for the static routing case(notice the logarithmic scale). Also, we verify that there isno longer a strict relationship between UDP delivery ratioor TCP throughput and delay values. This is particularlynoticeable for the dynamic routing case, where connec-tions CF and CH show little differences in terms of loss ra-tio, and significant differences for delay (especially forvoice).

Similar degradation trends were found for the three andfour-hops cases (see Figs. 12 and 14). It is also interestingto see that connections avoiding the weakest link (C–G)are able to maintain video and voice loss levels at a mini-mum, with a packet arrival ratio near 100%. To confirm thisjust check connections AD, BE and EH for the three-hopscase, as well as connection AE for the four-hops case. Suchresults clearly show that the IEEE 802.11e technology isable to sustain QoS at high levels, even when operating

10

100

250 500

1000 2000 4000

10000

AD AG BE BF BH DH EH

UD

P D

elay

(ms)


VoiceVideo


in a multi-hop fashion, as long as none of the links is weak,that is, as long as the data rate used in those links is suffi-ciently high to avoid it becoming a bottleneck. Otherwise,performance will drop down to unacceptable levels, withQoS traffic experiencing significant losses.

Focusing on the differences between static and dynamicrouting for the three and four-hops cases, we find that, asthe number of hops grows, route instability will tend to in-crease. In particular, the results we obtained show that thepacket delivery ratio is always below 90% when transmit-ting at a three hop distance, and it is below 60% whentransmitting at a four hop distance.

TCP-based best effort traffic is also greatly affected byroute instability, suffering from starvation (throughput of12 kbit/s in the best case) when transmitting at a four-hop distance.

Concerning delay results, Figs. 13 and 15 show that QoSperformance reaches critical levels. In fact, if the traffic hasreal-time constraints, video flows are in most cases experi-encing a delay considered too high (above 250 ms). Atfour-hops, neither solution offers acceptable QoS levels to

10

100

250 500

1000 2000 4000

10000

AD AG BE BF BH DH EH

UD

P D

elay

(ms)


VoiceVideo

n pairs at three hops using static (left) and dynamic (right) routing.

0

20

40

60

80

100

AE AF AH 0

0.2

0.4

0.6

0.8

1U

DP

Pack

et D

eliv

ery

Rat

io (%

)

TCP

Thro

ughp

ut (M

bits

)


VoiceVideo

TCP Traffic

0

20

40

60

80

100

AE AF AH 0

0.2

0.4

0.6

0.8

1

UD

P Pa

cket

Del

iver

y R

atio

(%)

TCP

Thro

ughp

ut (M

bits

)


VoiceVideo

TCP Traffic

Fig. 14. UDP packet delivery ratio (video and voice) and TCP throughput (best effort) for all source–destination pairs at four-hops using static (left) anddynamic (right) routing.

UD

P D

elay

(ms)


UD

P D

elay

(ms)


10

100

500 1000 2000 4000 7000

AE AF AH

VoiceVideo

10

100

500 1000 2000 4000 7000

AE AF AH

VoiceVideo

Fig. 15. Video and voice end-to-end delay values for all source–destination pairs at four hops using static (left) and dynamic (right) routing.


video or voice traffic, contrarily to what occurs in simula-tion at these hop distances, as was shown in Section 5. Dra-matic performance differences are also detected whencomparing delay for the static and dynamic routing cases,especially at three hops. We find that OLSR-based dynamicrouting will cause performance to be prohibitive for real-time streaming in terms of both loss ratio and delay values.Notice that a higher hop count is associated with a higherutilization, which increases the chances that routing pack-ets are dropped, thus causing end-to-end connections to bebroken during long periods (several seconds).

All the results presented above clearly show that rout-ing protocols for MANETs still require important improve-ments in order to achieve the stability and efficiencyrequired to operate in a multi-hop ad hoc network. In par-ticular, OSLR presents route misbehavior in indoor scenar-ios even in the absence of mobility.

7. Conclusions and future work

In this paper we assessed the performance of the IEEE802.11e technology in a multi-hop ad hoc network testbed.

We first performed a comparison of testbed resultsagainst the simulation outcome, as provided by the ns-2simulator in a specific setting. Our analysis focused ontwo different issues: (i) assessing the traffic differentiationcapabilities of the EDCA channel access mechanism in amulti-hop ad hoc network and (ii) determining the degreeof QoS robustness experienced by video and voice trafficcategories when varying the amount of best effort andbackground traffic. With respect to the first issue, experi-mental results showed a significant consistency in termsof overall performance trends. However, the traffic differ-entiation effectiveness of IEEE 802.11e is partially lost inthe testbed when compared to the simulation case in termsof both delay and throughput results. Concerning the sec-ond issue, we found that, contrarily to simulation results,the throughput values for video and voice traffic obtainedin the testbed are not stable when increasing the trafficfrom lower categories (best effort and background), experi-encing packet losses of up to 11.5% and 11.8%, respectively.In terms of delay, results showed that the high degree ofstability obtained via simulation is no longer achieved; infact, delay values for QoS traffic experience an increase ofabout two orders of magnitude compared to simulation re-


sults, achieving delay values up to 460 ms and 380 ms forvideo and voice traffic, respectively. In general we foundthat simulation-based results are always optimistic com-pared to real world performance values.

We proceed with our study by deploying an indoor adhoc network where terminals were distributed throughoutthe floor of an university building sized 60 � 40 m, and witha significant amount of physical obstacles. For this scenariowe relied on IEEE 802.11n radio communications for im-proved performance. Experimental results for a bidirec-tional videoconference between different pairs of stationshave showed that the IEEE 802.11e is able to retain its effec-tiveness, despite the multi-hop constraint, for communica-tion up to four-hops, as long as the quality of all links isreasonably good. However, delay values become excessivefor connections at more than three hops. Our analysis alsoshowed that, compared to static routing, OLSR-based dy-namic routing is prone to cause intermittent connectivityproblems that hinder QoS, despite no mobility is introduced.Such routing problems have in impact both on QoS traffic,which experiences significant delay and losses, and onTCP-based best effort traffic, which suffers from starvation.

Overall, we consider that to successfully achieve highlevels of QoS support when deploying real multi-hop adhoc networks based on IEEE 802.11e there are still severaltechnical challenges that must be solved. The most criticalare a stable and QoS-aware routing protocol implementa-tion, distributed admission control systems, and 802.11interface drivers that are optimized for the ad hoc modeof operation. Hence, in future works we will address theseresearch challenges.

Acknowledgments

This work was partially supported by the Ministerio deEducación y Ciencia, Spain, under Grant TIN2008-06441-C02-01, by the Generalitat Valenciana under Grant GV/2009/010.

References

[1] IEEE 802.11 WG, 802.11e IEEE Standard for Information Technology-Telecommunications and Information Exchange Between Systems –Local and Metropolitan Area Networks – Specific Requirements Part11: Wireless LAN Medium Access Control (MAC) and Physical Layer(PHY) specifications: Amendment 8: Medium Access Control (MAC)Quality of Service Enhancements, 2005.

[2] IEEE 802.11 WG, International Standard for Information Technology –Telecommunications and Information Exchange Between Systems –Local and Metropolitan Area Networks – Specific Requirements – Part11: Wireless Medium Access Control (MAC) and Physical Layer (PHY)Specifications, ISO/IEC 8802-11:1999(E) IEEE Std. 802.11, 1999.

[3] X. Chen, H. Zhai, X. Tian, Y. Fang, Supporting QoS in IEEE 802.11ewireless LANs, IEEE Transactions on Wireless Communications 5 (8)(2006) 2217–2227.

[4] Zhifeng Tao, S. Panwar, Throughput and delay analysis for the IEEE802.11e enhanced distributed channel access, IEEE Transactions onCommunications 54 (4) (2006) 596–603.

[5] Carlos T. Calafate, Pietro Manzoni, Manuel P, Malumbres, Assessingthe effectiveness of IEEE 802.11e in multi-hop mobile networkenvironments, in: 12th IEEE International Symposium on Modeling,Analysis, and Simulation of Computer and TelecommunicationSystems (MASCOTS’04), Volendam, Netherlands, October 2004.

[6] K. Duffy, D.J. Leith, T. Li, D. Malone, Improving fairness in multi-hopmesh networks using 802.11e, in: 4th International Symposium on

Modeling and Optimization in Mobile, Ad Hoc and WirelessNetworks, 2006, pp. 1–8.

[7] Anthony C.H Ng, David Malone, Douglas J. Leith, Experimentalevaluation of TCP performance and fairness in an 802.11e test-bed,in: 2005 ACM SIGCOMM Workshop on Experimental Approaches toWireless Network Design and Analysis, ACM Press, New York, NY,USA, 2005, pp. 17–22.

[8] Ian Dangereld, David Malone, Douglas J. Leith, Experimental evaluationof 802.11e EDCA for enhanced voice over WLAN performance, in: 4thInternational Symposium on Modeling and Optimization in Mobile, AdHoc and Wireless Networks, IEEE, 2006, pp. 1–7.

[9] A. Banchs, A. Azcorra, C. Garcia, R. Cuevas, Applications andchallenges of the 802.11e EDCA mechanism: an experimentalstudy, IEEE Network 19 (4) (2005) 52–58.

[10] Vasilios A. Siris, George Stamatakis, Optimal CWmin selection forachieving proportional fairness in multi-rate 802.11e WLANs:test-bed implementation and evaluation, in: 1st InternationalWorkshop on Wireless Network Testbeds, ExperimentalEvaluation & Characterization, ACM Press, New York, NY, USA,2006, pp. 41–48.

[11] MadWifi.net. <http://madwifi.org/> (accessed 30.01.09).[12] Ralink Technology Corporation. <http://www.ralinktech.com/>

(accessed 30.01.09).[13] V. Jacobson, C. Leres, S. McCanne. The Libpcap Packet Capture

Library, Lawrence Berkeley Laboratory, Berkeley, CA. <http://www.tcpdump.org>.

[14] Juan-Carlos Cano, José-Manuel Cano, Carlos T. Calafate, EvaGonzález, Pietro Manzoni, Trade-off between power consumptionand performance in bluetooth, Ad Hoc & Sensor Wireless Networks 8(1–2) (2009) 141–159.

[15] K. Fall, K. Varadhan, ns Notes and Documents, The VINT Project, UCBerkeley, LBL, USC/ISI, and Xerox PARC, February 2000.

[16] The netfilter.org iptables project. <http://www.netfilter.org/>(accessed 28.01.09).

[17] MetaGeek Wi-Spy. <http://www.wi-spy.com/> (accessed 31.03.10).[18] Sven Wietholter, Christian Hoene, Design and Verification of an IEEE

802.11e EDCF Simulation Model in ns-2.26, Technical Report TKN-03-019, Telecommunication Networks Group, TechnischeUniversitat Berlin, November 2003.

[19] Michele Garetto, Theodoros Salonidis, Edward W. Knightly,Modeling per-flow throughput and capturing starvation in CSMAmulti-hop wireless networks, in: Proc. IEEE Infocom, 2006.

[20] T. Clausen, P. Jacquet, Optimized link state routing protocol (OLSR),Request for Comments 3626, MANET Working Group, inpreparation. <http://www.ietf.org/rfc/rfc3626.txt>.

Alvaro Torres graduated Computer Scince atthe Technical University of Valencia, Spain, in2009. In 2008 he joined the ComputerNetworks Group (GRC) as a research assistant.He is currently pursuing the MSc degree inComputer Engineering. His research activity isrelated to multimedia networking (audio/video coding and network delivery) and wire-less technologies.

Carlos T. Calafate graduated with honors inElectrical and Computer Engineering at theUniversity of Oporto (Portugal) in 2001. Hereceived his Ph.D. degree in Computer Engi-neering from the Technical University ofValencia in 2006, where he works as anassistant professor since 2005. He is a mem-ber of the Computer Networks Group (GRC).His research interests include mobile andpervasive computing, security and QoS onwireless networks, as well as video coding andstreaming.

http://madwifi.org/

http://www.ralinktech.com/

http://www.tcpdump.org

http://www.netfilter.org/

http://www.wi-spy.com/

http://www.ietf.org/rfc/rfc3626.txt


Juan-Carlos Cano is an associate professor inthe Department of Computer Engineering atthe Technical University of Valencia (UPV) inSpain. He received the MSc and a Ph.D.degrees in Computer Science from the UPV in1994 and 2002, respectively. Between 1995–1997 he worked as a programming analyst atIBMs manufacturing division in Valencia. Hiscurrent research interests include poweraware routing protocols for mobile ad hocnetworks and pervasive computing.

Pietro Manzoni is an associate professor ofcomputer science at the Technical Universityof Valencia (SPAIN). He received the M.S.degree in Computer Science from the ‘‘Uni-versità degli Studi” of Milan (ITALY) in 1989and the Ph.D. in Computer Science from thePolytechnic University of Milan (ITALY) in1995. His research activity is related to wire-less networks protocol design, modeling andimplementation.