ICST Transactions Preprint On the Experimental …ICST Transactions Preprint On the Experimental Evaluation of Vehicular Networks: Issues, Requirements and Methodology Applied to a

ICST Transactions Preprint

On the Experimental Evaluation of VehicularNetworks: Issues, Requirements and MethodologyApplied to a Real Use CaseManabu Tsukada12,∗, José Santa34, Satoshi Matsuura5, Thierry Ernst6, Kazutoshi Fujikawa7

1INRIA Paris - Rocquencourt, Domaine de Voluceau Rocquencourt - B.P. 105 78153 Le Chesnay Cedex, France2The University of Tokyo, 1-1-1, Yayoi, Bunkyo-ku, Tokyo, 113-8656 Japan3University Centre of Defence at the Spanish Air Force Academy, MDE-UPCT , Murcia, Spain4University of Murcia, Campus de Espinardo, 30100 Murcia, Spain5Tokyo Institute of Technology, 2-12-1, Ookayama, Meguro-ku, Tokyo, 152-8850, Japan6Centre de Robotique, MINES ParisTech, Paris, France7Nara Institute of Science and Technology, Nara, Japan

Abstract

One of the most challenging fields in vehicular communications has been the experimental assessment ofprotocols and novel technologies. Researchers usually tend to simulate vehicular scenarios and/or partiallyvalidate new contributions in the area by using constrained testbeds and carrying out minor tests. In this line,the present work reviews the issues that pioneers in the area of vehicular communications and, in general,in telematics, have to deal with if they want to perform a good evaluation campaign by real testing. Thekey needs for a good experimental evaluation is the use of proper software tools for gathering testing data,post-processing and generating relevant figures of merit and, finally, properly showing the most importantresults. For this reason, a key contribution of this paper is the presentation of an evaluation environmentcalled AnaVANET, which covers the previous needs. By using this tool and presenting a reference case ofstudy, a generic testing methodology is described and applied. This way, the usage of the IPv6 protocol overa vehicle-to-vehicle routing protocol, and supporting IETF-based network mobility, is tested at the same timethe main features of the AnaVANET system are presented. This work contributes in laying the foundations fora proper experimental evaluation of vehicular networks and will be useful for many researchers in the area.

Keywords: Experimental Evaluation, Vehicular Ad-hoc Networks, Wireless Multihop Communication, NetworkMobility, Cooperative ITS, Intelligent Transportation Systems

1. IntroductionIntelligent Transportation Systems (ITS) are systemsdeployed to optimize the road traffic and realize safe,efficient and comfortable human mobility. There area number of research fields in ITS but cooperativeITS and vehicular communications have receivedan especial attention during the last years. Withinthis area various technologies are considered, suchas wireless communications, network management,communication security, navigation, etc. In cooperativeITS, multiple entities share information and tasks toachieve common objectives. Thus, data exchange existsamong vehicles, roadside infrastructure, traffic controlcenters, road users, road authorities and road operators,to support drivers, pedestrians, road authorities and

∗Corresponding author. Email: [email protected]

operators in different areas of safety, traffic efficiencyand infotainment. The European Commission (EC),for instance, published the action plan [1] in Europefollowed by ITS standardization mandate [2], to speedup the adoption of these systems in the EuropeanUnion, but there are a number of initiatives worldwideto encourage the research and development in ITS,mainly from the US Department of Transport and theJapan Ministry of Land, Infrastructure, Transport andTourism.

There are few barriers in the global road networkamong countries, and vehicles easily cross country bor-ders, especially in Europe. Thus there is a huge neces-sity that cooperative ITS relies on the same architec-ture, protocols and technologies. As such, standardiza-tion organizations are developing cooperative ITS stan-dards. The International Organization for Standardiza-tion (ISO) Technical Committee 204 Working Group

1ICST Transactions Preprint

arX

iv:1

504.

0442

6v1

[cs

.NI]

17

Apr

201

5

mailto:<[email protected]>

M. Tsukada, J. Santa, S. Matsuura, T. Ernst, K. Fujikawa

16 (TC204 WG16) (also known as CommunicationsArchitecture for Land Mobile (CALM)) is in charge ofstandardizing a communication architecture for coop-erative ITS. TC204 WG16 is specially working on acommunication architecture supporting all types ofaccess media and applications. In Europe, the EuropeanTelecommunications Standards Institute (ETSI) TC ITSis working on building blocks of the same architecturein harmonization with ISO TC204 WG16. In 2010, bothISO TC204 WG16 and ETSI TC ITS defined the ITSStation reference architecture [3, 4].

In cooperative ITS and, in general, in vehicular net-works, there are two main communication paradigms,vehicle to vehicle (V2V) and vehicle to infrastructure(V2I), depending on whether the communication isperformed directly between vehicles or using nodeslocally or remotely installed on the road infrastructure.When the V2V paradigm is considered, the researchfield is commonly called Vehicular Ad-hoc Networks, orVANET, as an especial case of Mobile Ad-hoc Networks(MANET) where nodes are vehicles. Although there area lot of works related to VANET applications and basicresearch at physical, MAC and network layers, thereis a significant lack of real evaluation analysis in thisfield, due to cost and effort implications. Many VANETsolutions and protocols could be considered as non-practical designs if they were tested over real scenarios,as it has been proved in MANET [5]. Performance ofVANET protocols based on a pure broadcast approachcan be more or less expected in simple configurations,even if they are not experimentally tested; but the num-ber of issues concerning the real performance of multi-hop designs is much more tricky. A similar problem canbe found in V2I, which has received a great attentionby the research community in the last years, due to theidea that V2I technologies and services will find a placein the market before V2V approaches. Nevertheless, anumber of experimentation works and supporting toolsshould be improved in the short term, in order to givereal evidences to car manufacturers and road operatorsof the benefits of vehicular communications.

Conventional network measurement tools (e.g. iperf,ping or traceroute) assume fixed networks and assessnetwork performances in an end-to-end basis. However,under dynamic network conditions such as in thevehicular networks case, it is difficult to analyzein detail the operation of networks by using solelythese tools, because vehicles are always changing theirlocation and the performance of wireless channelsfluctuates. In order to solve these issues, we havedeveloped a packet analysis and visualization toolcalled AnaVANET1, which considers the peculiarities ofthe vehicular environment for providing an exhaustive

1http://anavanet.net/

evaluation software for outdoor scenarios (Figure 1includes a preliminary screenshot of the visualization).Both V2V and V2I networks can be efficiently analyzed,thanks to the integrated features for collecting results,post-processing data, generate graphical figures ofmerit and, finally, publish the results in a dedicatedweb site (if desired). All tests and results are lateravailable in the form of an animated webpage whereboth researchers and the general public can accessthe evaluations. AnaVANET has been successfullyexploited for the moment in experimental evaluationcampaigns in the GeoNet [6] and ITSSv6 [7] projects.

Figure 1. Screenshot of AnaVANET viewer

In this paper, apart from presenting our tool forassessing the performance of vehicular networks, weanalyze in detail the problem of real testing in V2Vand V2I, identifying the main issues, requirements, andproposing a general methodology useful for furtherworks in the area. To sum up, the rest of the paper isorganized as follows. Section 2 introduces the readersin network layer protocols for vehicular networks.Section 3 reviews related works in the area of testingvehicular networks. Then, the issues and requirementsfor evaluating vehicular networks are listed in Section 4.The evaluation methodology desired in this frame isdescribed in Section 5 and, as a result of our analysis,the design and implementation of the AnaVANETevaluation tool is detailed in Section 6, together witha reference evaluation of a network testbed using thetool in Section 7. As a result of this evaluation, thefunctionalities provided by AnaVANET are analyzed inSection 8 according to the previously identified needs.Finally, Section 9 concludes the paper summarizing themain results and addressing future works.


http://anavanet.net/

http://anavanet.net/demo-vienna/?analysis=1264516500

On the Experimental Evaluation of Vehicular Networks: Issues, Requirements and Methodology Applied to a Real Use Case

2. Network protocols in vehicular networks

Network protocols in vehicular networks can beclassified in infrastructure-less scenarios, i.e. V2V, andinfrastructure-based scenarios, i.e. V2I, as showed inFigure 2.

The infrastructure-less scenario is well-known inthe research areas of VANET and MANET. Theseapproaches are designed to enable wireless commu-nications in dynamic topologies without any infras-tructure. Routing protocols here are further classifiedas topology-based and position-based routing protocols.Upon the appearance of vehicular communications, asecond class of infrastructure-less protocols added tothe list: VANETs. Most of the VANET solutions arebased on geographical routing, thus based on the node’sposition.

Topology-based protocols were divided into two mainbranches by the IETF MANET working group: reactive,where nodes periodically exchange messages to createroutes, and proactive, in which control messages areexchanged on demand when it is necessary to reacha particular node. Generally, proactive protocols havethe advantage of starting communication rapidly bymaking the routing table ahead, however, this makesbattery life shorter due to frequent signaling. If thetopology is highly dynamic and the data traffic isfrequent, a proactive protocol could be better. Reactiveprotocols, on the contrary, keep the battery life longerby reducing signaling messages when there is no data totransmit. The hybrid protocols that take the advantageof both proactive and reactive protocols by maintainingroutes to near neighbors regularly and searching thedestination in long distance on demand.

Some routing protocols are specified by the IETFMANET working group [8]. Both IPv4 and IPv6 aresupported in the working group. Ad hoc On-DemandDistance Vector Routing (AODV) [9] and DynamicSource Routing Protocol (DSR) [10] are specified asreactive routing protocols. And Optimized Link StateRouting (OLSR) [11] and Topology DisseminationBased on Reverse-Path Forwarding (TBRPF) [12] arespecified as proactive routing protocols. As an exampleof a hybrid MANET protocol, Zone Routing Protocol(ZRP) [13] is proposed.

VANETs are a particular case of MANETs, and arenot restricted by the battery of the communicationnodes and are also characterized by the high speedof nodes, the availability of GPS information, and aregular distribution and foreseeable movements. First,vehicles have a larger battery than mobile terminalsor sensor devices, which is also charged when theengine is running. Second, the speed of vehicles is alsohigher than common portable terminals, and relativespeeds can reach 300 Km/h; hence, the duration ofthe routing entries is extremely short. Third, a GPS

device and digital map can be assumed in many cases,whose information improves the network performancein some proposals.

Unlike topology based routing, position basedrouting does not need to maintain part of the networkstructure in order to forward packets towards thedestination node. When the routing is based onthe position, nodes forward the packets with theaim of reaching the nodes within a geographicallocation. Thus, position based routing can eliminatethe problem that appears in topology based protocolswhen routes become quickly unavailable in highmobility scenarios. In Greedy Perimeter StatelessRouting (GPSR) [14], for instance, intermediate nodesmake a decision based on the destination position andneighbor positions. The Car-to-Car CommunicationConsortium (C2CC) also specified the C2CNet protocol,which was later enhanced by the GeoNet project tosupport IPv6. Within the ITS standardization domain,GeoNetworking [15] is being completed by ETSI atthe moment, integrating several geo-aware strategies tobetter route packets in vehicular networks.

On the other side, infrastructure-based protocolshave been focused on the global connectivity of nodesto the Internet. Mobile IPv6 [16] solved the mobilityproblem for mobile hosts and, later, Network MobilityBasic support (NEMO) [17] provided a solution forthe mobility of a whole network (e.g. a vehicle orbus), which has been recommended by the ISO TC204WG16 to achieve Internet mobility for vehicles. NEMOmaintains a bi-directional tunnel between the routerin the vehicle, known as the mobile router (MR), and aserver in the fixed infrastructure, known as the homeagent (HA), in order to provide a unchanged networkprefix called mobile network prefix (MNP) to the in-vehicle network. All the in-vehicle nodes called mobilenetwork nodes (MNN) maintain a permanent addressderived from the MNP even when MR changes the pointof attachment to the Internet during the movement (i.e.handover).

3. Experimental evaluation of VANET approachesin the literatureBecause of equipment cost, logistic issues and, ingeneral, the necessary effort, literature in experimentalevaluation of vehicular network architectures is limited.However, these works are of key importance for theITS community. Up to now, there are several worksdealing with this issue, although most of them arestill focused on studying the operation of WiFi, DSRC(Dedicated Short Range Communications) or IEEE802.11p technologies in the vehicular field.

Communication between a vehicle and a staticterminal is important for some ITS services. In [18] acommunication scenario considering a static terminal



AODV,DSR

OLSR, TBRPF

ZRP GeoNetworking,C2CNet,GPSR

Mobile IPv6 NEMO

Network Layer Protocols for vehicular communications

Topology based PositionbasedReactive Proactive Hybrid

Internet Mobility

Network mobility

Hostmobility

Wireless multihop routingInfrastructure basedInfrastructure less (VANET or MANET)

Figure 2. Network protocols in vehicular networks

and a moving vehicle is studied in detail. Among allmetrics considered in this work, the transmission poweris the more original one, determining the maximumcommunication range. The type of data traffic used totest the performance of the communication channelis also of interest. Most VANET designs use UDPpackets, due to poor TCP performance over wirelesschannels [19, 20]. The evaluations performed in [21]with IEEE 802.11p reveal that the packet delivery ratioachieved by this technology is highly dependent onthe distance between sender and receiver. These resultsare also confirmed in [22], where it is also concludedthat the vehicle speed does not imply a noticeableperformance degradation of the communication. Asimilar evaluation is performed in [23], but this timecarrying out a great testing campaign in a city.

When V2V scenarios are considered, most ofthe previous works only consider two terminals inperformance tests, what is not too representativewhen multi-hop schemes are evaluated. In [24], theapplicability of 802.11b in V2V communications isevaluated over urban and highway scenarios, and itis demonstrated that a direct line of sight is one ofthe most important issues in the network performance.Two works evaluate a multi-hop VANET over realconditions, using three [25] and even six vehicles [26].These papers offer a wide study about a real VANETset-up, and the last one includes an interesting analysisdescribing the impact of the number of hops on thefinal performance. Nonetheless, static routes are used inthat work, presenting a non-realistic vehicular network.Our prior work [27], by contrast, considered a real andstandardized ad-hoc routing protocol to dynamicallymodify communication paths. The hardware testbedpresented is also suited for future ITS research, with aflexible in-vehicle and inter-vehicle IPv6 network basedon mobile routers.

To the best of our knowledge, there exists afew works dealing with the evaluation of IPv6-based communications at network level in vehicular

communications, and some of them are withinour research line [28, 29]. However, our priorevaluations are only focused on IPv6 network mobility.In this work, the operation of NEMO over aV2V protocol is evaluated, using an implementationof GeoNetworking. This way, an integrated V2Vand V2I approach is considered for providing anintegral vehicular connectivity using IETF and ETSIstandardized protocols. The novelty of this workis twofold, since not only this routing approach isexperimentally analyzed, but also an evaluation toolespecially designed for vehicular networks is used.As far as the authors know, no specific tools forassessing the performance of vehicular networks havebeen developed or used in previous research works.

4. VANET evaluation: issues and requirements

4.1. Issues

As said above, the experimental evaluations carried outin vehicular networks are mostly based on single-hopstudies. In the case of multi-hop experiments, a staticroute configuration is often employed, but dynamicrouting presents a more realistic view in vehicularcommunications.

Using multi-hop and dynamic routing strategiespresents a challenge in the evaluation of vehicularnetworks. Common end-to-end evaluation tools such asping6 and iperf are useless to track the effect of routechange, because they are unaware of the path takenduring a communication test. An additional lack ofthese tools is the possibility to measure the performanceof hop-by-hop links, since the study is carried out end-to-end. Also, geographical and external factors such asnodes position, distance between nodes or obstacles arenot linked with network performance figures of merit.Therefore, the performance comparison of variousdynamic routing protocols is essentially missing.



4.2. RequirementsWith the aim of summarizing these main requirementswhen evaluating multi-hop vehicular networks, thenext needs are found essential by the software toolsused in experimental campaigns for evaluating bothV2V and V2I:

Path detection. The topology and communication pathof a vehicular network changes frequently withdynamic routing as vehicles move. Thus, the tool shouldtake note of the communication path used in everymoment.

Communication performance in links. The communicationperformance between ends is the sum of the links onthe way between them. Once the communication path istracked, the tool should measure the performance linkby link as well as end-to-end.

Geographical awareness. The network performance ina link depends on various geographical factors. Forexample, the distance between the nodes affects thepacket loss probability of the link; the movementspeed and the direction are also important factorsfor the packet loss in the link; and the existence ofobstacles between the nodes may screen the wirelessradio propagation. Thus, the evaluation tool shouldtake the above geographical factors into account.

Intuitive visualization. It is important to visualize thegeographical factors such as node movement (speed,direction), distance and signal obstacles in order toanalyze which of them affect the network performance.For intuitive visualization, performance figures ofmerit and environmental information should be showntogether in a synchronized way. Moreover, the spatio-temporal data series should be available in post processto play them at different speeds, stop when desired, orreplayed freely as he or she wants.

Independence from network protocols. As shown inSection 2, there are many network layer protocols in theliterature for vehicular scenarios, both infrastructure-less and infrastructure based. The evaluation toolshould be independent from the network protocolsemployed in the target vehicular network. This includesthat the tool does not require changes to adapt toneither specific protocols nor special message or datatransported.

Independent from devices. Depending on the experiment,the configuration of the used devices may differ inboth vehicle and infrastructure sides. The devicesinclude the antenna, wireless chipset, CPU, memory,GPS and so on. The tools should not rely on any ofthe specific devices functionalities. Most favorably, thesame software and settings for an experimental testshould work on multiple devices.

Adaptation to various scenarios. There are a number ofpossible networking scenarios in vehicular communica-tions, such as using parked vehicles, slow speeds withsurrounding buildings in a urban situation, vehiclesmoving at higher speeds in a highway, overtaking, vehi-cles crossing in a two-way road, different topologicallocations of the ends in a V2I setting, etc. The softwareevaluation tool should accommodate to all of thesescenarios.

Easiness for data collection. In order to compare thenetwork performance obtained when using differentnetwork protocols, a lot of experiments could beneeded. This may require installing data collectorsoftware on many devices, depending on the scenario.Thus, the easiness of the installation of these softwaremodules is very important. Of course, the mostfavorable case is to employ common software in all ofthem, such as tcpdump or cat.

Flexible experimental data format. The experimental datashould be stored in a well-organized way. Therefore, thedata format needs to be flexible for future extension. Forexample, the user of the system could require addingnew attributes to the data format of evaluation results.We must consider flexible data formats in order not toimpact the process of adding new attributes.

5. Evaluation methodologyAs it is later described, the evaluation tool presentedin this work (AnaVANET) copes with the previousrequirements, but first it is important to identify ageneric testing methodology that allow a researcher tosuccess in a testing campaign with a vehicular network.

In general, the evaluation goals in computer networksare to analyze which testing conditions affect which dataflows or network protocols. For achieving this end it isnecessary to design a proper evaluation methodology.Within it, we should consider the tendency of resultsby repeating tests with the same settings or varyingparameters under study, such as the network protocol,the mobility of nodes or the data volume. A properevaluation tool, such as the later presented AnaVANET,should support the overall analysis. This sectionconsiders both the testing conditions and the possiblerouting protocols to consider in vehicular networks,as it is summarized in Figure 3, by introducing theconcept and presenting our real use case for testing theperformance of NEMO over IPv6 GeoNetworking.

5.1. Testing conditionsTestbed platform. The testbed used for the evaluationof network architecture should be carefully chosento implement most relevant nodes in real softwareand hardware. In vehicular communications, this is



Distance Static Urban Highway

Packet size, Send rate

TCP window size, Max segment size

Packet size, send interval

Testing Scenarios

Data Flows

Hardware (CPU, Memory), Antenna, Wireless setting(frequency, data rate)Testbed platform

Performance Indicators

AnaVANETPDR, throughput, Jitter,

Hop count Throughput RTT, PDR, Hop count

Infrastructure-basedInfrastructure less(NEMO)(OLSR / C2CNet / IPv6 GeoNetworking)

Test

ing

Con

ditio

nsTarget Network Protocols

(Examples)

Number of Vehicles

UDP TCP ICMPv6

Figure 3. Evaluation methodology

extremely important, since a good deployment could beneeded in case of testing V2V multi-hop networks.

In our particular case, the testbed comprises a set offour vehicles and two roadside stations, as illustrated inFigure 4. Each vehicle is equipped with a mobile router(MR), with at least two interfaces: an Ethernet link toconnect mobile network nodes (MNNs) within the in-vehicle network, and a wireless adapter in ad-hoc modeused for both V2V and V2I communications. On theroadside, access routers (ARs) are fixed on the top ofa building or any other elevated point near the road.Each one provides two interfaces: an Ethernet link for awired Internet access, and a wireless adapter in ad-hocmode to connect with vehicles in the surroundings. Ata backend point on the Internet, a home agent (HA) isinstalled to support Internet mobility of MRs by usingNEMO.

Among the various testbed conditions, the hardwarespecifications (CPU, memory, etc), antenna and wirelesssettings are important factors for the evaluation, sincethey will highly affect the results. In our case, MRsare Alix3d3 embedded boxes provided with a Linux2.6.29.6 kernel. Each MR has a mini-pci wireless cardAtheros AR5414 802.11 a/b/g Rev 0, and an antenna2.4GHz 9dBi indoor OMNI RP-SMA6 is used. Thefrequency used has been 2.422Ghz and the data rate hasbeen fixed to 6 Mbits/s.

Testing scenarios. Fixing the evaluation scenariosbeforehand is essential in the planning of a testingcampaign. In general, the main factors that determinethe possible scenarios are:

Mobility Vehicle mobility is a key issue to copewith realistic vehicular network conditions. Thisway, we can consider not only static scenarios,to test the network operation in a controlledway, but also dynamic scenarios under commonspeed situations. Of course, field operational tests

WirelessWirelessMR1

MNN1cable

MR2

MNN2cable

MR3 MR4

Vehicular Network

Infrastructure Network

AR1AR2

IPv6 InternetHA1

Figure 4. Reference network configuration

should be conducted to confirm the expectedresults, taking into account the proper handlingof mobility, i.e. Doppler shifting, fast fading, etc.

Location Urban and interurban environments affectcommunication performance in a different way,because the signal propagation can be interferedby buildings (among other elements), and theline of sight between vehicles is not alwayspossible. Two environments are considered inour tests: a semi-urban one located at INRIA-Rocquencourt, which contains a set of smallbuildings surrounded by streets, and a highwaystretch, the A-12 one, near INRIA-Rocquencourt.

Number of vehicles The number of hops between thesource and the destination vehicles affect thecommunication delay and the higher probabilityof packet looses, due to route changes or MACtransmission issues. Up to four conventional



vehicles (Citroën C3) are considered in our case.This testing fleet is showed in Figure 5.

Figure 5. Testing vehicles

A set of possible testing scenarios when evaluatingmulti-hop vehicular networks is summarized in Fig-ure 6. These have been divided into urban and highway.Mobility has been set to static, urban-like speed andhigh speed. In our particular evaluation, these scenarioshave been considered with our fleet of vehicles, withthe aim of covering a wide range of communicationconditions. The obstacles have been in our case a setof building blocks located at the Paris - Rocquencourtpremises. The chosen highway has been the French A13,near Versalles.

Distance Static

Urban Highway

parked ~10km/h

Obstacle Obstacle

parked parked

parked parked

packet

~100km/h

~100km/h

~100km/h

Obstacle~30km/h

~30km/h

~30km/h

Figure 6. Proposal of movement scenarios

5.2. Data flows and performance indicatorsA number or protocols and data flows can be set forevaluations, however, only the most representative andmore used in the literature should be considered tostudy concrete performance indicators. For instance, inour case UDP, TCP and ICMPv6 are used to measurethe network performance between two communicationend-nodes (MNN to MNN) mounted within twovehicles:

UDP is a connection-less unidirectional transmissionflow. The traffic is generated by iperf in our case.It is considered that with UDP the performanceindicators under consideration can be the packetdelivery ratio, throughput and jitter.

TCP is a connection-oriented bidirectional transmis-sion flow. This traffic is also generated by iperf inour case. The performance indicator under con-sideration here has been the maximum through-put.

ICMPv6 is a bi-directional transmission flow. Thetraffic is generated by ping6 in our case. Theperformance indicator under consideration can bethe road trip delay time and packet deliveries.

The set of performance indicators most used in theliterature are detailed next:

Round-Trip Time (RTT) can be measured usingICMPv6, as in our case. A host on the sourcevehicle, or located at an infrastructure point,sends ICMPv6 echo request to a host on thedestination vehicle, or located at an infrastructurepoint. The destination host replies with anICMPv6 echo reply. The period between the timethat the request is sent and the time that the replyis received can be obtained by using ping6.

Throughput can be measured using UDP or TCP. It canbe measured with a traffic generator tool, such asthe iperf tool in our case. In UDP, iperf is executedin both the sender and the receiver nodes. TheUDP packet transmission rate is set with a fixedrate and the sender is not able to see the resultbecause the communication is unidirectional fromthe sender to the receiver. The throughput isshown on the receiver side. On the other hand,when using a TCP transmission, the sending rateis automatically adjusted with the TCP congestioncontrol mechanism. The sending rate is adjusteddepending on the acknowledgement messagesreceived. The throughput appears in both thesender and receiver nodes.

Jitter is a measure of the variability over time of thepacket latency across a network. A network witha constant latency has a null jitter. In general, thejitter is expressed as an average of the deviationfrom the network mean latency, and can becalculated using the RTT, as in our case.

Packet Delivery Ratio (PDR) is the percentage ofpackets received by the target node as comparedwith the number of packets sent by the source.iperf, for instance, shows this value at the receiverside when using TCP in an end-to-end manner,but AnaVANET is also able to calculate the PDR



on each hop between the sender and destinationnodes.

6. System design and implementation ofAnaVANET

6.1. Overview of the softwareAnaVANET (initially standing for Analyzer of VANET)is an evaluation tool implemented in Java to assessthe performance of vehicular networks. It takes asinput the logs generated by the iperf, tcpdump and/orping6, together with navigation information in NMEAformat, to compute the next performance metrics:network throughput, delay, jitter, hop count and list ofintermediate nodes in the communication path, PDRend-to-end and hop-by-hop, speed, and instantaneousposition.

In this part of the work AnaVANET is put in thecontext of the evaluation scenario described in theprevious section in Figure 7, showing also the maininputs and outputs of the tool. The sender MNN (leftmost vehicle) is in charge of generating data traffic,and both the sender and the receiver (right mostvehicle) MNNs record a high level log, according tothe application used to generate network traffic (iperfand ping6 for the moment). All MRs record informationabout forwarded data packets by means of the tcpdumptool, and log the vehicle position continuously. All thisdata is post-processed by the AnaVANET core softwareand then analyzed. The tool traces all the data packetstransmitted from the sender node to detect packetlosses and calculate statistics for each link and end-to-end, and then merge all these per-hop information withtransport level statistics of the traffic generator. As aresult, AnaVANET outputs a JSON file with statisticson a one-second basis (see Section6.2 for details), anda packet trace file with the path followed by each datapacket.

Once generated, performance metrics can be graph-ically showed through plots generated by gnuplot anda website where all tests are available. The screenshotof the website is shown on the left bottom cornerof Figure 7 (which is also enlarged in the previousFigure 1). Accessing the website one can replay thetests on a map to see momentary figures of merit.Previous experiments can be chosen to monitor themain performance metrics at any time of the tests. Userscan play and stop at any arbitrary point of the testwith the control buttons on the upper left part of thewindow. The player speed, one step forward and onestep backward are also implemented. On the map, theposition and movement of the vehicle are depicted withthe speed of each vehicle and the distance betweenthem. The transferred data size, bandwidth, packet lossrate, RTT and jitter, for each link and end-to-end are

displayed. The network performance is visualized bythe width of links and the colors used to draw them.

6.2. Data format of experimental resultsIn this part, we describe the problems of the formerAnaVANET data format [30], which was based onXML, and we detail the recent changes to improve theflexibility of the results using the JSON format [31].

There is a fundamental trade-off between flat dataformat and structured data format. Flat data format ismore flexible than structured one, because if developerswant to add a new attribute, they just put the attributenext to the other attributes. On the other hand, ina structured format, developers have to consider thelayers and relationships to add a new attribute and theysometimes cannot add the new attribute because of itsstructure. However, when a flat data format is used,developers have to revise and adjust their applications,since the relationships among attributes can vary. Inthis line, a normalized way of calling the attributes isalso important. If there is no rule of normalization,developers have to handle differences of an attributename (e.g., temperature, Temperature, temp).

AnaVANET was initially developed to analyzethe real operation of VANETs. The initial dataformat used as output of an evaluation had someproblems regarding its flat format and the dynamiccolumns available per each data record. Hence ittook several hours to check the results after carryingout new experiments. To solve these problems, wehave designed a structured and normalized format,considering the features of vehicular networks. Ourformat is extensible and independent from concreteexperimental environments and visualization tools.We have also adapted the initial visualization toolwith an internal converter module within the webapplication, and an additional command line toolhas been implemented to process the output logs ofAnaVANET in a text-based basis. The new data formatand tools enable us to check the results in severalseconds after carrying out experiments, consideringthat users could require a fast evaluation to continuewith new experiments.

The new structured and normalized data formatof AnaVANET considers three layers, as it is shownin Figure 8. AnaVANET summarizes data on eachtime slot following the next scheme. The top layeris the “experiment” layer. This layer mainly managesstatic attributes (e.g. ID of experiment or the name ofexperiment). The second layer is the “data” layer. Thislayer manages results of an experiment on each timeslot. This layer has time, total packet delivery ratio(PDR), total RTT and other attributes. The third layeris comprised by the “node” and “link” parts. The nodepart manages each node’s statuses, whereas the link part



MNN1(Sender)

MNN2(Receiver)MR1 MR4 MR2

Cable CableWireless Wireless

UDP / ICMPv6traffic generation

iperf / ping6log

tcpdumplog

GPSlog

tcpdumplog

GPSlog

tcpdumplog

GPSlog

iperflog

AnaVANET

(optional)(optional)

JSONstatistics

Packettrace

GnuplotWeb font-end(Google maps)AnaVANET Web viewer

Graphshttp://anavanet.net/

MR3

tcpdumplog

GPSlog

Wireless

Figure 7. Overview of AnaVANET

manages each link’s statuses. Link means a relationshipbetween two nodes and especially represents wirelesslink statuses. An experiment has a series of data andeach data has several nodes and links. We have alsonormalized the names of the attributes considered ineach layer.

This data format based on time-series for saving nodeand link information is an abstracted representationthat can be used to collect results from any kindof network. Moreover, by using this three-layerrepresentation, the system can be easily adapted tofuture requirements.

7. Evaluation of NEMO over IPv6 GeoNetworkingEarly versions of AnaVANET were designed forevaluating infrastructure less network protocols, asused in our previous works for analyzing OLSR invehicular environments [32] and later tests of IPv6 overC2CNet [33] in the FP7 GeoNet project. The currentversion of AnaVANET can also analyze infrastructure-based network protocols such as NEMO.

In this section, we report a summary of the resultscollected in the evaluation of NEMO over IPv6GeoNetworking when a vehicle connects with a nodelocated in the Internet using two roadside units asaccess routers. The umip.org2 implementation of NEMOis used, whereas the cargeo6.org3 software is used for

2http://umip.org3http://www.cargeo6.org

{ "exp_id": 1303883952, "name": "[ICMP] 27/4/2011 5:59:12”, "exp_type": "ICMP”, "timezone": 9, "data": [ {"time": 1303883952, "jitter": 0, "bandwidth": 0, "bytes": 64, "rtt": 3.78, "req_pdr": 0, "nodes": [ { "node_id": "MR1”, “lat”: 34.73193383216858, “lng”: 135.73369348049164, “speed”: 1.433448 }, {"node_id": "MR2”, “lat”: 34.731985330581665, “lng”: 135.73398900032043, “speed”: 0.40373600000004 }, {} ], [] "links": [ {"src": "MR1”, "dest": “MR2”, “name”: "MR1_MR2”, “distance”: 27.639465545, “req_pdr”: 1.0, “res_pdr”: 1.0 }, {} ], [] ], []}

Experiment

exp_id, name, exp_time, …

datatime, jitter, bandwidth, ….

data

data

data

nodenode_id, lat, lng, …

node

node

linksrc, dest, name, ….

link

data

Figure 8. Three-layer model of the structured data format usedby AnaVANET

IPv6 GeoNetworking. ICMPv6 and UDP evaluations inhandover scenarios were performed at INRIA Paris-Rocquencourt campus with the two ARs previouslypresented in the testbed description. The speed of the


http://umip.org

http://www.cargeo6.org


vehicle was limited to less than 15 km/h, like in a lowspeed urban scenario.

The reader can directly click in from Figure 9 toFigure 12 to see the correspondent results in theAnaVANET web viewer, to further perceive the detailsof the gathered results.

7.1. ICMP evaluation in a handover scenarioICMPv6 echo requests (64 bytes) are sent from theMNN to a common computer located in the wirednetwork twice a second, which replies with ICMPv6echo replies. The results collected in the ICMPv6 testsare plotted in Figure 9. The lower part shows theitinerary of the vehicle and the locations of AR1 andAR2 on the map, whereas the upper part shows theRTT, the packet loss and the result of the mobilitysignaling. The X-axis and the Y-axis of the upper partare the latitude/longitude of the vehicle, correspondingto the road stretch indicated in the lower part of thefigure. When either the request or the reply is lost,the RTT is marked with a zero value and, at the sametime, a packet loss is indicated. A binding registrationsuccess is plotted when the NEMO binding update (BU)and the corresponding binding acknowledgment (BA)are successfully processed. On the contrary, if either ofthem is lost, a binding registration fail is plotted at theposition.

Figure 10 shows the same result of the test, butreferred to the test time. The upper graph shows theRTT and the distance to the two ARs; the middle oneshows the PDR obtained with the two ARs; and, finally,the lower plot shows the status of the NEMO signaling.A NEMO success means that the binding registrationhas been successfully performed, and a fail indicatesthat either the BU or the BA has been lost.

As can be seen in Figure 9 and Figure 10, the RTTis stable at the beginning of the test near AR2, witha value of around five milliseconds. AR2 is installedat about 100 meters away the road. It sends constantBU messages and, consequently, the MR successfullyperforms the binding registration every twelve seconds,without any packet loss. Soon, after the vehicle turnsthe first corner (north west of the square), packets startto be dropped until the second corner. This is because anear building screens the wireless radio. The bindingsignaling is dropped as well in the period. Then itrecovers when the vehicle comes to the straight roadon the south. The mobility signaling is successfully sentagain with a regular interval.

The lower straight road of the stretch is less stablethan the one on the north, because of two reasons. First,the location of the south straight road is 250 metersfurther to AR2 than the one on the north. Thus thesignal strength is weaker now. Second, the trees at thislocation interfere the wireless radio, especially at the

AR2

AR1

MR

0

20

40

60

80

100

RTT (ms)

RTTPacket loss

Binding registration SuccessBinding registration Fail

RTT (ms)

Figure 9. Map-based RTT, packet losses and mobility signalingin an ICMP evaluation under a handover scenario

0 50 100 150 200 250 300

NE

MO

sta

tus

Time (seconds)

Fail

Success

0

20

40

60

80

100

PD

R (

%)

Echo Request via AR2 0

20

40

60

80

100

PD

R (

%)

Echo Request via AR1

0

100

200

300

400

0

5

10

15

20

25

30

Dis

tance (

m)

RT

T (

ms)

Distance between MR and AR1Distance between MR and AR2

RTT

Figure 10. RTT, packet looses and mobility signaling in an ICMPevaluation under a handover scenario

end of this part of the circuit, as can be seen with thethree consecutive binding registration fails. When theMR fails to receive a valid matching response within theselected initial retransmission interval, the MR shouldretransmit the message until a response is received.The retransmission by the MR must use an exponentialback-off in which the timeout period is doubled uponeach retransmission, until either the MR receives a





response or the timeout period reaches the value ofmaximum timeout period as specified in [16]. In ourparticular case the mobility daemon tries to deliver theBU one second after the first failure of the binding.Then, when it fails, it increases the retransmission timein two, four, eight seconds, and so on.

The performance in the final part of the testing circuitis more stable, and no binding messages are lost. In thisperiod the vehicle approaches AR2 and then leaves itturning right at the end of the test.

The MR starts receiving router advertisement (RA)messages from AR1 when the distance to AR1 is 50meters, however, the RA messages from AR2 alsoreaches the zone. As the result, the vehicle triggers themovement detection, and sends the mobility signalingvia the AR where it receives the RA. When the MRassociates with AR2 some ICMP packets and mobilitysignaling messages are lost because of the distance anda near building. When the MR later switches to AR1,the packets are more stably transmitted.

7.2. UDP evaluation in a handover scenarioThe results collected in the UDP tests are plotted inFigure 11. UDP packets are sent from the MNN to thewired node at a rate of 1 Mbps and a length of 1250bytes. The lower part of the figure shows the itineraryof the vehicle, and the upper part corresponds to thePDR obtained with the ARs and the binding registrationresults, as in the previous case. The road stretch is thesame one used above, but the vehicle moves on thecontrary direction in this case.

Figure 12 shows the time-mapped results of the sameUDP test. The upper graph shows the UDP throughputfrom the MNN to the wired node, the middle part showsthe PDR to the two ARs, and the lower plot includes thestatus of the NEMO signaling.

The throughput of the UDP traffic is below 30% ofthe sending rate of 1Mbps (i.e. 300Kbps), however thePDR with the two ARs reaches 100%. This is becausethe throughput is measured between end nodes (MNNand a node in the Internet) by iperf and the PDR in thewireless links are calculated hop-by-hop by AnaVANET.In this case, it shows that more than 70% of the UDPpackets are dropped outside the wireless links. In fact,the CarGeo6 software experimented a bottleneck inthe processing of so many UDP packets at that time.This also explains the phenomenon where the bindingregistration messages are lost while none of the UDPpackets are lost (this can be seen in the straight road inthe south part of the circuit). In this case, the BUs arelost in the CarGeo6 software and are not transmittedfrom the wireless interface. We can detect where apacket is lost, especially the loss in a wireless link,thanks to the AnaVANET system (although the causeof the packet losses was not in the wireless links in the

AR2

AR1

MR

0

20

40

60

80

100

PDR (%)

PDR to AR1PDR to AR2

Binding registration SuccessBinding registration Fail

PDR (%)

Figure 11. Map-based PDR of UDP evaluation using NEMOover IPv6 GeoNetworking

0 20 40 60 80 100 120 140 160 180

NE

MO

sta

tus

Time (seconds)

Fail

Success

0

20

40

60

80

100

PD

R (

%)

PDR to AR2 0

20

40

60

80

100

PD

R (

%)

PDR to AR1

0

100

200

300

400

0

100

200

300

400

500

Dis

tan

ce

(m

)

Th

rou

gh

pu

t (K

bits/s

ec)

Distance between MR and AR1Distance between MR and AR2

Throughput

Figure 12. PDR of UDP evaluation using NEMO over IPv6GeoNetworking

present case). This is because AnaVANET is capable ofmeasuring both the hop-by-hop network performanceand the end-to-end one.

As can be seen in Figure 11, AR2 is available mostof the test period (especially, around the square) exceptfor the end of the test. When the vehicle moves inthe first straight road in the east, the PDR to AR2 isalmost 100%. During this period, no binding message





is dropped. The BUs are sent regularly at intervals oftwelve seconds.

The packets start being dropped on the west of thesquare because the building on the north west corner ofthe square blocks the wireless radio. When the beaconsexchanged between GeoNetworking nodes twice in asecond are dropped, the correspondent entry of thelocation table expires in five seconds.

As can be seen in Figure 12, after the southwestcorner, the end-to-end throughput drops to zero and thebinding registration fails, while the hop-by-hop PDR toAR2 is still almost 100%. This shows that the mobilitysignaling packets are lost in CarGeo6 as explainedearlier. Since the binding life time is configured as 24seconds, the binding entry in the HA expires 24 secondsafter the last successful binding registration. After theexpiration of the binding, HA discards all the packetfrom the MR. During the period, the MR try to send theBUs in exponentially increased interval from 1 secondto 32 seconds (1, 2, 4, 8, 16 and 32 seconds).

Then, at time 139 seconds, when the vehicle is 20meters away from AR1, the first binding registrationthrough AR1 successes. UDP packets are switched toAR1 from this moment. Then at time 155 seconds, thebinding registration is successfully performed via AR2again. During the handover from AR1 to AR2, fromtime 155 seconds to time 158 seconds, three secondsof disconnection are present in the iperf log. At time166 seconds, the path to the Internet is switched to AR1again. In this handover, UDP packets are lost duringfour seconds from time 166 seconds.

8. Qualitative evaluation of the systemAs a result of the experience working with the recentversion of AnaVANET, including the results presentedabove, we have revisited the requirements for anefficient testing environment in vehicular networksdetailed in Section 4, with the aim of evaluating theadvantages of the system. Table 1 summarizes the mostimportant features, which demonstrate that AnaVANETfulfills the most important requirements and it is anefficient evaluation tool.

9. Conclusions and future workThe paper has presented the peculiarities of evaluatingvehicular networks experimentally, through presentingthe most used protocols and detailing the needs ofthe software tools to be used for this task. Afterthat, the importance of the testing methodology isdescribed, and a reference design of a vehicularnetwork evaluation is used to exemplify it. The testbeddesign and implementation, testing scenarios, routingprotocols and data flows, are found essential to be fixedbeforehand to avoid improvisation during the testingcampaign. The AnaVANET platform is then presented

Requirement Proposal

Path detection AnaVANET can track the nodes ofthe communication path for eachtransmission

Communicationperformance inlinks

The system can measure the PDR ofeach link as well as the end-to-endPDR

Geographicalawareness

The system outputs the performanceindicators in a geo-referenced way,which facilitates the analysis ofresults

Intuitivevisualization

The movement of vehicles is showedusing Google Maps in a Webapplication, together with the graphsof the desired performance metrics.It allows a step-by-step visualanalysis of the results.

Independencefrom networkprotocols

The system adopts the MAC addressfor packet tracing. Therefore anykind of network layer protocol canbe evaluated.

Independentfrom devices

The system does not require specifichardware.

Adaptation tovarious scenarios

The system can be used in a numberof scenarios, including distance,static, urban and highway tests. Alsoit allows both V2V and V2I tests.

Easiness for datacollection

The system does not require specialsoftware to gather experimentaldata. Packet dumps are taken withtcpdump, and GPS NMEA data isobtained directly from a serialinterface, to finally generate results.

Flexibleexperimentaldata format

We have adopted a structured andnormalized format defining athree-layer model in order toincrease the flexibility for futureextension.

Table 1. Qualitative evaluation of the system

as an efficient evaluation software to process the datagathered by common testing tools, and then generatelots of performance indicators of the trials. All of theseperformance parameters are put in the spatio-temporalcontext, through the collection and correlation of GPSinformation, and most important figures of merit can beexported in the form of graphics or showed interactivelyin a web front-end.

The capabilities of AnaVANET are exploited in anovel evaluation of NEMO over IPv6 GeoNetworking,using the tool to gather RTT, PDR and channelthroughput information. The results reveal that mobileIPv6 connectivity can be maintained in a V2I case using



GeoNetworking over WiFi to pass NEMO IPv6 trafficbetween vehicles and infrastructure.

Our future work includes, first, a link layer extensionof the system to analyze the channel quality (RSSI)and load ratio. This data will allow the development ofcoverage maps for the communication nodes. Second,it is considered the support for multicast data flows,since it is essential for the dissemination of events invehicular networks. Third, we plan to evaluate a realapplication developed for cooperative ITS.

AcknowledgmentThis work has been sponsored by the European 7th FP,through the ITSSv6 (contract 270519), FOTsis (contract270447) and GEN6 (contract 297239) projects, and theSpanish Ministry of Science and Innovation, throughthe Walkie-Talkie project (TIN2011-27543-C03).

References[1] Action plan for the deployment of Intelligent Transport

Systems in Europe, December 2008. COM(2008) 886final.

[2] Standardisation mandate addressed to CEN, CENELECand ETSI in the field of information and communicationtechnologies to support the interoperability of co-operative systems for intelligent transport in theeuropean community, October 2009.

[3] ISO 21217:2010 Intelligent transport systems – Commu-nications access for land mobiles (CALM) – Architecture,April 2010.

[4] Intelligent Transport Systems (ITS); CommunicationsArchitecture, September 2010. ETSI EN 302 665 V1.1.1(2010-09).

[5] C. Tschudin, H. Lundgren, and E. Nordstrom. Embed-ding MANETs in the real world. Lecture notes in computerscience, 2775(1):578–589, September 2003.

[6] T. Ernst, M. Goleva, I. Ben Jemaa, A. Kovacs, H. Menouar,C. Noguchi, S. Schulze, M. Tsukada, P. Zhang, andW. Zhang. GeoNet STREP No.216269 Deliverable 7.1GeoNet Experimentation Results. 2010. GeoNet-D7.1-ExperimentationResults-v1.0.

[7] J. Santa, A. Kovacs, A. Varadi, A.F. Skarmeta,P.J. Fernandez, F. Bernal, M. Tsukada, B. Cama,F. Pereniguez, R. Marin, C. Schulze, A. Fitzner,O. Shagdar, and Y. Bouchaala. ITSSv6 STREP GrantAgreement 210519 Deliverable 4.2 Final Validation& Evaluation Results. March 2014. ITSSv6-D4.2-FinalValidation&EvaluationResults-v1.1.

[8] IETF: Mobile Ad-hoc Networks (MANET) WorkingGroup, 1997. http://datatracker.ietf.org/wg/

manet/charter/.[9] C. Perkins, E. Belding-Royer, and S. Das. Ad hoc On-

Demand Distance Vector (AODV) Routing. RFC 3561(Experimental), July 2003.

[10] D. Johnson, Y. Hu, and D. Maltz. The Dynamic SourceRouting Protocol (DSR) for Mobile Ad Hoc Networks forIPv4. RFC 4728 (Experimental), February 2007.

[11] T. Clausen and P. Jacquet. Optimized Link State RoutingProtocol (OLSR). RFC 3626 (Experimental), October2003.

[12] R. Ogier, F. Templin, and M. Lewis. Topology Dissemi-nation Based on Reverse-Path Forwarding (TBRPF). RFC3684 (Experimental), February 2004.

[13] Zygmunt J. Haas, Marc R. Pearlman, and Prince Samar.The Zone Routing Protocol (ZRP) for Ad Hoc Networks, July2002. IETF work in progress, draft-ietf-manet-zone-zrp-04.

[14] Brad Karp and H. T. Kung. Gpsr: Greedy perimeterstateless routing for wireless networks. In 6thAnnual International Conference on Mobile Computing andNetworking, MobiCom 2000, August 6.-11., 2000, Boston,Massachusetts, USA, pages 243–254. ACM / IEEE, August2000.

[15] Intelligent Transport Systems (ITS); Vehicular Commu-nications; Part 4: Geographical Addressing and Forward-ing for Point-to-Point and Point-to-Multipoint Commu-nications; Sub-part 1: Media-Independent Functionality,June 2011. ETSI TS 102 636-4-1 V1.1.1 (2011-06).

[16] C. Perkins, D. Johnson, and J. Arkko. Mobility Supportin IPv6. RFC 6275 (Proposed Standard), July 2011.

[17] V. Devarapalli, R. Wakikawa, A. Petrescu, and P. Thubert.Network Mobility (NEMO) Basic Support Protocol. RFC3963 (Proposed Standard), January 2005.

[18] C. Wewetzer, M. Caliskan, K. Meier, and A. Luebke.Experimental evaluation of umts and wireless lan forinter-vehicle communication. In Proc. 7th InternationalConference on ITS Telecommunications ITST ’07, pages 1–6, 6–8 June 2007.

[19] F. Hui and P. Mohapatra. Experimental characterizationof multi-hop communications in vehicular ad hocnetwork. In ACM international workshop on Vehicularad hoc networks, pages 85–86, Cologne, Germany,September 2005.

[20] A. Festag, H. Fußler, H. Hartenstein, A. Sarma,and R. Schmitz. FLEETNET: BRINGING CAR-TO-CAR COMMUNICATION INTO THE REAL WORLD.Computer, 4(L15):16, 2004.

[21] Oyunchimeg Shagdar, Manabu Tsukada, MasatoshiKakiuchi, Thouraya Toukabri, and Thierry Ernst.Experimentation Towards IPv6 over IEEE 802.11p withITS Station Architecture. In International Workshopon IPv6-based Vehicular Networks (colocated with IEEEIntelligent Vehicles Symposium), Alcala de Henares, Spain,June 2012.

[22] Jia-Chin Lin, Chi-Sheng Lin, Chih-Neng Liang, and Bo-Chiuan Chen. Wireless communication performancebased on IEEE 802.11p R2V field trials. IEEECommunications Magazine, 50(5):184 –191, may 2012.

[23] J. Gozalvez, M. Sepulcre, and R. Bauza. IEEE802.11p vehicle to infrastructure communications inurban environments. IEEE Communications Magazine,50(5):176 –183, may 2012.

[24] Víctor González, Alberto Los Santos, Carolina Pinart,and Francisco Milagro. Experimental demonstrationof the viability of ieee 802.11b based inter-vehiclecommunications. In TridentCom 2008, pages 1–7, ICST, Brussels, Belgium, Belgium, 2008. ICST(Institute for Computer Sciences, Social-Informatics and


http://datatracker.ietf.org/wg/manet/charter/

http://datatracker.ietf.org/wg/manet/charter/


Telecommunications Engineering).[25] M. Jerbi, S. M. Senouci, and M. Al Haj. Extensive experi-

mental characterization of communications in vehicularad hoc networks within different environments. In Proc.VTC2007-Spring Vehicular Technology Conference IEEE65th, pages 2590–2594, 2007.

[26] M. Jerbi and S. M. Senouci. Characterizing multi-hop communication in vehicular networks. In Proc.IEEE Wireless Communications and Networking ConferenceWCNC 2008, pages 3309–3313, 2008.

[27] Manabu Tsukada, José Santa, Olivier Mehani, YacineKhaled, and Thierry Ernst. Design and ExperimentalEvaluation of a Vehicular Network Based on NEMOand MANET. In The special issue for Vehicular AdHoc Networks, EURASIP Journal on Advances in SignalProcessing, 2010.

[28] Jose Santa, PedroJ. Fernandez, Fernando Pereniguez,Fernando Bernal, Antonio Moragon, and AntonioF.Skarmeta. Ipv6 communication stack for deployingcooperative vehicular services. International Journal ofIntelligent Transportation Systems Research, 12(2):48–60,2014.

[29] J. Santa, F. Pereniguez-Garcia, F. Bernal, P.J. Fernandez,R. Marin-Lopez, and A.F. Skarmeta. A framework

for supporting network continuity in vehicular ipv6communications. Intelligent Transportation SystemsMagazine, IEEE, 6(1):17–34, Spring 2014.

[30] M. Tsukada, J. Santa, S. Matsuura, T. Ernst, andK. Fujikawa. Anavanet: an experiment and visualizationtool for vehicular networks. In 9th InternationalConference on Testbeds and Research Infrastructures for theDevelopment of Networks & Communities (TRIDENTCOM2014), May 2014.

[31] T. Bray. The JavaScript Object Notation (JSON) DataInterchange Format. RFC 7159 (Proposed Standard),March 2014.

[32] Jose Santa, Manabu Tsukada, Thierry Ernst, OlivierMehani, and A. F. Gomez-Skarmeta. Assessment of vanetmulti-hop routing over an experimental platform. Int. J.Internet Protoc. Technol., 4(3):158–172, September 2009.

[33] Manabu Tsukada, Ines Ben Jemaa, Hamid Menouar,Wenhui Zhang, Maria Goleva, and Thierry Ernst. Exper-imental evaluation for IPv6 over VANET geographicrouting. In IWCMC ’10: Proceedings of the 6th Inter-national Wireless Communications and Mobile ComputingConference, pages 736–741, New York, NY, USA, 2010.ACM.


Documents

ICST Transactions Preprint On the Experimental …ICST Transactions Preprint On the Experimental Evaluation of Vehicular Networks: Issues, Requirements and Methodology Applied to a