Transfer reliability and congestion control strategies in opportunistic networks a survey

Ad Hoc Networks 10 (2012) 1322–1345

Contents lists available at SciVerse ScienceDirect

Ad Hoc Networks

journal homepage: www.elsevier .com/locate /adhoc

Efficient and adaptive congestion control for heterogeneousdelay-tolerant networks

Milena Radenkovic ⇑, Andrew GrundySchool of Computer Science, University of Nottingham, Jubilee Campus, Wollaton Road, Nottingham NG8 1BB, UK

a r t i c l e i n f o

Article history:Received 15 August 2011Received in revised form 30 November 2011Accepted 20 March 2012Available online 20 April 2012

Keywords:Delay-tolerant networkingCongestion controlMulti-path forwardingLoad-dependant routingOpen-loop

1570-8705 � 2012 Elsevier B.V.http://dx.doi.org/10.1016/j.adhoc.2012.03.013

⇑ Corresponding author.E-mail address: [email protected] (M. Radenkov

Open access under CC

a b s t r a c t

Detecting and dealing with congestion in delay-tolerant networks (DTNs) is an importantand challenging problem. Current DTN forwarding algorithms typically direct traffictowards more central nodes in order to maximise delivery ratios and minimise delays,but as traffic demands increase these nodes may become saturated and unusable. We pro-pose CafRep, an adaptive congestion aware protocol that detects and reacts to congestednodes and congested parts of the network by using implicit hybrid contact and resourcescongestion heuristics. CafRep exploits localised relative utility based approach to offloadthe traffic from more to less congested parts of the network, and to replicate at adaptivelylower rate in different parts of the network with non-uniform congestion levels. We exten-sively evaluate our work against benchmark and competitive protocols across a range ofmetrics over three real connectivity and GPS traces such as Sassy [44], San Francisco Cabs[45] and Infocom 2006 [33]. We show that CafRep performs well, independent of networkconnectivity and mobility patterns, and consistently outperforms the state-of-the-art DTNforwarding algorithms in the face of increasing rates of congestion. CafRep maintainshigher availability and success ratios while keeping low delays, packet loss rates and deliv-ery cost. We test CafRep in the presence of two application scenarios, with fixed rate trafficand with real world Facebook application traffic demands, showing that regardless of thetype of traffic CafRep aims to deliver, it reduces congestion and improves forwardingperformance.

� 2012 Elsevier B.V.Open access under CC BY license.

1. Introduction

Over the recent years the pervasiveness of mobile com-puting devices has increased significantly and the possibil-ity of communication without an existing networkinfrastructure has become a reality. Hsu and Helmy [18]showed that real world traces of different university campuswireless networks node encounters are sufficient to build aconnected relationship graph that can be successfully usedfor data transmission despite the absence of contemporane-ous end-to-end paths. Routing in these networks is a chal-lenging problem and has been primarily concerned with

ic).

BY license.

providing maximum throughput and minimal delays whiletypically assuming unlimited storage and transfer band-width i.e. using forwarding algorithms that greedily selectbest connected nodes as their next hop [14,28]. As mobiledevices have limited resources, key nodes in the networkquickly become congested and unusable and cause evenmore disconnections and even lower delivery rates.

Newly emerging work on congestion control in DTNs at-tempts to rectify this by proposing adaptive forwarding[14,31,15,30] or adaptive replication management[39,38,26,36,37] techniques. In our earlier work we pro-posed an adaptive forwarding protocol, Café [14,31,15,30],that diverts the load away from its conventional central-ity-driven path by considering congestion heuristics thatroute the traffic away from the congesting areas. This ap-proach is particularly suitable to social opportunistic

http://dx.doi.org/10.1016/j.adhoc.2012.03.013

mailto:[email protected]

http://dx.doi.org/10.1016/j.adhoc.2012.03.013

http://www.sciencedirect.com/science/journal/15708705

http://www.elsevier.com/locate/adhoc

http://creativecommons.org/licenses/by/3.0/

http://creativecommons.org/licenses/by/3.0/

M. Radenkovic, A. Grundy / Ad Hoc Networks 10 (2012) 1322–1345 1323

networks as they have been shown to exhibit the pathexplosion phenomenon [9]. The main drawback of adaptiveforwarding is that it does not decrease the total level oftraffic in the network but only redirects it. This is subopti-mal at times when finding alternative non-congested partsof the network is not possible. Adaptive replication man-agement techniques [39,38] can decrease the traffic in re-sponse to congestion, but they cannot adaptively offloadthe traffic from more congested to less congested parts ofthe network. This means that they cannot efficiently dealwith scenarios when congestion affects some regions ofthe network (e.g. hotspots, roadside units) and not theothers.

In this work we propose to unify adaptive forwardingand adaptive replication into a common congestion controlframework for DTNs (CafRep) that manages to both de-crease the load on the network and offload the traffic tothe parts of the network that are less congested. Weachieve this by using a local based implicit heuristic basedon contact and resource statistics that extends our previ-ous work on adaptive forwarding [10–12]. More specifi-cally, we propose to dynamically combine three types ofheuristics: node centrality and contact analysis driven heu-ristics that exploits contact relationships to allow optimaldirectionality and delivery probability of a node; node re-source driven and ego network driven heuristics to adaptto the nodes and parts of the network’s buffer availability,delays or congesting rates. As a result, our new frameworkadaptively changes forwarding and replication behaviourto best manage tradeoffs across multiple contact and re-source attributes of nodes in real network scenarios withdifferent mobility, and connectivity patterns.

Previous work on resource aware DTN protocols haseither focused on the homogeneous networks where theauthors assumed nodes to have the same amount of re-sources on their devices [51] or looked at only one connec-tivity trace [15,39]. However, it is not realistic to assumenetwork homogeneity as mobile devices can be used in dif-ferent ways, move in different ways and do not consumetheir resources at the same rates. In this paper we explorehow a more intelligent resource aware model, reflective ofthe heterogeneity of devices’ resources, and non-uniformconnectivity patterns, and application demands can im-prove data delivery and dissemination in the network.

Section 2 discusses the lessons learned and shortcom-ings of DTN forwarding, replication, load distribution andcongestion control approaches further, together with stateof the art work from related areas such as MANET conges-tion control, resource pooling, Peer-to-Peer (P2P), as wellas an algorithmic game theory perspective of selfish for-warding and routing.

Section 3 specifies the particular challenges in opportu-nistic delay-tolerant networks that prevent successful datadelivery and identifies our criteria that guide our proposalfor congestion control framework for adaptive replicationand forwarding – CafRep. We present an analytical modelthat identifies fundamental problems regarding traffic dis-tribution and shortest path forwarding heuristics in rela-tion to the traffic flow and the price of anarchy.Modelling the behaviour of opportunistic networks is achallenging because of time varying network topology

and congestion, and thus we integrate the concepts of atime varying networks [12,35] and dynamic flows [13].We describe the design space of our proposal, give multi-layer architectural overview of our conceptual model andCafRep pseudo code. We identify and describe the corecongestion signals, heuristics and techniques that are atthe core of our proposal and allow adaptive disseminationof messages throughout the network such that we addressour criteria i.e. allow spreading the traffic across multiplepaths whilst avoiding congested regions and minimisingdelays and network overheads. We extend our previouswork by discussing the impact of a number of different Caf-Rep utility weighting schemes on the protocol perfor-mance over varying network topologies.

Section 4 describes our evaluation methodology. Asnodes’ encounter patterns can greatly differ for social andvehicular networks, it is important to evaluate our CafRepprotocol across different real traces from CRAWDAD [1] inorder to gain better understanding of our protocol perfor-mance. Section 4 begins by discussing the heterogeneityof the three chosen real connectivity and GPS traces. Wethen motivate the use of two different application scenar-ios: publish subscribe podcasting and real Facebookapplication.

Section 5 discussed our extensive evaluation of CafRepagainst some of the major state of the art DTN protocolsincluding a benchmark DTN forwarding algorithm Prophet[23], an adaptive forwarding algorithm Café [31], a fixedreplication algorithm Spray and Focus (SF) [37] and twoadaptive replication algorithms Encounter Based Routing(EBR) [26] and Retiring Replicants (RRs) [38]. We considerseven different metrics for the performance analysis of theprotocols including success ratio, delay, buffer availability,packet loss rates, number of forwarded and replicatedpackets, and delivery cost. We show that our protocol out-performs all four other algorithms across majority of met-rics across the three chosen heterogeneous traces that havedifferent mobility and connectivity patterns. We identifydifferences in protocols’ performances across the differenttraces and discuss multiple reasons for causing that. Webriefly describe the results with the different Facebooktraffic types that show that regardless of the type of trafficCafRep aims to deliver, it reduces congestion and improvesforwarding performance in the network.

Section 6 concludes the work and presents the openquestions for future work.

2. Related work

2.1. DTN forwarding, replication, load distribution andcongestion control

This section discusses the most recent advances in mes-sage replication and forwarding for DTNs. Early work in thisarea focuses primarily on the challenge of reducing thedelivery latency and cost with the underlying assumptionof unlimited transfer and storage capacity[40,23,17,8,36,37,25]. The majority of research in conges-tion control for DTNs is concerned with buffer management[6,22,24,34], but recent developments have been concerned

1324 M. Radenkovic, A. Grundy / Ad Hoc Networks 10 (2012) 1322–1345

with replication management [34,39,30] and distribution[26,28,15].

Encounter Based Routing (EBR) proposed in [26] is aquota based replication protocol that aims to forwardmore copies of a message to nodes that are better con-nected. The introduction of limited resources and increas-ing traffic demands cause key components in the networkto become congested, which in turn leads to messagesbeing dropped.

Retiring Replicas (RRs) [38] is an adaptive replicationprotocol that adjusts a nodes’ copy limit based on the lo-cally perceived global level of congestion. RR proposesthe nodes to create their own congestion view (CV) asthe ratio of drops and duplicate deliveries and compare itto the congestion threshold. Depending on the comparisonthe copy limit for new messages is lowered or raised fol-lowing a back-off algorithm. This work assumes a uniformnetwork with random waypoint mobility. In reality thenetworks are likely to be non-uniform and the level of con-gestion may vary between different regions of the network.This work has not proposed how the network adjustmentswould compensate for differing local conditions.

In Cafe [15] we proposed and examined several com-bined social and resources heuristics in order to detectcongested parts of the network and move the traffic awaytowards less congested parts. These heuristics include so-cial, delay and buffer metric of nodes and their ego net-works. The total combined utility function we propose isat the core of our adaptive forwarding protocol that is dy-namic and flexible as it operates as a pure social (contactdriven) protocol at times of low congestion but is highlyresource driven at times of high congestion. We show thatour single copy adaptive forwarding protocol achieves bet-ter performance in comparison to multi-copy protocolssuch as spray and wait (SW) [36], Spray and Focus (SF)[37], Prophet [20] and epidemic protocols in terms of de-creased delays, higher availability of nodes and higher suc-cess ratios.

In [30] we built an interest-driven P2P content dissem-ination overlay on the top of our congestion aware for-warding protocol. Both caching and forwarding policiesare decided based on the interest, availability, social close-ness and numbers of interested nodes. Our results showthat our adaptive overlay manages to maintain high suc-cess ratio of answered queries, high availability of interme-diary nodes and short download times for a P2P file castingapplication running in the face of increasing number of filepublishers and topic popularity.

FairRoute [28] argues that considering only contact his-tories to define contact duration, frequency and interactionstrength cannot achieve balanced traffic distribution.FairRoute proposes nodes queue length to be evaluatedin order to allow nodes to only forward to contacts witha bigger queue size, but this does not avoid congestingpopular nodes and leads to packets being dropped.

Storage Routing (SR) [34] avoids congested nodes drop-ping packets by sending a set of messages out to neigh-bours with available storage, when buffer capacity is freethese are retrieved. This protocol only temporarily allevi-ates congestion as it relies on contacts that are presentnot suffering from congestion themselves.

Autonomous Congestion Control (ACC) [6] implementscongestion control by apply a financial model to bufferspace management, in order to propagate buffer utilisationstress backwards through the network to the source nodes.This work does not overcome intermediary/source nodesfilling their own buffers and isolating peripheral nodes.

[38] propose DA–SW (Density-Aware Spray-and-Wait),that is a measurement-oriented variant of the spray-and-wait [36] algorithm that dynamically determines the num-ber of a messages disseminated in the network in order toachieve constant delay. DA–SW relies on the current aver-age node degree in the roller tour. Whenever a node has abundle to transmit, it computes its current connectivitydegree and refers to the abacus to determine the exactnumber of copies that is expected to lead to some expecteddelay. The authors did not address the impact of their staticmeasurement window (30 s) on the performance of theirsystem. This work does not consider dealing with resourceconstraints such as node buffers, bandwidth and energyconsumption.

2.2. Resource pooling

[42] believe that the natural evolution of the Internetshould be to harness multipath-capable end systems in or-der to achieve resource pooling. [9] show that opportunis-tic networks typically exhibit the path explosionphenomenon. In our work, the nodes benefit from poolingthe capacity of their many and varied contacts, makingeffective use of the network resources available to them.If traffic is spread across the resources of a node’s manyand varied contacts in the right way, with the right reac-tion to the right congestion signals from the network, thentraffic can quickly move away from congested regions.

2.3. Peer-to-peer replication strategies

We observe the similarities between content dissemi-nation in opportunistic networks and in the related fieldof peer-to-peer (P2P) content dissemination and storagesystems. Although P2P networks operate in the applicationlevel we believe lessons can be learned from the work inthis area. In applications such as BitTorrent, peers replicateeach other’s data in order to increase data availability [32],also resulting in the pooling of the upload capacity of manynetwork nodes [42].

[32] studied the problem of replica placement in a P2Psystem intending to optimise availability and/or the num-ber of replicas. [32] show that centralised control of re-source placement is a NP-hard problem and that if thecontrol is fully decentralised the peers selfishness cangreatly alter the results leading to performance inequitiesthat can render the system unreliable and thus ultimatelyunusable [32,29]. The most common approach to P2P rep-lication is the random distribution of copies [7,5]. [4] ana-lyse how many randomly placed replicas are required toachieve a desired level of availability. [32] argue that rep-lication should not be random, but be based on cliques ofpeers replicating each other’s data, limiting the selfishnessof the participants.

Table 1Overview of the techniques that deal with efficient data dissemination in DTNs.

EBR FairRoute Storage routing DA–SW RR ACC P2P Cafe

Adaptive forwarding No Yes Limited (offloading) Yes No No Yes YesAdaptive replication Yes No No Yes Yes No Yes NoCongestion aware No No Limited Limited (relies on the abacus) Yes Yes Yes YesSocial/contact aware Yes Yes Limited Yes Yes No No YesResource aware No No Yes No No Yes Yes YesFully localised Yes Yes Yes No (relies on the abacus) Yes No Yes YesDelay tolerant Yes Yes Yes Yes Yes No No Yes


Table 1 presents a summary of the techniques discussedin this section in terms of seven criteria. Note that none ofthe existing approaches have support for adaptive for-warding and replication, fully localised, social and resourceaware congestion control in DTNs.

3. Congestion aware forwarding and replication forDTNs

3.1. Challenges

Varying mobility patterns, topology changes, discon-nections and resource restrictions pose many challengesfor the design and implementation of congestion awaredata transmission in DTNs. This section systematically out-lines particular challenges that motivate our criteria de-scribed in Section 3.2 that guide our proposal.

3.1.1. Distributed decisionsThe limited connectivity and fragmented nature of

opportunistic networks mean that it is neither efficient toobtain and maintain a global level of knowledge of the net-work nor to rely on closed-loop (acknowledgement based)techniques for any decisions. We aim to allow each devicein the DTN network to act independently and base theirdecisions on the limited localised knowledge with theaim to enable better performance of the whole network.

3.1.2. Limited resourcesAs buffer capacity, transfer bandwidth and battery life

are limited, the messages can only be transferred success-fully if the efficient coordination between the devices com-peting for resources is provided.

3.1.3. Network density and localised surges in trafficNode connectivity in DTNs is sporadic and islands of

connectivity may range in size, from sparse to dense.Sparse networks present limited forwarding options atany given time, while dense networks are prone to suffer-ing from transmission collisions due to wireless interfer-ence. We aim to propose a protocol that could work wellacross these highly different network scenarios.

3.2. Criteria

In response to the challenges described in Section 3.1,we focus on the following research question:

How can disconnection prone nodes with differentmobility and connectivity patterns and with limited

resources, communicate in an efficient, adaptive and ro-bust manner when there are a large number of datasources and destinations?

We address this question by identifying the followingcriteria that guide our proposal.

3.2.1. EfficiencyWe define efficiency in terms of providing support for

minimising the traffic latency and optimising utilisationof network resources. More specifically, for DTN forward-ing algorithms traffic latency is a key concern, as the fresh-ness of data is important and it is persistently challengedby disconnections and congestion of intermediaries thatprevent efficient store-carry-and-forward routing. Simi-larly, DTN forwarding algorithms identify a subset of betterconnected nodes to which they forward the majority of thetraffic. As these nodes become overloaded, forwarding to alower ranked node may lead to more even spread of thenetwork load in the network and lower congestion rates,but also to increased number of intermediaries for the for-warded messages. We aim to provide a mechanism thatmanages to dynamically balance more even utilisation ofnetwork nodes while keeping the traffic delays and num-ber of forwarded packets low.

3.2.2. AdaptationDTN nodes typically have self organised fully distrib-

uted behaviour which means that they base their decisionson the knowledge gathered in their local environment. Aswe aim to optimise network wide behaviour based onnodes localised decisions, the question of how the individ-ual nodes can get the feedback about the remote networkstate and affect it becomes highly challenging. We aim toprovide an adaptive mechanism that achieves network-wide optimisations by only fully relying on localised nodeaggregate heuristics and multi-dimensional metrics.

3.2.3. RobustnessWe define robustness in terms of providing increased

packet resiliency and collaboration between the nodes.More specifically, due to nodes limited battery resourcesand high mobility, our aim is to investigate strategies forensuring that the effect of dropped packets is minimal onthe final delivery rates. As the closed-loop acknowledge-ment-based loss recovery is ineffective in DTN environ-ments, we look into packet replication techniques thatcan help with this. Similarly, in order to avoid greedy local-ised node-only behaviour that leads to decreased interme-diary resources and lower network-wide performance, we


aim to provide a mechanism that carefully balances oppor-tunistic usage of intermediary resources and cooperativebehaviour that leads to improved end to end delivery rates.

3.3. Analytical model

We model the network as a temporal graph G = (V,E)because the connectivity of the network E and the stateof the nodes V change over time. We model each of theseas time series and depict the vertices as V = {Vt: t 2 T} andthe edges as E = {Et: t 2 T}, where t is a member of the timeseries T. We assume that connectivity is bidirectional andtherefore the edges of the graph are undirected, the edgeconnecting nodes u,v 2 Vt we denote as {u,v} 2 Et.

The traditional representation of a path in a graph that iscommonly used to depict the route that a message is trans-mitted along is an alternating sequence of vertices andedges. In this work we model a path as a sequence of re-source locations a message occupies, where each index rep-resents a particular time interval e.g. P = (x0,x0,e1,x1, . . .,ek,xk) where ei = {xi�1,xi}. Intuitively the best route is thepath with the smallest resource cost, i.e. the path with theminimum number of transfers and shortest storage time.Each second a message occupies storage or transfer band-width it adds cost to a network resource, and it can be moreefficient for messages to travel via a greater number of hopswith smaller in-network delays in order to arrive at the des-tination, rather than to be kept in storage waiting for a highdemand resource to become available adding to the mes-sages latency. We aim to carefully manage the trade-off be-tween increasing the storage occupation and increasing thehop count. Using alternative paths is particularly suitablefor social DTNs due to the path explosion phenomenon [9].

The demand of a resource is dependent on the set of for-warding demands. The amount of demand for a resource Xat a time t is given by:

DtX ¼

X

S–X2V

X

D–X2V

FtSDðXÞ

where FtSDðXÞ denotes the number of paths between source

node S and destination D that contains the resource X attime t. Which paths contain X is influenced by the effectsof the forwarding strategy.

Each resource X in the network can have a differentstress level at any given time t as a result we denote stressas STt

x ¼Dt

xCx

which is a measure of demand DtX of a resource

X at a given time t against the capacity for that resource CX.Packet loss occurs when Dt

X > CX i.e. the level of demand Dtx

of a resource X at a given time t is greater than the resourcecapacity CX. The total cost of delivering a single copy of amessage is the sum of all storage and transmission occur-rences in the lifetime of a given message along a path be-tween the source and the destination. As messages canbe replicated, one or more copies of the message are trans-mitted, each following an independent path. The numberof paths is limited by a replication limit (M) and the costof delivery for a replicated message is the sum of all pathcosts in the replication path set.

In order to impartially evaluate the inherent load distri-bution of a network we observe the stress effects of a uni-formly non-biased forwarding strategy that selects the

next hop randomly. The result of selecting the next hoprandomly is congruent with a random walk and thereforenodes that are better connected are more likely to receivemessages. Forwarding based on a heuristic which favours anode because of its connectivity puts well connected nodesin even greater demand than when simply randomly se-lected. Connectivity observations such as: how recently anode has encountered the destination, the duration of con-nectivity a node has experienced with the destination, howfrequently a node encounters the destination and a nodesbetweenness or degree centrality are each used as heuris-tics within forwarding strategies. The better the node isconnected the greater the probability it has of fulfillingthe criteria of this type of forwarding strategy. Forwardingbased on connectivity is a method of seeking the shortestpath. Shortest path routing is greedy and it can be the bestsolution in a network with no opposing traffic. However,this is not realistic because it assumes no delay. Delay isrelative to the level of congestion experienced by a re-source X at a given time t, and is a measurement of the cur-rent demand D and buffered demand B over the number ofavailable outlets for the resource X at a given time dividedby the degree centrality of the node X at time t dt(X) whichcan be denoted as delaytðXÞ ¼ Dt

XþRm2Bt

dtðXÞ .We define resource consumption as subgraph G0 = (V0,E0)

where the set of vertices are defined as the set of verticesthat have a demand greater than 0 V 0 ¼ 8v t 2 V : Dt

v > 0and the set of edges is defined as the set of edges that havea demand greater than 0 E0 ¼ f8et 2 E : Dt

e > 0g. Networkutilisation can be measured as difference between theavailable resources G and the consumed resources G0 de-picted as U ¼ jG

0 jþkG0kjGjþkGk where |G| and |G0| denote the size of

the set of vertexes for G and G0, and ||G|| and ||G0|| denotethe size of the set of edges for G and G0. The relationship be-tween F and G is constrained such that 0 6 U 6 CG where CG

is the total capacity of the graph. The forwarding criteria Hinfluences the size of F (the number of paths that wish touse these resources) based on its utilisation of G. Giventwo forwarding criteria H and H0 each utilising the networkto the value of U and U0 respectively, H has a greater capac-ity than H0 if U > U0. H is therefore more effective as F canutilise more of the elements of G.

Existing DTN forwarding algorithms direct traffic to-wards the most desirable next-hop nodes as this is theoptimal solution when the network is free. When the traf-fic demands increase, these nodes become inundated andthis is made even more challenging as the flow of trafficis unpredictable and has a tendency to accumulate in someregions of the network [2] shows that the price of anarchyis unbounded if the optimal forwarding solution is basedpurely on minimising delay in a flow-independent model,but that this can be improved by forwarding based onhow congested the resource is in addition to the delay cost,resulting in the price of anarchy being at most 4/3 providedthe cost functions are all linear or d/logd if the cost func-tions are polynomials.

3.4. Design space and multi-layer overview of CafRep

We briefly describe the design space for CafRep andshow different dimensions of possible approaches that


can be useful for congestion aware dissemination in DTNsin Fig. 1. The vertical axis represents the Direction Influ-ence that refers to the connectivity dimension such as cen-trality, similarity, interaction strength and other socialcontexts that identify affiliation such as clique identifica-tion and delivery probability used to influence the socialforwarding decisions. The two horizontal axes representthe resource dimensions. The multi-path transport ap-proach, such as Café [14,31,15,30], introduces methods ofavoiding congested regions of the network. The replicationrestriction technique, such as Retiring Replicants [38],helps to increase network capacity by reducing the in-net-work occupancy of redundant messages. Our CafRep’s de-sign philosophy is to consider the combination of routeoptimisation with multi-path transport, sending raterestriction and social connectivity.

CafRep takes a multi-layered approach that is illus-trated in Fig. 2. The edges in the graph on the NetworkLayer illustrate connectivity and the vertices representthe nodes. In reality the connections go up and down overa period of time and the network can get disconnected forthe majority of time, but for simplicity the time seriesinformation has been flattened. We assume the following:

Fig. 1. Design space.

Fig. 2. Multi-layer view.

source node and destination node belong to the sameinterest group, multiple paths exist between the two nodesand the socially optimal route is also the most congestedpath. The Interest Layer, a part of the Application Layer,maps users into areas of interest as each application canhave its own topics and interest groups. CafRep Layer com-bines Congestion Layer and Social Layer in order to broad-en the next hop selection criteria allowing nodes withcapacity, or less direct routes, to receive messages bymonitoring both social and congestion signals and dynam-ically balancing between them. The Network Layer, on thePhysical Layer, illustrates the actual route a message wouldtake through this example network, given the trade-off be-tween shortest path and resource-driven routing, in orderto redistribute load to avoid congestion as identified inour criteria.

3.5. Congestion aware forwarding and replication: CafRep

3.5.1. CafRep design overviewWe describe unified adaptive forwarding and adaptive

replication management approach into a commoncongestion control framework for DTN routing, CafRep(Congestion Aware Forwarding and Replication). CafRepworks as a local adaptive forwarding and replication proto-col that diverts the load from its conventional social awarepath at times of congestion and directs it via a differentpath that decreases the load of hotspots and end-to-enddelays while keeping high success ratios. It dynamicallycombines three types of implicit congestion heuristics: so-cial (contact) driven heuristics that exploits contact rela-tionships among nodes to allow optimal directionalityand delivery probability of a node; node resource drivenheuristics that aim to adopt to the nodes’ buffer availabil-ity, delays and congesting rates; and ego network (regio-nal) driven heuristics that aim to detect and adapt bufferavailability, delays and congesting rates of different partsof the network. Selecting the node that represents the bestcarrier for the message and deciding on the optimal num-ber of replicas to forward are both multiple attribute deci-sion problems across multiple measures, where the aim isto select the node and number of messages that providethe maximum utility for carrying a certain number of mes-sages. To achieve this we propose CafRepUtilD heuristic(Formula (1)) that is responsible for determining the over-all improvement an encountered node represents whencompared to the sending node, and deciding on whichnodes will be selected as next hop and how many copiesof the message are to be disseminated. Formula (1) showsthe CafRepUtilD utility calculation node (X) uses when for-warding a message towards a destination (D) in order toevaluate each encountered node (i) within its contact set(C) as the sum of the set of carefully selected equallyweighted utilities of different congestion heuristics:

CafRepUtilDðXÞ ¼X

h2H

UtilhðXÞ ð1Þ

CafRepUtil gets calculated for all contacts of the sendingnode (node X) and selects the best next hop as the nodewith the highest CafRepUtil value if its CafRepUtil is higherthan the sending node’s. The number of messages to be


sent to it is decided by Formula (2). If node X has more thanone copy of the message ReplRate shows how the numberof copies can be divided between the two nodes. This divi-sion is dependent on the CafRepUtil utility value of eachnode; therefore, the division of the replication rate forthe destination d between node X and node C(X) is givenby Formula (2) where M is the number of replicas of a mes-sage m at node X.

ReplRateDðXÞ ¼ M

� CafRepUtilDðCðXÞÞCafRepUtilDðXÞ þ CafRepUtilDðCðXÞÞ

ð2Þ

There are seven non-trivial heuristics h(X) that coverdifferent dimensions of the problem and when combinedthey allow managing a number of trade-offs between dif-ferent challenges we identified in Section 3. For each ofthe heuristics we define their respective utilities as mea-surements of their relative gain, loss or equality, calculatedas pair-wise comparison between the node’s own conges-tion heuristics and that of the encountered contacts. Morespecifically, we use a pair-wise comparison matrix on thenormalised relative weights of the social and resourcesheuristics of nodes and their ego networks in the followingway.

UtilhðXÞ ¼hðCðXÞÞ

hðXÞ þ hðCðXÞÞ ð3Þ

The set of our selected congestion heuristics includes:node retentiveness (Ret), node receptiveness (Rec), nodecongesting rate (CR), and their weighted ego-networkcounterparts (WENRet, WENRec, WENCR), along with a nodesocial heuristic (Social), and is given in Formula (4). In thispaper, we use SimBetTS as our social heuristic that wasintroduced in [8].

h 2 H ¼ fRet;Rec;CR; Social;WENRet;WENRec;WENCRg ð4Þ

Before we move to describing CafRep algorithm in agreater detail we briefly describe the congestion heuristicsused in CafRep for completeness purposes. More detaileddescription of these heuristics is given in [31,48].

Retentiveness (Ret) refers to the node’s available stor-age for the new packets that are sent to them. Retentive-ness is an important attribute to consider because of thestore and forward nature of opportunistic DTN networks.Nodes with limited storage, either due to popularity orsimply due to more limited hardware constraints, are moresusceptible to packet loss. Retentiveness is calculated as anexponentially weighted moving average of a node’sremaining storage. Formula (5) shows retentiveness of Xis calculated as the sum of all message occupancy sub-tracted from the node’s buffer capacity (Bc(X)).

RetðXÞ ¼ BcðXÞ �XN

i¼1

misizeðXÞ ð5Þ

Receptiveness (Rec) refers to the node’s ability to re-ceive packets and forward them on. This is an importantobservation as increasing in-network delays is an indica-tion that the volume of traffic a node or region is receivingis greater than the bandwidth available to it for offloading.The total current message delay is calculated as the sum of

the difference between the time each message in a node’sbuffer was received and the current time. The delay be-tween receiving a message and forwarding a message isconstrained by the size of the buffer and the bandwidthavailable for a node to offload the messages. Nodes withlarge amounts of storage are more susceptible to receivingmore messages than they are capable of offloading. For-mula (6) shows receptiveness is the total current messagedelay, calculated as the sum of differences between thecurrent time (Tnow) and the time each message was re-ceived (Mreceived).

RecðXÞ ¼XN

i¼1

ðTnow �mireceivedðXÞÞ ð6Þ

Congesting rate (CR) refers to a measure of how fast anode is likely to congest, an important observation as itindicates stability. Congesting rate is calculated as the per-centage of time a node or region has been congested di-vided by the average time between fullness periods forthe node or region, given in Formula (7). This congestionsignal indicates the likelihood of traffic spikes that couldcause the message to be dropped. Each node keeps trackof the percentage of time it has been full (T%Full(X)) inFormula (7a), and the average time between its fullnessperiods (TAT(X)) in Formula (7b).

CRðXÞ ¼ T%FullðXÞTATðXÞ

ð7Þ

T%FullðXÞ ¼ 100 � TFullBufferðXÞTTotalTimeðXÞ

ð7aÞ

TATðXÞ ¼1N�XN

i¼1

TiendðXÞ � TistartðXÞ ð7bÞ

Note that TFullBuffer(X) is the time the buffer has been andTTotalTime(X) is the total time duration. Tistart(X) and Tiend(X)are the start and end of between full buffer periodsrespectively.

Ego Network retentiveness, receptiveness and congest-ing rate (WENret WENrec WENcr) refer to congestion heuris-tics of the node’s ego network. Ego network (EN) is definedhere as a network consisting of a single node together withthe nodes they have encountered and gives each node theirown perspective of the network. CafRep allows nodes toaggregate resource observations disseminated by encoun-tered nodes in order to form an ego-network perspectiveof the network. Ego-network information can be aggre-gated in many different ways and we have explored anumber of models for weighting the contacts within anodes ego-network in order to improve the accuracy ofprediction of the EN congestion levels. This is highlyimportant as it leads to better performance for both for-warding and replication optimizations and making thenodes less selfish. More specifically, we have consideredtechniques such as simple average, weighted moving aver-age (EWMA) and social weighting of the nodes ego net-work congestion heuristics. Our experiments have shownthat EWMA gives better performance than the simpleweighting and the social weighting across diverse network


topologies. Formula (8) shows how we use EWMA toaggregate congestion heuristic information in order to al-low the short-term fluctuations to be smoothed out andlonger-term trends to be highlighted making it suitablefor forecasting. This is updated at each new encounter foreach congestion heuristic (h).

WENhðX;CiðXÞÞ ¼ ð1� kÞ �WENhðXÞ þ k � hðCiðXÞÞ ð8Þ

k is a fraction that represents the responsiveness of thesmoothing, this is typically around 0.8 [38,26]. Relatedstate of the art work on congestion control in DTNs[38,26] that uses implicit local congestion signals to esti-mate global congestion levels uses EWMA with values of0.85 or 0.9 for the most recent value a node observes. In[38], each node independently calculates a local approxi-mation of the current congestion level as the ratio of dropsand replications collected in the last time interval. [38] cal-culate new congestion value using an estimated weightedmoving average (EWMA) with the new congestion viewbeing weighted 0.9 to keep node’s congestion view fresh.Similarly, in [26], the locally calculated encounter value(EV) used to track node’s rate of encounter uses the expo-nentially weighted moving average that places an empha-sis proportional to 0.85 on the most recent completeCurrent Window counter. The authors argue that theirexperiments showed that k of 0.85 and update interval ofaround 30 s allow for reasonable results in a variety of net-works. In our experiments we have chosen a value of 0.8even though results were similar for 0.85 and 0.9.

3.5.1.1. CafRep algorithm. Our CafRep algorithm functionsas follows. When a forwarding node (X), meets contactson its way, it exchanged relevant heuristics and calculatesthe CafRepUtil of each contact. The CafRepUtil allows thenode X to detect how well connected its contact Y is andhow available Y and Y’s ego network are in terms of buffer,delay and congesting rate parameters. If there are multipleencounters, node X sorts them in the reverse order (fromthe highest to the lowest) in terms of their CafRepUtil, cal-culates respective sending and replication rates for eachnode and sends the appropriate ReplRate copies of themessages to each one of them invoking the pseudo codefor interest-driven content transmission over CafRep givenin Fig. 4. If the receiving node still has available resourcesafter receiving all the topics of interest, node X will sendadditional messages it has in its buffer (in a FIFO manner)as long as they are not duplicates. The reason for this isthat the node X should offload the messages to nodes withbetter resources even if they are not directly or indirectlyinterested in it as they will have better opportunities toforward these messages on. If two contacts are equally sui-ted to forward a message, they each receive half of theavailable copies. A contact receives more or fewer mes-sages depending whether it has greater or smaller utilityrespectfully. The replication rate is rounded to the nearestinteger so that in the single copy case messages are prop-agated provided a minimum of equivalent utility is met.In this way, CafRep adapts the initial number of copies(M) in order to carefully manage the trade-off betweenthe network size and traffic demands.

As CafRepUtil of a node moves up and down, the repli-cation limit grows to take advantage of all available re-sources, but backs-off when congestion increases, similarto how TCP updates its congestion window [22]. As a re-sult, our protocol is able to replicate at adaptively lowerrates in the parts of the network that have low buffer avail-ability, increased node delay and are likely to congest fas-ter. As CafRep node discovers parts of the network withhigher buffer availability, lower node delays and slowercongesting rates, it replicates at higher rates. Using socialutilities together with resource utilities as part of CafRep-Util enables little (or no) replication at high rates on nodesand network regions that available but not on the directpath to the destination. At the same time, using ResourcesUtilities allows CafRep to replicate at nodes that do nothave high social utilities but have high available resources.With the use of combined adaptive forwarding and replica-tion, we allow the sender to stop sending until it finds theright node that it can redirect the traffic to without incur-ring additional packet loss. Using ego network resourceutilities in addition to social utilities allows CafRep to ac-count for a wider view of the network resources and con-nectivity patterns while allowing differing localconditions. This is very suitable for the DTNs where thenodes and parts of the networks can be highly heteroge-neous in terms of connectivity and resource parameters.We argue that CafRepUtil congestion metric is more effi-cient for heterogeneous DTNs than the metrics that esti-mates global congestion parameters in DTNs proposed inEBR and RR as it conveys information about which partsof the network are more congested than the others, andcan opportunistically use parts of the network that areavailable while the others are busy. In this way, CafRep en-ables replication at different rates at different parts of thenetwork that are dissimilar from each other and thus havedifferent social and resource characteristics and patterns.

3.5.1.2. Content transmission over CafRep. We describe anexample of an interest driven overlay for content dissemi-nation on the top of CafREP in Fig. 4. We assume that thecontent contains topics and each topic has chunks thatcan be exchanged when the two nodes meet via summaryvector. Each chunk has a unique ID. For easier bootstrap-ping, we assume that some subscribers that are known tothe publishers, while others are discovered in an ad hocmanner. When propagating resource and centrality infor-mation, our nodes also propagate their interests byexchanging their interest profiles (topics they are inter-ested in) and summary vectors in order to avoid sendingchunks to nodes that are already carrying the same chunks.The Summary Vector contains a list of chunk IDs per topicthat a node currently carries per topic.

As described in Fig. 3, when two nodes meet, the send-ing node scans for neighbouring nodes’ and calculates theirrespective relative CafRep Utilities. Each neighbouringnodes’ CafRep Utility is weighted and sorted in a list so thatthe node with the largest weight appears first in the list.When the sending and replication rates are determinedfor a node, pseudo code in Fig. 4 is invoked per node totransmit the content to each node. Before the sending nodestarts sending the chunks to its contact, the receiving node

Algorithm CafRep List CafeRepUtils = {} For each Contact in scan do: Contact.NodeHeuristics = exchangeNodeHeuristicsInfo(Contact) Contact.CalculcateNodeUtilities(Contact.NodeHeuristics) Contact.CafeRepHeuristics = exchangeCafeRepHeuristicsInfo(Contact) Contac.CalculateCafeRepUtility(Contact.CafeRepHeuristics) CafeRepUtils.Insert(Contact) End For

For each Contact in reverse sortedCafeRepUtils do: Contact.SendingRate = CafeRep.ComputeSendingRate(Contact) ...Contact.ReplRate = CafeRep.ComputeReplRate(Contact)...Contact.Sent = TransmitContent(Contact, Contact.SendingRate, Contact.ReplRate)End For

For each Contact in reverse sorted CafeRepUtils do: If Contact.Sent < Contact.SendingRate then: TransmitFifoTo(Contact, Contact.SendingRate – Contact.Sent, Contact.ReplRate) End For

Fig. 3. CafRep pseudo code.


is queried about its topics and chunks that it already hasper topic. When the queried node responds, the sendingnode finds the intersection of the topics between the twonodes and then it calculates for each topic the complementof the two summary vectors the nodes have to determinethe list of topics and associated chunks to be sent withoutduplication.

3.5.2. Exploring different weighting for congestion heuristicsCafFRep adaptively moves the traffic away from the

more overloaded parts of the network to freer parts ofthe network while avoiding greedy choices of choosingthe currently more available nodes that may congest lateron. The decisions of how cautious (versus how greedy) a

Algorithm: TransmitContent over CaFunction TransmitContent(Contact, Rate, Re Vector ContactTI = Contact.obtainTopicInt...Buffer SendBuffer = {} For each Topic in Node.TI • ContactTI do: ...Vector ContactTopicSV = Contact.obtainSu

...For Chunk in Node.Topic.SVcomplement Chunk.ReplCount = Repl SendBuffer.append(Chunk) ......If SendBuffer.Length >= Rate then: .........Break ......End If

End For End For ChunksSent = Transmit(SendBuffer) Return ChunksSent

Fig. 4. Content o

node should be when choosing the next hop and decidingon the number of packets are not trivial to make and coulddepend on the state of the network and relative weights ofthe CafRep utilities. At times of low congestion, social util-ity is the primary utility for forwarding and replication pol-icy but as the congestion increases the social utility plays asmaller role while resource considerations becomeincreasingly important. More specifically, social utility al-lows the nodes to find the most direct route to the destina-tions and gives more copies to the nodes with higher socialutility value but can congest them and cause them to beunusable. When congestion happens, the combined utilityallows CafRep to de-cluster individual nodes and parts ofthe network by leveraging social metric with resource

fRep pl)

erestList(Contact)

mmaryVector(Contact, Topic)

ContactTopicSV do:

ver CafRep.


constraints. In Section 3.5.1 we defined CafRep utility asthe sum of the equally contributing utility values. Comput-ing optimal weights that adaptively favour different utili-ties differently at different times is a very difficultproblem even with the complete knowledge about theenvironment [47]. In the case of inherent uncertainty ofthe DTN environments, extreme heterogeneity of the con-nectivity patters and no feedback, related research [47] hasruled out provably efficient online routing algorithms. In[46], the authors analysed the impact of storage and trans-mission limitations on DTN message routing by and aimedto provide a comprehensive formalisation of this problembased on the first principle of Wardrop, but their basedtheir work on the Oracle based routing algorithm. This isnot applicable to our work as we are interested in fully dis-tributed opportunistic approach.

Even though we do not aim to theoretically analyse dif-ferent utility weighting models, we have, in our earlierwork [31], considered assigning different weights to eachof the utilities for Café in order to empirically explore theirrelative importance on the adaptive forwarding. Our exten-sive experiments over RollerNet trace [21] revealed a num-ber of interesting findings. We showed that utilising only asingle forwarding strategy based on one node utility leadsto suboptimal next hop choices. For example, in [15] reten-tiveness was weighted significantly higher than receptive-ness and our results showed increased delays anddecreased success ratios over Infocom 2006 trace [33] com-pared to when they were equally weighted [31]. Similarly,when we weighted the receptiveness significantly higherthan the retentiveness, Café performance was lower dueto the increased packet loss rates [31] as a result of greedystorage utilisation. Another interesting finding was thatusing ego network retentiveness only was highly valuablefor forwarding in social opportunistic networks such asInfocom trace [33]. In particular, even a simple ego networkretentiveness utility for the majority of time performedbetter than more sophisticated analysis of the node only re-sources statistics. This was expected as Infocom is a socialtrace with high degree of reoccurring contacts. Both ofour empirical results with Café over RollerNet and Infocom2006 traces [21,33] revealed that combined metric of sim-ilar weights allows the forwarding protocol to be suffi-ciently dynamic and flexible to operate as mainly socialprotocol at times of low congestion and as a mainly re-source driven protocol at times of high congestion.

In this section, we describe our experiments with differ-ent CafRep weightings across three topologically differenttraces: one highly social (Infocom 2006 [33]), one sparsesocial (Sassy [44]) and one highly sparse vehicular (SF Cabs[45]). These traces are described in Section 4.1. As withCafé evaluation, extreme differences between CafRep util-ities do not result in performance gain, so here we lookat the finer differences between the CafRep utilities’weightings. As per each individual trace, we could not de-tect dramatic differences in the performance results be-tween different CafRep weightings. Fig. 5 shows no morethat 5% difference in success ratios between the best andworst performer per trace. However, we could clearly de-tect that some utilities were significantly more dominantthan the others across different traces.

More specifically, in order to understand the impact ofthe SocialUtil, we looked into High SocialUtil (0.6 of theCafrepUtil), Low SocialUtil (0.3 of CafRepUtil) and EqualSocialUtil (0.5 of the CafRepUtil) while the node utilitiesend ego network utilities were ranked equally. Figs. 5and 6 show that CafRep with lowest social weighting(0.3) has the highest success ratio (46–48%) and the lowestdelay (29–45 h) for San Francisco Cab trace [45]. This is ex-pected as this trace has the sparsest topology with leastrepetitive connectivity patterns. CafRep with high Social-Util achieved highest success ratios (90–81%) and lowestdelays (67–132 s) for Infocom 2006 and Sassy (73–64%and 44–83 min). CafRep with equal SocialUtil and lowSocialUtil closely follow the high Social utility for therespective traces. This is expected as both of these tracesare social and have more repeating contact patterns soSocialUtil is able to utilise the complex graphs statistic inorder to make better predictions about the most directroute to the destination. Even though both include humanmobility patterns, compared to Infocom 2006 trace, Sassytrace [44] has much less regular and sparser connectivity,no highly central points (as the ones deployed by Infocom2006). This results in lower performance results for allweightings of SocailUtils for Sassy compared toInfocom.2006.

In order to understand the relative importance of egonetwork utilities and node utilities, we looked into CafRepwith low SocialUtil (0.2 of CafREpUtil) and unequal weigh-tings between ego network utilities and node utilities: firstwe weighted ego network utilities 0.6 and node utilities as0.2 of CafREpUtil (we refer to it as HighEN Util) and thenwe weighted ego network utilities 0.2 and node utility as0.6 of CafRepUtil (we refer to it LowEN Util). It is interest-ing to see that ego network resources play less importantrole than node resources in San Francisco Cab trace [45]compared to Sassy [44] and Infocom 2006 [33] traces.More specifically, CafRep with HighEN Util achieves lowersuccess ratio than CafRep with LowEN Util for the SF Cabstrace [45] (and vice versa for SASSY and Infocom 2006traces). Fig. 5 shows CafRep with HighEN Utils success ratioranging from around 47% to 44% for SF Cab trace, followedby 68–60% for Sassy trace and 86–76% for Infocom 2006trace. This lowest performance of SF Cab trace [45] is dueto it having the least repeating mobility patterns and thusnot benefiting significantly from the nodes’ ego networkresources predictions for future forwarding. So the mostgreedy approach of low EN Util and low Social works thebest for SF Cab trace [45] while its performance it the worstfor the other two social traces. In the case of both socialtraces, HighEN Util performs better than LowEN Util butworse than higher SocialUtil version of CafRep. This is ex-pected as ego networks play a more important role in so-cial traces.

4. Evaluation methodology

We perform extensive evaluations of CafRep againststate of the art DTN protocols across a range of metrics,three realistic topologies, and two application scenarios.As mobility and connectivity patterns of nodes have major

Fig. 5. Success ratios of varying CafRep weighting models.


impact on the performance of communication protocols inDTNs, we choose three real-life connectivity and GPS tracesfrom CRAWDAD [1] to ensure sensible transmission rangesand realistic movement patterns of mobile users and vehi-cles. Our selected traces exhibit vastly different connectiv-ity patterns and we describe them in Section 4.1 We usetwo different application scenarios that have different traf-fic patterns: publish subscribe podcasting applicationwhere publishers publish messages with fixed sizes at con-stant bit rate, and Facebook social networking applicationthat has heterogeneous users that generate content at avariety of different rates and sizes. We induce varying lev-els of congestion by increasing the percentage of randomlychosen publishers for the podcasting application and bydecreasing buffer sizes for the Facebook application.Increasing the number of randomly chosen publishers al-lows us to have non-uniform temporal and spatial conges-tion rates across the network topology. This examplepodcasting application is appropriate for the realisticphoto/video uploading application scenarios that were

shown to have a larger number of publishers than sub-scribers [41].

We compare CafRep to two benchmark DTN routingprotocols and three competitive (adaptive) routing DTNprotocols. All our experiments are done in the ONE simula-tor [20] and our performance metrics include: success ra-tio, end to end delay, node buffer availability, number offorwarded packets, packet loss rates, number of deliveredpackets and number of replicated packets. We aim to showthat CafRep adapts well to the topologically different net-works and two applications while being efficient in termsof network resources.

4.1. Real-life connectivity and GPS trace datasets

To evaluate CafRep in topologically different networks,we choose to use three real world connectivity and GPStraces from CRAWDAD: Infocom 2006 [33], Sassy 2011[44] and San Francisco Taxi Cabs 2011 [45] that have dif-ferent mobility and connectivity patterns. We show that

Fig. 6. Delays of varying CAFERP weighting models.


CafRep is successful independently of the connectivity pat-terns of the underlying networks. The connectivity and GPStraces we use are briefly described below:

Infocom 2006 trace [33] consists of a 4-day long tracethat is based on a human mobility experiment conductedat Infocom 2006. A total of 78 volunteers joined the exper-iment and each was given an iMote device capable of con-necting to other Bluetooth-capable devices. In addition 20static long-range iMote devices were placed at variouslocations of the conference venue; three of these weresemi-static as they were placed in the building lifts. Thisdataset has been shown to exhibit strong communitystructure [16].

Sassy trace [44] consists of a 79 day long trace that isbased on a human mobility experiment conducted at St.Andrews in 2011. A total of 27 volunteers joined the exper-iment and each was given a TMote Invent sensor mote andencounters were tracked in their day-to-day activates for aperiod of 79 days. The rage of these devices was about

10 m and encounters were uploaded to a base station reg-ularly (but the base station did not have any role in for-warding). Similarly to Infocom 2006, this dataset hassome social structure but unlike Infocom 2006, it has noinfrastructure-like nodes, is much sparser and results inmuch longer disconnection periods.

San Francisco Cab Trace [45] are live traces that recordthe GPS coordinates of 550 cabs, logged approximatelyevery 10 s, over a period of 30 days, in the San FranciscoBay Area. We have downloaded the most recent at the timeof writing traces for the period of September 20th 2011 toOctober 20th 2011 via the Cabspoting.org API. These tracesare part of the Cabspotting project [45] that aimed to inferand visualise Taxi Cab collocation information from GPScoordinates in the San Francisco Bay Area. We have as-sumed that two cabs are collocated if their physical dis-tance is less than 50 m, furthermore, as cab clocks arenot synchronised we have assumed a 60 s interval duringwhich, if the distance between two cabs is less than


50 m, those cabs are assumed to be collocated. We use 100taxis that where logging their location data most fre-quently for higher confidence and accuracy of the GPStraces. This trace has shown to exhibit long periods of dis-connections, short periods of connectivity and islands ofconnectivity that are rarely populated by more than twonodes.

Fig. 7 draws a comparison between the three describedmobility traces Infocom 2006 [33], Sassy [44], and SF Cabs[45], highlighting their topological differences.

Fig. 7 shows that SF Cab trace [45] is the most challeng-ing trace with very short connectivity durations, very highdisconnections and low number of connected nodes duringconnected times. Both Sassy [44] and SF Cabs [45] are sig-nificantly more challenging compared to Infocom 2006[33] both due to the smaller numbers of neighbours duringconnectivity times (connectivity sets), and short connec-tions combined with long disconnections.

Fig. 7a shows that both Sassy [44] and SF Cab [45] tracesexhibit predominantly short contact durations (a mean of33 s and 31 s, a median of 27 s and 24 s and a maximum

Fig. 7. Data traces c

of 2.3 min and 4 min respectively) while Infocom 2006 dis-plays substantially longer contact durations (a mean of3 min, a median of 2.5 min and a maximum value of7 min).

Fig. 7b illustrates three different trends regarding nodeisolation periods. SF Cabs trace suffers the longest periodsof isolation (a mean of 10 h, a median of 6.5 h and a max-imum value of 4.5 days); while Sassy is significantly moredisconnected than Infocom 2006 but much more con-nected than SF with mean 1.5 h, median 14 min and max-imum 11 h; and Infocom 2006 experiences substantiallylower periods of isolation (a mean of 5.5 min, a medianof 6.4 min and a maximum value of 1.5 h).

Fig. 7c displays the difference in node connectivity be-tween the three datasets. This is calculated as the numberof active connections a node has at the time a new connec-tion is established. SF Cabs trace has the lowest observednode connectivity (a mean of 2, a median of 1.9 and a max-imum value of 4.47 connections), Infocom 2006 has thehighest observed node connectivity (a mean of 6.19, amedian of 6.44 and a maximum value of 10 connections)

haracteristics.


and Sassy experiences a mean of 2.95, a median of 2.23 anda maximum value of 7.57 connections).

Each of these datasets provides a different and challeng-ing environment for the algorithms to perform in: SF Cabsdue to the very long periods of isolation, short periods ofconnectivity and small connectivity sets; Infocom 2006due to the longer periods of connectivity, moderate isola-tion periods and larger connectivity sets. Sassy due to shortconnectivity periods, longer isolation periods and smallconnectivity sets. In Section 5 we show that CafRep adaptswell to the dynamics of all three datasets as it keeps highsuccess ratio and availability, low delay and packet lossrates, and outperforms the other protocols across all data-sets. We compare the performance of CafRep againstbenchmark protocols: Prophet [23], Spray and Focus (SF)[37] and competitive adaptive protocols Café [15], Encoun-ter Based Routing (EBR) [26] and Retiring Replicas (RRs)[38]. Prophet has become a prevalent benchmark forward-ing algorithm, SF as it is a prominent fixed replication for-warding algorithm, EBR due to replication placement beingvariable and RR because of its distinct adaptive replicationcapping.

4.2. Application models

In order to evaluate CafRep in the presence of conges-tion at different rates and locations, we have designedand built a fully distributed, interest driven overlay filecasting application as described in Section 3. We randomlyassign topic interest and choose varying number of pub-lishers and subscribers. The data is published by the pub-lishers at the constant bit rate (5 Mb/s) and messages ofuniform sizes (1 MB) are sent to the neighbours. We as-sume fixed, limited buffer 1 GB for this set of experiments.We run eight increments of congestion levels induced byincreasing number of publishers ranging from 1/9 to 8/9of total number of nodes in that connectivity dataset. Allsimulations are repeated a ten times with different random

Fig. 8. Facebook traffic

subscribers and publishers. Related work on publish-sub-scribe data dissemination in DTNs in [43] explicitly relieson detecting communities and does not consider conges-tion control. [19] proposes content-based forwarding andbuffer management based on content popularity, addingexplicit application hints to messages that are visible toeach intermediary node, allowing them to cache content,act as distributed storage, or perform application-specificforwarding, but they do not consider congestion awarenessor multiple sources. [27] allows generic functions such asbundle routing to be performed differently per application,operation, or resource, but particularly enables applicationsupport by means of caching or distributed storage, butdoes not consider congestion aware forwarding.

In order to consider the impact of the real world socialapplication on the behaviour of CafRep, we designed ourFacebook Application that allows us to model the traffictypical of communication in social networking applica-tions. We extracted the friendship graphs and statisticaldata regarding the size and frequency of posts for everyuser from the Facebook application and used it to drivethe publish and subscribe application on the top of CafRep.We have sampled the usage patterns of 95 Facebook usersand extracted list of friends associated with each user,their 20 most recent wall post messages and their list ofinterests. The size of a wall post is calculated as the totaldownload cost (e.g. if a message contains a http link or aphoto then this is measured and appended to the messagesize).

Fig. 8a shows the non-uniform sizes of messages gener-ated for each user with the Mean size of 1 MB, a Mediansize of 82 KB and a maximum size of 268 MB. We observethree types of message, text, picture and link that areposted at different frequencies: text messages are the mostcommonly posted (78%), followed by posts containing pic-tures (19%), with only 3% of posts encompassing links. Wefound that the most frequent message types (text) formeda small proportion of total traffic (153 KB in total and 82B

characteristics.


on average out of total 2.3 GB observed data) while theremainder of the data was almost equally split betweenpicture messages (1.2 GB in total and 3 MB on average)and links (1.1 GB in total and 18 MB on average). Fig. 8billustrates the duration of time between posts for eachusers. The most prolific participant posted every 3 min onaverage, whilst the most inactive profile posted contentonce every 25 days on average. We observed that nodesare connected to three friends on average and at most 2%of all other nodes.

We define ‘‘user profile’’ that contains statistical infor-mation such as: the ratio of text, picture and link messages,the average and standard deviation of message sizes foreach message type and the average and standard deviationof message frequency information. Formula (9) illustrateshow we use the average and standard deviation values inorder to generate a new and meaningful value for the nextmessage generation time and message size. k is the averageof x, r is the standard deviation of x and W is a normallydistributed random number between �1 and 1.

PðxÞ ¼ b0:5þ kðxÞ þ ðrðxÞ �WÞc ð9Þ

When a message is generated (or as the application starts)the next message is scheduled for generation. When a newmessage is generated a message type is assigned, as per theobserved message type distribution. When the messagetype has been selected the message size can be assignedbased on the relevant statistics. We run ten incrementsof congestion levels, decreasing the size of buffers from100 MB down to 10 MB at 10 MB increments. We ran-domly select traffic profiles for the nodes to follow andsimulate multiple runs with different random traffic pro-files. Each experiment is emulated over 10 runs, with eachrun having a different random seed.

5. Evaluation

In this section we first discuss our findings from theinterest driven file casting application experiments acrosseach of the three CRAWDAD datasets described in Section4.1. We then briefly report on the observations from ourFacebook application over RollerNet trace [21] that hashighly challenging connectivity patterns due to the Accor-dion Effect described in [39]. For more consistent compar-isons between the datasets, we use CafRepUtil for all of ourexperiments as defined in Section 3 with SocialUtil, noderesource utilities and ego network resource utilities beingequally weighted. We show that CafRep adapts well to dif-ferent mobility and connectivity patters and outperformsfive major competitive and benchmark protocols.

Compared to the traces we used in our earlier work[15,30,31,48], in this work, we use two new, very differentdata traces (social and vehicular) that are much sparserand have much less frequently recurring connectivity pat-terns (Sassy [44] and SF Cabs [45]). It is interesting to seethat CafRep performance is slightly worse for the new so-cial trace (Sassy traces [44]) compared to the old socialtrace (Infocom traces [33]). Similarly, CafRep also performson average worse for the new vehicular trace (SF Cab traces[45]) than for the old vehicular (DieselNet) traces. This is

due to Sassy [44] and SF Cab traces [45] having much spar-ser topology and no infrastructure-like nodes that are tak-ing part in the forwarding process. Both Infocom 2006 [33]and DieselNet [3] traces had the motes and APs that weretaking part in the forwarding decisions and were highlycentral nodes. Without such nodes, CafRep’s performanceis lower but still better than for other comparative proto-cols. As expected, the non-adaptive benchmark Spay andFocus [37] and Prophet [23] protocols perform worse com-pared to other adaptive protocols across all the measures.Both of these protocols start dropping the packets at thecongested nodes early into the simulation that results inhigher delays and lower success ratios. Prophet [23] per-forms worse than SnF [37] because it is a non-adaptive sin-gle copy protocol.

5.1. Success ratio and delay

Across all three data connectivity traces CafRepachieves higher success ratio than all other protocols as itmore efficiently detects new parts of the networks thathave more resources and avoids the parts of the networkthat are congesting. Fig. 9 shows comparative success ra-tios across five protocols and three traces in the presenceof increasing number of publishers. Fig. 9 shows that Caf-Rep has on average 10–15% higher success ratio than RR[38] and EBR [26] and more than two times higher successratios than for SnF [37] and Prophet [23] across all traces.This is due to the combined CafRep utility, its ego networkutilities in particular, that manage to predict resourceavailability in different regions of the network with goodaccuracy. Contrary to this, RR [38] is concerned with globalcongestion measures do not sufficiently account for vary-ing regional behaviours and have negative impact on thesuccess ratio when regional congestions arise. More specif-ically, RR [38] reacts to detected packet loss in a uniformway so that nodes that are congesting still receive packets(even though fewer copies) that they may drop and avail-able nodes do not receive sufficient number of packets.This overutilisation and underutilisation of resources in-creases the delays and results in lower success ratios forRR [38]. Fig. 9 shows success ratios ranging from 77% to60% for Infocom2006, from 67% to 50% for Sassy, and from46% to 39% for SFCab. In EBR [26], the respective rates ofencounter between two nodes determine the appropriatefraction of message replicas the nodes should exchange.Because EBR [26] relies only on the prediction of the futurerate of encounters for each node to decide on the probabil-ity of successful message delivery, EBR [26] does not detectcongestion and results in highly central nodes congestingat even faster rate which causes increased delays(Fig. 10) and lower success ratios (Fig. 9). Fig. 9 shows suc-cess ratios ranging from 72% to 58% for Infocom2006, from63.6% to 50% for Sassy, and from 44% to 37% for SFCab.Non-adaptive protocols do not check for resource con-strains but aggressively keep the most direct routes tothe destination that leads to the nodes quickly getting con-gested, increasing delays and having low success ratios(Figs. 9 and 10). Fig. 9 shows that Prophet’s success ratiosranges from 36% to 24% for Infocom2006, from 29% to19% for Sassy, and from 33% to 13% for SFCab. Similarly,

Fig. 9. Comparative success ratios across three traces.


SnF ranges from 55% to 26% for Infocom2006, from 44% to20% for Sassy, and from 38% to 21% for SFCab. SnF [37] hashigher success ratios than Prophet [23] because it is a rep-lication based protocol. It is interesting to see that Café hashigher success ratios than SnF and Prophet – more that 50%higher that SnF and more than 200% higher than Prophetacross all data traces. The only exception to this is at timesof low levels of congestion in SF Cab Trace where the dif-ferences are down to 25%.

Between the three connectivity data traces, CafRepachieves lowest success ratio in the SF Cab traces [45](48–44%) due to three reasons: first due to the SF Cab[45] topology described in Section 5.1, the social utility inCafRep cannot identify nodes that are significantly morecentral; second as the isolation periods are very high, thenodes cannot offload their content for a very long time;and third as the connectivity periods are very short, thepublishing nodes that generate data at a constant ratecause increased dropped packets. It is interesting to see

that CafRep achieves around 20% lower success ratio overSF Cab trace [45] than over the DieselNet [3] vehiculartrace (that we used in our earlier work [48] and showed70–65% success ratio). This is due to DieselNet trace [3]having a few access points that take part in the forwardingof packets and also having significantly lower isolationperiods than SF Cab trace [45] and longer connectivitydurations. SF Cab trace [45] that was generated for the pur-pose of tracking cabs in the SF Bay Area and has much spar-ser topology that covers a significantly wider area (400 and1600 square miles) without using any access points (APs)or road-side units (RSUs). We believe that is important thateven with such challenging and sparse trace, CafRep man-ages to outperform other protocols. CafRep achieves highersuccess ratio over Sassy [44] data trace then with SF Cabtrace [45] but much lower than when compared with Info-com data trace. Sassy [44] trace is more challenging thanInfocom 2006 because it does not include any infrastruc-ture-like nodes (such as semi-static and static nodes such

Fig. 10. Comparative delays across three traces.


as those deployed during Infocom2006 [33]) that partici-pate in forwarding, has significantly shorter connectivitydurations and sparser network. Because Sassy trace [44]does not contain nodes that are significantly better con-nected and has much less repetitive pattern than Infocom2006 (where the participants used the lift or often cameto the registration desk where the semi-static and staticnodes were deployed) social utility (as part fo CafRep)could not be as helpful as with this trace as with Infocom2006 trace in identifying the most direct routes to destina-tions. However, Sassy is significantly more connected thanSF Cab trace [45] and thus has much higher success ratios.

Across all the three connectivity traces, CafRep managesto keep lower delays compared to the other protocols dueto the following two reasons. First, CafRep is able to predictregional in-network delays with good accuracy because ofchecking for the in-network delays of both the nodes andtheir ego networks. In this way CafRep is able to morequickly identify (potentially) longer but less congested

paths with lower delays than the other protocols. NeitherRR [38] nor EBR [26] are able to efficiently discover longerroutes with smaller queue sizes but instead only decreasetheir replication rate and remain in the congested regionsfor longer periods of times. Fig. 10 shows delays for RRranging from 100 to 220 s, 70–120 min and 45–75 h forCafRep across Infocom, Sassy [44] and SF Cab traces [45]respectively. For the majority of time this is about twotimes as high delay as for CafRep across the three traces.Non-adaptive protocols (Prophet [23] and SnF [37]) haveaggressive forwarding strategy that quickly leads into con-gesting the en-route nodes that increase delays. Fig. 10shows delays for SnF and Prohpet ranging from 180 to320 s, 80–160 min and 60–100 h for the three tracesrespectively. This is between 1.5 and 4 times higher thanthe delays for CafRep. Second, using Social utility as partof the CafRep utility, CafRep ensures best prediction ofthe most direct route to the destination especially for Sassyand Infocom 2006 traces. Neither of the RR [38], EBR [26],


Prophet [23] and SnF [38] use social metrics and resourcemetrics together and are thus not able to adjust to thedynamics in both of these dimensions.

Infocom 2006 delays are significantly lower than for theother two datasets across all protocols (seconds versusminutes and hours). This is due to topology characteristicssuch as short disconnection times, long connection times inInfocom 2006 [33], moderate connectivity sets and fewinfrastructure-like nodes that allow for more efficient datadissemination. For similar reasons, delays across all proto-cols are significantly higher for SF Cab Traces [45] thanfor delays for Sassy [44] and Infocom 2006 traces. Still, be-cause CafRep considers the relative in-network delays, it isable to maintain lower delays than the rest of the protocols.

5.2. Availability and packet loss

Across all three traces, CafRep sustains higher availabil-ity than other protocols. This is because CafRep is able to

Fig. 11. Comparative availability

make good predictions of node and ego network availabil-ity which avoid depleting the storage resources of fre-quently used nodes and regions in the network that droppackets. Fig. 11 shows CafRep availability ranging from88% to 70% for Infocom 2006, 70–60% for Sassy [44] and60–35% for Sf Cabs [45]. EBR [26] results in lower availabil-ity as it congests the regions that are highly central andwhere the nodes cannot offload the traffic faster than thetraffic is generated (that is the example application sce-nario we are considering). Fig. 11 shows that EBR [26]manages only 70–20% availability over Infocom [33] andSassy [44] traces and 40–20% for SFCabs trace [45]. It isinteresting to see in Fig. 12 that RR [38] has higher packetloss rates than CafRep despite RR [38] tracking and adapt-ing to the global packet loss congestion signals. We believethat this is due to RR [38] not being fast enough to detectsudden opportunities of longer but more available pathsas it awaits for the global signs of increased packet lossto act locally. RR [38] manages to keep availability levels

across three data traces.

Fig. 12. Comparative packet loss across three data traces.


at 80–20% for Infocom 2006 [33] and Sassy traces [44] andonly 45–30% for SF Cabs [45]. RR [38] maintains packet lossrates 33–63%, 49–83% and 75–95% for Infocom 2006 [33],Sassy [44] and SF Cabs traces [45] (given in Fig. 12). RR ismore suitable for even traffic distribution and connectivitypatterns, such as constant bit rate traffic with a randomway point mobility model, as the nodes in such networksbecome congested at a uniform rate. In heterogeneousDTNs that we consider, uniformity is not common, so itis more likely that traffic hot spots trigger messages to bedropped while other areas of the network remain highlyavailable. For such scenarios, when RR [38] uniformly re-duces the number of replicas entering the network, it re-sults in underutilisation of existing network resourcesand overutilisation of some network parts.

The low availability of CafRep over SF Cabs traces [45]comes primarily from the very long node isolation periodscombined with intense traffic generation during short con-nectivity durations. This means that, despite discovering

more paths than other protocols can discover, with the in-crease of publishing nodes (above 70% congestion ratewhere most nodes are publishing), publishing nodes areforced into decreased availability and dropping the packetsthat they cannot offload. With increased congestion levels(above 50% of publishing nodes), Fig. 11 shows that CafRepmanages to keep two times higher availability than EBR[26], and above 50% higher than RR [38]. It is interestingto see that the differences in availability between the threeadaptive protocols are smallest for low congesting rates inSassy [44] and Infocom 2006 data traces [33], but as thecongestion levels increase, CafRep starts to dominate overRR and EBR starts to fall behind more dramatically. For SFtraces [45], CafRep is significantly better than RR [38] andEBR [26] across all congestion levels. This is because EBR[26] and RR [38] are slower to utilise already very infre-quent opportunities to divert the traffic along diverse paths.

All protocols manage highest availability over theInfocom 2006 trace [33] because this trace is highly social


(allows repeating patterns of connectivity) that allows foraccurate social/connectivity predictions that CafRep, EBR[26], RR [38] and SnF [37] use. In addition to this, this tracedeploys infrastructure-like nodes that facilitate interactionamong nodes, has longer contact times, shorter smallerisolation periods and bigger connectivity islands. Availabil-ity for both non-adaptive protocols, Prophet and SnF, is lowacross all the three traces and it ranges from 0.6 to 0.13,0.6–0.14 and 0.3–0.1 across Infocom 2006, Sassy and SFCab traces respectively. With the constant bit rate datatransmission of the publishing nodes, these two protocolsquickly saturate the en-route nodes that increase packetdrop rates ranging from 0.59% to 98%. 0.71–96% and0.62–95% across Infocom 2006, Sassy and SF Cabs tracerespectively. Cafe, even though being a single packet proto-col, maintains significantly better availability: up to twice

Fig. 13. Comparative performance c

the availability of and around three times lower packet lossrates than non-adaptive protocols. This is due to Cafe’sadaptive forwarding that predicts identifies highly con-gesting nodes and regions, and avoids them.

5.3. Performance costs: forwarded, replicated packets anddelivery cost

We assess performance costs in terms of total averagenumber of delivered, forwarded and replicated packets.Fig. 13a shows that even though CafRep may longer pathsbut less congested paths, it does not generate significantlymore forwarded packets compared to the two adaptivereplication protocols RR [38] and EBR [26]. For example,we observe similar average numbers of forwarded packetsfor CafRep, EBR [26] and RR [38] across Infocom 2006 [33]

osts across three data traces.


and Sassy [44] but higher for CafRep over SF Cabs [45]. Thisis due to the following two reasons. First, even though Caf-Rep uses greater diversity of available paths, its Social Utilallows it to keep as direct route to the destination as pos-sible over the social traces. EBR and RR do not exploit morecomplex connectivity relationships and may thus result inlong paths especially in the presence of increasing conges-tion. Since SF Can Trace is not a social trace and CafRepbenefits less from the sophisticated contact relationshipanalysis, it results in marginally higher number of for-warded packets. Second, even though EBR, RR and CafRepforward similar rates of packets, EBR and RR drop morepackets as the forwarding is not appropriately distributedto match the non-uniform varying resources in the net-work. The number of forwarded packets for Café [15] andProphet [23] is smaller than for the other protocols as theydo not use replication. As expected, SnF [37] forwards onaverage highest number of packets as it is non-adaptivereplication-based protocol.

More interestingly, CafRep and RR replicate similarnumber of packets across Sassy trace [44] while CafRepreplicates 15–30% more than RR [38] and EBR [26] for SFCab trace [45] (Fig. 13b). We believe that higher replicationrates for CafRep in SF Cab trace [45] are due to CafRep dis-covering more forwarding opportunities that the other twoadaptive replication protocols do not discover. Comparedto CafRep, EBR [26] replicates 15–20% more packets overInfocom 2006 trace [33] because that trace contains highlycentral nodes that EBR [26] uses heavily for the replicationcontrol and as EBR does not check for resource constraintsthis replication results in increased packet loss rates andlower availability.

Fig. 13c shows the cost of delivery that we define as thenumber of messages forwarded over the number of mes-sages delivered across six protocols and over the threetraces. This is a significant metric as it relates to the aver-age number of hops required to deliver a message. As ex-

Fig. 14. Average delivery delay

pected, Café [15] has the lowest cost of delivery acrossthe three traces as it adaptively forwards to offload net-work hotspots in a delay-aware manner and does not in-clude replication. Out of the replication-based protocols,CafRep outperforms EBR [26], Prophet and SnF [37] acrossall the three data traces and is only marginally outper-formed by RR [38] in the SF Cab dataset [45]. As discussedin Section 3, we believe that this higher cost for CafRep inSF Cab trace [45] comes from high weighting of ego net-work resources and social utility in a trace that does nothave node connectivity with good pattern of regularity.We expect better results for CafRep over this trace if we de-ployed low weighting of social utilities and ego networkresources so that the results do not suffer from the poorquality for the ego networks formed in unstructured envi-ronments. Our results also show that SnF is consistentlythe worst performer with approximately double the costof all the other algorithms, this is due to the static natureof SnF’s [37] replication scheme, which under populatesthe network with traffic when demands are low and doesnot reduce the volume of traffic when contention is high.Across all the protocols, the cost of message delivery isthe lowest for Infocom 2006 trace [33], followed by Sassytrace [44] and then SF Cab trace [45]. This is due to signif-icantly better connected Infocom 2006 trace than the othertwo traces, and most isolated and more random connectiv-ity patterns for SF Cab trace [45].

5.4. Facebook application

This section explores the effects that real world socialnetworking traffic usage patterns have on the performanceof CafRep. Fig. 14 illustrates three experiments with differ-ent traffic profiles: low traffic profiles, randomly selectedtraffic profiles and high traffic profiles, which are indicatedby 1, 2 and 3 respectively. We compare CafRep with bench-mark DTN protocols Direct Delivery (DD) [49] and

(a) and success ratio (b).


Epidemic (EPI) [50], and the state-of-the-art competingcongestion control algorithm Retiring Replicas (RRs) [38].

In Fig. 14a we evaluate the average number of secondsdelay each algorithm experienced across the three differ-ent traffic profiles. We observe higher delays for less activeprofiles (that generate smaller messages at low frequen-cies) as delivery opportunities are low and message gener-ation is infrequent. This in turn causes nodes to store theirmessages within their buffer for extended periods of time.These user profiles cause lower variance for the majority ofalgorithms which reflects the decreased contention in thenetwork. We observe that CafRep has lower delays thanall other algorithms excluding epidemic for these profiles.RR [38] is the exception to the reduced variance, as it dis-plays high delay variance and much larger delays thanother algorithms. This is because RR [38] assumes that con-gestion is uniformly spread and do not identify congestingregions in the network, which occur more frequently whentraffic patterns are less uniform. As the message sendingrates intensify the probability of a node having generateda message for an encountered destination increases, as aresult message delays are consistently lower in compari-son to the less active profiles. The increased traffic de-mands from these profiles lead to a higher level ofcontention in the network, which results in a higher vari-ance in delays. This is best illustrated by the Direct Deliv-ery [49] results that show significantly reduced delaysand increased contention. We also observe that in the hightraffic experiments CafRep even outperforms Epidemic[50]. This best illustrates the benefits CafRep offers by alle-viating contention in the network through the use of mul-ti-path forwarding. Our results show that in the mixedtraffic profile experiments delays experienced are betweenthe low and high traffic profiles, but the degree of varianceis the highest of all three experiments. This is due to theheterogeneity of the user activity, which leads to volatiletraffic demands. Despite the challenges presented by thesemixed demands CafRep continues to outperform the otheralgorithms, while RR [38] experiences increased conten-tion due to its assumption of homogeneous user patterns.

In Fig. 14b we show the success ratio of CafRep in com-parison to DD [49], EPI [50] and RR [38]. We observe thatCafRep supports real application traffic, as our results forreal application traffic are consistent with our previousexperiments. CafRep has the highest success ratio in theless active profile experiments, in comparison with mixedand high traffic profiles as the levels of congestion are low-er. CafRep’s success ratio decreases as the levels of conten-tion increases and success ratio for low traffic profiles ishigher than mixed, which is higher than the high trafficprofile experiments. However, CafRep always has highersuccess ratio than all other protocols excluding directdelivery [49], which has 100% success ratio as it only for-wards messages directly to the destination nodes.

6. Conclusion and future work

We proposed CafRep that uses a combined local social,buffer and delay metrics for congestion aware message for-warding and replication that maximises message delivery

ratio and availability of nodes while minimising latencyand packet loss rates at times of increasing congestion lev-els. At the core of CafRep is a combined relative utility dri-ven heuristic that allows highly adaptive forwarding andreplication policies by managing to detect and offload con-gested parts of the network and adapting the sending/for-warding rates based on resource and contact predictions.We empirically investigated a number of weighting modelsfor analysing the impact of different utilities on the CafRepperformance. We have done extensive performance analy-sis of CafRep in three CRAWDAD real connectivity and GPStraces with different mobility and connectivity patterns:Infocom 2006, Sassy and San Francisco Cabs. We show thatCafRep outperforms five other state of the art DTN adaptiveand non-adaptive routing protocols across the majority ofmetrics across all the traces. We also show that CafRepmaintains performance levels in two application scenarios:publish subscribe constant bit rate podcasting and Face-book traffic application. We believe that CafRep providesa useful generic and highly adaptive congestion controlframework suitable for different types of resource con-straint DTN application scenarios. We also plan to investi-gate the efficiency of CafRep in the context of more realisticanycast and multicast applications.

Acknowledgement

The work was supported by the Engineering and Phys-ical Sciences Research Council UK (EPSRC) Grant NumberEP/D062659/1.

References

[1] Community Resource for Archiving Wireless Data at Dartmouth(Crawdad). <http://crawdad.cs.dartmouth.edu>.

[2] E. Anshelevich, S. Ukkusuri, Equilibria in dynamic selfish routing,Algorithmic Game Theory (2009) 171–182.

[3] N. Banerjee, M. Corner, D. Towsley, B.N. Levine, CRAWDAD data setumass/diesel/transfer/fall2007 (v. 2008-09-14). <http://crawdad.cs.dartmouth.edu/umass/diesel/transfer/fall2007>, 2008.

[4] R. Bhagwan, D. Moore, S. Savage, G. Voelker, Replication strategiesfor highly available peer-to-peer storage, Future directions indistributed computing (2003) 153–158.

[5] R. Bhagwan, K. Tati, Y.C. Cheng, S. Savage, G.M. Voelker, Otal recall:system support for automated availability management, In NSDI(2004).

[6] S. Burleigh, E. Jennings, J. Schoolcraft, Autonomous congestioncontrol in delay-tolerant networks, American Institute ofAeronautics and Astronautics (2007).

[7] L.P. Cox, C.D. Murray, B.D. Noble, Pastiche: making backup cheap andeasy, ACM SIGOPS Operating Systems Review 36 (SI) (2002) 285–298.

[8] E.M. Daly, M. Haahr, Social network analysis for information flow indisconnected delay-tolerant MANETs, IEEE Transactions on MobileComputing (2008) 606–621.

[9] V. Erramilli, A. Chaintreau, M. Crovella, C. Diot, Diversity offorwarding paths in pocket switched networks, in: Proceedings ofthe 7th ACM SIGCOMM Conference on Internet Measurement, ACM,2007, pp. 161–174.

[10] V. Erramilli, M. Crovella, A. Chaintreau, C. Diot, Delegationforwarding, in: Proceedings of the 9th ACM InternationalSymposium on Mobile Ad hoc Networking and Computing, ACM,2008, pp. 251–260.

[11] H. Falaki, R. Mahajan, S. Kandula, D. Lymberopoulos, R. GovindanandD. Estrin, Diversity in smartphone usage, in: Proceedings of the 8thInternational Conference on Mobile Systems, Applications, andServices, ACM, 2010, pp. 179–194.

[12] A. Ferreira, Building a reference combinatorial model for MANETsNetwork, IEEE 18 (5) (2004) 24–29.

http://www.crawdad.cs.dartmouth.edu

http://www.crawdad.cs.dartmouth.edu/umass/diesel/transfer/fall2007

http://www.crawdad.cs.dartmouth.edu/umass/diesel/transfer/fall2007


[13] L. Ford, D. Fulkerson, Flows in Networks, Princeton University Press,Princeton, NJ, 1962. p. 10.

[14] A. Grundy, M. Radenkovic, Decongesting opportunistic social-basedforwarding, in: Seventh Annual Conference on Wireless on demandNetwork Systems and Services (WONS), vol. 2, Kranjska Gora,Slovenia, 2010.

[15] A. Grundy, M. Radenkovic, Promoting congestion control inopportunistic networks, in: IEEE International Conference onWireless and Mobile Computing, Networking and Communications(WiMob’2010), Niagara Falls, Canada, 10, 2010.

[16] P. Hui, J. Crowcroft, How small labels create big improvements, in:Proceedings of the 2006 ACM CoNEXT Conference, ACM, New York,NY, USA, 2006.

[17] P. Hui, J. Crowcroft, E. Yoneki, Bubble rap: social-based forwarding indelay tolerant networks, in: Proceedings of the 9th ACMInternational Symposium on Mobile Ad hoc Networking andComputing, ACM, New York, NY, USA, 2008, pp. 241–250.

[18] Wei jen Hsu, Ahmed Helmy, Impact: Investigation of Mobile-userPatterns Across University Campuses using Wlan Trace Analysis.Technical Report, IEEE INFOCOM, 2005.

[19] Y. Jiao, Z. Jin, Y. Shu, Data dissemination in delay and disruptiontolerant networks based on content classification, in: 5thInternational Conference on Mobile Ad-hoc and Sensor Networks,2009, MSN’09, IEEE, 2010, pp. 366–370.

[20] A. Keranen, J. Ott, T. Karkkainen, The ONE simulator for DTN protocolevaluation, in: Proceedings of the 2nd International Conference onSimulation Tools and Techniques, ICST (Institute for ComputerSciences, Social-Informatics and Telecommunications Engineering),2009, pp. 1–10.

[21] J. Leguay, F. Benbadis, CRAWDAD data set upmc/rollernet (v. 2009-02-02). <http://crawdad.cs.dartmouth.edu/upmc/rollernet>,February 2009.

[22] Y. Li, L. Zhao, Z. Liu, Q. Liu, N-drop: congestion control strategy underepidemic routing in DTN, in: Proceedings of the 2009 InternationalConference on Wireless Communications and Mobile Computing:Connecting the World Wirelessly, ACM, 2009, pp. 457–460.

[23] A. Lindgren, A. Doria, O. Schelen, Probabilistic routing inintermittently connected networks, Lecture Notes in ComputerScience (2004) 239–254.

[24] A. Lindgren, K.S. Phanse, Evaluation of queuing policies andforwarding strategies for routing in intermittently connectednetworks, Proceedings of IEEE COMSWARE (2006).

[25] M. May, V. Lenders, G. Karlsson, C. Wacha, Wireless opportunisticpodcasting: implementation and design tradeoffs, in: Proceedings ofthe Second ACM Workshop on Challenged Networks, ACM, 2007, pp.75–82.

[26] S.C. Nelson, M. Bakht, R. Kravets, A.F. Harris, Encounter: basedrouting in DTNs, ACM SIGMOBILE Mobile Computing andCommunications Review 13 (1) (2009) 56–59.

[27] J. Ott, M.J. Pitkanen, Dtn-based content storage and retrieval, in: IEEEInternational Symposium on a World of Wireless, Mobile andMultimedia Networks, 2007, WoWMoM 2007, IEEE, 2007, pp. 1–7.

[28] J.M. Pujol, A.L. Toledo, P. Rodriguez, Fair routing in delay tolerantnetworks, in: Proceedings of IEEE INFOCOM (2009).

[29] M. Radenkovic, C. Greenhalgh, Multi-party distributed audio servicewith TCP fairness, in: The 10th ACM International Conference onMultimedia, ACM, 2002, pp. 11–20.

[30] M. Radenkovic, A. Grundy, Poster: congestion aware datadissemination in social opportunistic networks, SIGMOBILE MobileComputing and Communications, Review (MC2R) 14 (3) (2010) 31–33.

[31] M. Radenkovic, A. Grundy, Congestion aware forwarding in delaytolerant and social opportunistic networks, in: Eighth AnnualConference on Wireless On demand Network Systems and Services(WONS), 2011.

[32] K. Rzadca, A. Datta, S. Buchegger, Replica placement in p2p storage:complexity and game theoretic analyses, in: 2010 InternationalConference on Distributed Computing Systems, IEEE, 2010, pp. 599–609.

[33] J. Scott, R. Gass, J. Crowcroft, P. Hui, C. Diot, A. Chaintreau, CRAWDADdata set Cambridge/haggle (v. 2009-05-29). <http://crawdad.cs.dartmouth.edu/cambridge/haggle>, May 2009.

[34] M. Seligman, K. Fall, P. Mundur, Storage routing for dtn congestioncontrol, Wireless Communications and Mobile Computing 7 (10)(2007) 1183–1196.

[35] A.N. Shiryaev, B. Rozovskii, G. Grimmett, Optimal Stopping Rules,Springer Verlag, Berlin, Heidelberg, 2008.

[36] T. Spyropoulos, K. Psounis, C.S. Raghavendra, Spray and wait: anefficient routing scheme for intermittently connected mobilenetworks, in: Proceedings of the 2005 ACM SIGCOMM Workshopon Delay-tolerant Networking, ACM, 2005, pp. 252–259.

[37] T. Spyropoulos, K. Psounis, C.S. Raghavendra, Spray and focus:efficient mobility-assisted routing for heterogeneous and correlatedmobility, in: Fifth Annual IEEE International Conference on PervasiveComputing and Communications Workshops, 2007, PerComWorkshops’ 07, IEEE, 2007, pp. 79–85.

[38] N. Thompson, S. Nelson, M. Bakht, T. Abdelzaher, R. Kravets, Retiringreplicants: congestion control for intermittently-connectednetworks, in: Proceedings of Infocom, 2010.

[39] P.U. Tournoux, J. Leguay, F. Benbadis, V. Conan, M.D. De Amorim, J.Whitbeck, et al., The accordion phenomenon: analysis,characterization, and impact on dtn routing, in: Proceedings ofIEEE Infocom, Citeseer, 2009.

[40] A. Vahdat, D. Becker, Epidemic Routing for Partially Connected Adhoc Networks, Technical Report, Citeseer, 2000.

[41] M. Valafar, R. Rejaie, W. Willinger, Beyond friendship graphs: a studyof user interactions in flickr, in: Proceedings of the 2nd ACMWorkshop on Online Social Networks, ACM, 2009, pp. 25–30.

[42] D. Wischik, M. Handley, M.B. Braun, The resource pooling principle,ACM SIGCOMM Computer Communication Review 38 (5) (2008) 47–52.

[43] E. Yoneki, P. Hui, S.Y. Chan, J. Crowcroft, A socio-aware overlay forpublish/subscribe communication in delay tolerant networks, in:Proceedings of the 10th ACM Symposium on Modeling, Analysis, andSimulation of Wireless and Mobile Systems, ACM, 2007, p. 234.

[44] G. Bigwood, D. Rehunathan, M. Bateman, T. Henderson, SaleemBhatti, CRAWDAD data set st_andrews/sassy (v. 2011-06-03).<http://crawdad.cs.dartmouth.edu/st_andrews/sassy>, 2011.

[45] San Francisco cabs data traces, downloaded from the Cab Spottingproject, <http://cabspotting.org/api> for the period 20 Sep 2011– 20Oct 2011.

[46] P.U. Tournoux, V. Conan, J. Crowcroft, J. Leguay, M.D. de Amorim, F.Benbadis, Wardrop equilibrium formulation of resource-constrainedDTN routing in public safety networks, in: Proceedings of the 8thIEEE International Conference on Mobile Adhoc and Sensor Systems,MASS 2011, Valencia, Spain, October 17–22, 2011.

[47] A. Balasubramanian, B. Levine, A. Venkataramani, DTN routing as aresource allocation problem, In ACM SIGCOMM ComputerCommunication Review (2007).

[48] M. Radenkovic, A. Grundy, Framework for utility driven congestioncontrol in delay tolerant opportunistic networks, in: Proceedings ofthe 7th International Wireless Communications and MobileComputing Conference (IWCMC), 2011.

[49] T. Spyropoulos, K. Psounis, C.S. Raghavendra, Single-copy routing inintermittently connected mobile networks, in: First Annual IEEECommunications Society Conference on Sensor and Ad HocCommunications and Networks, IEEE SECON, 2004, IEEE, 2004, pp.235–244.

[50] A. Vahdat, D. Becker, Epidemic Routing for Partially Connected Adhoc Networks, Technical Report, Citeseer, 2000.

[51] A.J. Mashhadi, S.B. Mokhtar, L. Capra, Fair Content Dissemination inParticipatory DTNs. Ad Hoc Networks, Elsevier, 2011.

Milena Radenkovic has received her PhDDegree in 2002 from the University ofNottingham, UK and her Dipl Ing from theUniversity of Nis, Serbia in 1998. Her researchspans the areas of mobile and delay tolerantnetworking, P2P systems, and their applica-tion to pervasive gaming, social networking,and environmental monitoring. She has beenan Investigator of four EPSRC grants. She hasorganised and chaired multiple conferences,served on many program committees andeditorial boards and published in premium

venues including ACM MC2R, IEEE WONS, IEEE Multimedia, MIT PressPRESENCE, ACM Multimedia, ACM VRST, ACM CCGRID.

http://crawdad.cs.dartmouth.edu/upmc/rollernet

http://www.crawdad.cs.dartmouth.edu/cambridge/haggle

http://www.crawdad.cs.dartmouth.edu/cambridge/haggle

http://www.crawdad.cs.dartmouth.edu/st_andrews/sassy

http://cabspotting.org/api


Andrew Grundy is currently a final year Ph.D.researcher at the University of Nottingham.He received his BSc degree in Computer Sci-ence from the University of Leicester in 2008.He has published work in venues includingACM MC2R, IEEE WiMob and IEEE WONS. Hisresearch interests include delay tolerant, self-organising, distributed and mobile systems,social network analysis and network theory.

Education

Transfer reliability and congestion control strategies in opportunistic networks a survey