Design of Link and Routing Protocols for Cache-and-ForwardNetworks·

Shweta Jain, Ayesha Saleem, Hongbo Liu, Yanyong Zhang, Dipankar Raychaudhuri

both short-term and long-term path metrics along with in­network storage availability.

Abstract---Cache-and-Forward (CNF) is a future Internetarchitecture designed for content delivery to mobile users overwireless networks with varying link quality and intermittentconnectivity. The CNF protocol is based on strict hop-by-hoptransport of media files with in-network storage at each routeror wireless access point. The protocol also incorporates contentcaching capabilities for efficient delivery of popular media files.In this paper, we briefly describe the CNF architecture, presenta survey of prior work, and describe new CNF link and routingprotocols. Throughput results comparing CNF with TCPIIPare summarized for an example wide-area Internet scenariowith wireless access networks. The design of a reliable CNFlink layer protocol is discussed and performance results aregiven for multihop wireless scenarios. The paper concludeswith an outline of dynamic CNF routing algorithms whichconsider both short-term and long-term path quality along withavailable in-network storage.

l\Julti-hop 'Vir~I~5.~

.-\cce~~ n~",'ork

Reliable Link L.n·er In Network StorllKe~/:->/~

Figure 2: CNF Protocol Stack

Although proposed as a "clean slate" design, CNFcan also be implemented as an overlay over IP. CNF routershave sufficient storage capacity for temporary storage whileawaiting connectivity with mobile destinations and for in­network caching of content. The network is robust tointermittent disconnections, a feature similar to disruptiontolerant networks (DTN) [9]. CNF's hop-by-hop transportprotocol contrasts with the connection-oriented TCP model,and has the potential for significant throughput and delayimprovements even in various wired and wireless networkscenarios [10].

CNF also supports content and information centricapplications by supporting a mode in which the end-user candirectly request content files from the network using acontent identifier tag (CID). Caching and content retrievalare thus as an integral part of the underlying network and therouting protocol in CNF. The content delivery protocol isresponsible for searching the in-network caches to find thenearest copy of the content rather than fetching the contentfrom the original source. This requires efficient contentcaching and cache advertisement protocols as discussed in[11 ].

In the following sections we present a briefsummary of the CNF protocol architecture. We then present

I. INTRODUCTION

Future Internet usage is expected to involve anincreasing proportion of content de~ivery to m~bi~e

end users. The mobile content delIvery scenano IS

associated with wireless access networks of varyingquality, with the likelihood of occasional disconnection.The end to end connection model of today's TCP/IP isunsuitable for mobile and wireless Internet access [1], andthis has motivated a variety of transport protocolenhancements such as I-TCP [2] and CLAP [3]. Morerecently, the networking community has started to considerso-called "clean-slate" protocol solutions under researchprograms such as NSF (FIND [4] and GENI [5]) in the US,and FP7 Future Networks [6] and FIRE [7] in Europe. Acompletely new protocol design for mobile content deliveryhas the potential to significantly improve performance androbustness relative to existing TCP/IP. In this paper, wepresent preliminary results on such a clean-slate networkdesign from an ongoing NSF FIND research project calledthe "cache-and-forward" architecture.

Cache-and-forward (CNF) is a clean slateinformation centric architecture specifically optimized formobile content delivery. The architecture is based on in­network storage at routers along with hop-by-hop transportof large media files. Figure 1 shows the conceptual view ofthe CNF architecture as envisioned in [8]. As illustrated inthe figure, a media file is routed in a hop-by-hop mannertowards its destination. If the path quality is poor, or the end­user is temporarily disconnected, the network stores the filein transit, delivering it at a later time when conditionsimprove. The routing protocol incorporates awareness of

1 Research supported by the National Science Foundation's NeTS-FINDprogram under collaborative grant # CNS-0626959.

Content Retriel'al Interface Content Retriel'al Protocol

Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on April 05,2010 at 02:10:39 EDT from IEEE Xplore. Restrictions apply.

tagged with content ID and offset tuples (CID, offset). Thesechunks are then transmitted in a hop by hop fashion. Errorcontrol and congestion control are delegated to the link andnetwork layer keeping the transport protocol much simplerthan TCP. Figure 3 illustrates the CNF transport protocolwith an example. Here the file is divided into four chunks.Each chunk is identified by the <CID, offset> tuple. Thetransport layer at S sends the chunks to D and transport layerat D reassembles the file from the chunks. The routing layerinitially picks the path S-A-C-D but switches to an alternatepath A-B-E-D for <CID, 2>. This might happen if theoriginal route does not have enough storage space(congestion), the link layer reports a broken link or therouting layer detects a better route in the short term. If thedestination is unavailable for a long time, the intermediaterouter stores the content in its storage space. These routechanges and storage decisions are transparent to thetransport layer.

In summary, the CNF transport protocol servescontent query, fragments large content into smaller chunks,reassembles the chunks into the original file, retrievesmissing chunks from the network and maintains a contentcache. The network protocol discovers routes to thedestination after the content has been located and the linklayer provides hop by hop reliability for contenttransmission.

In the following sections, we will present the designand evaluation of the CNF link layer followed by priorresults in CNF transport and future work in CNF routingprotocols.

III. CNF LINK PROTOCOL

In CNF the link protocol must receive the entire file orcontent chunk before sending it to the routing layer.Therefore, in this context we introduce per hop file transferreliability which may be used instead of or in addition to perpacket reliability provided by an 802.11 like MAC protocol.When a file to be transferred arrives at the link layer, it isassigned a temporally unique identifier and the file isdivided into batches of a fixed number of packets. The linklayer then reliably transfers each batch to the next hopbefore moving onto the next batch. To achieve thisreliability we introduce two new link layer control packets,Request for Batch (RFB) and negative acknowledgementNACK. The RFB contains the MAC addresses to identifysending and receiving routers, the file and batch identifiers,the file size, number of packets in a batch and a timestampindicating when the batch transmission started. The senderrelies upon the MAC layer acknowledgement to ensuresuccessful reception of the RFB by the next hop. Thesending node must wait for the receiver to transmit a NACKthat contains a bitmap indicating the packets from theprevious batch that were not received. The receiving nodemight be in the middle of another batchreception/transmission and therefore the sender might haveto wait for as much as the transmission time for themaximum size batch. When the receiving node is ready toreceive a batch, it first checks whether any part of the file

(C'ID.J)

('m.2)

(CID, 1)

«'ID.O)

II. CNF ARCHITECTURE

Figure 3: Hop By Hop Transport in CNF

the design and evaluation of CNF link layer protocol thatprovides reliable hop-by-hop file transfer. We then presentresults from prior work showing the network performance ofend-to-end CNF transport using this CNF link protocol andstatic routing. We conclude with an outline of a dynamicCNF routing protocol that finds the best path to thedestination taking into account short-term and long-termpath quality along with availability of in-network storage.

The CNF network [8] consists of wired core as well aswireless access clouds. Each network router or wirelessaccess point/base station has large storage capacity whichcan be used for packets in transit as well as to provide in­network caching service for popular content. The networkprovides traditional client server communication as well ascontent and file delivery. The content delivery may be ineither "push" or "pull" mode. In the push mode, the serviceprovider unicasts or multicasts content to the end-userswhile in the pull mode an end user may request a specificcontent from the network. Each query and content istransported in a strictly hop-by-hop fashion through the CNFnetwork.

CNF introduces the concept of a Post Office (PO)specifically to improve content delivery to mobile users. ThePO associates content in transit to its mobile requestor. If adestination becomes unavailable during content retrieval, thecontent is cached at an intermediate location and the PO isinformed. When the mobile reconnects after an interruption,it may query the PO to obtain a pointer to the intermediatecache and resume content retrieval where it was left offinstead of making a new request to the content source.

The CNF protocol stack (Figure 2) consists of acontrol plane that consists of file or content name resolutionservice (CNRS) and content routing service. The data planecontains CNF application, transport, network and linkprotocols. The CNRS protocol provides naming service forthe content and resolves a content name to its actuallocation. The content routing protocol provides a contentdiscovery service that searches in-network caches to find thecontent at nearest location from the requestor.

I~'~

The CNF application is an information centriccontent service that transmits large files (,....,1O's of GB) to therequestor. In the CNF transport protocol, a very large file(,...., 1O's of GB) is first fragmented into smaller chunks(,...., 1OOMB-l GB) at the original source and each chunk is

Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on April 05,2010 at 02:10:39 EDT from IEEE Xplore. Restrictions apply.

IV. SIMULATION RESULTS

p = A * file _ size *8*number _ of _ sources *average _ hop _length / 3600

1800

0.5 file/hr (2.64l\Ibps)1 file/hr (5.28 l\Ibps)1.5 file/hr (7.93l\.lbps)

2 file/hr(10.57l\.lbps)

600 800 1000 1200 1400 1600

Dela~' (seconds)

200 400 600 800 1000 1200 1400 1600 1800

200 400

The results presented in Figure 4 show that CNF

achieves 71 % higher network capacity compared to TCP

(10.6Mbps vs. 6.2 Mbps). Next we study the file transfer

delay and plot CDF of the average delay at various offered

loads with CNF and TCP in Figure 5 and Figure 6. We see

that at offered load of I file/hr (5.28 Mbps), 60% of the files

incur less than 200 seconds delay for both CNF and TCP.

However, the next 40% incur another 350 seconds delay in

CNF and 1200 seconds with TCP. The flows that traverse

larger number of hops or through bottleneck links are flow

controlled in TCP hence the increased delay. This TCP

unfairness has been studied in

[1 ].

Dela)· (seconds)Figure 5: CDF of file transfer delay with TCP

Figure 6: CDF of file transfer delay when using CNF LL and CNF transport

We have presented in Figure 7 results from priorwork [10] that shows the composite performance of CNFtransport protocol used with CNF link layer protocol incomparison with TCP over IP routing and link protocols.Figure 7(a) shows that even in a purely wired network, CNFtransport and link layers can sustain a much higherthroughput with lower delays. Figure 7(b) shows the resultswhen one wireless link is added to the same route. Here TCPincurs 100s delays when the offered load is 5Mbps whileCNF can sustain a 20Mbps load with the same delay

1110678O&red Load (Mltps)

3

i f~CNF

t--'rCPIjIJ

! ----f-.I

II

!JI

i! II I

I ---J .a-.

I~

....--__-a,....-----~.

=+-== -r-~-T

~ 10008~.:it soo-;Q

Figure 4: Perfonnance comparison ofUDP with CNF LL against TCP.

indicated in the RFB has been received already. If this is anew file, the receiver creates an entry for the file in its ownmemory and sends a NACK with all bits set to 1. Otherwise,the receiving node creates a bitmap to appropriately indicateall un-received packets from the last batch. When thesending node receives a NACK, it (re)transmits any packetthat was marked as not received or moves on to the nextbatch if all packets were received. This reliability may be inaddition to the per-packet MAC layer retransmission andreliability. We have evaluated the CNF link layer withoutthe additional MAC layer per packet reliability.

We present the performance evaluation of the CNF linklayer in an NS2 simulation of a wireless multi-hop networkof 49 nodes placed in a uniform 7x7 grid. Adjacent nodes inthe grid are 200m apart, the transmission range is 250m andthe carrier sensing range is 550m. The channel isimplemented as a two-ray ground path loss model and theMAC layer data rate is fixed at 11 Mbps.

Each node in the network transmits a 10MB file toa random destination at the rate of A per hour, where A is aPoisson variable. We define offered load (p) as

1500

We vary A to change the offered load and note the

average file delay. We define the capacity of the network asthe offered load for which the average file delay remains

below 500s. We set up the CNF network with UDP at the

transport layer, IP at the routing layer, CNF link layer and

IEEE 802.11 at the MAC layer. We compare this with TCP

over normal network and link layers. To allow the network

to stabilize, we wait for the first 450 files to enter the

network before we start measurements. We measure the

average delays incurred by the next 550 files as they enter

and exit the network. We observe that on an average the hop

length from file source to destination is 4.63.

Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on April 05,2010 at 02:10:39 EDT from IEEE Xplore. Restrictions apply.

Algorithm 1: Algorithm to build the short term routing table

based upon ETX [12], ETT [13], LQI [14], error rate or acombination of several cost metrics. We have used expectedtransmission time (ETT) for our initial study. We alsoinclude the minimum available storage space along the routein both the long and short term routing tables.We have used OLSR as the baseline routing protocol toimplement our routing metric. OLSR uses topology control(TC) and Hello messages to disseminate connectivityinformation throughout the network. We will now describethe routing table building algorithm using two hop neighbor

tables received in the TC messages. At node x, let N(2) k

be the set of two hop neighbors n(2) ki that appear in the

TCk received from node k. Let n(1) kj be the one hop

neighbor that are advertised as the nexthop to reach n(2) ki .

ETT(k, n(2) ki) is the short term link cost in terms ofETT

to reach k from n(l) Iq" and Space(k, n(2) ki) is the

minimum storage space along the route

k ~ n(l) kj ~ n(2) ki' Algorithm 1 illustrates the short

term routing table building process using this information.Similarly we build the long term table by taking a

moving average of link costs from several TC messages. Thelong term table consists of several alternate paths. Wecurrently keep the five best paths in the long term tablewhere the goodness of the path is determined by their longterm costs. We resolve route to a destination by looking upthe long term cost of the route obtained from the short termtable. We map the route in the two dimensional decisiongraph shown in Figure 8.

This decision graph classifies routes in threeregions based upon their long term and short term costs. Thestable region contains routes in which there is little deviationfrom the long term cost. The store region is comprised ofroutes with slower short term links and the forward andstore region that have good short term links but slow longterm links. As the names suggest, if a route in the storeregion is selected, the data needs to be stored until the routequality improves. This implies the routes in the store regionmust have sufficient storage space. Similarly in routes thatfall in the forward and store region, it is desirable for the

downstream router to have enough storage space. When aroute is in the stable region, storage might not come in play

end ifend for

end for

for ':;'n(2 lkl ~V (2)k do~st1llatioll:= n! 2)/"-1

Cost:= jll1'

for vq E .V (1).1' doif Cost £TT(f. 1/(1 Lrq) + £TT(n(1 ).1'q' k) +E:'TT I: k_11 (2 )kl) then

Cost:= £TT(.f. nil ).rq) + ETT('11{ 1 )xq. k) + ETT(k. n(2)kJ)N~xt Hop:= 11 ( 1).1'q

Storage:= S'rH1Cf'\l'. n(21I\1', = IBin Spacfi .... :1.

'7 nod~s .... in th~ rout~ frotu .1' to 1} i: ~ 11.~1

iii

20 30 40 50 60 70Offered traffic load (Mbps)

--e-CNF delay--'-CNF delay 1% noise

F-····,,·······:················:···'-+-CNF delay 5% noise.......- CNF delay 10% noise......-rcp delay--.-rcp delay 1% noise

~:S!M~.....-"" ··· ... ·············;·················1 -+-rcp delay 5% noise

.......- rcp delay 10% noise

C11111i1 iii iii

2

6

V. CNF ROUTING PROTOCOL

In this section we describe the design of a dynamic CNFrouting protocol which is intended to improve theperformance even further relative to static routingconsidered in Section IV. We also incorporate the storagecapacity available in CNF routers into our routing algorithmdesign so that content can be temporarily stored in thenetwork until the end-to-end path quality is consideredacceptable. The design is also robust to link qualityvariations and is opportunistic in utilizing short termimprovements in the end-to-end path quality.

We propose a routing metric design in which eachrouter maintains two routing tables: a long-term routingtable that contains the average link costs over long timeduration and a short-term table that maintains link costsaveraged over shorter time duration. The link cost may be

We have seen benefits of using CNF link andtransport layer and we believe that CNF routing layer wouldimprove these results further. We will present thepreliminary design of this routing philosophy in the nextsection.

~o 100 150 200 250 300 350 400 450Offered traffic load (Mbps)

Figure 7: CNF and TCP file transfer delays with varying number ofwireless links in the route (8) (a) no wireless links, (b) one wireless link.

isl-··.··.···.··-:··· .. ·· ·····.···············:······· Jr .

.!!!,

~CD'lJ

i 4(IJ

c:as~

~ 31-··························;·········,······

LL

constraint.500

450

400

u- 35O•~i;' 300

"i)

~ 250 ..!(IJ

g200

.!!u:: 150

100

50 .

00 10

7

Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on April 05,2010 at 02:10:39 EDT from IEEE Xplore. Restrictions apply.

but the path selection protocol should favor a route withmore storage when comparing two routes in the stable regionwith equivalent long term costs.~UJ

By selecting a path from the forward and storeregion, the routing protocol opportunistically utilizes bettershort term paths. By choosing to store the data and wait forthe better link to become available, the routing protocolavoids using links that are poor in the short term. Whilewaiting for a link to improve, the router may either serveanother route or defer the medium access to other routersthat have better links. This scheme has the promise toimprove the overall system performance while preservingindividual performance objectives.

At the time of writing this paper, we are in theprocess of implementing this protocol in the ORBIT testbedas well as in the NS2 simulator.

VI. CONCLUSION

The CNF project is currently at an intermediate stage ofdevelopment. We have completed initial design andevaluation and we are working on routing protocoloptimization and proof-of-concept prototyping on theORBIT testbed at WINLAB [15]. Our long term goal is tomove toward real world deployment of the CNF networkusing the outdoor ORBIT deployment covering the Rutgerscampus with open Wi-Fi access points and WiMax/cellularbase stations. It is also noted here that although CNF wasdesigned as a clean-slate architecture, we are also in theprocess of evaluating strategies such as networkvirtualization [16] [17] that would enable evolutionary CNFdeployment and coexistence with existing services such asTCP/IP and VOIP.

Communication Systems Software and Middleware COMSWARE,Bangalore, India, 2007.

[4] "Networking Technology and Systems: Future Internet Design(FIND)," 07-507, 2007.

[5] "Global Environment for Networking Innovations (GENI)," 06-601,2006.

[6] "FP7 Information and Communication Technologies:Pervasive andTrusted Network and Service Infrastructures," FP7-ICT-2007-2, 2007.

[7] M Lemke, "Position Statement: The European FIRE Initiative," ,Washington DC, 2007.

[8] Roy Yates, Dipankar Raychaudhuri, Jim Kurose Sanjoy Paul, "TheCache-ADd-Forward Network Architecture for Efficient Mobilecontent delivery services in the future internet," in First lTU-TKaleidoscope Academic Conference on Innovations in NGN: FutureNetwork and Services, 2008, pp. 367-374.

[9] Kevin Fall, "A delay-tolerant network architecture for challengedintemets," SIGCOMM '03: Proceedings of the 2003 conference onApplications, technologies, architectures, and protocols for computercommunications, Karlsruhe, Germany, 2003.

[10] Yanyong Zhang, and Dipankar Raychaudhuri Hongbo Liu,"Performance Evaluation of the Cache-and-Forward (CNF) Networkfor Mobile Content Delivery Services,"Under Submission, 2008.

[11] Yanyong Zhang, Dipankar Raychaudhuri, Sanjoy Paul, Hongbo LiuLijun Dong, "Content Retrieval in a Cache-and-Forward Network,"WINLAB technical report, New Brunswick, New Jersey, 2008.

[12] Daniel Aguayo, John Bicket, Robert Morris Douglas S. 1. De Couto,"A high-throughput path metric for multi-hop wireless routing," inMobiCom '03: Proceedings ofthe 9th annual international conferenceon Mobile computing and networking, San Diego, 2003.

[13] R., Padhye, 1., and Zill, B Draves, "Routing in multi-radio, multi-hopwireless mesh networks," in In Proceedings ofthe 10th Annualinternational Conference on Mobile Computing and Networking,Philadelphia,USA, 2004.

[14] Erol Gelenbe and Ricardo Lent, "Link quality-aware routing," PE­WASUN '04: Proceedings of the 1st ACM international workshop onPerformance evaluation of wireless ad hoc, sensor, and ubiquitousnetworks, Venezia, 2004.

[15] 1. Seskar, M. Ott, S. Ganu, K. Ramachandran, H. Kremo, R. Siracusa,H. Liu and M. Singh D. Raychaudhuri, "Overview of the ORBITRadio Grid Testbed for Evaluation of Next-Generation WirelessNetwork Protocols," Proceedings of the IEEE WirelessCommunications and Networking Conference, 2005.

[16] Kumar Reddy Victor Moreno, "Network virtualization," Cisco Press,Indianapolis, USA, 2006.

[17] Rui Zhang-Shen, Sampath Rangarajan, and Jennifer Rexford YapingZhu, "Cabernet: Connectivity architecture for better network services,"Proceedings of the Workshop on Rearchitecting the Internet, 2008.

Figure 8: Routing Metric Design

Decreasi ng order of short term En

Store regi on (long term En « short term En)-111111111~ Stable region(Iongterm En == short term En)

Forward and Store reqi on(long term En» short term En)

E2boOC

.Q

'0"­QJ

"EoboOC"V;~

~oQ.Io

REFERENCE

[1] A. DeSimone, Mooi Choo Chuah, and On-Ching Vue, "Throughputperformance of transport-layer protocols over wireless LANs," GlobalTelecommunications Conference, 1993, including a CommunicationsTheory Mini-Conference, Houston, Texas, 1993.

[2] A. V. Bakre and B. R. Badrinath., "1-TCP: indirect TCP for mobilehosts," Proceedings - International Conference on DistributedComputing Systems, 1995.

[3] S. Paul, S. Raychaudhuri, D Gopal, "Leveraging MAC-layerinformation for single-hop wireless transport in the Cache and ForwardArchitecture of the Future Internet," 2nd International Conference on

Authorized licensed use limited to: Univ of Calif Los Angeles. Downloaded on April 05,2010 at 02:10:39 EDT from IEEE Xplore. Restrictions apply.