Evaluation of a comprehensive P2P video-on-demand streaming system

Computer Networks 53 (2009) 434–455

Contents lists available at ScienceDirect

Computer Networks

journal homepage: www.elsevier .com/locate /comnet

Evaluation of a comprehensive P2P video-on-demand streaming system

Karl-André Skevik *, Vera Goebel, Thomas PlagemannDepartment of Informatics, University of Oslo, P.O. Box 1080, Blindern, N-0316 Oslo, Norway

a r t i c l e i n f o

Article history:Available online 13 November 2008

Keywords:P2PVoDStreamingFreeloadersFirewalls

1389-1286/$ - see front matter � 2008 Elsevier B.Vdoi:10.1016/j.comnet.2008.09.025

* Corresponding author. Tel.: +47 22852713.E-mail addresses: [email protected] (K.-A. Skevik)

Goebel), [email protected] (T. Plagemann).

a b s t r a c t

Video-on-demand (VoD) streaming has recently become a popular service on the Internet,with several companies offering videos to a global audience. However, traditional client/server based VoD streaming systems can be very bandwidth intensive and expensive tomaintain, especially for high quality video content. To improve the scalability these sys-tems, the use of peer-to-peer (P2P) networking has been proposed, but despite the effi-ciency of applications such as BitTorrent for downloading of large files, it is not simpleto use P2P techniques for streaming. Problems such as firewalls and freeloaders reducethe efficiency of both types of P2P systems, but for real-time services such as streaming,the result can be reduced playback quality. Other issues include the traffic load imposedon ISPs by P2P networks, which can motivate ISPs to interfere with the P2P traffic. Finally,protecting against malicious modification of content can increase overhead, responsetimes, and startup delays.

We consider these issues to be fundamental to the problem of P2P based VoD, butdespite the large amount of research that has been done in this field, these issues have lar-gely been ignored. To address this, we present an evaluation of the Streaming P2P Protocol(SPP) architecture. By studying the problem as a whole we have found a simple and com-prehensive solution that addresses all the four issues listed above. To show that the systemis not only scalable, but also that it can be implemented efficiently, we have used both sim-ulations and experiments on PlanetLab for evaluation. The results show that the combina-tion of cache nodes and use of end-user resources found in the SPP architecture can give alow load on servers and ISPs, even when firewalls are taken into consideration. Further-more, we observed low startup delays and few playback errors during the PlanetLab exper-iments. The scalable and low-cost distribution of content possible with the SPParchitecture should be suitable for both large-scale commercial distributors and users ofcommunity networks with limited resources.

� 2008 Elsevier B.V. All rights reserved.

1. Introduction

The increased tolerance of major content owners to-wards Internet based video distribution, and an increasingnumber of households with broadband Internet accesshave resulted in the availability of several legal videodownload services from companies such as Apple and Ama-zon. However downloading incurs a delay before playback.

. All rights reserved.

, [email protected] (V.

Streaming reduces this delay, ideally allowing users tostart watching immediately.

Peer-to-peer (P2P) networks offer a tempting solutionto the scalability problems of video-on-demand (VoD)streaming, and the potential scalability of P2P networkshas been studied in great detail. However, the challengelies in creating a system that is scalable despite the practi-cal problems that exist on the Internet: malicious usersthat try to modify the video stream, traffic patterns thatput a large part of the cost of distributing the data on inter-net service providers (ISPs), firewalls that limit connectiv-ity, and freeloaders that reduce the efficiency of the system

mailto:[email protected]



http://www.sciencedirect.com/science/journal/13891286

http://www.elsevier.com/locate/comnet

K.-A. Skevik et al. / Computer Networks 53 (2009) 434–455 435

by not contributing resources. These problems can exist inboth small-scale community networks with limited re-sources and large-scale commercial distribution scenarios:failing to address them can result in excessive startup de-lays or frequent playback errors, making streaming infeasi-ble. Some of these problems have been considered inexisting research; e.g., PROMISE [21] can provide incen-tives for users to contribute, and COPACC [21] incorporatescaching, but much research in this area focuses primarilyon single issues such as scalability and relies on simula-tions that ignore many of these issues.

Rather than examining only a single issue, we have ta-ken a comprehensive approach with the design of thestreaming peer-to-peer (SPP) [39] architecture that at-tempts to address all of these issues, and by doing thiswe have arrived at a unified solution that achieves this goalin a simple manner. The SPP architecture has been de-signed for large-scale commercial VoD services throughefficient use of available resources, but the efficiency ofthe system also makes it suitable for scenarios such ascommunity networks with limited hardware and networkresources. In this article, we present the results of our sys-tem evaluation, and to demonstrate that the system is notonly scalable but also possible to use for video streamingon a real network, we have used both simulations andPlanetLab experiments. The contributions of this articleare the examination of the problems that occur whenP2P networking is used to build scalable, trustworthyVoD streaming systems, and the evaluation of the solutionsfound in the SPP architecture to address them.

The structure of this article is as follows: Section 2 de-scribes the challenges for P2P based VoD streaming andSection 3 examines related work. Section 4 presents theSPP architecture. Simulation results are presented in Sec-tion 5, and the PlanetLab experiments are covered in Sec-tion 6. Section 7 discusses how the SPP architectureaddresses the problems described in Section 2. The articleis concluded in Section 8.

2. Challenges for VoD streaming

VoD streaming systems have traditionally been basedon the client/server model, with the content stored on aserver, or server cluster, and a content distribution net-work (CDN) used to increase scalability by reducing net-work traffic. A drawback with this distribution model isthe significant investments that are required for equip-ment and network capacity, creating high barriers to entry.In this article, we examine an alternative approach; usingthe resources of end-users in a P2P fashion to increase sca-lability and eliminate most of the investments needed tooffer server based VoD streaming services. The primaryusage scenarios for this model is large-scale commercialVoD streaming, but with a sufficiently efficient system italso becomes increasingly realistic for private end-usersto offer VoD services in community networks or similarsystems. The first scenario needs scalability due to a largenumber of users, while the second scenario needs scalabil-ity due to the limited resources of the end-users that servecontent.

A third scenario, which we do not examine in this arti-cle, is VoD streaming from P2P based file sharing systemssuch as eDonkey. For VoD streaming we argue that serverbased systems are better suited than file sharing networks.A legal video distribution service does not need the ano-nymity provided by some P2P sharing networks; it israther, as we discuss below, desirable for users to be ableto identify the content provider. For commercial servicesthe content provider will also need to identify the usersand keep track of content usage. Both of these require-ments are easier to satisfy if the content provider main-tains a server, as opposed to the entire distributioninfrastructure being maintained by a third party.

2.1. Content trustworthiness

With P2P based video distribution, users retrieve mediadata not only from the server, but also from other usersthat have already downloaded the same data. The contentprovider no longer has control of the all the machines thatdistribute the content and it becomes necessary to con-sider the trustworthiness of the content: what happens ifusers try to modify the video stream? This is not merelya theoretical problem. Content owners have used poisoningor pollution [5,32] to reduce the usability of P2P networksfor illegal content distribution by adding fake versions ofpopular movies or music files, illustrating the vulnerabilityof these systems to this kind of attack. A study by Lianget al. [32] found that as many as 50% of copies of some pop-ular files were polluted. Without an effective protectionmechanism it is difficult for users to know if a file containsthe expected content and is safe to download or not. A pro-vider of popular content can become a target for malicioususers that wish to distribute their own content to a largenumber of users. To prevent these problems, P2P basedVoD systems need a way to identify the content providerand detect if content has been modified.

2.2. Caching of P2P traffic

In addition to trust issues, there are more general prob-lems with P2P networks that also affect VoD streamingsystems. The outgoing traffic from end-users creates trafficpatterns that are significantly different from what is thecase with client/server based systems. The media data nolonger passes in only one direction, from the server tothe clients, but can also be sent between any of the clients.For ISPs, the result can be increased outgoing traffic toother ISPs and multiple transfers of the same content. Tra-ditional VoD services can use CDNs with caches to improveperformance and reduce resource requirements, but P2Pbased file distribution networks are rarely designed to beused with caches. Even storing data at end-user machinesdoes not guarantee that popular content will not be trans-mitted multiple times over the same link because of thelimited resources of client machines.

Several studies of popular P2P based file distributionsystems have noted the potential for caching of P2P traffic.At the University of Washington, Saroiu et al. [38] mea-sured the P2P traffic created by applications such as Kazaa.Through simulations based on the measured traffic they

Fig. 1. Firewalls and P2P networks.

436 K.-A. Skevik et al. / Computer Networks 53 (2009) 434–455

discovered that the 300 most popular files were responsi-ble for 42% of the outgoing Kazaa traffic. Caching theseitems would require only a 180 GB cache [38]. Similar re-sults have been obtained by Leibowitz et al. [30] during astudy of Kazaa traffic at a large Israeli ISP. A 300 GB cachewould achieve a byte hit ratio of 67% on this data. Simula-tions performed by Gummadi et al. [15] suggest a 63%cache hit ratio for P2P traffic. These results are for down-load based file distribution networks, and there are indica-tions that VoD traffic on the Internet might not be Zipf-distributed [18], but there will still be benefits from cach-ing. Many ISPs already cache P2P traffic [29] and failing toconsider caching will make it more difficult for ISPs to re-duce the resource requirements of P2P based VoD systems.

2.3. Firewalls

Another challenge for P2P networks is the connectivitylimitations imposed by firewalls. Traditionally found nearthe border between networks, but increasingly even onpersonal computers, firewalls enforce restrictions on trafficpassing through them. P2P networks rely on having directconnections between end-user machines and firewalls canprevent these connections from being established. A clientlocated behind a firewall that blocks incoming connectionscan make outgoing connections but cannot itself receiveany connections.1 Two such clients, like nodes A and N inFig. 1, will be unable to communicate directly at all. Firewalltraversal schemes can in some cases allow communicationacross firewalls, but schemes that try to exploit limitationsin existing firewalls [14] cannot be said to be suitable forcommercial systems. Interfaces such as theinternet gatewaydevice (IGD) protocol [12] that can be used to configurecompliant firewalls are a more appropriate way of allowingincoming connections in a controlled manner, but must besupported by both the firewall and the application.

The limitations on connectivity described above applyto P2P networking in general, but P2P based VoD stream-ing systems are further limited by the communication pat-terns resulting from the linear nature of streaming. In VoDsystems, clients are unlikely to simultaneously request thesame file, and as a result, each client requires different

1 A similar inability to receive incoming connections can occur for clientson a network where network address translation (NAT) is used.

parts of the file at any given time. Normal video playbackresults in sequential block requests, first for block 0, thenfor block 1, etc., and a client that is currently requestingblock 17 will normally only have blocks in the range be-tween 0 and 16 stored locally. The result is that each peerrequests blocks from one set of peers and receives requestsfrom another set of peers. These two sets will be entirelydisjoint unless there is a high degree of interactivity, butresearch on client behavior indicates that playback issequential for most clients [45]. In Fig. 2, Client B can onlyreceive blocks from Client A, and Client C is the only nodethat needs the blocks that Client B has downloaded.

This contrasts with download based P2P networks suchas BitTorrent, where clients generally request the rarestblock first and peers can exchange blocks with each otherin any order. A client that is behind a firewall can connectto a peer that is not behind a firewall, and both clients willbe able to benefit by exchanging blocks with each other be-cause the file is used only after all blocks have been down-loaded. With stream based P2P networks, the situation isdifferent; only one client will benefit from a connection.If Client A in the figure is behind a firewall, it will only bepossible for Client B or Client C to requests blocks fromClient A if Client A initiates the connection, but Client Band Client C do not have any blocks that Client A needs. Thisunderlying limitation on VoD systems increases the impactof firewalls compared to other P2P systems, and makestechniques for increasing connectivity and creating incen-tives for clients to share resources even more important.Failing to do this can in the worst case eliminate most ofthe scalability benefits from P2P networking; as many as35–80% of clients being unable to receive incoming con-nections has been observed by Ganjam and Zhang [13]during P2P broadcasting experiments.

Fig. 2. Data block movement.


2.4. Incentives

The efficiency of P2P networks is the result of end-userscontributing resources, primarily in the form of uploadcapacity. However, studies have shown that there aremany users of systems such as Gnutella who contribute lit-tle or nothing [16], and this can become an even greaterproblem if ISPs stop using fixed-rate pricing schemes. Sev-eral techniques have been developed to prevent freeload-ing, but the limitations from linear playback that affectfirewalls also prevent many incentive schemes from beingused for VoD streaming. The direct exchange of blocks inapplications such as BitTorrent creates incentives becauseparticipants can reduce contributions to peers that give lit-tle back [7]. In broadcast systems, it is also possible to havenodes monitor each other [49], but for VoD there is noguarantee that there will be nodes available that can per-form this task; Client A in Fig. 2 might have disconnectedby the time Client C arrives. The problem is again that re-quests are received from peers that have nothing to giveback.

For streamed movies there are two other additionalimportant limitations. First, the playback quality of adownloaded movie is not affected by the download time,but a streamed movie that is not received sufficiently fastwill at best only be watchable at reduced quality. Second,the linear nature of playback means that a client cannotcontribute anything unless new clients arrive after it, be-cause the existing clients will have all the blocks it re-trieves. Locality awareness can further reduceopportunities for sharing resources, because clients cannotuse peers that are too slow. For commercial systems thereis a further limitation in that paying users are unlikely toaccept reduced quality or interruptions in playback,regardless of what they contribute.

Designing efficient incentive creation schemes for P2Pbased VoD streaming systems is more difficult than fordownload based P2P systems, but at the same time, argu-ably more important due to the real-time performancerequirements of streaming and the loss of efficiency causedby firewalls.

2 http://www.joost.com.

3. Related work

In recent years, several VoD systems that make use ofP2P techniques to achieve scalability have been proposed.While the basic principles are the same, there are manyvariations.

3.1. P2P based VoD streaming systems

P-Stream [35] uses a proxy on the local host to allowstandard media players to be used. BASS [9] combinesBitTorrent and a normal server based VoD system bydownloading blocks needed for playback from the server,unless they have already been retrieved via BitTorrent. ACDN/P2P hybrid system is proposed by Xu et al. [48]. Atraditional CDN architecture is used initially to serve files,but as the streaming capacity of clients increases, the sys-tem gradually shifts the load of serving to the clients. CO-

PACC [27] combines P2P networking and proxy caches ina similar way but builds the P2P network from the prox-ies, rather than from the clients. CoopNet [34] is a anotherserver based solution where end-user resources are onlyused to handle flash crowds. DirectStream [20] uses adirectory server to store content and maintain informa-tion about clients. GridCast [4] is a server based systemthat uses clients to offload the servers, which has beendeployed and tested with a large number of users. Joost2

is a commercial streaming system that also has been de-ployed. The company running the service has not publisheda detailed description of the system architecture, but someanalysis work has been done by Hall et al. [22]. In P2 VoD[10], clients cache recently received content and can for-ward it to new clients if sufficient outgoing bandwidth isavailable. Nodes are grouped into generations based onthe data that they have cached. Other members of a gener-ation can replace a failing peer to avoid contacting themain server. P2Cast [19] adapts patching [26] to achieveP2P based streaming. In PALS [36], adaptive layered stream-ing is used to serve heterogeneous clients. PBA [23] uses apull-based approach to simplify the operation of the server.A request is first made to the server, which serves the vi-deo directly if there is sufficient capacity, otherwise theserver gives the client a list of randomly chosen peers thatcan be contacted instead of the server. PROMISE [25] is astreaming system built upon CollectCast [24], which moni-tors the network and the status of other peers. PROP [17]uses caching proxies to organize clients at a site into aP2P network. The proxy retrieves content from the server,but the content can also be shared between the clientsfor which the proxy is responsible. The PeerStreaming [31]system is also client driven and has support for protectionof content using digital rights management (DRM). Push-to-peer [43] primarily targets networks with set-top boxesunder the control of the content provider, and spreadsthe content among the available set-top boxes before it isrequested by users.

3.2. Related work analysis

Despite the significant research interest in this topic, weargue that the problems outlined in Section 2 have not beenadequately addressed. The use of end-user resources givesmost P2P based systems the potential for scalability, andwe assume that all the systems described have this quality.To analyze the related work, we use the following factors.

� Trustworthiness. To what extent is it possible to trust thesource of a media file? The evaluation of this characteris-tic is divided into two parts. The first relates to whetherthe content is retrieved from an easily identifiable source,and the second to whether it is difficult for users to detectmodification or corruption of the distributed content.

� Caching. Can the system make use of caches to reduceresource consumption? By caching we do not meancaching at clients but integration of more permanentCDN-like caches deployable at ISPs and large networks.

http://www.joost.com


� Firewalls. Does the system try to reduce the negativeeffects of firewalls on P2P networking?

� Sharing incentives. Does the system try to discouragefreeloaders?

Table 1 shows the result of this analysis. It is based onthe published description of each system, so some of the is-sues listed above may have been addressed without beingcovered in published literature. Parenthesis indicate sys-tems that do not appear to have been designed with theindicated functionality as a goal but where there are as-pects of the design that provide some support, or thatcan likely be modified to do so. A question mark is usedin the case of Joost, where the available information ismainly based on reverse engineering and unfortunatelyincomplete.

Most systems have a certain level of trust, in that thecontent is retrieved from a well-defined server, but lessthan half have a mechanism to protect it. As for the otherthree issues – firewalls, incentives, and caching – theyare ignored by most of the systems.

Table 1P2P based VoD system functionality overview.

Name Trust Caching Firewalls Incentives

P-Stream X/– (X) – –BASS X/X – (X) (X)CDN-P2P hybrid X/– (X) (X) –COPACC X/X X (X) –CoopNet X/X – (X) (X)DirectStream X/– – – –GridCast X/– – (X) –Joost X/X ? X –P2VoD X/– – – –P2Cast X/– – – (X)PALS –/– – – –PBA X/– – X –PROMISE –/– – – XPROP X/– X (X) –PeerStreaming X/X – – –Push-to-peer X/(X) – – –

Fig. 3. WWW and SP

4. The SPP architecture

To address the problems analyzed in Section 2, we havedesigned a P2P based streaming system called the SPParchitecture. The application area for the system is legalVoD streaming of popular content, where it is desirableto be able to verify rather than hide the identity of the con-tent distributor. The system must be not only scalable andefficient, but able to protect the distributed content. Twobasic design guidelines have been followed to reach thisgoal. First, a permanent infrastructure can be used if it isavailable, but should not be relied on. Second, participatingpeers can be used for load distribution, but should not betrusted.

These two guidelines are related. Instead of having torely on information provided by clients we use infrastruc-ture elements in which some trust can be placed. The infra-structure might include multiple redundant servers andCDN-like caches, or be as simple as the machine of a userdistributing a single video file.

4.1. System structure

As seen in Fig. 3a, the structure of the SPP architecture isin many ways similar to that of the World Wide Web,where content providers use HTTP servers to make filesavailable to users. Users request content from the servers,with requests possibly being obtained from a web cachethat in some cases might be configured to cooperate withother caches [3]. The SPP architecture follows a similarstructure, with servers that serve as the origin of publishedcontent, site content caches (SCCs) that cache content, andusers that request content. The primary difference be-tween the two systems is the use of P2P networking inthe SPP architecture, not only between the caches but alsobetween the clients.

Files distributed by the SPP architecture are logically di-vided into blocks, and the server and SCC nodes keeps trackof block downloads by clients. Blocks can be received fromeither the server, an SCC node, or other clients. Each client

P comparison.

Fig. 4. SPP architecture – system structure.

Table 2SPP command summary.

Command Description

HEAD Returns metadata such as file size from server/SCCPEERS Returns a list of peer nodes from the server/SCCGET Requests a byte range from another nodeIGOT Reports the reception of a block to the server/SCCCRED Requests information on crediting from server/SCCSTARTTLS Starts initiation of a secure connectionPING Null operation, can be used for latency estimationREVCONN Used to increase node connectivity


is responsible for obtaining the blocks it needs in correctorder and in time to ensure error free playback.

The system structure, as seen from the viewpoint of aclient is shown in Fig. 4; the server on the right, the SCCnodes in the middle, and the client to the left. The serverand SCC nodes form the fixed infrastructure, and during astreaming session, each client has either an SCC node ora server as a parent node. Client nodes run a local hostcache (LHC) that acts as the interface between the mediaplayer and the streaming system. For sites with an SCCnode, the SCC address can either be configured into theLHC or connections can be automatically redirected, withthe SCC acting as a transparent proxy. The operation ofthe LHC is the same regardless of whether there is a hier-archy of SCC nodes or a direct connection to the server. Po-tential challenges related to cache deployment at ISPs arediscussed in Section 7.2.

The server and SCC nodes maintain a list of child nodes;adding an SCC node results in increased network localityby clustering close nodes together. Clients request a listof peers from the parent node and when this is an SCC nodeinstalled at the local site or ISP, it only returns the ad-dresses of other local peers in addition to its own address.The server maintains a list of the top level SCC nodes andclients without an SCC node.

4.2. Block selection

Most actions in the system occur as a result of user ac-tions, starting at the left of Fig. 4 and propagating towardsthe right. The user controls the media player and can per-form operations such as starting movie playback, pausing,or changing the current playback location. To fulfill theserequests the media player requests a continuous datastream from the LHC, which obtains the correspondingblocks from either another client or the SCC, which againobtains the block from another client, another SCC node,or the main server. The LHC will continue to retrieve blockssequentially from the system as long as the user continueswatching the data stream and the media player continuesto consume the media data. If the user pauses the video,the media player will stop consuming data and the LHCwill stop requesting additional blocks until playback isstarted again.

Both LHC and SCC nodes can prefetch blocks to reducethe latency of requests, but barring interactive operations,block selection will start at the first block and end with thelast block in the file. For performance reasons, the LHC andSCC nodes need to be network aware and able to distin-guish between fast and slow peers, because requestingdata from peers that are too slow can make it impossible

for the media player to show the requested content atthe playback rate.

4.3. Protocol commands

Communication between nodes in the SPP architectureis done with a protocol that is partly based on HTTP, withsome commands such as GET and HEAD used almost iden-tically. Additional commands have been added to handleP2P operations: the PEERS command to requests a list ofpeers from the parent node, IGOT to report block receptionto the parent node, CRED to verify peer submitted informa-tion, STARTTLS to verify the identity of clients, REVCONNto allow connections to peers behind a firewall, and PINGto estimate latency. Table 2 gives a short summary of allthe commands.

The media player is the interface between the user andthe SPP architecture. A file located at a given URI is speci-fied by the user, and a streaming session is initiated by themedia player contacting the LHC. HTTP is supported by theLHC to allow media players with support for HTTP stream-ing to be used without modification, but SPP can be useddirectly by media players that support this.

Upon receiving a file request, the LHC uses the HEADcommand to obtain metadata for the file from the parentnode. The PEERS command is then used to request a listof potential download locations for the first block fromthe parent node. The LHC selects one of the peers on the listand requests the first block with the GET command. Afterreceiving the entire block, the LHC calculates a message di-gest based on the content, and uses the IGOT command toreport the result to the parent node. It also informs the par-ent node of the identity of the node it obtained the blockfrom. If the message digest is correct, the parent node addsthe client to the list of nodes returned with the PEERS com-mand. To reduce the number of operations that involve theparent node, a single PEERS command can contain infor-mation about multiple subsequent blocks, and it is possible


to report the reception of multiple blocks with one IGOTcommand. The other commands are optional optimiza-tions, and described in more detail below.

4.4. Protocol operations

To illustrate how the protocol works, we describe theoperation of the commands in relation to the problemsidentified in Section 2.

4.4.1. Content trustworthiness and cachingThe SCC nodes are an important part of the system and

Fig. 5 shows an example of how data exchange can occur ina scenario with two clients and one SCC node, where theclients, LHC A and LHC B, both request the same block. Afterhaving obtained the file metadata with HEAD, the firstPEERS request is issued by LHC B, shown in Fig. 5a. The re-quest is sent to the SCC, and because it is the first requestfor this file, the SCC requests a list of peers from the server.The SCC then returns a peer list containing only the SCC ad-dress. With only one peer on the peer list, peer selection issimple for LHC B, which proceeds to send a GET request tothe SCC node, as shown in Fig. 5b. The data is received bythe SCC from the server, and retransmitted to LHC B.Fig. 5c shows the actions executed after the entire blockhas been received. The SCC computes the message digestand uses IGOT to report the result to the server, which con-

Fig. 5. Protocol intera

firms that the reported message digest is correct. This pro-cedure is repeated at LHC B, which reports the receivedblock to the SCC. After these steps the block is located atboth the SCC node and LHC B.

After this first protocol exchange, a request for the peerlist for the same block is made by LHC A, as shown inFig. 5d. The SCC now returns a peer list with two entries:the SCC node itself and LHC B. From this list, LHC A requeststhe block from LHC B, as shown in Fig. 5e, and after havingreceived the block, it notifies the SCC. The client also re-ports that it has received the block from LHC B, as shownin Fig. 5f. The peer list at the SCC now contains three en-tries for this block: the SCC, LHC A, and LHC B. The serveris not involved in the second exchange.

4.4.2. FirewallsAs Section 2.3 indicated, firewalls represent a particular

problem for P2P based VoD streaming systems. The SPParchitecture includes the REVCONN command, which canbe used to increase connectivity between nodes.

The way a connection is normally established betweentwo nodes is for the node that wishes to request a block tosend a request to a peer with the block available, but thisis not possible if the peer is behind a firewall. When theREVCONN command is used, the opposite occurs: thenode that has the block instead connects to the node thatwishes to request it. The main challenge with this ap-

ction example.


proach is that clients do not know which peers need theblocks that they have, but by exploiting the sequentialnature of video, a client can find potentially interestedpeers. A client located behind a firewall first contactsthe parent node to retrieve information on blocks alreadyretrieved by other peers. The goal is for the client to iden-tify peers that have fewer blocks than itself. It then opensa connection to one or more of the identified nodes anduses the REVCONN command to indicate that the purposeof the connection is to allow requests to be received. Itthen becomes possible for the node that received the con-nection to request blocks over the connection.

4.4.3. IncentivesThe SCC nodes and servers can keep track of resource

sharing among clients because clients report downloadedblocks with the IGOT command. The challenge is prevent-ing peers from falsely claiming to have made contributions,or from not reporting the contributions that they have re-ceived from other peers. A two-step reporting and verifica-tion process is used for this purpose. Each client knowshow much it has sent to other peers and can use the CREDcommand to verify that this information has been correctlyreported to the parent node. The parent node returns a listof the information reported by other peers, and this infor-mation can be compared against what the client knows ithas contributed. Subsequent requests from peers that havenot properly reported the data they have received from theclient can then be refused.

A freeloader can still try to cheat by regularly changingthe address that it is using, or by creating fake nodes thatfalsely credit the freeloader. More permanent identitiesare needed to prevent this [11], and the SPP architecturecan make use of a standard public key infrastructureand secure sockets layer (SSL) based verification of certif-icates with the STARTTLS command, which initiates theestablishment of a secure connection. Key distribution isnormally difficult in P2P networks but for legal contentdistribution scenarios where there is a commercial rela-tionship between the content provider and the users, itis less difficult to distribute keys on, for example, smartcards. SCC nodes propagate information on client sharingto the server, creating an accounting mechanism whichgives the server overview of resource sharing in the sys-tem. Both SCC owners and content providers can use theaccumulated information to provide incentives for shar-ing, or punishments for freeloaders. The SCC can prioritizeclient access to resources such as network bandwidth,whereas content providers can offer free content.

4.4.4. Playback qualityWhile download based P2P systems only need to ensure

that files are downloaded correctly, it is necessary for P2Pbased streaming systems to try to obtain the media data intime for error free playback. There is no guarantee that thiswill always be possible because transfer rates depend onthe capacity of available peers and the traffic on the linksbetween them, but the nodes in the system can alwaystry in a best effort manner to obtain data as fast as possible.To achieve this it is necessary to estimate the relativespeed of the available nodes, and ideally be able to do this

as fast as possible, because any time spent on doing esti-mation, before requesting data, increases the startup delayobserved by users.

Many techniques for peer or server selection, includ-ing the work by Carter and Crovella [2], work by measur-ing the response to ICMP or UDP packets, but due totraffic limitations imposed by firewalls, there is no guar-antee that they always can be used. For the SPP architec-ture, the PING SPP command exists to ensure that therewill always be a way to at least estimate the Round TripTime (RTT) to a peer. A node receiving this command willreply as fast as possible, making it possible to estimateRTT by measuring the response time. By exploiting thefact that there is often a relationship between RTT andthroughput for protocols such as TCP [33], this commandprovides a fallback for throughput estimation when moreaccurate techniques cannot be used. Because the opera-tion is performed at the application level, the result isless accurate than a datagram based approach wouldbe, but it can be used when other techniques do notwork. For example, a client receiving a REVCONN requestfrom a peer where a firewall prevents datagram basedtechniques from being used, can still use the PING com-mand to estimate latency.

5. Simulations

To evaluate the potential for scalability in the SPP archi-tecture we have used simulations. The goal of these simu-lations is to determine the minimum resources required atthe server, the ISPs, and the clients, in order to achieve er-ror free streaming to each client. We compare the resourcerequirements for the SPP architecture to a client/serversystem, a CDN, and a server based P2P network.

5.1. Simulation approach

In all four distribution models it is possible to identifythree primary entities that carry the majority of the traf-fic: the content provider, the clients, and the ISPs withmany clients. To examine the load distribution betweenthese entities we use the same workload with all fourmodels and calculate the resource requirements; the totalserver load, the aggregated contribution of clients, and theaggregated load on the large ISPs. All the resources avail-able in each system are used in turn, starting with the cli-ents, then the caches, and finally the server. For example,in the P2P model, other clients are preferred before theserver, but the server must supply the resources for re-quests that are not fulfilled by other elements in the sys-tem. A high server load shows that a model is not scalablebecause most requests must be served from a single point.A low server load shows that the system can be scalable,depending on the load on the other elements in thesystem.

Creating a realistic simulation environment is difficult,especially one that attempts to consider the unpredictablebehavior of users of an Internet based P2P network. Thesimulation approach described above focuses on findingthe system capacity in the different design alternatives,


which depends less on the behavior of individual clients.Furthermore, because the scalability problems for stream-ing are caused by the large size of the media data, we use asimplified network model that does not consider factorssuch as packet loss and protocol overhead. These issuesare instead studied with the prototype described in Section6.

5.2. Simulation models

In the client/server model, all data is retrieved from asingle server or server cluster. Scalability is limited be-cause the bandwidth consumption on the link from theserver increases linearly with the number of clients. Clientsnever communicate directly so firewalls are not an issue,but ISPs are affected by redundant traffic due to the lackof caching. The CDN model is essentially the client/servermodel with caches located at strategic points in the net-work. Fig. 6a shows a network with a cache located at ISP B.

The P2P based VoD streaming model we simulate usesend-user resources to reduce the load on the server butdoes not integrate caching. Content still originates at theserver, but clients can request blocks from peers that havepreviously downloaded the same block, as shown inFig. 6b. The last model is the SPP architecture, shown inFig. 6c; the caches used in the CDN model are combinedwith the end-user resources of the P2P model, and cluster-ing of nodes at the same ISP.

5.3. Simulation workload

The simulations examine a usage scenario with a med-ium sized video server and clients with link capacitiesclose to current end-user broadband Internet links. Manyusers have asymmetric Internet connections, but for P2Pnetworking, the most important parameter is the upload

Fig. 6. Simulation mode

capacity, to which content providers have to adapt. To ben-efit from P2P networking, the encoding rate of distributedmovies should be based on the link capacity of the usersthat are likely to watch them. In these simulations, we as-sume that the playback rate of the video corresponds to theupload capacity of the users, and that the download capac-ity is not lower than the upload capacity. Some simulationvalues, such as the server link capacity, are not inputparameters but output values.

An artificial workload spanning seven days was gener-ated with MediSyn [44], which uses a generalized Zipf-likedistribution based on requests to a real-world VoD server.The server hosts 1000 700 MB files with a playtime ofroughly 93 min, for a playback rate of 1 Mbit/s. The blocksize is 512 KB, giving 1400 blocks and roughly 4 s betweeneach downloaded block. The network topology was gener-ated with Inet [46] and includes 9221 clients, 27 ISPs, and asingle server. We have defined a subtree in the generatedtopology with 50 or more leaf nodes as belonging to a sin-gle large ISP. Approximately half of the clients belong toone of these ISPs, which as a result carry a large portionof the network traffic. File requests are distributed ran-domly among the clients while avoiding multiple requestsfor the same file from the same client. As many as 40% ofthe clients are randomly classified as having a firewall [39].

The resource contribution and capacity of clients is oneof the simulation parameters. The ISPs have caches in theCDN model and SCC nodes in the SPP architecture model,both with a disk capacity of 180 GB[38]. Clients cache atmost 1 GB of media data, which is enough to store onecomplete movie. All caches use a FIFO based cache replace-ment algorithm. Clients have at most one outgoing trans-fer, while the SCC nodes are expected to have highercapacity and allow four simultaneous external clients. Peerselection is random and based on all peers with resourcesavailable. The REVCONN command is used in the P2P and

l data movement.


SPP models, and assumes an optimal client search algo-rithm that allows accessible clients to request blocks frompeers behind firewalls.

5.4. Simulator

The workload is taken as input by a simple simulatorthat we have written to compare the four distributionmodels and examine the effects of changing the simulationparameters. The file request patterns specify when file re-quests are made and the simulator converts each of theseto a sequence of block requests issued by a client. A simpleinteractivity model loosely based on observations made byVilas et al. [45] is used to model the behavior of each client,and it can change the sequence of block requests made by aclient. For most clients the blocks will be requestedsequentially, starting at block 0, but interactivity changesthis for some clients. A backward jump in the file resultsin some of the blocks being requested twice, a forwardsjump results in a gap in the sequence of requested blocks,a pause introduces a break in the time between two blockrequests, and a client leaving early results in only the firstpart of the sequence being requested. The simulator runsuntil all requests have been completed and calculates thenetwork resources required for clients to receive the re-quested blocks in time for playback.

The amount of data that must be received by clients atany time in the simulation is given by the sequence ofblock requests, but the resources required in the networkto achieve this depends on the simulation model. For theclient/server model the blocks must be transmitted toevery client all the way from the server. For the CDN modelblocks might be cached at an ISP cache. For the P2P modelanother peer might have the blocks, and in the SPP modelblocks can be cached at another peer or the parent SCCnode. In this manner, we use the simulator to examine re-source requirements and the potential for resource reduc-tion in each of the models.

Peer selection and factors such as whether a node is be-hind a firewall or will perform an interactive operation aredetermined randomly, with the standard rand() system calland a fixed seed used to generate random values in areproducible manner.

5.5. Model comparison

We first compare the scalability of the different modelswhen no incentive mechanism is used. Clients contributeresources while they are active but leave shortly after hav-ing received the entire file. The resource requirements are

Table 3Baseline simulation results.

Model Server Ext. (%) Client caches

Int. (%) Ext. (%)

Client/server 99.95 0.05 –CDN 60.73 0.05 –P2P 30.49 0.05 69.46SPP 5.76 0.05 21.60

shown in Table 3, with load distribution on the left and theresulting ISP traffic on the right. A value of 100% corre-sponds to all of the 13.432 TB requested by the clients dur-ing the seven days. Blocks are requested from either theServer, the Client caches, or the ISP caches. The client cachescan receive multiple internal requests from the mediaplayer as a result of interactivity. The ISP caches receiveinternal requests from clients at the same ISP and externalrequests from clients at other ISPs. In the SPP architecture,the ISP caches correspond to the SCC nodes. The ISP trafficcolumns show the resulting incoming and outgoing trafficat the ISPs with the incoming ISP traffic included; the firstfive columns show only where the requested content orig-inates, with the sum of all values being 100%.

For the client/server model, 99.95% of the data is re-quested from the server, and 47.49% enters the ISP networkfrom external nodes, in this case the server. The server loadis 99.95% rather than 100.0% due to client interactivity. The0.05% not served by the server is retrieved from client ca-ches after a backwards jump in playback causes previouslyretrieved content to be requested again. In the client/servermodel, there is no caching at ISPs and data is not sharedbetween clients, so the other values are not relevant forthis model. When caches are added at ISPs, the server loadfalls to 60.73% in the CDN model, which is still relativelyhigh, but for the ISP nodes the load from incoming trans-fers falls from 47.49% to 8.26%. The load is moved fromthe server to the ISP caches, which have a load of 39.22%.

With the P2P model, the server load falls to 30.49%,which is half of the value for the CDN model. The loadreduction is a result of clients sharing data, but for the ISPs,this leads to outgoing traffic at 32.09%. The lack of cachingadditionally increases the incoming traffic from 8.26% to46.52%, but a topology-aware peer selection algorithmcan reduce this value. The SPP architecture combines bothcaching and client sharing, giving a server load of only5.76%. This corresponds to a reduction in the averagethroughput of the traffic leaving the server from186.30 Mbit/s, for the client/server model, to 10.73 Mbit/s. The SCC nodes receive requests from both internal ISPclients and external nodes. Compared with the P2P model,the outgoing ISP traffic is increased from 32.09% to 44.38%,while the incoming traffic is decreased from 46.52% to8.26%. The outgoing traffic is caused by external requeststo the SCC nodes. The requests served by clients fall from69.46% to 21.60% because the SCC nodes now serve manyof the requests, and caching reduces the overall traffic.

These simulations use a simple network model, and allclients are assumed to be homogeneous, but the resultsconfirm the expectations for the different content distribu-

ISP caches ISP traffic

Int. (%) Ext. (%) In (%) Out (%)

– – 47.49 –39.22 – 8.26 –– – 46.52 32.0928.21 44.38 8.26 44.38

Fig. 7. Random packet corruption.

Fig. 8. Coordinated attack block corruption.


tion models. The client/server model is limited by the ser-ver capacity and is not scalable. A cache-based CDN im-proves scalability, but the server still carries a high load.A P2P based approach further reduces the server load butincreases traffic for the ISPs. Overall, the SPP model givesthe lowest server load and eliminates a large proportionof the incoming transfers for the ISPs. The increase in out-going traffic for ISPs in the SPP model is examined furtherin Section 5.8.


In Section 2.1, we listed content corruption as a poten-tial problem for P2P systems. Message digests can protectagainst both random corruption during transmission andmodification by malicious users that deliberately try tointroduce corrupted content into the system. An attackerwill need to generate digest collisions for every block in afile to make an effective attack. When message digestsare used there is a fairly good guarantee that clients willreceive a data stream identical to the one originating fromthe servers because the content provider is free to choose adigest algorithm that provides an appropriate level ofprotection,

The drawback is increased system complexity and extraCPU overhead for clients, which need to verify that the re-ceived content is valid. To determine the extent to whichthe use of digests is necessary, we have examined the ef-fects of data corruption in our simulation environment. Aconsequence of the nature of P2P networks is that any cor-rupted block received by a client without detection can bespread to a large number of peers. According to Stone andPartridge [42], the number of packet errors that are unde-tected by both link level and TCP checksums are betweenone data packet in 16 million and one packet in 10 billion.Despite being infrequent, this type of error becomes prob-lematic due to the large amount of data transmitted in astreaming system. Assuming a packet size of 1500 bytes[28], it is possible to calculate the frequency of per-packetcorruption, and in the simulations we have examined theeffects of packet corruption on the P2P and SPP models,when no application level message digests or checksumsare used to detect errors.

We have run two simulations, one for the high probabil-ity end of the range, when the probability of undetectedblock corruption corresponds to one data packet in 16 mil-lion, and one for the low probability end of the range, whenthe probability corresponds to one packet in every 10billion.

For the low probability end of the range, no packet cor-ruption occurs in the simulations, but for the high proba-bility end of the range, this becomes a problem. Anoverview of the percent of block transfers that involve acorrupted block is shown in Fig. 7. In the P2P case, as manyas 0.379% of all transmitted blocks contain an error unde-tected by link or transport layer checksums after sevendays, while the corresponding number is 0.048% for theSPP architecture. The reason for the difference betweenthe two models is the higher degree of isolation betweendifferent parts on the network in the SPP architecture.While all nodes are visible to a client in the P2P case, only

nodes belonging to the same SCC node communicate witheach other in the SPP architecture. Furthermore, SCC nodesnever retrieve content from their own internal clients,making it more difficult for corrupted blocks to spread be-tween ISPs.

More problematic is a coordinated attempt by an at-tacker to distribute modified content. In this scenario, wehave assumed that an attacker has gained control of 1%of the client machines, and uses these to distribute contentthat is not identical to the one distributed by the contentprovider. Each controlled client announces the availabilityof all blocks and all files offered by the server and servethese to requesting clients from the start of the simula-tions. The results are shown in Fig. 8, and apart from theinitial spike, when the corrupted clients make up a signif-icant portion of all active clients, the number of corruptedblock transfers stay at around 7% for the P2P model, givinga high probability of corrupted blocks reaching clients, andmaking this kind of attack very effective. The SCC nodes inthe SPP model again reduces the redistribution of cor-rupted blocks, with around 0.5% of all transmitted blocksbeing corrupted when no message digest verification isused.

A simple application level checksum can protect againstrandom packet errors, but a cryptographic message digestis required to protect against a malicious attacker. Thedrawback is the overhead added by this approach, but it


is difficult to see how this can be avoided, especially in theP2P model; a service provided by a popular content pro-vider would become a tempting target due to the relativeease with which modified content can be distributed to alarge number of users.

5.7. Firewalls

Connectivity in the P2P and SPP models is affected byfirewalls, which the simulations confirm can increase re-source requirements. Fig. 9 shows the effect on the serverload, relative to client/server model, where most of theload is carried by the server. ISP caching eliminatesapproximately 40% of the server load in the CDN model,while the P2P model gives a reduction of almost 80% butis less efficient when there are firewalls. The SPP architec-ture produces the lowest server load (below 10%) and withfirewalls at 40% of the clients, the difference between thetwo SPP lines is so small that the two lines appear to over-lap in the figure. The load distribution values are shown inTable 4, and for the P2P model, firewalls increase the loadon both the server and the ISPs, but the changes are onlyapproximately 5%. For the SPP architecture, the changesare even smaller.

These results are for the baseline configuration, whenboth the P2P model and the SPP architecture model useREVCONN to reduce the impact of firewalls. The REV-CONN command is not initiated by the nodes that re-quests the blocks, but by nodes that wish to receiverequests but are prevented from doing so by firewalls;therefore, in a real environment there is no guaranteethat a connection from an inaccessible node with the re-

Fig. 9. Server load relative to client/server model.

Table 4Firewall influence on resource requirements.

Model Server Ext. Client caches

Int. (%) Ext. (%)

P2P (no firewalls) 25.68 0.05 74.27P2P (40% firewalls) 30.49 0.05 69.46P2P (no REVCONN) 52.88 0.05 47.07SPP (no firewalls) 5.70 0.05 26.18SPP (40% firewalls) 5.76 0.05 21.60SPP (no REVCONN) 5.76 0.05 18.46

sources available will exist when it is required. The base-line configuration with 40% firewalls represents theoptimal case, whereas disabling REVCONN gives the worstcase scenario. The impact of firewalls is greater withoutREVCONN for the P2P model, with the load on the serverincreasing from 30.49% to 52.88%, and the amount of datashared by clients falling by the same amount. For the SPParchitecture, the changes are smaller; the server loadstays the same, while the load on the SCC nodes frominternal clients increases from 28.21% to 31.36%. TheSPP architecture can still benefit from REVCONN, butthe P2P model is much more reliant on an efficient imple-mentation of this technique to achieve a low server load.The efficiency of the P2P model is overall very sensitive tofirewalls. If communication between peers becomesimpossible, the P2P model will degenerate into the cli-ent/server model, while the SPP architecture still hasthe SCC nodes and will degenerate into the CDN model,which has higher scalability.

The danger of ignoring firewalls in the design of a P2Pstreaming system is obvious from the results; the serverload doubles from 25.68% to 52.88% for a system that doesnot include REVCONN, or similar functionality, when 60%of all nodes are able to receive incoming connections.Moreover, even this value might be considered high; asfew as 29.6% of the hosts observed by Xie et al. [47] duringa live streaming event on the Internet were able to receiveincoming connections. Firewalls can have a significant im-pact on scalability, but despite this it is a problem which isoften ignored in research on P2P streaming.

5.8. Incentives and client sharing

The capacity of clients determine how much they areable to contribute, but there is no guarantee that all usersof a P2P system will contribute even if they are able to doso. Introducing incentives is one way to address this, andthe simulation results show that there are clear benefitsfrom doing this.

Table 5 shows the resource requirements when clientscontribute as much as they receive, and when they contrib-ute twice as much as they receive. In these simulations,only clients that quit as a result of user interactivity leaveimmediately, all other clients stay connected until theyhave fulfilled the contribution requirements.

For the P2P model, increased contributions result in asignificantly lower server load, down from 30.49% withthe baseline configuration, to 5.52%, almost the same asthe server load when the SPP architecture is used. The

ISP caches ISP traffic

Int. (%) Ext. (%) In (%) Out (%)

– – 46.48 34.29– – 46.52 32.09– – 46.79 21.7926.34 41.73 8.26 41.7328.21 44.38 8.26 44.3831.36 44.38 8.26 44.38

Fig. 10. Server upload rate, CDF distribution.

Fig. 11. ISP upload rate, CDF distribution.

Fig. 12. ISP download rate, CDF distribution.

Table 5Client contribution influence on resource requirements.

Model Server Ext. Client caches ISP caches ISP traffic

Int. (%) Ext. (%) Int. (%) Ext. (%) In (%) Out (%)

P2P (baseline) 30.49 0.05 69.46 – – 46.52 32.09P2P (up = down) 5.52 0.05 94.43 – – 47.09 18.01P2P (up = 2 � down) 5.43 0.05 94.52 – – 47.07 18.43SPP (baseline) 5.76 0.05 21.60 28.21 44.38 8.26 44.38SPP (up = down) 5.12 0.05 72.50 7.08 15.25 8.26 15.25SPP (up = 2 � down) 5.05 0.05 75.42 5.08 14.40 8.26 14.40


main disadvantage with the P2P model in this configura-tion is the ISP load: the incoming traffic load is 47.09%,compared with only 8.26% for the SPP architecture. Reduc-ing this value for the P2P model will require efficient local-ity aware peer selection in the clients.

Significant benefits are also seen for the SPP architec-ture. Client contributions are increased from 21.60% to72.50%, resulting in a lower load on the ISP caches; theinternal requests are reduced from 28.21% to 7.08%, andthe external requests fall from 44.38% to 15.25%. Comparedto the P2P model, both the incoming and outgoing ISP traf-fic values are lower with the SPP architecture. For incom-ing traffic, the reduction is quite significant: from 47.09%to 8.26%. Doubling the contributions of clients results inonly minor changes for either model.

With the SPP architecture, there are only minor benefitsfor the content provider from adding incentives. However,if ISPs choose to block external requests, the costs for con-tent providers will be significantly increased. The result is asituation where content providers have an interest inkeeping ISPs sufficiently happy by using incentive mecha-nisms to reduce the ISP traffic load.

5.9. Caching

One of the more important observations it is possible tomake from these simulation results is the effect cachingand node clustering can have on the resource requirementsof P2P based VoD systems. The combination of parameterswe have examined above which is most feasible, is for REV-CONN to be used and for users to contribute as much asthey receive, and we have analyzed the results from thisscenario in some greater detail.

Fig. 10 shows the cumulative distribution function(CDF) for the server load, the distribution of server uploadrates for the P2P and SPP models. The P2P model has aslightly higher number of high transfer rates, but the dif-ference in server load is not large. The difference is some-what larger for outgoing data from ISPs, shown in Fig. 11,but again, it is not here the most significant differencecan be found. Rather, the reduction in incoming trafficcaused by node clustering and content caching is whatgives the largest difference between the two models, asseen in Fig. 12. The distribution of incoming traffic forthe SPP and CDN models are almost identical and overlapin the figure. Similarly, there is little difference betweenthe P2P and client/server models. The benefits for the ISPsfrom doing data caching and node clustering are quitesignificant.


Efficiency for a real-world system will depend on fac-tors such as content popularity, file and cache size, requestdistribution, and the number of servers, but despite thesecaveats, the results indicate that designers of P2P basedstreaming systems would do well to consider the cache-ability of the data traffic.

6. PlanetLab experiments

We have examined the scalability and resource require-ment of the SPP architecture using simulations, but to ver-ify that the system can satisfy the real-time requirements ofvideo streaming when it is deployed on the Internet, wehave performed several experiments using PlanetLabnodes.

The use of PlanetLab is not without caveats and lim-itations, such as the hardware and network capacity ofPlanetLab machines not being representative of typicalend-user machines [40], and PlanetLab being a resourcethat is shared with other researchers, potentially result-ing in interference from other experiments and limita-tions in the amount of resources that can be used.Despite this, with PlanetLab it is possible to examinethe behavior of a system when nodes are widely distrib-uted geographically and have huge variations in RTT andtransfer rates.

A critical issue for a VoD streaming system is whether itcan deliver a video stream to users at sufficient speed forerror free playback, without excessive startup delays, andthese two factors represent the playback quality experi-enced by users. The purpose of these experiments is toexamine these factors, and to demonstrate the behaviorof the SPP architecture.

6.1. SPP Prototype

The SPP prototype is written in C, with LHC, server, andmost of the SCC functionality having been implemented.Both SPP and HTTP can be used by media players to requestfiles from the LHC, allowing applications that supportstreaming over HTTP to be used without modification. Asimilar approach is used by Piotrowski et al. [35] to sup-port HTTP clients. The SPP block size is 512 KB. The SCCuses a FIFO based disk cache replacement algorithm toemulate the behavior used by the nodes in the simulations.The prototype supports the use of OpenSSL3 for message di-gest calculation and certificate verification. The sendfile()system call is used on Linux machines to reduce the over-head of serving data.

6.2. Experiment overview

Two sets of experiments were performed: one to exam-ine playback quality, and one to examine SCC overhead. Allexperiments were performed with a single non-PlanetLabmachine running as a server and multiple PlanetLab nodesused as clients. For the playback quality experiments starttimes were randomized, but within a couple of minutes of

3 http://www.openssl.org.

each other for all clients. A media player4 requested a med-ia file from the LHC and decoded the received media data,allowing playback quality to be evaluated. The media playerdid not display the video but decoded it at the correct play-back speed when sufficient data was available. A wrapperaround the media player simulated user interaction withthe same interactivity model as was used in the simulations(see Section 5.4). The video requested by the clients was theMPEG1 encoded movie ‘‘The Corpse Vanishes”,5 which runsfor 63 min and 9 s and has a file size of 256 MB, for an aver-age required reception rate of 0.54 Mbit/s. In total, 24PlanetLab nodes were used as clients together with oneadditional machine located behind a firewall on the sameLAN as the server. A 26th client, running on a PlanetLab ma-chine that experienced consistent DNS problems, onlystarted once during the experiments. The server for thesePlanetLab experiments was a 3 GHz Pentium 4 desktop PCconnected to the network of the University of Oslo, and run-ning OpenBSD. Of the selected PlanetLab nodes, 21 were lo-cated in Europe, one in Taiwan, and two at the same locationin Japan. This gives a scenario where most of the clients arelocated relatively close to each other in Europe, and wheresome clients are located far away. To address the variabilityof network traffic and the load on the PlanetLab nodes weused an approach similar to the one used by Chu et al. [6].Every 12 h, an experiment was started and kept runningfor 55 min. The first experiment was started at around mid-night, Wednesday, September 6, 2006, and the last experi-ment was started on Tuesday, September 12, 2006, for atotal of 12 trials.

The SCC node experiment data was obtained in a singleexperiment started on August 14, 2008, having a similarcomposition of 27 PlanetLab machines used as SCC nodes.Rather than executing the experiment multiple times, thisexperiment used a workload with a significantly largernumber of requests, based on the workload used in thesimulations. The server used in these experiments was a64 bit Linux server with two dual core 1800 MHzprocessors.

6.3. Playback quality

For users, the performance of the system is representedby the startup delay and the state of the media playercache. If the rate of the data stream received by the mediaplayer is lower than the movie playback rate, the cache willempty and errors are likely to occur during playback.Fig. 13 shows the frequency of these cache zero events atthe clients. Overall, 75.1% of the clients experience no play-back problems. Fig. 14 shows the distribution of the stateof the media player cache; how much of the cache has beenfilled. The media player application used on the clientnodes will usually maintain the cache at around 40% full,and the figure shows that most of the values are between40% and 50%, indicating that the media player at the clientsis rarely starved of data.

4 We used MPlayer version 1.0pre7, available from http://www.mplayerhq.hu.

5 Obtained from The Internet Archive, located at http://www.archive.org.

http://www.openssl.org

http://www.mplayerhq.hu

http://www.mplayerhq.hu

http://www.archive.org

Fig. 14. Media player cache fill rate distribution.

Fig. 13. Client playback errors.


A combined overview of both startup delay and cachezero occurrences is given in Fig. 15, with the Z-axis show-ing the frequency of the different values. In total, 71.4% ofthe clients have startup delays below 10 s and no playbackerrors. Playback problems are more frequent than longstartup delays and there appears to be no correlation be-tween the two. Only one client experienced both longstartup delay and a high number of playback errors. Thisnode is examined further in Section 6.9. The clients withmost playback problems are, generally, the same in eachexperiment; some of the nodes located in Asia often expe-rienced initial playback problems. For the nodes located inEurope, only a small number of playback problems oc-

Fig. 15. Video playback quality.

curred, and in some cases the playback problems werelikely caused by load on the PlanetLab machines becausethey occurred despite the LHC having sufficient data forplayback available. Fig. 14 also shows that the cache israrely empty.

Overall, these results indicate that a media player cachesize of 512 KB is too small. The startup delay is low formost clients, and doubling the cache size is likely to reduceplayback errors while still retaining acceptable startuplatencies. A cache size of 1024 KB would only result inthe number of clients with startup delays of 10 s or lessfalling from 98.9% to 90.0%.

The ability of the LHC to obtain sufficient data for themedia player is related to the rate at which data is ob-tained. Fig. 16 shows the distribution of block transferrates, the average playback rate of the video, and the ratelimits used by the clients. When the LHC requests a blockfrom another node, it will abort the transfer if it does notfinish within a time limit based on an estimate of thetransfer rate of the node. The rate limit is the averagetransfer rate required to transmit the data within the timelimit, and as the figure shows, the distribution of rate limitsand actually measured download rates follow each otherclosely, indicating that the LHC is able to make a good esti-mate of peer transfer rates. The gap between the two linesis higher for speeds above 10 Mbit/s. For these speeds thepeer will likely be on the same LAN as the client and theLHC uses a rate limit that is less strict in these cases.

The transfer rates are above the playback rate for 99.2%of the block transfers, indicating that the LHCs nodes arequite efficient at choosing fast nodes. While most of thenodes are well connected, there are still some nodes thatwould give low transfer rates.

The rate limit is below the playback rate for the moviein some cases. This is due to the best effort nature of theblock downloading mechanism. In a given streaming ses-sion there might not be any nodes that are able to sendthe media data fast enough for playback. The peer selectionmechanism chooses nodes based on estimated transferrates and a block transfer will not be aborted unless thetransfer rate is slower than the rate limit, which is basedon previous transfer rates and not the playback rate.

Fig. 16. Block transfer rate distribution.

Fig. 17. CPU load distribution. Fig. 18. Contribution ratio distribution.

6 Only one client was behind a firewall.



The SHA-1 message digest algorithm is used to ensurecorrectness of transmitted content. The cost of doing thisis the CPU load from message digest verification for all datareceived by clients and the additional protocol overheadneeded to transmit the message digests. For the serverthere is no CPU overhead because the message digestscan be generated before files are published on the server,but clients still need to calculate digests. If the message di-gest algorithm is too CPU intensive it can interfere withmedia decoding and playback.

Fig. 17 shows the percent of total CPU time used by theLHC on both clients and the server, as reported by getru-sage(). The server line is somewhat ragged due to the lowernumber of data values compared to the larger set of clientdata, but for both the server and clients, the CPU load is be-low 1% for as much as 99% of the time, despite the serverbeing a normal desktop PC. Furthermore, a SPP client run-ning on a normal set-top box has been demonstrated bySrebrny et al. [41], using MD5 as the message digest algo-rithm. SSL was not used on the set-top box, but we donot believe this would have added significantly to theCPU load because most of the overhead from SSL comesfrom the initial handshake operation [37]. No corruptedblocks were received by the clients during theexperiments.

The total protocol overhead is modest; in one experi-ment it was on average 484 bytes per completed block of512 KB, not including SSL data overhead.

6.5. Firewalls and incentives

The simulation results show that contributions from cli-ents are an important factor in achieving efficiency, but asdiscussed in Section 2, there are practical limitationscaused by the real-time requirements and linearity ofstreaming. The practical consequences of these limitationscan be seen in Fig. 18, which shows the distribution of con-tributions made by clients, relative to how much data theyreceived. A client that contributes nothing has a value of�1, a client that contributes the same amount of data asit receives has a value of 0, and a client that contributestwice as much as it receives has a value of 1. Nothing is

contributed by 7.5% of the clients, as many as 66.1% con-tributed less than they received, and only 9.3% contributedmore than twice as much as they received.

These modest contributions occur despite each clientactively trying to share resources with other clients. Withfreeloaders and a higher number of nodes made inaccessi-ble by firewalls, these numbers would be even lower. Thelow contributions are caused by network awareness inthe clients, the linear nature of streaming, which leavethe last nodes to arrive with no other clients to send datato, and firewalls, to a lesser extent.6 The consequence isthat freeloaders are difficult to identify during short timespans, such as the streaming of a single movie. Rather thanpunishing freeloaders, the P2P VoD streaming applicationscenario is more suited to rewarding users that contributeresources over time, such as by offering content at reducedcost. The accounting mechanism in the SPP architecturecan be used for this, but it will be up to content providersto offer incentives that are effective in increasing user con-tributions and that, consequently, reduce the server load.

Ensuring connectivity across firewalls is another factorin keeping the system efficient. With only a single node be-hind a firewall in each of these experiments, the purposehas been mainly to show that it is possible for this nodeto contribute data, which it succeeds at, contributing al-most 50% more data than it receives.

6.6. Caching

Properly testing the SCC nodes requires a higher loadthan the one used to evaluate playback quality, but thisis difficult due to the load it would place on the networkand the need to have access to a large number of clients.To avoid these problems we can reduce the file and blocksize of the media data and emulate the behavior of SCC cli-ents. This makes it possible to measure the response timeof SCC nodes to a set of client requests based on the work-load used in the simulations. We used the load generatedfor the simulator when clients contribute as much as theyreceive, examined in Section 5.9, and extracted the re-quests issued by the clients to the SCC nodes. The networkused in the simulations has 27 SCC nodes located at ISPs,

Fig. 20. External block request distance distribution.


and we ran these SCC nodes on 27 PlanetLab nodes. At eachnode, a client request emulator sends the block requeststaken from the simulation workload to the SCC node,which requests these blocks from the server or other SCCnodes. The block size used in the simulations is 512 KB,which in these experiments was reduced to 512 bytes.The file size was reduced correspondingly from 700 MBto 700 KB. This eliminates most of the load from datastreaming, but preserves the protocol interaction andkeeps the number of blocks the same, allowing the latencyintroduced by the caches to be examined with a more real-istic workload. The time frame of the simulations is sevendays, but because the block transfer time is significantlyreduced, it is possible to run the emulated clients at a high-er speed, and we reduced the delay between block requeststo one tenth.

The only part of the SCC operation which has not beenimplemented is the propagation of data on client sharingup to the server, but this operation does not have any influ-ence on the command or data delay introduced by the SCCnodes, which we examine here.

In total, the data set contains the response time of357,859 client block requests, corresponding to roughlyhalf of the requests made during the first day of the simu-lation workload. The distribution of these values is shownin Fig. 19. The command latency is low for most requests,with 99.1% being below 0.1 s. The SCC replies to commandrequests immediately upon having read the entire requestdata; variations in the command response time indicateeither that the SCC is highly loaded, or that the machineit is running on is highly loaded. The second case is notuncommon on PlanetLab and in some cases we observeddelays likely caused by process scheduling. The data la-tency is larger and represents three cases: the requesteddata is already cached in memory, the data is cached ondisk, and the data needs to be retrieved over the network.In the last case, the measured delay might include the timerequired to obtain a list of peers before the data can be re-quested. The result is that the data response time is higherthan the command response time. Only 27.4% of the re-quests have a response time below 0.1 s, but as many as93.5% have a response time below one second. There arehowever some cases where the response time is quitehigh; 2.3% of the requests have a response time above 10 s.

Fig. 19. SCC command and data response time.

Another issue related to the latency of the caches is theusability of techniques such as interval caching [8], thataim to reduce latency by exploiting the sequential accesspatterns of streamed video to buffer data in memory. In aP2P system, blocks are still requested sequentially by cli-ents, but not necessarily always from the same peers. Theresult is that the request patterns seen by each node areless predictable. Fig. 20 shows part of the distribution ofthe request distance for the LHC and server nodes in theplayback quality experiments, where the same file was re-quested by all clients, and the SCC nodes and the server inthe SCC experiment, where multiple files where requestedby the clients. The distance value is calculated based on thedifference between subsequent block requests from eachclient; sequential request have a distance of one (e.g., block1, block 2), while skipping a block gives a distance of two(e.g., block 2, block 4), etc. The point in the graph wherethe request distance is one is marked with a vertical line.

In both of the data sets, the server receives a large num-ber of sequential requests, mainly because most of the cli-ent requests are mostly sequential and the server is usedwhen no copies of a file exist at any of the other clients.In the multiple files data set, 87.2% of the server requestsare sequential, and 50.7% of the requests are sequentialin the single file data set. For the LHC and SCC nodes, themajority of requests are not sequential.7 For the LHC andSCC nodes, the values are as low as 26.3% and 26.6%,respectively. The consequence is that it is more difficultto predict if a client that requests a block will request addi-tional blocks, and if so, which blocks. We have currentlynot implemented an algorithm that attempts to addressthis, but while the request patterns in a P2P system makesusing techniques like interval caching more difficult, theyalso makes them less necessary, due to the load sharingin the system which gives a lower load on each node.

6.7. Load distribution

The efficiency with which P2P networks can reduce theserver load is one of the primary reasons for research on

7 The SCC nodes in addition have a load from the internal nodes, notshown here, similar to the one seen for the server, because all requests forfiles that are not available at any other clients pass through the SCC nodes.

Fig. 21. Data serving load distribution.


P2P based VoD systems. This load reduction comes fromdata being served by client nodes instead of only fromthe server. Fig. 21 shows the distribution of non-zerotransfer rates at the server and the clients, in addition tothe rate at which data is sent to the media player, for theplayback quality PlanetLab experiments. Most of the loadis carried by the clients, however a lower server load mightbe expected, considering that the server load in the base-line simulation configuration in Section 5.5 is 5.76%. How-ever, these values cannot be compared directly. Thesimulations include the effects of SCC caching, and showthe minimum server resources necessary. With no SCCnodes, a higher server load is to be expected, but the sys-tem still moves the majority of the load to the clients. Thiscan be seen more clearly in Fig. 22, which shows the rela-tionship between how many peers with a given block areknown to a client, and how often a client successfully re-trieves a block from the server. For a random peer selectionalgorithm, the frequency with which the server would bechosen is given by the 1/peer count line. The figure showsthat the load on the server is in proportion to the numberof available peers; the higher the number of peers avail-able, the less likely that the server will be used. For the casewhen only a single peer is known, the server is actuallyonly used in 82.6% of the cases. The reason for this is thatthe server under-reports the number of blocks it has storedin the reply to peers requests. The server has all blocks in afile, but will never return information which indicates that

Fig. 22. Server usage frequency.

it has more blocks than any of the other peers in a givenPEERS reply. When a client requests a block from a peer,it will receive updated information about any subsequentblocks the peer has available, which ensures that clientsdo not need to issue additional PEERS requests if they man-age to locate a good set of peers to retrieve blocks from.

6.8. Fault tolerance

There is inherently a certain level of fault tolerance inthe system, because having multiple peers adds redun-dancy, but the server can become a single point of failure.The server is needed when a child node first requests a file,but it is possible for existing clients to operate indepen-dently of the server for shorter time intervals. A longerdown time will cause the content to gradually becomeunavailable, but this is a desirable characteristic of the sys-tem. The SPP architecture is not meant to be a file sharingnetwork like eDonkey, and files should become unavailableif they are deleted from a server or if the server is takendown.

To demonstrate what happens when the server fails, weperformed an experiment on a LAN, using one server andthree clients. Fig. 23 shows the block request patterns forthis experiment. Client A arrives first and requests allblocks from the server. Client B can use the server andClient A, whereas Client C can use both the two other clientsand the server. The server process is terminated after fourminutes using kill -9 to simulate a power failure, or similarproblem that prevents the server from terminating prop-erly. Client A becomes unable to retrieve additional blocksbecause only the server has the blocks it needs. Client Bcan still obtain blocks from Client A and keeps running untilit has received all these blocks. Client C is able to keep run-ning even longer by using blocks located at the two otherpeers. After approximately two minutes, the server is re-started, and the streaming session eventually continuesfor all three clients. The third client is almost unaffectedby the server failure because it can use blocks located atthe other peers.

6.9. Remaining issues

The overview of playback quality in Fig. 15 has one out-lying value with both high startup delay and a high num-

Fig. 23. Client fault tolerance.

Fig. 24. Prefetched data at planetlab1.iiitb.ac.in. Fig. 26. LHC parallel transfer limit at planetlab1.iiitb.ac.in.


ber of playback error events. The client responsible for thisvalue is planetlab1.iiitb.ac.in. Consistent DNS problems onthis machine resulted in the client starting only once.These problems were also responsible for the long startupdelay; if the time from all the failed DNS resolution at-tempts are subtracted, the startup delay is only 13 seconds,which includes 5.6 s of DNS lookup time. However, theplayback errors have another cause.

The prototype attempts to automatically determinehow many parallel transfers are required, by graduallyincreasing the upper limit on simultaneous transfers. Thisis a simple approach that will eventually work for all cli-ents, but it can result in initial playback problems for someclients, which is what occurred on this machine.

Fig. 24 shows the state of the LHC buffer containing datafor the media player. When the media player has con-sumed all the data received by the LHC, there is no pre-fetched data in the LHC. This occurs for this client duringthe first five minutes. Sufficient data to start playback issent to the media player 71 seconds after the request is re-ceived, but it is not until after 266 s that the LHC has moredata available than is consumed by the media player. Themedia player cache fill rate is shown in Fig. 25.

The reason for the playback problems can be seen inFig. 26, which shows the upper limit on parallel transfersused by the LHC. This number is gradually increased as aresult of the high consumption rate of the media player rel-ative to the rate of data received from other peers. By using

Fig. 25. Media player cache fill rate at planetlab1.iiitb.ac.in.

multiple parallel transfers, the LHC is eventually able to re-ceive data fast enough to supply the media player at anadequate rate. The approach used in the implementationworks in most cases, but can clearly be improved.

7. Proposed solutions

Section 2 identifies trust, caching, firewalls, and incen-tives as problem areas for P2P based VoD streaming, prob-lems that are often ignored in P2P streaming research. Thissection discusses how the SPP architecture addresses theseproblems.


The pollution problems seen in P2P file sharing net-works are made easy by the search mechanism used to lo-cate files. The name of a file gives no guarantee as to whatthe content actually is if malicious users can trivially dis-tribute files with misleading file names or metadata. TheSPP architecture avoids these problems by having a tightconnection between the content publisher and the content.For commercial distribution this involves integration withthe WWW, where it is simple to identify the origin of a fileby looking at the host part of the uniform resource identi-fier (URI) [1]. The same kind of identifiers are used by theSPP architecture to identify the content.

A cryptographic message digest is used to detect modi-fication and corruption of media data. The need for this isexamined in Section 5.6, and the simulation results showthat corrupted blocks can easily spread in a P2P networkthat does not use an application level message digest.The SPP architecture calculates message digests for eachblock, and to effectively modify the content, a malicioususer must either break the message digest algorithm orcompromise the security of the server; attacking an SCCnode will only have an impact on the child nodes of thatSCC node. The results from the PlanetLab experiments de-scribed in Section 6.4 show that the overhead from mes-sage digests do not prohibit real-time streaming.

7.2. Caching

One of the main ideas in the SPP architecture is to usecaches when they are available, and the benefits of using


caching can be clearly seen in the simulation results in Sec-tion 5. For content providers, caching results in reducedserver load, reduced network traffic, and consequently, re-duced costs for offering content. Caches installed by ISPs orcompanies wishing to reduce SPP P2P traffic lead to re-duced costs for content providers, without content provid-ers having to pay for an expensive CDN infrastructure. Thehierarchical structure created by the caches gives the sys-tem increased scalability and reduces intra-ISP traffic byclustering together clients at the same ISP. However, asthe SCC nodes, which perform the caching, are integral tothe scalability and efficiency of the system, it is naturalto ask whether they actually would be deployed by ISPs.

One of the primary benefits from P2P based streaming isthe reduction in cost for the content provider. If the playbackquality of a system is sufficiently good, it will be possible forcontent providers that cannot afford a large infrastructure tocreate scalable VoD services. If these content providers man-age to attract a large number of users, the result will be a sig-nificant amount of traffic. Especially ISPs are affected by thetraffic patterns of P2P applications, as confirmed by the sim-ulations results. ISPs can at this point choose to install ca-ches to reduce the traffic, but this is not the only responsepossible; it would be possible to block the P2P traffic en-tirely or use traffic shaping to reduce transfer rates. The de-gree to which ISPs will be able to do this will in part dependon the outcome of the currently ongoing net neutrality de-bate, but even if ISPs were free to do this, it would be difficultin the case of a legal service. The SPP architecture is designedto be used for commercial VoD streaming, and customersthat pay for access to a video service are unlikely to choosean ISP that make the service unusable. Organizations withlarge networks have a similar choice; if video streaminggenerates a lot of traffic, it is possible to either block it, or in-stall a cache in the same way that many organizations haveused web caches to reduce HTTP traffic.

The SCC nodes increase latency for clients that requestfiles that are not cached, but for subsequent requests forthe same content by other clients, the latency will be smal-ler because the files will then be cached close to the clients.

7.3. Firewalls

Due to the integration of the proxy-like SCC node, theSPP architecture does not need to rely on firewall traversaltechniques to reduce firewall impact. The SCC is the onlynode that needs to be accessible in the SPP architecture.Allowing incoming connections to a single service main-tained by the network administrator is more secure thanallowing connections to all internal user machines. Legally,there should be no issues because the server based SPParchitecture is not suited for illegal content distributionas the content provider is always easily identifiable. Beingplaced on the border between the internal network and theInternet in order to cache content, the SCC will workequally well at sites that use NAT.

Clients at sites that do not have an SCC node can still re-ceive requests from peers that are not behind a firewallwith the REVCONN command. Two peers that are behinddifferent blocking firewalls still cannot communicate, butthe simulation results in Section 5.7 show that pure P2P

networks with only end-user nodes are more adversely af-fected by this than the SPP architecture.

7.4. Incentives

P2P based VoD streaming systems face the challenge ofresource contributions by users being important for perfor-mance, as seen in the simulation results, but adding incen-tives for sharing is much more difficult than for, e.g.,BitTorrent, as illustrated in Sections 2.4 and 6.5. However,the simulation results in Section 5.8 show that user contri-butions are important to reduce the load on ISPs. The solu-tion for incentive creation in the SPP architecture is anaccounting mechanism that keeps track of user contribu-tions over time, and which can be used by content provid-ers to provide external incentives such as access toadditional content for free.

8. Conclusions

P2P based VoD streaming has recently become a popu-lar research topic, but much of this research is based onlyon simulation models that ignore important problems,such as the traffic load imposed on ISPs. Firewalls, free-loaders, and content corruption are other problems com-mon to P2P networks, but the linearity of streamed videomake them difficult to solve. The contributions of this arti-cle are the identification of these issues for P2P based VoDstreaming, and the evaluation of the SPP architecture,which is our proposed solution.

We have taken a comprehensive approach rather thanfocusing on only a single part of the puzzle, and therebyidentified a simple solution that goes a far way towardsaddressing all of the identified problems. Recognizing theneed for integration of caching in order to achieve scalabil-ity, we have made full use of the cache element in the de-sign of the SPP architecture. The SCC proxy/cache nodescan be added to the system incrementally, and increasescalability by forming a hierarchical structure and cluster-ing nodes. The proxy functionality of the SCC nodes reducethe impact of firewalls, and the REVCONN command fur-ther addresses this issue by allowing peers that are behinda firewall to receive requests from other peers. Crypto-graphic message digests are used to detect modificationof content, and an accounting mechanism provides the ba-sis for incentive creation.

The simulation results show the importance of consid-ering firewalls, incentives, and ISP traffic. Firewalls andISPs can in the worst case make P2P networking impossi-ble; firewalls by preventing connectivity between clients,and ISPs by using bandwidth shaping or interrupting P2Ptraffic. With sufficient user contributions, the load on ISPscan be significantly reduced, while ignoring these issuesmakes CDNs look attractive, especially for ISPs, which haveno outgoing traffic when they are used.

During testing of our prototype on PlanetLab, we ob-served a startup delay of below 10 s and no playback prob-lems at most clients. The prototype is able to avoid slowpeers and can contribute resources even if used on a clientthat is located behind a firewall. The PlanetLab experi-ments illustrate the importance of implementing an effi-


cient P2P based video streaming system and testing it on areal network. Simulations alone will in most cases not besufficient to convincingly demonstrate the feasibility of asystem.

As for future work, the implementation can still beimproved with regards to request scheduling and peerselection. Section 6.9 identifies problems with parallelrequests that we hope to solve in the future. Additionalsimulations can also be performed to examine the effectsof doing only transparent caching at ISPs, or only doingclustering of P2P clients at the same ISP, but not doingany caching.

Acknowledgments

This work has been supported by the Network of Excel-lence CONTENT.

References

[1] Tim Berners-Lee, Roy T. Fielding, Larry Masinter. Uniform resourceidentifier (uri): Generic syntax. RFC 3986 (STD: 66), January 2005.

[2] Robert L. Carter, Mark E. Crovella, Server selection using dynamicpath characterization in wide-area networks, in: INFOCOM’97:Proceedings of the Sixteenth Annual Joint Conference of the IEEEComputer and Communications Societies. Driving the InformationRevolution, 1997, p. 1014.

[3] Anawat Chankhunthod, Peter Bernard Danzig, Chuck Neerdaels,Michael Franklin Schwartz, Kurt J. Worrell, A hierarchical internetobject cache, in: USENIX Annual Technical Conference, 1996, pp.153–164.

[4] Cheng Bin, Stein Lex, Jin Hai, Zhang Zheng, Towards cinematicinternet video-on-demand, ACM SIGOPS Operating Systems Review42 (4) (2008) 109–122.

[5] Nicolas Christin, Andreas S. Weigend, John Chuang, Contentavailability, pollution and poisoning in file sharing peer-to-peernetworks, in: EC’05: Proceedings of the Sixth ACM Conference onElectronic Commerce, ACM Press, 2005.

[6] Yang Chu, Sanjay Rao, Srinivasan Seshan, Hui Zhang, Enablingconferencing applications on the internet using an overlaymulticast architecture, SIGCOMM Computer CommunicationReview 31 (4) (2001).

[7] Bram Cohen, Incentives build robustness in bittorrent, in Workshopon Economics of Peer-to-Peer Systems, June 2003.

[8] Asit Dan, Daniel M. Dias, Rajat Mukherjee, Dinkar Sitaram RenuTewari. Buffering and caching in large-scale video servers, in:COMPCON’95: Proceedings of the 40th IEEE Computer SocietyInternational Conference, IEEE Computer Society Washington, DC,USA, 1995, pp. 217–224.

[9] Chris Dana, Danjue Li, David Harrison, Chen-Nee Chuah, Bass:Bittorrent assisted streaming system for video-on-demand, in:MMSP’05: Proceedings of the Seventh IEEE Workshop onMultimedia Signal Processing, October 2005, pp. 1–4.

[10] Tai T. Do, Kien A. Hua, Mounir A. Tantaoui, P2vod: Providing faulttolerant video-on-demand streaming in peer-to-peer environment.Technical Report, School of Electrical Engineering and ComputerScience, University of Central Florida, 2003.

[11] Feldman Michal, Chuang John, Overcoming free-riding behavior inpeer-to-peer systems, ACM SIGecom Exchanges 5 (4) (2005).

[12] UPnP Forum. Internet gateway device (igd) standardized devicecontrol protocol v 1.0. <http://www.upnp.org>, November 2001.

[13] Aditya Ganjam, Hui Zhang, Connectivity restrictions in overlaymulticast, in: NOSSDAV’04: Proceedings of the 14th InternationalWorkshop on Network and Operating Systems Support for DigitalAudio and Video, ACM Press, 2004.

[14] Saikat Guha, Paul Francis, Characterization and measurement of TCPtraversal through NATS and firewalls, in: IMC’05: Proceedings of theInternet Measurement Conference, October 2005.

[15] Krishna P. Gummadi, Richard J. Dunn, Stefan Saroiu, Steven D.Gribble, Henry M. Levy, John Zahorjan, Measurement, modeling, andanalysis of a peer-to-peer file-sharing workload, in: SOSP’03:Proceedings of the 19th ACM Symposium on Operating SystemsPrinciples, ACM Press, 2003.

[16] P. Krishna Gummadi, Stefan Saroiu, Steven D. Gribble, Ameasurement study of napster and gnutella as examples of peer-to-peer file sharing systems, Computer Communication Review 32(1) (2002).

[17] Lei Guo, Songqing Chen, Shansi Ren, Xin Chen, Song Jiang, Prop: Ascalable and reliable p2p assisted proxy streaming system, in:ICDCS’04: Proceedings of the 24th International Conference onDistributed Computing Systems, IEEE Computer Society, 2004. pp.778–786.

[18] Lei Guo, Enhua Tan, Songqing Chen, Zhen Xiao, Xiaodong Zhang,Does internet media traffic really follow zipf-like distribution? in:SIGMETRICS’07: Proceedings of the 2007 ACM SIGMETRICSInternational Conference on Measurement and Modeling ofComputer Systems, New York, NY, USA, 2007, ACM.

[19] Yang Guo, Kyoungwon Suh, Jim Kurose, Don Towsley, P2cast: peer-to-peer patching scheme for vod service, in: WWW’03: Proceedingsof the 12th International Conference on World Wide Web, ACMPress, 2003. pp. 301–309.

[20] Yang Guo, Kyoungwon Suh, Jim Kurose, Don Towsley, A peer-to-peeron-demand streaming service and its performance evaluation, in:ICME’03: Proceedings of the 2003 International Conference onMultimedia and Expo, 2003.

[21] Ahsan Habib, John Chuang, Incentive mechanism for peer-to-peermedia streaming, in: IWQOS’04: Proceedings of the 12th IEEEInternational Workshop on Quality of Service, IEEE, 2004.

[22] Hall Yensy James, Piemonte Patrick, Weyant Matt, Joost: Ameasurement study, Carnegie Mellon University, 2007. May.

[23] Anwar Al Hamra. Approaches for Scalable Content Distribution inthe Internet. PhD Thesis, UNIVERSITE DE NICE-SOPHIA ANTIPOLIS –UFR SCIENCES, Ecole Doctorale de Sciences et Technologies del’Information et de la Communication, December 2004.

[24] M. Hefeeda, A. Habib, D. Xu, B. Bhargava, B. Botev, Collectcast: apeer-to-peer service for media streaming, ACM/Springer MultimediaSystems Journal (2003). October.

[25] Mohamed Hefeeda, Ahsan Habib, Boyan Botev, Dongyan Xu, BharatBhargava, Promise: peer-to-peer media streaming using collectcast,in: MULTIMEDIA’03: Proceedings of the 11th ACM InternationalConference on Multimedia, ACM Press, 2003. pp. 45–54.

[26] Kien A. Hua, Ying Cai, Simon Sheu, Patching: a multicast techniquefor true video-on-demand services, in: MULTIMEDIA’98:Proceedings of the Sixth ACM International Conference onMultimedia, ACM Press, 1998. pp. 191–200.

[27] Alan T.S. Ip, Jiangchuan Liu, John C.S. Lui, Copacc: A cooperativeproxy-client caching system for on-demand media streaming, in:Raouf Boutaba, Kevin C. Almeroth, Ramón Puigjaner, Sherman X.Shen, James P. Black (Eds.), NETWORKING, vol. 3462 of Lecture Notesin Computer Science, Springer, 2005, pp. 1383–1387.

[28] Wolfgang John, Sven Tafvelin, Analysis of internet backbone trafficand header anomalies observed, in: IMC’07: Proceedings of theSeventh ACM SIGCOMM Conference on Internet Measurement, ACM,New York, NY, USA, 2007, pp. 111–116.

[29] Thomas Karagiannis, Andre Broido, Nevil Brownlee, Kimberly C.Claffy, Michalis Faloutsos, Is p2p dying or just hiding? in:GLOBECOM’04: Proceedings of the 2004 IEEE GlobalTelecommunications Conference, vol. 3, December 2004.

[30] Nathaniel Leibowitz, Aviv Bergman, Roy Ben-Shaul, Aviv Shavit. Arefile swapping networks cacheable? characterizing p2p traffic, in:WCW’02: Proceedings of the Seventh International Workshop onWeb Content Caching and Distribution, August 2002.

[31] Jin Li, Yi Cui, Bin Chang, Peerstreaming: design and implementationof an on-demand distributed streaming system with digital rightsmanagement capabilities, Multimedia Systems Journal 13 (3) (2007)173–190.

[32] Jian Liang, Rakesh Kumar, Yongjian Xi, Keith W. Ross, Pollution inp2p file sharing systems, in: INFOCOM’05: Proceedings of the 24thAnnual Joint Conference of the IEEE Computer and CommunicationsSocieties, vol. 2, March 2005.

[33] Matthew Mathis, Jeffrey Semke, Jamshid Mahdavi, The macroscopicbehavior of the tcp congestion avoidance algorithm, SIGCOMMComputer Communication Review 27 (3) (1997) 67–82.

[34] Venkata N. Padmanabhan, Kunwadee Sripanidkulchai, The case forcooperative networking, in: IPTPS’01: Revised Papers from the FirstInternational Workshop on Peer-to-Peer Systems, Springer-Verlag,2002. pp. 178–190.

[35] Tadeusz Piotrowski, Suman Banerjee, Sudeept Bhatnagar, SamratGanguly, Rauf Izmailov, Peer-to-peer streaming of stored media: theindirect approach, in: SIGMETRICS’06/Performance’06: Proceedingsof the Joint International Conference on Measurement and Modelingof Computer Systems, ACM Press, 2006.

http://www.upnp.org


[36] Reza Rejaie, Antonio Ortega, Pals: peer-to-peer adaptive layeredstreaming, in: NOSSDAV’03: Proceedings of the 13th InternationalWorkshop on Network and Operating Systems Support for DigitalAudio and Video, ACM Press, 2003. pp. 153–161.

[37] Eric Rescorla, SSL and TLS: Designing and Building Secure Systems,Addison-Wesley Professional, 2000.

[38] Stefan Saroiu, Krishna P. Gummadi, Richard J. Dunn, Steven D.Gribble, Henry M. Levy, An analysis of internet content deliverysystems. ACM SIGOPS Operating Systems Review, 36(SI), 2002.

[39] Karl-André Skevik, Vera Goebel, Thomas Plagemann, Design of ahybrid cdn, in: MIPS’04: Proceedings of the Second InternationalWorkshop on Multimedia Interactive Protocols and Systems,November 2004.

[40] Neil Spring, Larry Peterson, Andy Bavier, Vivek Pai, Using planetlabfor network research: myths, realities, and best practices, ACMSIGOPS Operating Systems Review 40 (1) (2006).

[41] Piotr Srebrny, Karl-André Skevik, Vera Goebel, Thomas Plagemann,Demo spp: A demonstrator for a scalable p2p vod infrastructure, in:MULTIMEDIA’07: Proceedings of the 15th International Conferenceon Multimedia, ACM, New York, NY, USA, 2007, pp. 156–157.

[42] Jonathan Stone, Craig Partridge, When the crc and tcp checksumdisagree, in: SIGCOMM’00: Proceedings of the Conference onApplications, Technologies, Architectures, and Protocols forComputer Communication, ACM, New York, NY, USA, 2000.

[43] Kyoungwon Suh, Christophe Diot, Jim Kurose, Laurent Massoulié,Christoph Neumann, Don Towsley, Matteo Varvello, Push-to-peervideo-on-demand system: design and evaluation, IEEE Journal onSelected Areas in Communications 25 (9) (2007) 1706–1716.

[44] Wenting Tang, Yun Fu, Ludmila Cherkasova, Amin Vahdat, Medisyn:a synthetic streaming media service workload generator, in:NOSSDAV’03: Proceedings of the 13th International Workshop onNetwork and Operating Systems Support for Digital Audio andVideo, ACM Press, 2003.

[45] Manuel Vilas, Xabiel G. Pañeda, Roberto García, David Melendi,Víctor G. García, User behaviour analysis of a video-on-demandservice with a wide variety of subjects and lengths, in: EUROMICRO-SEAA’05: Proceedings of the 31st EUROMICRO Conference onSoftware Engineering and Advanced Applications, IEEE ComputerSociety, 2005.

[46] Jared Winick and Sugih Jamin, Inet-3.0: Internet topology generator.Technical Report CSE-TR-456-02, University of Michigan, 2002.

[47] Susu Xie, Bo Li, Gabriel Y. Keung, Xinyan Zhang, Coolstreaming:Design, theory, and practice, IEEE Transactions on Multimedia 9(2007) 1661–1671. December.

[48] Dongyan Xu, Sunil Suresh Kulkarni, Catherine Rosenberg, Heung-Keung Chai, Analysis of a cdn-p2p hybrid architecture for cost-effective streaming media distribution, Multimedia Systems 11 (4)(2006) 383–399.

[49] Song Ye, Collaboration-aware peer-to-peer media streaming, FilliaMakedon, Collaboration-aware peer-to-peer media streaming, in:

MULTIMEDIA’04: Proceedings of the 12th Annual ACM InternationalConference on Multimedia, ACM Press, 2004.

Karl-André Skevik is currently doing a PostDoc at the University of Oslo. He completedhis Ph.D. at the same institution in April 2007.His research interests include networks andoperating systems, including multimediastreaming.

Vera Goebel is Professor at the Department of
Informatics of the University of Oslo whereshe works in the Distributed MultimediaSystems Group. Her research interests includedatabase systems, operating systems, mid-dleware, QoS, distributed systems, and datastream management systems.
Thomas Plagemann is Professor at the
Department of Informatics of the University ofOslo where he heads the Distributed Multi-media Systems Group. His research interestsinclude multimedia middleware, QoS, oper-ating system support for distributed multi-media systems, and data stream managementsystems for network monitoring.

Documents

Evaluation of a comprehensive P2P video-on-demand streaming system