Optimizing Streaming Server Selection forCDN-delivered Live Streaming

Zhenyun Zhuang∗ and Pan Zhou† ‡

∗ LinkedIn Corp., 2029 stierlin court, Mountain View, CA 94043, USA† panzhou@hust.edu.cn, Department of Electronics and Information Engineering,

Huazhong University of Science and Technology, 1037 Luoyu Road, Wuhan, 430074, P. R. China‡ Correspondence author

Abstract—Content Delivery Networks (CDNs) are increasinglybeing employed to facilitate the delivery of live streaming. DNSis the standard practice for enabling dynamic assignment ofstreaming servers. GeoDNS, which provides DNS resolutionby tracking the geographical locations of end-users and CDNservers, effectively redirects users to nearest CDN edge serversfor traditional web-objects, but the new application type of livestreaming exposes unique characteristics and challenges thatrequire more advanced design of GeoDNS. Unlike traditionalweb-objects fetching, which allows Edge Servers to cache contentsand thus typically only involves edge servers for deliveringcontents, live streaming requires a real-timefull CDN-streamingpath that spans across an Ingest Server, an Origin Server andan Edge Server.

Though the nearest Origin Server can be easily determinedby regular GeoDNS, it may not be the optimal one from theperspectives of both CDN transit cost and the viewer’s experience.In this work, we propose a live-streaming specific GeoDNS designfor selecting optimal Origin Servers to serve Edge Servers,whichcan save transit cost for CDNs and achieve better end-viewer’sexperience. Named Sticky-DNS, the new design primarily aimsat optimizing CDN delivery for less-popular live streams, but canalso adapt to accommodate popular live streams.

Keywords: Live streaming, CDN, Server selection

I. I NTRODUCTION

Today’s web contents are increasingly being delivered byContent Delivery Networks (aka Content Distribution Net-works, or CDNs) [1]–[4]. a typical CDN infrastructure con-sists of two types of servers based on their functionalities:Ingest Servers for accepting customers’ web contents; and (ii)Delivery Servers for serving end web users. Live streamingis becoming increasingly popular, and it is expected to bemore pervasive in the near future. Major CDN providers suchas Akamai [1] and Level3 [2] are supporting delivering livestreaming with their CDNs. When supporting live streaming,CDN providers typically deploy a layered infrastructure whichconsists of three types of streaming servers: (i) Ingest Stream-ing Servers (ISS) which accept customers’ live streams; (ii)Origin Streaming Servers (OSS) which transit live streamsfrom ingest servers to Edge Streaming Servers; and (iii) EdgeStreaming Servers (ESS) that directly serve the end viewers.Depending on the scale of a CDN, the number of OriginStreaming Servers varies from several to hundreds, and EdgeStreaming Servers can be even thousands.

Unlike other traditional web-based applications, live stream-ing is associated with unique characteristics and in turnpresents special challenges to CDN infrastructure.First, livestreaming has to be delivered from the CDN Ingest Serverto Edge Servers in a real-time fashion, which is in sharpcontrast with traditional CDN-based web delivering wheretypically only the last hop (i.e., from CDN Edge Servers toweb users) needs to be optimized. Such uniqueness requiresthe optimization of the entire CDN transit path which consistsof two transit hops of Ingest-Origin and Origin-Edge.Second,live streaming often features high-bandwidth requirement. Atypical live videos can easily consumes up to 3 Mbps perstream, thus there is a higher need for reducing transit coston CDN networks.Third, live streaming typically is time-sensitive, and the CDN infrastructure design needs to considersuch timeliness by choosing the shortest path from the stream-ing source to the end users.

After a live stream is ingested into the CDN infrastructurethrough an Ingest Server, when an end viewer requests to playa live stream, an appropriate Edge Server and an Origin Serverneed to be chosen, so that the player pulls the stream fromthe Edge Server and the Edge Server pulls the stream fromthe Origin Server. DNS is the standard practice for makingthe above server selections. GeoDNS [5], typically used byCDNs, is a specialized DNS service by taking into accountgeographical locations of CDN Origin/Edge servers and endusers. It translates host names into IP address and redirects therequests to optimal Origin/Edge servers. Though many factorssuch as server loads contribute to this selection process, thecritical factor is the proximity (i.e., geographical) information.For instance, in a CDN that deploys Edge Servers on bothcoasts of USA, an end user coming from California mostlikely will be served by an Edge Server located in west coastrather than east coast. Traditional GeoDNS solutions workeffectively by offering the end users lower response time andhigher throughput. Inside CDN infrastructure, GeoDNS is alsoused to determine the optimal Origin Server to serve EdgeServer requests, with the goal of offering lower transit costand response time.

When supporting live streaming, GeoDNS is used for dy-namical assignment of Origin Streaming Servers in responseto Edge Streaming Server requests. In this process, traditionalGeoDNS only considers the properties (e.g. proximity) of

(a) Default Delivery (b) Optimized Delivery

Fig. 1. Live streaming delivered with CDN

Origin Servers with respect to a particular Edge Server, whichmay result in sub-par performance. Consider an exampleshown in Figure 1 where SourceS is ingesting a live streaminto CDN’s Ingest ServerI . Assuming Edge ServerE needsto serve UserU the live stream originating from Ingest ServerI . E has two optional Origin Servers to retrieve the streamfrom: O1 and O2. SinceO1 is closer toE, default GeoDNSwill assign O1 to E’s DNS request, thus resulting in a CDNtransit path ofI −O1−E, as shown in Figure 1(a). However,as shown in Figure 1(b), there exists a better CDN transit pathof I −O2−E, which has smaller CDN transit cost in termsof stream delivery distance. In addition to less transit cost,U ’s viewing experience is also expected to be better, as theentire live stream path ofS− I −O2−E−U is shorter. TheCDN transit path shown in Figure 1(b) can be achieved byan optimized GeoDNS solution which assigns toE the OriginServer ofO2, as opposed toO1.

Though effectively addressing all challenges associated withlive streaming requires advanced design/redesign of manyaspects of CDN infrastructure, in this work, we focus onoptimizing the Origin Streaming Server selection throughGeoDNS, and propose a design calledSticky-DNSfor support-ing live streaming. The key idea of Sticky-DNS is that whenperforming GeoDNS and selecting Origin Servers to serveEdge Servers, the optimal Origin Server is determined withthe consideration of two CDN hops of both Ingest-to-Originand Origin-to-Edge rather than the single hop of Origin-to-Edge as in traditional GeoDNS. Sticky-DNS primarily aimsat optimizing CDN delivery for less-popular live streams1,but can also adapt to accommodate to popular live streams.

In the following paper, we first provide some backgroundinformation and motivate our design of Sticky-DNS in SectionII. We then present the problem definition, detailed design

1In this work, a live stream is considered to be “less-popular” if theconcurrent viewers are less than the number of CDN streamingservers.

and deployment scenario of Sticky-DNS in Section III andSection IV, respectively. We perform prototype-based evalu-ation and show the results in Section VI. We then present aextended version of Sticky-DNS to handle scenarios whereCDN providers use flat infrastructure (as opposed to the strictlayered infrastructure focused in this paper) in Section VII. Wealso present related works in Section VIII. Finally we concludethe work in IX.

II. BACKGROUND AND MOTIVATION

In this section, we first provide some backgrounds aboutCDN, live streaming and GeoDNS. We then present some ini-tial results about the popularity of live streams on Ustream.tv,a major live streaming web site. Finally, we summarize thissection by providing insights into the design of new GeoDNSfor live streaming.

A. Backgrounds

CDN: Content Delivery Networks (CDNs) can be used todeliver both static web contents and live streams. A typicalCDN infrastructure consists of Ingest Servers (for acceptingcustomer contents), Origin Servers (for serving edge servers)and Edge Servers (for serving end users directly), forming atree-like structure. Various servers are organized into POPs(i.e., Point Of Presence).

GeoDNS:CDNs aim to deliver contents directly from EdgeServers that are closer to end users, thus minimizing theadverse impact of dynamic network path and reducing networktransit cost. For this, a specialized DNS service is typicallyused for redirecting requesting end users to appropriate EdgeServers. Static web requests benefit most from the regular DNSservice, since static contents cached on Edge Servers can bereutilized by successive requests.

Live Streaming: Live streaming is a new type of applicationthat is increasingly delivered by CDNs. Featuring differentproperties from static contents, live streaming typicallyhas

0

200

400

600

0 10 20 30 40 50 60

Ustream.TV Live Streams

Fig. 2. Number of viewers for the first 55 streams on Ustream.tv

high bandwidth requirement which exert much pressure to theCDN infrastructure in the form of high transit cost. Unlikestatic web contents, live streams cannot be cached and haveto maintain a full path from the customer source to the endviewers (i.e., end users).

B. Popularity of live streams

On today’s Internet, vast amount of live streams are beingsupported by CDNs. Though the sizes of viewer set varygreatly, many live streams today feature small viewer sets.Specifically, we consider a major live streaming website ofUstream.tv and study the live streams that fall into the categoryof “most-total-views” at a particular time. We count thenumbers of viewers for each of the first 55 streams, andthe results are shown in Figure 2. Note that the figure onlyshows the active live streams, and other streams are off-airatthe particular time. We observe that though the most popularstream has more than 500 viewers, only 6 of them have morethan 100 viewers. The average number of live viewers is onlyabout 53.

C. Observations

As we observed from the illustrate example shown inFigure 12, CDN transit cost (i.e., Ingest-Origin and Origin-Edge paths) can be saved by assigning appropriate OriginServers to requesting Edge Servers with a special GeoDNSfor live streaming applications. Instead of always assigningthe nearest Origin Server to the Edge Server that issues DNSrequest, a specially-designed GeoDNS could determine theassignment based on the entire CDN delivery path. Such aspecial GeoDNS can significantly save CDN transit cost forscenarios where a live stream is not popular as they incurshorter CDN delivery paths.

Though the popularity of streams (i.e., viewer set size)varies greatly, many live streams today feature small viewersets. On the other hand, today’s major CDN providers typicallyhave hundreds or even thousands of Edge/Origin streamingservers for supporting live streaming. The mismatch betweenthe sizes of viewer sets and server sets motivates our designof a specialized DNS for less-popular live streams, with thebenefit of reduced CDN transit cost.

For popular live streams, however, the same approach needsto be adapted to reutilize delivery path at Ingest-Origin level.Specifically, default GeoDNS practically encourages path re-utilization at Ingest-Origin level since each Origin Servercan aggregate the streaming requests from closer-by EdgeServers, thus reducing the CDN transit cost. Allowing eachEdge Server to connect to far-away Origin Servers, on theother hand, discourages path reutilization, thus could result inlarger aggregate delivery distance inside CDN. Therefore,anappropriate design of GeoDNS has to consider the popularityof a live stream, and adapts its behavior correspondingly.

III. D ESIGN OFSTICKY-DNS

In this section, we first formalize the problem we attempt toaddress. We then present the design overview of Sticky-DNS,its software structure and the entire process.

A. Problem Definition

We consider scenarios where a CDN provider has threetypes of streaming servers for supporting live streaming:Ingest Streaming Servers, Origin Streaming Servers, and EdgeStreaming Servers. We useSSI , SSO and SSE to denote thethree server sets respectively. The CDN provider delivers aset of live streams denoted by{Lm}. We further assume eachlive streamLm has a single ingesting source which connectsto an Ingest ServerSSIm.

The problem we are addressing with Sticky-DNS design canbe defined as follows. For a live stream denoted byLm and aviewer Vn, we assume the Edge Server that serves viewerVn

is SSEmn, which is determined by regular GeoDNS. TypicallySSEmn is the closest Edge server to the end viewer. Sticky-DNS needs to decide the optimal Origin ServerSSOmn toserveSSEmn such that the CDN transit cost can be minimizedand the end viewers’ experience is not compromised. We useCIO andCOE to denote the transit costs corresponding to thepaths of Ingest-Origin and Origin-Edge, respectively. Withthese assumptions, the input, output and objective functionof Sticky-DNS are presented in Figure 3.

Input:1 {Lm}: The delivered live stream set of individual streamLm;2 Vm = {Vmn}: The viewer set for live streamLm;3 SSIm: The Ingest Server for live streamLm;4 SSOm: CDN Origin Server set forLm;5 SSEm= {SSEmn}: The CDN Edge Server set forLm; // Note that each Edge Server6 // Emn for each individual viewerVn is determined by regular GeoDNS;7 CIO: Cost arrays of all Ingest-Origin pairs;8 COE: Cost arrays of all Origin-Edge pairs;

Output:1 Optimal Origin Server setSSOm= {SSOmn} for Edge Server setSSEm,

where eachSSOmn is serving the Edge Server ofSSEmn ;

Objective Function:1 Total CDN transit costΣ(CIO +CEO) for all live streams{Lm} is minimized;2 Non-compromised viewer experience;

Fig. 3. Problem Definition

B. Design Overview

Sticky-DNS is a specialized DNS service designed for livestreaming applications. It aims at saving CDN transit cost butnot compromising end user experience.2 Sticky-DNS achievesthe cost savings by assigning proper Origin Servers to serveEdge Servers with shorter CDN delivery paths.

Sticky-DNS’s operations are per-stream based, thus it main-tains appropriate states for all live streams. At a higher level,Sticky-DNS adjusts its behavior as the particular live streamchanges popularity. Specifically, when a live stream is notpopular, Origin Servers are chosen based on the lengths of en-tire Edge-Origin-Ingest paths. For this, Sticky-DNS maintainsthe cost values of every Edge-Origin and Origin-Ingest pairs.Initially, all the costs are assigned according to certain cost-determination methods. Whenever an Origin Server is arealdyserving the live stream, the corresponding Origin-Ingest costis set to zero to reflect the fact that serving an additional EdgeServer with this Origin Server will incur no additional costonthe Origin-Ingest path.

However, as a stream becomes more popular, Sticky-DNSneeds to adapt back to the default behavior determined byregular DNS for reusing Ingest-Origin paths. The reason is thatwhen serving popular streams, since Origin Servers are morelikely serving that live stream, the behavior of Sticky-DNSwillmore likely be that of regular GeoDNS, and the proximity-closest Origin Server is typically chosen for each Edge Serverso that the Ingest-Origin paths are reutilized. To encouragethis, we introduce a concept of “would-be” behavior. When anOrigin Server is determined by regular DNS for Origin requestissued by an Edge Server, the particular Origin is the “would-

2Note that for ease of presentation, we use the distance as theprimary costmetric; but it would be easy to extend the cost metric to include other factorssuch as pricing; more discussions are in Section IV.

be” Origin for the Edge, while the Edge is the ”would-be”Edge for the Origin. Even if a different Origin is determinedby Sticky-DNS, the “would-be” state is recorded. The “would-be” state is used to adjust the corresponding Origin-Ingestcost.Specifically, with more “would-be” Edge Servers retrievingfrom a Origin Server, the corresponding Origin-Ingest costis reduced further. The idea behind this treatment is that theOrigin-Ingest cost is shared by all downstream Edge Servers,thus encouraging the reuse of the particular Origin-Ingestpath.

These cost values will be used to choose optimal OriginServers for incoming Origin Server DNS requests so that thesmallestΣ(CIO +CEO) will result. To achieve this, Sticky-DNS uses the notion of “Additional Cost” when deciding onan Origin Server. Specifically, given an incoming viewerVn

and the corresponding Edge ServerSSEmn, Sticky-DNS alwaysreturn the Origin Server that causes minimal “AdditionalCost”. The additional cost is defined as theextra CDN transitcost that needs to be paid when serving the Edge Serverwith a particular Origin Server. Recall that CDN transit costcontains two parts: the Ingest-Origin cost and Edge-Origincost. Whenever an Origin Server first connects to an IngestServer to retrieve a live stream, CDN pays Ingest-Origin cost,but succeeding retrieval of the same live stream from thesame Ingest Server to the same Origin Server will incur zeroadditional cost.

C. Software Architecture

The software architecture of Sticky-DNS is shown in Figure4. The heart of Sticky-DNS is Sticky-DNS Engine (SDE),which takes the DNS request, interacts with other compo-nents, and then decides the optimal Origin Server. Sticky-DNS assigns Origin Servers based on the cost associatedwith the live streaming delivery path, and the cost is CDN-specific and determined by another component of Cost Metric

Fig. 4. Software Architecture

Determination (CMD) and updated by Server Cost Probing(SCP). Meanwhile, to accommodate network dynamics, acomponent of Edge-Origin Connection Monitoring (ECM)is used to update the cost state. Sticky-DNS maintains theserver streaming state which basically records information ofwhich Edge Server is serving which streams and which OriginServer is retrieving which streams. This state is updated bymonitoring the events of viewers joining and leaving.

The overall process of how Sticky-DNS works is as below.When a new live streaming DNS request comes, the optimalEdge Server will be determined by regular GeoDNS. CDNwill then attempt to determine the optimal Origin Server forthe determined Edge Server. For this, the Edge Server issuesDNS request to DNS service asking for the proper OriginServer for retrieving the live stream from. Sticky-DNS takesthe Origin-Server request and returns the determined OriginServer. Meanwhile, it updates the server streaming states.

IV. STICKY-DNS COMPONENTS

We now describe the detailed component design of Sticky-DNS.

A. Cost Metric Determination (CMD)

In order to perform Sticky-DNS operations, each deliveryhop (i.e., Ingest-Origin and Origin-Edge) on CDN infrastruc-ture has to be associated with quantifiable cost. Note thatthough physical distance is often used as the path cost, costmetrics may also include network delay, price, server load,

1 If (NV[i][ j]≥ 1)2 AD[i][ j] = 0;3 Else4 If ( WB[i][ j] ==0 )5 AD[i][ j] = initial cost;6 Else7 AD[i][ j] = initialcost

WB[M][T]+1)

Fig. 5. Algorithm for updating additional cost

reliability, loss rate, etc. Since different cost metrics oftenbelong to distinct dimensions and they are often prioritized.Though how to prioritize them varies with different CDNproviders and deployment scenarios, in this work we providethe following guidelines: (i) Server Load is prioritized over allother metrics. An Origin server is put into the DNS pool (i.e.,available to be returned by sticky-DNS) only if it can takemore loads (i.e., serving more requests); (ii) Loss rate hastobe under certain threshold. Since most live streaming (e.g.,RTMP, HTTP) use TCP to transmit, large loss rate cannotsustain a reasonable level of performance; (iii) Reliability hasto be above certain threshold. Live streaming requires a real-time transmission path, thus reliability is important during thestream playback; (iv) Pricing is prioritized over network delayand physical distance. Though pricing is often correlated withthe latter two metrics, it is prioritized over them if they arenot identical; (v) Network delay and physical distance. Theimpact of these two metrics are buffering time and playbackdelay. In reality, they can be used as the substitute for pricingmetric when the latter is not available or hard to define.

Thus, with the above guidelines in mind, we proposeto enforce the cost determination by keeping track of thefollowing state information.

• An array of server load for the Origin Servers, denotedby SL[M], whereM is the number of Origin Servers.SL[i]represents the current load of Origin ServerSSOi;

• A 2-D array of loss rate for each Origin-Edge path,denoted byLR[M][N], whereM is the number of OriginServers andN is the number of Edge Servers.LR[i][ j]represents the loss rate of the path between Origin ServerSSOi and Edge ServerSSE j; Another array of loss rate foreach Ingest-Origin pair, denoted byLRI[M], assuming asingle Ingest Server;

• A 2-D array of reliability for each Origin-Edge path,denoted byRE[M][N], whereM is the number of OriginServers andN is the number of Edge Servers.RE[i][ j]represents the reliability of the path between OriginServer SSOi and Edge ServerSSE j; Another array ofreliability for each Ingest-Origin pair, denoted byREI[M];

• Pricing is represented byPR[M][N] for each pair of

1 Initialize all cost states;2 For an incoming new viewerVn requestingLm, determine the Edge ServerSSEmn;3 If SSEmn is already servingLm;4 Return; //No Origin Server request since the Edge Server isservingLm now;5 Else6 Determine the candidate Origin Server set based on load, loss rate, etc.;7 For every Origin ServerSSOi, if (SL[i]≤ α andLRj ,k) ≤ β andREj ,k ≥ γ, where8 α, β,γ are the respective threshold values),SSOi is a candidate Origin Server;9 Determine the optimal Origin ServerSSOq that has the smallest aggregate10 cost ofCIO andCOE for SSEmn;11 WB[q][m]++;12 NV[q][m]++;13 UpdateAD[q][m] according to Figure 5;14 If Edge ServerSSEi is disconnecting fromSSO j for live streamLm:15 WB[ j][i]−−;16 NV[ j][m]−−;17 If (NV[ j][m]≥=1)18 AD[ j][m] = Initialcost

(NV[ j ][m]+1) ;19 Else20 AD[ j][m] = {initialcost}.

Fig. 6. Core Algorithm

Origin-Edge; Another array ofPRI[M] for each pair ofIngest-Origin;

• Similarly, network delay and physical distance are rep-resented byND[M][N] and PD[M][N], respectively. Twoother arrays ofNDI[M] andPDI[M] are also used.

B. Server Cost Probing (SCP)

SCP updates the various cost state by proactively or reac-tively monitoring/probing the network. Specifically,

• Origin Server Load(SL[M]). The load can be obtainedby installing services which monitor CPU/Memory uti-lization;

• Loss rate(LR[M][N] andLRI[M][N]). Loss rate monitor-ing between servers can be obtained by observing existingconnections’ behaviors (e.g., TCP loss recovery behav-iors) or proactively sending test packets. When a networkpath (e.g., Ingest-Origin-Edge) consists of two networkhops, the aggregate loss rate of pathSSEi−SSO j−SSI canbe calculated asLRi j = (1− (1−LR[ j][i])(1−LRI[ j]));

• Similarly, reliability (RE[i][ j] and REI[i][ j]) can be ob-tained by monitoring the path up/down time. The aggre-gate reliability of pathSSEi−SSO j−SSI can be calculatedasREi j = (RE[ j][i]∗REI[ j]);

• Pricing (PR[M][N] and PRI[M][N]) can be adjusted bybusiness logics. Whenever the CDN operators change thenetwork pricing rates, the pricing used by Sticky-DNScan be correspondingly adjusted.

• Network delays (ND[M][N] and NDI[M]) and physicaldistances (PD[M][N] andPDI[M]) are easy to obtain withknown techniques.

C. Edge-Origin Connection Monitoring (ECM)

The component of ECM is used to keep the server streamingstate maintained by OSS fresh, in the sense that the additionalcost associated with each Origin Server is up-to-date, and thepopularity state of a stream on all Origin Servers is up-to-date. For all Origin Servers, there is a server streaming state.The server streaming state is an array, and each element cor-responds to an Origin Server and is defined as the AdditionalCost of using the particular Origin Server to serve a stream.The state can be realized by a 2-D array of AD[M][T], whereM is the number of Origin Servers and T is the number ofstreams. AD[i][j] records the additional cost of using OriginServerSSOi to serve streamL j . The values of AD[M][T] arebased on whether an Origin Server serves a stream or not.

Two additional 2-D array of NV[M][T] and WB[M][T] areused. NV[i][j] records the number of Edge Servers OriginServerSSOi is serving streamL j to. WB[M][T] is used toindicate the stream popularity on a particular Origin Server.The indicator uses the number of “Would-be” Edge Servers foreach Origin Server. Specifically, the assignment and updatingof AD[M][T] are shown in Figure 5.

ECM monitors the connecting/disconnecting events betweenOrigin and Ingest Servers for all live streams. Wheneverthe events occur, CVM updates NV[M][T], WB[M][T] andAD[M][T] correspondingly.

D. Sticky DNS Engine (SDE)

Sticky DNS Engine (SDE) performs the actually computa-tion and comparison of costs, then selects the most appropriate

(a) Original, no viewers (b) E1 is on (c) E2 is on

Fig. 7. An illustrative example (I)

Source: S

Ingest: I

Origin: O1

Origin: O2

Edge: E1

Edge: E2

Edge: E3

(a) E3 is on

(b) E4 is on

Fig. 8. An illustrative example (II)

Origin Server given a live streaming DNS request. The algo-rithms are shown in Figure 6. SDE handles the joining of anEdge Server (i.e., begins to serve a stream) and the leaving(i.e., begins to drop a stream) separately. For both events,itupdates theAD andNV states.

E. An illustrative example

We use the following example to illustrate how Sticky-DNS works. Assuming there are one Ingest Server (I ) andtwo Origin Servers (O1 andO2), as shown in Figure 7. Whenthe first end-viewer joins and is bound to Edge ServerE1,E1 needs to decide which Origin Server (O1 or O2) for E1.Assuming the distance is the cost metric,O2 is chosen sincethe additional cost of pulling fromO2 (i.e., sum of cost ofI −O2 andO2−E1) is smaller.

When the second Edge ServerE2 requests an Origin Server,though it is closer to Origin ServerO1, the additional cost ofpulling from O1 (i.e., I −O1 andO1−E2) is larger than thatof pulling from O2 (i.e, onlyO2−E2, sinceI −O2 has cost ofbeing zero). ThusE2 choosesO2 as the optimal Origin Server.Note that sinceE2 is the “would-be” edge ofO2, the cost ofI −O1 is reduced in half to reflect the benefit of path reuse.When the third Edge ServerE3 joins, as shown in Figure 8, itcompares the additional costs of pulling fromO1 and pullingfrom O2. Since the cost ofI −O1 is not in half, assumingthe cost sum ofI −O1 and O1−E3 is smaller than the cost

of O2−E3, E3 choosesO1 to be the optimal Origin Server.Finally, when Edge ServerE4 joins, it choosesO1 as the OriginServer.

V. DEPLOYMENT

Sticky-DNS can be easily integrated into CDN GeoDNSoperations. Since Sticky-DNS is designed for live-streamingapplications at Origin level only, to accommodate other appli-cations and scenarios, CDN providers need to deploy a DNS-service frontend to distribute different types of DNS requests.We refer to this type of frontend as DNS Distributor, as shownin Figure 9. It takes the input of all DNS requests, and ifthe request is a DNS request from an Edge Server askingfor assignment of an Origin Server and the DNS request isfor live streaming application, it will redirect the request toSticky-DNS. For all other types of requests including EdgeDNS requests and DNS requests for other applications, theyare handled by default GeoDNS.

In order for live streaming applications to use Sticky-DNS,the hostname in the DNS request has to be formatted in afashion that the DNS service can recognize live streamingapplications and the specific live streams. For instance, a DNSrequest for live stream live-1 of customer customer-1 wouldbe live-1.customer-1.cdnprovider.com.

To capture the events of connecting and disconnecting,Edge or Origin Servers should have the ability to monitor

0.00

5.00

10.00

15.00

Origin Transit

Cost

Edge Transit

Cost

Aggregate

Transit Cost

Default GeoDNS Optimized GeoDNSKM

Fig. 9. CDN Transit Cost

such events. The capability can be implemented inside variouslive streaming servers. Examples of live streaming serversareWowza Media Server, Adobe Media Server and IIS smoothstreaming server.

VI. EVALUATION

To gain concrete understanding of GeoDNS regarding theimpact on both CDN and viewers, we consider the followingsimplified scenario to quantify the saving on CDN transitcost and the performance experienced by end viewers. Weassume an USA mainland CDN provider which has 1 IngestPOP (Point of Presence) and 10 Origin POPs, where eachPOP has only a single streaming server. We further assumethat Edge Servers have to retrieve live streams from OriginServers, which then retrieve from Ingest Server. For simplicity,we consider a performance metric of physical distance in theunit of thousand-miles (i.e., KMiles) for both CDN transit-costsaving and live-stream viewers. Thus, the larger the distancesare, the higher cost both CDN and viewers pay.

We built a prototype with Adobe Flash Media Server (FMS)4.0. The prototype consists of 11 servers deployed in 11 cities,all running Windows Server 2008 R2. One of the servers

Fig. 10. Deployment Illustration

serves as the Ingest Server and receives a live stream from astream source, and the other 11 servers serve as Origin Serversand are relaying the live stream to end viewers. We then invoke10 stream viewers from different locations. Since viewers arealways bound to the closest Edge Servers, we use viewers tomimic the edge servers in typical CDN setup as they need todecide the origin servers to pull stream from.

We first consider the default regular DNS, which directseach viewer to the closest Origin Server. For comparison, wealso consider an optimized DNS which directs each Edge POPto the closest Origin POP that leads to smallest CDN transitpath (i.e., from Ingest Server to Edge Server).

CDN Transit Cost:For both scenarios, we count the lengthsof Ingest-Origin paths, Origin-Edge paths, and aggregatedCDN delivery path. As shown in Figure 9, the blue bars showthe default scenario, while the red bars show the optimizedscenario. The aggregated CDN delivery distance is the sum ofall Ingest-Origin and Origin-Edge paths. Specifically, first, theaggregated Ingest-Origin distances are 10.48 Kmiles and 1.81Kmiles for two scenarios, respectively. The reason for reduceddistance in optimized-DNS scenario is that some Origin POPsare not serving any Edge Servers, thus they are free fromstream delivery. Second, the aggregated Origin-Edge distancesare 2.92 Kmiles and 5.40 Kmiles, respectively. Since EdgeServers in optimized-scenarios may connect to far-away OriginPOPs, the Origin-Edge paths are inflated. Third, the aggregatedpath distances, which correspond to the CDN transit cost, are13.41 Kmiles and 7.17 Kmiles, respectively. We see that forthe particular setup, optimized-DNS results in much smallerCDN transit cost (more than 46% reduction).

These results suggest that CDN is able to gain significanttransit cost saving by optimizing GeoDNS inside CDN. Notethat in this simple scenario, we assume each POP only has asingle Edge Server. With more servers, the saved transit cost

0.00

0.50

1.00

1.50

2.00

2.50

3.00

C D F H J K N O Q T

Default OptimizedFig. 11. Stream Delivery Paths

can be more significant as each server will incur a separatedelivery path. Note that with shorter delivery paths, endviewers also see shorter playback buffers and better viewingexperiences.

Viewers’ Experience: Since each live stream has to de-livered from the stream source all the way to the end usersin a real-time fashion, the end-users’ performance is heavilyimpacted by the length of the entire delivery path. In otherwords, though there are other factors impacting live-streamusers’ performance, it is generally acceptable to claim that thelonger path an end user relies on, the worse performance theuser experiences. For both scenarios, since the distances fromstreaming sources to the Ingest Server and from Edge Serverto end viewer are the same, we now focus only on the CDNdelivery distance from Ingest Server to Edge Server for eachviewer and show them in Figure 11. We observer that all pathsin optimized-DNS scenario are shorter, which is not surprisingsince Edge Servers choose the Origin Servers that result inshortest paths. The aggregated distances are 13.41 Kmiles and10.35 Kmiles, respectively, or 23% decrease. These resultssuggest that end viewer’s experience can also improved withoptimized GeoDNS.

Money Talks: Taking a more direct business perspective,we now compare the transit cost with respect to pricing ofbandwidth usage. Depending on subscription packages, CDN

# of Daily BW Sav. Yearly BW Sav. Yearlystreams (TB*KMiles) (PB*KMiles) Cost Sav.

1 0.20 0.07 $7K100 20 7.37 $700K

2,000 400 147.47 $14.74M10,000 2K 737.35 $73.7M

TABLE IBANDWIDTH /COSTSAVING OF STICKY-DNS

providers may pay ISPs for the bandwidth used. We assumeeach live stream is 3Mbps. In the above scenario where only4 Origin Servers are delivering the same live stream, thesaved bandwidth per day would be 0.20 TB*Kmiles (or 73.74TB*KMiles per year). For a small-scale CDN with 100 con-current streams, the daily saved network bandwidth will be 20TB*Kmiles (or 7.37PB*Kmiles per year). For a medium-scaleCDN with 2K concurrent streams, the daily saved bandwidthwould be 400 TB*Kmiles (or 147.47 PB*Kmiles per year).And for a large-scale with 10K concurrent streams, the dailysaved bandwidth will be 2 PB*KMiles and yearly saving of737.35 PB*KMiles. Simply assuming the transit cost of 0.10per GB/Kmile, optimized-DNS can help save $14.74 Millionper year with 2000 streams, or about $7K per stream, as shownin Table I.

VII. M ODIFIED STICKY-DNS FOR FLAT INFRASTRUCTURE

So far the proposed design of Sticky-DNS for selectingoptimal origin streaming servers works on CDN infrastructuresthat have the strict 3-layer (i.e., Ingest, Origin and Edge)servers. Though most of today’s CDNs feature the 3-layerinfrastructure, we do envision smaller-scale CDN providersmay deploy 2-layer (i.e., Ingest and Edge) infrastructures.For differentiating, we refer to the 2-layer network asflatinfrastructure.

To handle the scenarios where flat CDNs are used todeliver live streaming, we also propose a modified version ofSticky-DNS to optimize the CDN infrastructure. The proposedapproach is termed as Constrained Server Chaining (CSC) forsupporting live streaming.CSC’s key idea is to allow CDNstreaming servers to form a constrained chain structure forpulling live streams from the ingest server and delivering toend viewers.

Motivation: Consider an example where SourceS is ingest-ing a live stream into CDN’s ingest serverI . Assuming bothstreaming serversS1 andS2 serve live streams to users. With

CDN

Ingest:

I

Server:

S2

User

Source:

S

Server:

S1

User

CDN

Ingest:

I

Server:

S2

User

Source:

S

Server:

S1

User

(a) Default CDN Delivery (b) Optimized CDN Delivery

Fig. 12. Live streaming delivered with CDN

strictly layered CDN structure, bothS1 andS2 have to pull thestream fromI , resulting CDN transit hops ofIS1 and IS2, asshown in Figure 12(a).

However, as shown in Figure 12(b), there exists anothertransit structure which allowsS2 to pull the stream fromS1 asopposed toI . The two CDN transit hops ofIS1 andS1S2 havea smaller aggregated CDN transit cost, asS1S2 is shorter thanIS2. Note that though the shown example is trivial at a firstlook, the cost saving of a CDN with many streaming serverscan be huge. Specifically, the layered CDN will always resultin a star-like delivery map, while a tree-like delivery map canhave significantly shorter delivery path.3

Problem definition: We consider the scenarios where aCDN provider has two types of servers: ingest servers whichaccept published live streams from customers, and streamingservers that transit the streams to end users. We useIS andSSto denote the two server sets, respectively. The CDN providerdelivers a set of live streams denoted byLM. We furtherassume each live streamLMi has a single ingesting sourcewhich connects to the ingest server ofISi .

The problem we are addressing with CSC can be de-fined as follows. For each of the live streams (denoted byLMi and ingested toISi), CSC needs to decide the optimalstreaming server setOSSi such that the CDN transit costcan be minimized, and the end viewers’ experience is notadversely compromised (which is captured by imposing adelivery length cap). We useCi to denote the aggregated transitcost corresponding to the deliveringLMi acrossISi andOSSi.

With these assumptions, the input, output, and objectivefunction of CSC are:• Input: (i) Live streamLMi , (ii) CDN ingest serverISi , (iii)

CDN streaming server setSS, and (iv) Delivery length cap;

3Though the full delivery paths for some users are longer withtree-likestructure, the lengths can be easily capped by exerting somerestrictions onthe sever selection process. We will address this concern inSection III.

VariablesUi : New live stream userLMk: requested live streamSSei: The streaming server that servesUi

CSSi: The candidate streaming server setOSSo: The optimal upstream streaming server

1 Received a new live stream playback request fromUi

2 Determine the streaming serverSSei that will serveUi

3 Check whetherSSei is servingLMk or not4 If SSei is servingLMk

5 Begin playback6 Else7 Retrieve state information8 Determine candidate streaming server setCSSi

9 Retrieve state of cost values and streaming10 Determine the optimal upstream serverOSSo11 (The cost betweenSSei andOSSo is minimum)12 Update server streaming state13 End

Fig. 13. Pseudo code of CSC

• Output: Optimal streaming server setOSSi;• Objective function:Total CDN transit cost is minimized.CSC Design: As we observed from the illustrate example,

CDN transit cost (i.e., between CDN servers) can be savedby allowing streaming servers to retrieve live streams fromother streaming servers rather than always retrieving fromtheCDN ingest server. Such a design essentially converts thelayered infrastructure most CDN providers are using toflatinfrastructure.

CSC is a specialized server-selection mechanism designed

for live streaming applications. It aims at saving CDN transitcost but not compromising end user experience. CSC achievesthe cost savings by allowing streaming servers to pull livestreams from other closer streaming servers. In other words,rather than simply retrieving from the ingest server as ina layered structure, streaming servers can connect to otherstreaming servers to form a flat and layer-less structure. Indoing so, however, the resulting delivery path for certainviewers playing the particular stream could be inflated in theform of larger distances, which could cause other undesirableeffect such as larger playback-buffering time. To address this,CSC imposeslength-capwhen making decisions of selectingupstream servers. It always choose streaming servers thatsatisfy the length-cap requirement. Whenever necessary, it canalso fall back to the default behavior of directly pulling fromingest servers.

CSC’s operations are per-stream based, thus it maintainsappropriate state for every live stream. Specifically, for eachlive stream, the active streaming servers that currently areserving it are recorded so that other streaming servers canretrieve from them. In addition to the per-stream state, CSCalso maintains a cost state that is used by all live streams. Coststate maintains the server-server transit cost for each pairs ofCDN servers (i.e., ingest-streaming and streaming-streaming).

Software Architecture: The heart of CSC is the CSCEngine, which takes the user request, interacts with othercomponents, and then decides the optimal upstream streamingserver. CSC selects streaming servers based on the cost associ-ated with the live streaming delivery path, and the cost is CDN-specific. CSC maintains the state of the cost values of eachCDN hops and the server streaming state for each live stream.The server-server cost is determined by another componentof Cost Determination (CD), and the server streaming stateis adjusted by Server Connection Monitoring (SCM), whichupdates state whenever a connection between CDN servers issetup or dismantled.

Overall Process: The overall process of how CSC worksis as follows. When a new live streaming viewer begins toplayback a stream, the direct CDN streaming server willbe determined (e.g., by CDN’s GeoDNS service [5]). If thestreaming server is actively serving the stream, then it sendsthe stream to the user. Otherwise, CDN will then attempt todetermine the optimal upstream streaming server. For this,the CSC engine retrieves the cost values and streaming stateof streaming servers, and it then determines the candidatestreaming server set. Each of the streaming server in thecandidate set has to satisfy three requirements: (i) activelyserving the requested stream; (ii) available for more retrieval(e.g., not overloaded); and (iii) retrieval from it does notcompromise the length cap. Based on the candidate set, CSCengine then selects the optimal streaming server that incurs thesmallest transit cost (i.e., the transit cost between the selectedupstream server and the direct streaming server). Finally,itupdates the streaming server state and begins stream retrieval.CSC’s pseudo code is shown in Figure 13.

VIII. R ELATED WORK

CDN technologies: Content Delivery Networks (CDNs)have been carried out by various providers to expedite the webaccess [1], [2]. The techniques used by CDNs for deliveringconventional web contents are explained in related writings[6], [7]. Despite the popularity and pervasiveness of CDNs,many research problems and system-building challenges per-sist for optimizing CDN infrastructure and addressing chal-lenges associated supporting various types of applications. Forinstance, as we demonstrated in this work, how to efficientlysupport the application of live streaming with CDNs deservesmore study.

New Internet architecture: Though various network ar-chitectures have been proposed and utilized for differentdeployment scenarios with the rapid development of Internettechnology [8]–[10], CDN providers typically apply layeredstructure (e.g., ingest-layer, origin-layer and edge-layer) due tothe caching-property of delivering static web contents tracingback to Day-1 of CDN industry. Realizing that CDNs andP2P can complement each other, many research efforts alsoattempt to integrate these two realms into one [11]. Unlikethese works that combine the merits of both CDNs and P2P,our work takes an inside perspective and attempts to refineCDN delivery infrastructure.

CDN-delivered live streaming:Live streaming as well asVoD (Video on Demand) streaming have been studied andanalyzed from various perspectives [12]–[19]. The uniqueproperties of live streaming, coupled with CDN-assisted de-livery, justifie a specialized design of CDN infrastructurethatserves live streaming and saves CDN transit cost. One work[20] proposes to combine the benefits of CDN and P2P toenhance live streaming. To our best knowledge, this work isthe first to consider and address the problem of saving transitcost for delivering live streaming with traditional layered-infrastructure CDNs.

DNS-related:Being one of the fundamental building blocksof Internet, DNS system has deserved significant researchattentions [21]–[26]. However, most of the works aim atoptimizing the regular DNS service which serves all typesof applications. Meanwhile, both live streaming and CDNshave been studied and analyzed from various perspectives.The unique properties of live streaming, coupled with CDN-assisted delivery, justifies a specialized design of DNS servicethat specifically serves live streaming and saves CDN transitcost. To our best knowledge, this work is the first to considerand address the problem of saving transit cost for deliveringlive streaming with CDNs.

IX. CONCLUSION

In this work, we optimize CDN streaming server selection tobetter support live streaming with CDN. The proposed designdynamically selects streaming servers based on the popularityof the live stream. Significantly improving both the CDNtransit cost saving and end viewers experience, the design canbe deployed with a specialized GeoDNS.

REFERENCES

[1] “Akamai technologies,” http://www.akamai.com/.[2] “Level 3 communications, llc.” http://www.level3.com/.[3] “Limelight networks,” http://www.limelightnetworks.com/.[4] “Internap network services corporation,” http://www.internap.com/.[5] “Bind geodns,” http://www.caraytech.com/geodns/.[6] K. Park, W. W. (editors of H.T. Kung, and C. Wu), “Content networks:

Taxonomy and new approaches,” 2002.[7] D. C. Verma, S. Calo, and K. Amiri, “Policy based management of

content distribution networks,”IEEE Network Magazine, vol. 16, pp.34–39, 2002.

[8] I. Stoica, R. Morris, D. Karger, M. F. Kaashoek, and H. Balakrishnan,“Chord: A scalable peer-to-peer lookup service for internet applications,”in SIGCOMM ’01: Proceedings of the 2001 conference on Applications,technologies, architectures, and protocols for computer communications,San Diego, CA, USA, 2001.

[9] S. Ratnasamy, P. Francis, M. Handley, R. Karp, and S. Schenker, “Ascalable content-addressable network,” inSIGCOMM ’01: Proceedingsof the 2001 conference on Applications, technologies, architectures, andprotocols for computer communications, San Diego, CA, USA, 2001.

[10] I. Stoica, D. Adkins, S. Zhuang, S. Shenker, and S. Surana, “Internetindirection infrastructure,”IEEE/ACM Trans. Netw., vol. 12, no. 2, pp.205–218, 2004.

[11] H. Yin, X. Liu, T. Zhan, V. Sekar, F. Qiu, C. Lin, H. Zhang,and B. Li,“Livesky: Enhancing cdn with p2p,”ACM Trans. Multimedia Comput.Commun. Appl., vol. 6, pp. 16:1–16:19, August 2010.

[12] K. Sripanidkulchai, B. Maggs, and H. Zhang, “An analysis of livestreaming workloads on the internet,” inProceedings of the 4th ACMSIGCOMM conference on Internet measurement, ser. IMC ’04, 2004.

[13] J. He, A. Chaintreau, and C. Diot, “A performance evaluation of scalablelive video streaming with nano data centers,”Comput. Netw., vol. 53,pp. 153–167, February 2009.

[14] C. Vicari, C. Petrioli, and F. L. Presti, “Dynamic replica placement andtraffic redirection in content delivery networks,”SIGMETRICS Perform.Eval. Rev., vol. 35, December 2007.

[15] T. Hisada, Y. Hirota, H. Tode, and K. Murakami, “P2p network systemfor private live-streaming distribution,”Int. J. of Multimedia Intelligenceand Security, vol. 2, pp. 36–53, 2011.

[16] C.-Y. Chen, T.-Y. Wu, W.-T. Lee, H.-C. Chao, and J.-C. Chiang, “Qos-based active dropping mechanism for ngn video streaming optimization,”Knowledge Engineering Review, 2012.

[17] C.-F. Lai and H.-C. Chao, “A network and device aware qosapproachfor cloud-based mobile streaming,”IEEE Transactions on Multimedia,2012.

[18] K.-D. Chang, J.-L. Chen, W.-M. Chen, H.-C. Chao, and C.-F. Laih,“Cloud multimedia system with dynamic adjustable streaming service,”World Automation Congress 2012, 2012.

[19] J. Tu, J. Wu, and C. Hsu, “A cross-multilayer mechanism design forimproving video quality of svc coded video streams over mobile wimaxnetworks,”Int. J. Internet Protoc. Technol., vol. 6, no. 1/2, pp. 123–134,June 2011.

[20] Z. H. Lu, X. H. Gao, S. J. Huang, and Y. Huang, “Scalable andreliable live streaming service through coordinating cdn and p2p,” inProceedings of the 2011 IEEE 17th International Conferenceon Paralleland Distributed Systems, ser. ICPADS ’11, Washington, DC, USA, 2011.

[21] B. Ager, W. Muhlbauer, G. Smaragdakis, and S. Uhlig, “Comparing dnsresolvers in the wild,” inProceedings of the 10th annual conference onInternet measurement, ser. IMC ’10, 2010.

[22] J. Jung, E. Sit, H. Balakrishnan, and R. Morris, “Dns performance andthe effectiveness of caching,”IEEE/ACM Trans. Netw., pp. 589–603,2002.

[23] V. Pappas, D. Wessels, D. Massey, S. Lu, A. Terzis, and L.Zhang,“Impact of configuration errors on dns robustness,”IEEE J.Sel. A.Commun., vol. 27, April 2009.

[24] A. Broido, E. Nemeth, and K. C. Claffy, “Spectroscopy ofdns updatetraffic,” in Proceedings of the 2003 ACM SIGMETRICS internationalconference on Measurement and modeling of computer systems, ser.SIGMETRICS ’03, San Diego, CA, USA, 2003.

[25] D. Dagon, M. Antonakakis, P. Vixie, T. Jinmei, and W. Lee, “Increaseddns forgery resistance through 0x20-bit encoding: security via leetqueries,” in Proceedings of the 15th ACM conference on Computerand communications security, ser. CCS ’08, Alexandria, Virginia, USA,2008.

[26] V. Pappas, Z. Xu, S. Lu, D. Massey, A. Terzis, and L. Zhang, “Impactof configuration errors on dns robustness,” inProceedings of the 2004conference on Applications, technologies, architectures, and protocolsfor computer communications, ser. SIGCOMM ’04, Portland, Oregon,USA, 2004.