IEEE TRANSACTIONS ON BROADCASTING 1 QoE-Aware …bentaleb/files/papers/journal/BMS.pdf · BENTALEB et al.: QoE-AWARE BANDWIDTH BROKER FOR HTTP ADAPTIVE STREAMING FLOWS IN SDN-ENABLED

This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

IEEE TRANSACTIONS ON BROADCASTING 1

QoE-Aware Bandwidth Broker for HTTP AdaptiveStreaming Flows in an SDN-Enabled HFC Network

Abdelhak Bentaleb , Member, IEEE, Ali C. Begen, Senior Member, IEEE,and Roger Zimmermann, Senior Member, IEEE

Abstract—This paper proposes a software defined networkingbased bandwidth broker solution for improving viewer experi-ence for any type of content delivered to any type of consumerdevice using HTTP adaptive streaming (HAS) in a hybrid fibercoax network. This solution is designed to meet per-sessionand per-group quality-of-experience objectives, to avoid commonHAS culprits such as video instability, unfair and unequal qualitydistribution and network resource underutilization, and to scaleto a large number of concurrent HAS sessions without introduc-ing too much overhead. The mathematical framework behind oursolution solves a convex optimization problem, which relies on aconcave network utility maximization function. Results confirmthe effectiveness of the proposed solution over the state-of-the-artbitrate adaptation and bandwidth allocation schemes.

Index Terms—QoE, SDN, HAS, DASH, HFC network, convexoptimization, bandwidth broker.

I. INTRODUCTION

TODAY, cable TV providers (also known as multiplesystem operators—MSOs, or multichannel video pro-

gramming distributors—MPVDs) are trying to increase theefficiency and capacity of their hybrid fiber coax (HFC) accessnetworks by analog/spectrum reclamation, plant upgrades,service group reductions and using newer codecs for the dis-tribution of video. At the same time, these providers aretransitioning to IP-based delivery to offer enhanced and per-sonalized experiences to their customers in not only the set-topboxes but also all types of connected devices. Instead ofthe traditional and all-broadcast model, a mixed model ofmulticast and unicast (switched digital video) is rapidly beingadopted. For on-demand services, the content is now mostlydelivered using HTTP adaptive streaming (HAS). Currenttrends [1] estimate that HAS traffic will grow threefold from

Manuscript received December 30, 2017; revised February 26, 2018;accepted March 7, 2018. This work was supported in part by the NationalNatural Science Foundation of China under Grant 61472266, and inpart by the National University of Singapore (Suzhou) Research Institute,Suzhou Industrial Park, Jiang Su 215123, China. (Corresponding author:Abdelhak Bentaleb.)

A. Bentaleb and R. Zimmermann are with the School of Computing,National University of Singapore, Singapore 119077 (e-mail:[email protected]; [email protected]).

A. C. Begen is with Computer Science Department, Özyegin University,34794 Istanbul, Turkey, and also with Networked Media, Inc., Cekmekoy,34794 Istanbul, Turkey (e-mail: [email protected]).

Color versions of one or more of the figures in this paper are availableonline at http://ieeexplore.ieee.org.

Digital Object Identifier 10.1109/TBC.2018.2816789

2016 to 2021, and reach 82% of the overall Internet trafficby 2021 due to the tremendous increase in viewer demandsfor more and higher-quality video. Each MSO has to meetthe quality-of-experience (QoE) expectations of its customersto stay competitive, reduce churn, gain more customers andincrease revenues. Achieving this with a reasonable cost andeffort is, however, not a straightforward task in a shared HFCnetwork due to the surge in bandwidth consumption stemmingfrom the unicast services and the rival over-the-top (OTT)services.

HAS (also dubbed adaptive bitrate, ABR, in the industry) istypically a client-driven pull-based approach where the HASserver hosts the segmented media encoded at multiple repre-sentations (that have different bitrate levels, content resolutionsand perceptual qualities), and the HAS client requests thesesegments from the server based on the network and play-back buffer conditions [2]. By design, each HAS client strivesindividually to fetch the media segments encoded at the high-est bitrate level. This selfish behavior introduces a numberof problems when a set of clients share a fixed total capac-ity [3], [4]. In competing for the shared bandwidth, the clientsmake incorrect bitrate decisions that may lead to either videostability issues, unequal and unfair quality distribution (qual-ity unfairness), or network resource underutilization. The MSOcannot solely rely on designing an ultimate HAS client to dealwith these issues since there can always be different HASclients in the network that are not under MSO’s control [5].The MSO may alternatively attempt to solve these issues byallocating the bandwidth equally among the competing clients,however, the equal-bandwidth allocation approach can workeffectively only if every client has similar capabilities and isinterested in the same content. Said differently, if a small-screen device were allocated the same amount of bandwidthfor streaming as a large-screen device, we would not be ableto accomplish the desired outcome since what we seek is fair-ness in quality, which is almost never the same as fairness inbandwidth [6]. Empirical and experimental results [4], [7], [8]have shown that the issues mentioned above aggravate withan increasing number of clients.

The survey by Seufert et al. [9] suggests that the viewerQoE can be improved among the competing HAS clients byusing a centralized component that controls the total state ofthe network and its entities. For example, software definednetworking (SDN) [10], which recently received significantattention from both the industry and academia, allows a flex-ible and programmatic approach for managing the network.

0018-9316 c© 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.See http://www.ieee.org/publications_standards/publications/rights/index.html for more information.

https://orcid.org/0000-0002-5382-6530


2 IEEE TRANSACTIONS ON BROADCASTING

Fig. 1. The SDN-based bandwidth broker takes content features, device capabilities and service plan types into account for representation decision, bandwidthallocation and provisioning, resource management, reservation and monitoring, flow prioritization and service differentiation in an HFC network. For brevity,only one CMTS is shown, however, in practice there could be several CMTSs, each connected to a subset of the fiber nodes inside the HFC network. Themedia flows coming from the streaming servers are shown in red.

SDN decouples the network’s control logic (control plane)from the forwarding devices that transmit the data (data plane),and implements the control logic in a separate programmablecentralized controller.

In this paper, we propose a QoE-aware SDN-based band-width broker solution for HAS flows in an HFC network,termed BMS (Bandwidth Management Solution). We assumethat the HFC network has a fixed capacity and there is afixed total number of customer premises. Generally speak-ing, a big portion of this total capacity is used for analogand digital TV services, which includes the linear channelsand on-demand services, while the remaining capacity is allo-cated for upstream/downstream high-speed data for generalInternet access and IP-based voice and video services. Ideally,analog spectrum should be totally reclaimed and only the lin-ear channels that are currently popular should be multicastleaving more capacity for IP access, which can be used tostream less popular linear channels, and more on-demandand OTT content. Nevertheless, in this paper, we are notconcerned about the time-varying popularity of the linearchannels, rather we assume that there is a time-varying resid-ual capacity available for IP access, and it is that capacitywe want to allocate more diligently so that the MSO canachieve one or more of the following objectives depend-ing on its needs: (i) ensure a minimally acceptable qualityfor each HAS session, that is, avoid starvation of any HASflow, (ii) improve viewer QoE when the number of HASsessions is fixed, and (iii) support a larger number of con-current HAS sessions without degrading QoE for any HASsession. Our solution builds upon the insights of our priorwork, SDNDASH [4] and SDNHAS [11], and addressesthe following four issues that were previously left as futurework:

• BMS avoids extra message overhead that may beintroduced in our prior methods due to explicit orimplicit interactions between different architectural enti-ties. In contrast, BMS leverages existing communicationinterfaces in HFC systems. In particular, it employs both

in-band event messages and a feedback channel to collectstatuses from different entities.

• BMS supports three types of simultaneously running QoEoptimizations, namely per-session, per-channel and per-cluster while our prior work was limited to two types anddid not support per-channel optimization.

• BMS requires no modifications at the client-side to enablethe players to read decision recommendations of the SDNapplication. Furthermore, BMS supports two operatingmodes that allow BMS-enabled and traditional clients toco-exist with good overall system performance.

• To solve the QoE optimization problem, BMS uses a con-vex relaxation mechanism [12] of online decompositionmethods, which allows the parallelization of calculationsduring the solving process. This is in contrast to our ear-lier methods that used either a dynamic programmingor online reinforcement learning [13], both of whichintroduce significant complexity.

In improving the viewer QoE or the concurrency, ourapproach takes content features (e.g., video type, encodingbitrate, content resolution and perceptual quality), device capa-bilities and service plan types into account. As shown in Fig. 1,these input parameters are fed into the bandwidth managementapplication (BMA) to trigger various control and manage-ment mechanisms such as representation decision, bandwidthallocation and provisioning, network resource management,reservation and monitoring, flow prioritization and servicedifferentiation. Our solution goes beyond the well-known sta-tistical multiplexing (statmuxing)1 [14], [15] solutions thatdynamically allocate bandwidth only for the linear channelsby also brokering bandwidth for HAS flows. Its effectivenessover the existing bitrate adaptation schemes and the traditionalnetwork resource allocation solutions is shown via extensivetrace-driven and emulation-based experiments across a wide

1By definition, statmuxing is a type of communication link sharing wherethe channel is divided into a random number of variable bitrate digital channelsor data streams.


BENTALEB et al.: QoE-AWARE BANDWIDTH BROKER FOR HTTP ADAPTIVE STREAMING FLOWS IN SDN-ENABLED HFC NETWORK 3

variety of QoE metrics, video complexities and properties, andnetwork variability.

The rest of the paper is organized as follows. Related workon some existing ABR and bandwidth allocation schemes arepresented in Section II. Our SDN-based solution’s overviewand details are given in Sections III and IV, respectively.Section V presents the performance evaluation of the proposedsolution. Conclusions and future directions are highlighted inSection VI.

II. RELATED WORK

To date, many bitrate adaptation schemes have beenproposed in the context of improving viewer QoE. Below, wereview the controller-based approaches. For the client-drivenschemes please refer to [16].

In the controller-based approaches, the client uses networkassistance in its ABR heuristics to select the particular seg-ments. Network assistance could be provided from variousentities such as the forwarding devices, DASH server, SDNcontroller, home gateways, proxies, etc. The controller-basedsolutions roughly fall into two classes:

• Without Bandwidth Reservation: The controller onlyguides the HAS clients in their decisions without anybandwidth allocation (e.g., laissez-faire). Mok et al. [17]designed a QoE-aware proxy-based architecture forDASH, called QDASH. The proxy implements a prob-ing technique to estimate the total bandwidth in thesystem and then returns a suitable bitrate level foreach DASH client. Similarly, Jiang et al. [18] proposedthe Pytheas system. Pytheas describes the benefits ofusing a centralized data-driven approach to improve thethroughput prediction and optimize the viewer QoE.Thomas et al. [19] described a standardized architecturefor server and network assisted DASH, called SAND.SAND offers an asynchronous interaction between theDASH client and network elements, which allows theexchange of recommended bitrate information amongmany other types of assistive information.

• With Bandwidth Reservation: The controller guides theHAS clients in their decisions through bandwidth allo-cations (e.g., equally or based on other criteria). Thebandwidth reservation is a function of the suggestedbitrate. The central manager assigns a bandwidth sliceto a client or a group of clients, referred to as per-session or per-group QoS provisioning, respectively.Bentaleb et al. [4], [11] leveraged SDN and reinforcementlearning (RL) capabilities to design robust architecturesfor intelligent DASH and HAS delivery. Both solu-tions use SDN flexibility in programming the forwardingdevices to dynamically allocate the required bandwidthfor HAS clients and improve the overall viewer QoE.However, both architectures were designed for a specificsystem with particular assumptions as shown in Section I.In contrast, our presented solution works well for alltypes of HAS delivery systems with both per-sessionand per-group QoE optimizations. Benefiting from SDN,Cofano et al. [20] built a video control plane (VCP)

to allocate and monitor the network resources for eachclient. Their work aimed to maximize quality fairness andnetwork resource utilization among competing clients.Similarly, Ramakrishnan et al. [21] designed a band-width management solution for optimizing viewer QoEof multiple clients. The proposed solution formulated theper-flow bandwidth allocation as a convex optimizationproblem that intended to allocate the bandwidth for eachclient based on its viewer QoE.

The described controller-based solutions show a highperformance. Nevertheless, most of them introduce large over-head during the interaction between different entities of thesystem due to the implicit or explicit and extra signaling mes-sages. Hence, their deployment in a large-scale system remainsan open question.

III. BANDWIDTH MANAGEMENT SOLUTION

In this section, we describe the architecture of the proposedSDN-based solution, referred to as Bandwidth ManagementSolution (BMS), and define the system model. BMS differsfrom the existing controller-based approaches in three points:

• BMS is designed to support any type of existing HASdelivery systems over an HFC network, and thus, itperforms consistently in all environments.

• BMS optimizes both per-session and per-group QoEobjectives (see Section IV-D). Further, BMS is able tobalance different QoE metrics and ABR goals.

• BMS scales to accommodate a massive number of HASclients that are distributed geographically in a wide areawithout introducing additional overhead benefiting fromexisting communication interfaces.

A. Architecture Overview

BMS uses SDN’s conceptual design and its centralizedentity capabilities to enable a dynamic bandwidth allocation,slicing and management in an HFC access network (fromthe Cable Modem Termination Systems (CMTS) to cablemodems in Fig. 1). It improves the viewer QoE among het-erogeneous competing clients (e.g., smartphones, tablets, TVs,etc.) that have different device capabilities such as screenresolution, CPU and memory, by selecting the optimal jointrepresentation (and including the respective bandwidth alloca-tion) decisions. This way, BMS avoids the drawbacks of thetraditional bandwidth allocation approaches noted in Section I.

The overview of our proposed BMS architecture is depictedin Fig. 2. The three-layered BMS architecture consists of theapplication, control and infrastructure planes. As is com-mon with SDN, the control plane resides within the SDNcontroller itself. The infrastructure plane below consists ofdata forwarding devices that interface with the SDN controllervia its southbound, OpenFlow protocol. The application planeat the top encompasses the bandwidth management applica-tion (BMA) that includes two interfaces that communicate(a) with the SDN controller via its northbound interface and(b) with the set of HAS clients, server(s) and MSO entities.



SDN Controller

Application Plane

Control Plane

InfrastructurePlane

HAS Clients

OpenFlow

BMA

Policies

HTTP REST API

HFC Network

REST API

HAS Server(s)

Headend /Video Hubs

OS

S/B

SS

Sub

. Mgm

t.In-band EventStream

MSO Core

Fig. 2. The proposed BMS architecture.

The external application components may be hosted on a sep-arate computer or in the cloud to provide maximum flexibility.Next we will describe each plane in more detail.

1) Application Plane: The BMA contained in this planeconsists of multiple components that support monitoring,network programming and assisting competing HAS clientsin their ABR decisions. It functions as a proxy between theHAS clients and server(s) and its key mission is to optimallycompute the per-session or per-group joint representation deci-sions. By monitoring and collecting the network and clientstatuses it also detects all relevant events such as clientsjoining or leaving the system, or playback state changes. Itscomponents are summarized as follows:

(i) The online Data Collector Service collects and recordsthe required statistics from different entities and for-wards them to the Viewer QoE Optimizer (component(4)). From each HAS client the buffer occupancy, viewerQoE, requested video ID, device capabilities, subscrip-tion plan type, etc., are gathered. From the MSO Core,the OSS/BSS subscriber management server contributessubscriber and service differentiation information (orsubscription plans), the HAS servers supply the videocontent with their manifest files, and the headend/videohubs provide the service type offered (video on-demand(VoD), live or both) and the channels with their IDs.From network devices (CMTS) queue lengths, QoSstates, congestion levels, network resource states, andtraffic types (video or cross) are recorded. Due to dig-ital rights management (DRM) some traffic may beencrypted and the BMS architecture is modular to sup-port the end-to-end encryption employed by the MSOs.

(ii) The offline MAP Model defines a set of static map-pings that are predetermined by the MSO and pro-vided as auxiliary inputs to the Viewer QoE Optimizer(component (4)) to help in the selection of the optimaljoint representation decisions. The model describes therelationship between bitrate levels, perceptual quali-ties, device resolutions, content types and subscriptionplan types. Such mapping relationships were calculated

offline and stored in hard-coded tables. For instance,we leverage four mapping tables based on recommen-dations proposed in [22] and by YouTube,2 Twitch,3

and JWPlayer4: (a) Bitrate level ↔ device resolution:This defines the mapping between the encoding bitrateof the content and the device resolution of the client,and it helps select a suitable bitrate that fits with thedevice resolution. (b) Bitrate level ↔ perceptual qual-ity: This defines the mapping function between theencoding bitrate of the content and the perceptual qual-ity, and it helps provide a consistent quality during astreaming session (i.e., at the same bitrate, we mayhave different values of perceptual quality, e.g., simplescene vs. complex scene). (c) bitrate level ↔ subscrip-tion plan type: This defines the mapping between theencoding bitrate of the content and the subscriptionplan type, and it helps offer a level of resources (i.e.,intuitively a customer is allocated a certain amount ofresources at a given service plan level). (d) Perceptualquality ↔ subscription plan type: We deduced thismapping from (c). Additionally, we used the StructuralSimilarity Index plus (SSIMplus) [23] mapping modelfrom Bentaleb et al. [11] which maps three features—device resolution (DR), content type (CT) and sub-scription plan type (SPT)—into one common perceptualquality space. The authors created this model based on aset of offline perceptual video quality measurements andit allows MSOs to group the HAS clients into five clus-ters. Thus, a per-group optimization can be applied fordifferent streaming services. Also, it helps BMS to sup-port large-scale deployments with a significant reductionin complexity and optimization overhead. Further, BMSis designed modularly, which enables an implementer tochoose other factors to perform the clustering if desired.

(iii) The Bandwidth Estimator employs the PANDA algo-rithm [7] as a periodic passive probe-based technique toeither estimate the available bandwidth coarse-grainedin the whole network or fine-grained for each path. Inthree phases the bandwidth share is estimated, smoothed,and quantized. PANDA has been shown to provide anaccurate and efficient throughput and total availablebandwidth estimation.

(iv) The Viewer QoE Optimizer is the heart of the BMAand computes the optimal joint representation deci-sions that optimize viewer QoE. It implements threemain functionalities. First, the optimization granular-ity is determined. For unicast VoD or OTT servicesa per-session optimization is applied. Else, for broad-cast/multicast services a per-group QoE optimization isperformed. A group may consist of all the clients ofa channel or be based on some other clustering crite-ria [11], [24]. Frequently a subscription plan representsdifferent business strategies for service differentiationthat are imposed by an MSO, such as payment-based

2[Online]. Available: https://goo.gl/pXW1NW3[Online]. Available: https://goo.gl/Rk4acM4[Online]. Available: https://goo.gl/PGqT25



tiers where a higher monthly subscriber fee ensures bet-ter service. For example, an SPT may be structuredinto platinum, gold, silver, bronze and basic clients.Second, it formulates the joint representation decisionsas a convex optimization problem, which relies on aconcave network utility maximization (NUM) function.The objective functions of each case are defined inSection IV. Third, it solves the optimization problemusing optimal online decomposition methods [12], theSDN optimization language (SOL) [25] and fast modelpredictive control (fastMPC) [26].

(v) The Traffic Monitor & Control keeps track of the dif-ferent interactions to detect real-time event-based actionssuch as client playback play, stop, buffering, etc. Itdynamically differentiates between HAS video, crossand encrypted traffic using the deep packet inspectiontool (DPI) sFlow [27]. Further, it monitors the networkresources for any violations that may occur when thedemands of all clients exceed the total network capac-ity. If there is insufficient bandwidth available then anew request is refused (i.e., admission control).

(vi) The Bandwidth Reservation component takes the per-session or per-group bandwidth allocations which arecalculated in the (4) Viewer QoE Optimizer moduleand forwards them to the SDN controller to performthe respective bandwidth slicing. While BMS currentlyimplements a QoE-to-Bandwidth translation the archi-tecture supports the implementation of more generalQoE-to-QoS provisioning [28] with additional QoS met-rics.

(vii) The Communication Service defines the interactioninterfaces and enables the communication between theabove detailed application and control plane entities asfollows. (i) HAS clients ↔ BMA: it leverages DASHevent messages (InbandEventStream) [29] to collecteach client’s status and send decision recommenda-tions to the corresponding client. (ii) MSO and CMTS↔ BMA: it uses the feedback channel or a libcurl5

REST API to collect the required information from boththe MSO6 and CMTS. (iii) BMA ↔ SDN controller:it exchanges QoS requirements with the clients via adefined northbound interface such as a REST API. (v)BMA ↔ SAND: this interface is used only when aSAND architecture [19] is enabled, where it allows theinteraction between BMS and SAND entities (e.g., aDANE server). Please note that our solution leveragesexisting control communication interfaces and thus noadditional overhead is introduced in the system.

In addition to the BMA, two more entity types are part ofthe application plane.

2) HAS Clients: BMS is designed to work with differentexisting heterogeneous HAS clients. These clients may differin terms of device capabilities (e.g., screen resolution, CPUand memory), requested video content type, subscription plan

5[Online]. Available: https://curl.haxx.se/libcurl/6All required information such as the content types, service offered and its

type, service differentiation, subscriber information, channel numbers, etc.,should be published to the SDN-based application beforehand.

Fig. 3. Modifications to the dash.js client.

type and ABR schemes. To fully support BMS, only minormodifications in the client are required. We include one addi-tional class into the ABR scheme of a client, which is theDecision Recommendation Receiver. This accepts the repre-sentation decision recommendations that are provided by theBMA. Subsequently, the client combines these readings withits own ABR heuristics for better decisions. For the evaluationof BMS (Section V), we modified the dash.js [30] referenceplayer as illustrated in Fig. 3 (modifications are highlighted indotted gray boxes). Further, we added the well-known existingABR schemes for comparison.

3) MSO: The MSO offers video services, including VoDand live streaming, to a set of HAS clients. It provides one ormore HTTP-based HAS servers that encode and store mediasegments at various bitrate levels and resolutions for eachvideo content. A description of all the segment representa-tions is stored in a manifest (MPD) file. BMS further includesa simple XML file that lists the perceptual qualities of all seg-ments. Thus, during the decision process, BMS considers boththe bitrate level and the perceptual quality.

4) Control Plane: It orchestrates the main functionalitiesrelated to bandwidth allocation, slicing, and monitoring. Inaddition, it offers a well-defined API to control the networkinfrastructure including the forwarding devices. It simplifiesthe interaction and resource abstraction with the applicationplane via a northbound interface (REST API), and the pro-gramming of the infrastructure plane through a southboundinterface (OpenFlow [31]). It is comprised of an SDN con-troller and an internal SDN-based bandwidth managementcomponent.

5) SDN Controller: It represents the entity of BMS thatacts as a broker between the application and infrastructureplanes. It manages and controls network flows in real time.The controller programs and installs forwarding rules and poli-cies directly on the infrastructure devices via the OpenFlowprotocol. Nowadays, many implementations of SDN con-trollers exist [10]. We utilize the Python-based open-sourceRYU [32] controller which has good support for all OpenFlowversions (v1.0–5.0). In BMS, the RYU controller receivesbandwidth allocation policies from the application plane andthen forwards them to the SDN-based bandwidth managementcomponent.

6) SDN-Based Bandwidth Management: This componentis situated within the SDN controller and receives its infor-mation from the BMA. It implements the actual dynamic



bandwidth allocation, slicing and monitoring for one or groupsof sessions. The received per-session or per-group bandwidthallocation policies are translated into OpenFlow messages asADD rules [Flow Mod: OFPFC_ADD]. Subsequently theserules are installed in forwarding devices via OpenFlow forthe purpose of bandwidth slicing. Each bandwidth allocationrule contains a timeout field that is used to control and moni-tor the bandwidth slices. When the timeout reaches zero, oneof two actions is taken: an OpenFlow message with typeDELETE [Flow Mod: OFPFC_DELETE] informs the for-warding devices to delete the corresponding rule from theirflow and meter tables. Otherwise, an OpenFlow message withtype MODIFY [Flow Mod: OFPFC_MODIFY] is sent to theforwarding devices to apply changes accordingly.

Note that BMS is designed with the best-effort nature ofthe existing HFC networks in mind. It accurately allocates therequired bandwidth for both HAS and best-effort (non-HAS)traffic, even under congested network conditions. BMS consid-ers traditional applications such as email, Web browsing andfile transfers as best-effort services since they do not requireguaranteed latencies. In our solution, such best-effort servicesare admitted generically as cross traffic and taken into consid-eration in the model formulation, Section III-B. BMS ensuresdynamic bandwidth allocation for best-effort traffic throughthe following steps: (i) It uses the Traffic Monitor & Controlcomponent to detect when the best-effort traffic is active, then(ii) it leverages the proper scheduling type [Scheduling Type:BEST_EFFORT] of the QoS parameters in the CMTS. (iii)It generates a bandwidth allocation policy of Best-Effort type,which represents the total bandwidth required for the best-effort traffic. Finally, (iv) when the SDN-based bandwidthmanagement receives such a policy, it generates an OpenFlowmessage as ADD rule without any QoS support for properbandwidth provisioning and slicing. Subsequently, this rulewill be installed in the CMTS.

7) Infrastructure Plane: It represents the OpenFlowenabled forwarding devices, which in the proposed archi-tecture are the CMTS, that carry the network traffic. Thesedevices are programmable from the control plane and theyreceive the forwarding and per-flow network resource provi-sioning rules (i.e., the control policies) that are establishedby the SDN controller via OpenFlow. In our emulation-basedexperiments we used forwarding devices such as Lagopus [33],that support OpenFlow 1.3, CMTS capabilities, and metertable installations for QoS provisioning.

B. System Model

We consider a network topology that is represented by adirected or undirected graph G = (V, E) where V is the setof vertices (i.e., network entities) and E ⊆ V × V is the setof links that connect the network entities. Each link e ∈ Econnects two endpoints x, y ∈ V and is associated with a band-width capacity C(x, y) denoted as BW(x,y). We only considerconnected graphs. The network entities V are divided into sixtypes, including HAS clients P ⊂ V where P = {p1, . . . , pNP},an MSO with its components denoted as CP ⊂ V , a BMA ∈ V ,an SDN controller with SDN-based bandwidth management

SC ∈ V , a network device infrastructure including networkoperator management NI ⊂ V . For brevity we consider a sim-ple HAS delivery system over an HFC broadcast network thatis comprised of NP HAS clients sharing one bottleneck linkwith a total capacity of BWtotal. The CP offers two types ofstreaming services, VoD and live. Further, the system employsa payment-based subscription plan type (SPT) for its clients,where SPT = {platinum, gold, silver, bronze, basic}, and mayoffer an IPTV service with NCH channels where the set ofchannels is defined as CH = {ch1, . . . , chNCH }. Let BWalloc

chand BWalloc

p be the bandwidth allocation for a channel and aclient, respectively. The CP should satisfy the requirements (interms of QoE) of the clients such that the sum of bandwidthdemands (ABR decisions) does not exceed BWtotal or the sumof the allocated bandwidth for CH and their clients’ demandsdoes not exceed BWtotal, i.e.,:

⎧⎪⎨

⎪⎩

∑NPu=1 lpu

≤ BWtotal,∑NP

u=1 BWallocpu

≤ BWtotal,∑NCH

m=1 BWallocchm

≤ BWtotal.

(1)

The clients pu ∈ P can either request VoD or live streamingservices from CP (HAS servers of MSO). In a hetero-geneous system, each client pu, where u ∈ [1, . . . , NP],is characterized by a device screen resolution (DRpu ∈DR = {240p, 360p, 480p, 720p, 1080p}), one subscriptionplan type (SPTpu ) from SPT , and an ABR scheme. Theclient pu first requests the manifest file from the CPand then sequentially starts downloading the segments ofthe corresponding video v with content type CTv

pu∈

CT = {animation, documentary, sport, movie, news} [34].Each video v ∈ CT of total duration T is split into K segmentsand each segment i ∈ [1, . . . , K] is encoded into several repre-sentations with a fixed duration τ = T/K and a variable bytessize si. Each video representation is characterized by its bitratelevel li ∈ Lv, resolution ri ∈ Rv and perceptual quality qi ∈ Qv,where Lv, Rv, and Qv are lists of bitrate levels, resolutions,and perceptual qualities for segments of video v, respectively.Periodically at each segment downloading step i, the BMAestimates the total available bandwidth BWe considering thecross traffic in the network and then finds the optimal jointrepresentation (including the respective bandwidth allocation)for the next segment i + 1 to be downloaded. Thereafter, theclient pu uses its throughput estimator to predict the through-put bwi

pu, the buffer controller to track its buffer occupancy

Bipu

∈ [0, . . . , Bmax] and its representation recommendationgiven by BMA as an upper bound. Finally, it selects the suit-able representation (i.e., bitrate level li+1

puwith its perceptual

quality qi+1pu

and resolution ri+1pu

) using the ABR controllerfor the next segment i + 1 to be downloaded. The contentresolution of the selected representation should fit the devicescreen resolution of pu, where ri+1

pu≈ DRpu . We assume that a

client always requests a representation with a suitable contentresolution, which is not higher than its device screen resolu-tion. During the ABR decision process, the client pu choosesa representation from the available representation lists Lv, Qv,and Rv of segments for video v which are extracted from the



manifest file, where:⎧⎨

⎩

Lv = {l1, . . . , li, . . . , lK

},

Qv = {q1, . . . , qi, . . . , qK

},

Rv = {r1, . . . , ri, . . . , rK

}.

(2)

Here l ∈ [l1, . . . , lη], q ∈ [q1, . . . , qη], and r ∈ [r1, . . . , rη],with η expressing the total number of available representa-tions. Let Qv(Lv, Rv) be the non-linear correlation that mapsthe bitrate levels and content resolutions to the SSIMplus [23]perceptual qualities for each video v. Thus, the selected rep-resentation by pu for the segment i + 1 can be described asqi+1

pu(li+1

pu, ri+1

pu).

We assume a persistent streaming session between the HASclients P and the HAS servers of CP. Let si(li) denote the sizeof segment i that is encoded at bitrate level li. The HAS deliv-ery system uses a variable bitrate (VBR) for segment encodingwith a binary relation si ∼ li which may differ across seg-ments. Thus, the download time for the selected segment i bypu is denoted as ti(si, li) = si(li)/bwi

pu. Furthermore, the client

pu computes its current buffer occupancy Bipu

as:

Bipu

= max

((

Bi−1pu

− si(li)

bwipu

)

+ W − �ti, 0

)

, (3)

where Bi−1pu

is the buffer occupancy of the previous step i − 1,W is the increase (in seconds) of the buffer occupancy whensegment i − 1 is fully downloaded and the decrease as theclient watches the video, �ti (i.e., the off period of HAS) is thewaiting time between the download end of the current segmentand the download start of the next one. In our study we set�ti = 0 and let the client begin fetching the next segmentimmediately when the previous one is completely downloaded.Hence, we alleviate the inaccuracy of throughput estimation.However, when the buffer occupancy is full (i.e., it reachesBmax), the client should wait until the buffer occupancy reducesand thus avoid a buffer overflow.

We assume that periodically at each time slot �f ≈ τ , theSDN-based bandwidth management SC can slice and updatethe network resources based on the bandwidth allocation deci-sions sent by BMA. Also, for IPTV services we assume thatthe clients switch between existing channels following a Zipfdistribution [35] which assigns a high probability to popularchannels followed by a quick decay for non-popular channels.The Zipf distribution probability Prchm of channel chm ∈ CHis calculated as:

Prchm = 1

ch�m ×

NCH∑

ρ=1

(1/ρ�

),∀m ∈ [1, . . . , NCH], (4)

where ρ is the rank of the corresponding channel chu and �

is the Zipf parameter that determines the degree of popular-ity of a channel. When � = 0, the popularity is equal for allchannels. Otherwise, a channel with a lower rank is more pop-ular. We set the Zipf parameters based on the measurementsin [35].

IV. BMS DESIGN

Next we describe the steps that the BMS system relies on.

A. Bitrate-Quality Map Function

BMS uses both the segment bitrate level and its perceptualquality while solving one of the objective functions describedin Section IV-D. This is because the correlation betweenbitrate and perceptual quality is non-linear [4], [11] and thusat the same bitrate level the perceptual quality may varyconsiderably (e.g., complex vs. simple scenes). Further, thesegment quality differs from one video content to another.Thus, streaming video at low quality especially in the case ofdeclining throughput may significantly reduce the viewer QoE.In addition, knowing the segment quality allows the optimizer(i) to avoid quality oscillations and (ii) to aim for a fairlyhigh, consistent quality [36]. Thus, we adopt the bitrate-to-quality function q(l, r) from [11] that is based on the SSIMplusindex [23] and we use this model to create segment qualitymanifest files for each video. Thus, before starting any stream-ing session, the HAS client requests both MPD and segmentquality manifests of the corresponding video.

B. BMS Decision Recommendation Mechanisms

BMS implements two primary operational modes to assistthe concurrent HAS clients in their ABR decisions: (i)representation assistance and bandwidth allocation, and (ii)bandwidth allocation only.

1) Representation Assistance and Bandwidth Allocation:BMS uses the Viewer QoE Optimizer component (SeeSection III) to compute the per-session or per-group optimaljoint representation decisions according to the objective func-tions described in Section IV-D. This mechanism requires afew small modifications to the ABR schemes of the HASclients to allow them to receive representation recommenda-tions sent by the BMA as shown in Fig. 3. The interactionsbetween various entities of BMS work as follows:

1 A HAS client initializes its streaming session andrequests the manifest files from the MSO servers.Simultaneously, the BMA is collecting the required datafrom all HAS clients (i.e., client statuses), CP (HASserver, OSS/BSS subscriber management server, andheadend/video hubs) and CMTS.

2 During each segment request, the BMA computes theoptimal joint representation decision for each client orgroup of clients according to the service type requestedand the QoE objective function (See Section IV-D).Thereupon, it sends the representation recommenda-tions for the next segments to the corresponding clientsusing DASH event messages. Then, every client usesits recommendation as an upper bound during the ABRprocess.

3 Concurrently the BMA uses its Bandwidth Reservationcomponent to generate a data structure of the QoSrequirements which represent the per-session or per-group bandwidth allocation (via the QoE-to-QoS map-ping model). Thereafter, it forwards these QoS require-ments to the SDN controller through a REST API forproper QoS provisioning.

4 The SDN controller receives the QoS requirementsand forwards them to the SDN-based bandwidth



management component. This component buildsOpenFlow messages that contain the bandwidth alloca-tion values. It then sends these messages to the CMTSfor proper bandwidth slicing.

5 The forwarding devices install the per-session or per-group QoS requirement rules. Periodically, these rulescan be updated or deleted according to OpenFlowmessage types and their timeout.

The above steps are repeated until the streaming sessionends. It is important to note that the communication betweenBMS entities is performed by leveraging existing interfacesand messages. Thus, the system does not produce additionaloverhead that could impact the performance of the network.

2) Bandwidth Allocation: The BMS triggers this mech-anism when the client does not implement the DecisionRecommendation Receiver within its ABR logic. In this case,the client uses its ABR scheme to independently select thebitrate levels. Accordingly, the BMA is not required to send anyexplicit representation recommendations to the correspondingclients. However, BMS finds the optimal bandwidth allocationdecisions and then follows steps 3 , 4 and 5 to install andprovision the optimal bandwidth allocation computed for theseclients, thus forcing these HAS clients not to exceed theirmaximum bandwidth slice during the ABR process.

C. QoE Function and Metrics

Quality of experience (QoE) represents the level of usersatisfaction. It may be computed using subjective or objectivemeasures. Moreover, viewer QoE is usually expressed usinga Mean Opinion Score (MOS) ranging from 1 (‘Bad’) to 5(‘Excellent’) [37]. In the literature on viewer QoE, severalobjective models have been proposed to measure the satisfac-tion of a client [9]. Recent studies in viewer QoE for HASdelivery systems [4], [9], [38] have clearly identified four crit-ical metrics that can significantly impact the client satisfaction,namely: the average video quality (AvgQT), the startup delay(ST), the average of the quality oscillations (AvgQO), and thenumber of stall events with their durations (SE). The QoEfunction [4], [38] of client pu is computed as follows:

QoEipu

= δ1AvgQTipu

− δ2AvgQOipu

− δ3SEipu

− δ4STipu

, (5)

where:⎧⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎩

AvgQTipu

= ∑Ki=1 qi

pu

(lipu

, ripu

),

AvgQOipu

= ∑K−1i=1

∣∣∣qi+1

pu

(li+1pu

, ri+1pu

)− qi

pu

(lipu

, ripu

)∣∣∣,

SEipu

= Stalls(i, pu),

STipu

= StartupDelay(i, pu).

(6)

Here K is the total number of segments, i ∈ [1, . . . , K] isthe current step (or segment to be downloaded), q(l, r) isthe SSIMplus-based non-linear relationship function betweenbitrate level, perceptual quality and content resolution adoptedfrom [4] and [11], and

∑4m=1 δm = 1 are non-negative factors.

To ensure a trade-off between QoE metrics, we set thesefactors to be equal (0.25 each). Equation (5) returns a valuethat is in the range between zero and one. For the MOSrange, we scale it up to the range between one and fiveusing a normalized QoE as shown in [4]. For per-group QoE

optimizations (i.e., per-channel or per-cluster), we definethe average QoE of the HAS clients belonging to the samecluster or channel as:

AvgQoEig =

∑Ngu=1 QoEi

pu

Ng, (7)

where, g is a channel or cluster, and Ng is the total numberof HAS clients in g with u = [1, . . . , Ng].

D. Objective Functions

We formulate the joint representation decisions for viewerQoE maximization as a convex optimization problem whichrelies on a concave network utility maximization (NUM) func-tion. Our formulation finds the optimal decisions (i.e., optimalrepresentation q�(l�, r�) and bandwidth allocation BWalloc,�)that maximize the viewer QoE. We introduce three objectivefunctions, namely: per-session, per-channel, and per-clusterQoE optimizations. Our objective functions are formulatedcarefully to be strictly monotonically increasing concave andflexible enough to accommodate a set of dynamic constraintsincluding:C.1 Buffer Occupancy: The selected joint representation

decisions should avoid buffer occupancy underruns oroverflows and maintain the data level between two pre-defined thresholds (Bmin and Bmax). For a per-clusteroptimization, we define (C.1a) AvgBclα , AvgBmin, andAvgBmax as the average of the buffer occupancy, thebuffer minimum and the buffer maximum thresholds ofHAS clients grouped in clα ∈ CL, respectively.

C.2 Total Capacity: The sum of the selected decisions forall clients should not exceed the total capacity of thenetwork with respect to the fact that li,�pu

≈ BWi,alloc,�pu

.We define two additional constraints for per-channel andper-cluster optimizations, namely that (C2a) the sum ofthe selected decisions for the HAS clients in the samechannel or cluster should not exceed the bandwidth allo-cated for the corresponding channel or cluster, and that(C2b) the sum of the bandwidth allocated for each chan-nel or cluster and their sum should not surpass the totalnetwork capacity.

C.3 Mapping: The selected decisions should fit and satisfyeach client’s device resolution, requested content type,and subscription plan type. In addition, we define (C.3a)the mapping function for the used business strategy bythe MSO for per-cluster optimization. For example, fora payment-based strategy, the selected recommendationsfor clients who paid for platinum service should be highcompared to other service plan types (this relies on theMAP Model component (See Section III) and businessstrategy).

C.4 Segment Downloading Time: The selected decisionsshould not exceed the throughput estimations for eachplayer. To avoid any inaccuracy in a player’s through-put estimation, we use the segment downloading timewhere the maximum required time (ti,max(li,�. )) to down-load segment i with bitrate level li,�. should not exceedthe segment duration (τ ). We note that we define (C.4a)



Avgbwiclα

which is the average throughput estimations ofHAS clients in cluster clα ∈ CL.

C.5 Available Bandwidth and Congestion Level: The sumof the throughput estimations of HAS clients should notexceed the total available bandwidth BWe that is esti-mated by BMA. This is essential to avoid issues whenthe cross traffic increases in the network. Furthermore,to alleviate any inefficiencies in bandwidth utilizationin the presence of cross traffic, we include the conges-tion constraint C = (BWHAS + BWcross)/BWtotal, whereBWHAS and BWcross are the sum of the bandwidth allo-cated for HAS clients and the bandwidth required for thecross traffic, respectively. When C ≈ 1 then the band-width is fully utilized. When C > 1 or C � 1 then thenetwork is oversubscribed or underutilized, respectively.

Depending on the service type selected by the HAS clients,and the business strategy applied by the MSO, the objectivefunction to solve is defined accordingly. At each step i, foreach client pu ∈ P, the per-session objective function is definedas:⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩

find qi,�pu

(li,�pu

, ri,�pu

)& BWi,alloc,�

pu,∀i,∀pu

arg maxqi∈Qv,li∈Lv,ri∈Rv

QoEipu

s.t. Bminpu

≤ Bipu

≤ Bmaxpu

C.1NP∑

u=1

(li,�pu

, BWi,alloc,�pu

)≤ BWtotal C.2

MAP(

qi,�pu

(li,�pu

, ri,�pu

),{

CTpu, DRpu , SPTpu

})C.3

li,�pu≤ bwi

pu⇔ ti,max

(li,�pu

)≤ τ C.4

NP∑

u=1bwi

pn≈ BWi,e, Ci ≤ 1 C.5

(8)

The above formulation aims to find the per-session optimaljoint representation qi,�

pu(li,�pu

, ri,�pu

) and the respective bandwidthallocation BWi,alloc,�

pudecisions that maximize the per-session

viewer QoE.Analogous, in the per-channel QoE optimization, BMS

aims to find the per-channel optimal joint representationdecisions that optimize the per-channel viewer QoE (i.e.,arg max

qi∈Qv,li∈Lv,ri∈RvQoEi

pu, AvgQoEi

chm). In this case, the group

of clients connecting to the same channel will receive thesame bandwidth slice BWi,alloc

chm, which is divided between

them based on the business strategy and the viewer QoE ofeach. The MSO provides NCH channels, and the HAS clientsrequest these channels and switch between them based ona Zipf distribution (See (4)). At each step i, for each client∀pu ∈ chm, ∀chm ∈ CH, where u = [1, . . . , Nchm ], Nchm

denotes the total number of clients connecting to the chan-nel chm. The per-channel objective function differs from (8)in the C.2 constraint, where:

C.2 :

⎧⎪⎪⎪⎨

⎪⎪⎪⎩

Nchm∑

u=1

(li,�pu

, BWi,alloc,�pu

)≤ BWi,alloc

chmC.2a

NCH∑

m=1BWi,alloc

chm≤ BWtotal C.2b

The per-cluster QoE optimization is performed when theMSO applies a business strategy to group the HAS clientsinto a set of clusters denoted as CL, where the clients thatshare similar features such as CT , SPT , DR, IP address, ISP,etc., will be gathered into the same cluster clα ∈ CL withα = [1, . . . , NCL] and NCL and Nclα denoting the total num-ber of the created clusters and the number of HAS clients incluster clα , respectively. In BMS, we integrate five businessstrategies (i.e., clustering-based logical structures) including:(i) payment-based: higher paying customers (e.g., platinum)receive better quality, (ii) CT-based: premium or popular con-tent is delivered with better quality, (iii) DR-based: deviceswith higher screen resolutions receive better quality, (iv)SSIMplus mapping-based (MAPSSIMplus(.)) [11]: CTpu , DRpu

and SPTpu values of each client are mapped into one commonSSIMplus-based space, where the HAS clients are groupedinto five non-overlapping clusters, and (v) based on the CS2Pcluster identification algorithm from [24]: video sessions ofthe HAS clients that match based on defined characteristicssuch as IP, ISP, region, city and MSO are aggregated.

In the above cases, BMS solves the QoE optimizationproblem for each cluster (i.e., arg max

qi∈Qv,li∈Lv,ri∈RvAvgQoEi

clα,

with the average QoE of the clients within the clusterclα represented), where the clients within the same clus-ter receive a similar per-cluster optimal joint representationqi,�

clα(li,�clα

, ri,�clα

) and respective bandwidth allocation BWi,alloc,�clα

decision. Furthermore, the bandwidth slice reserved for eachcluster BWi,alloc,�

clαwill be divided equally among its clients.

Thus, the per-cluster objective function differs from (8) in theC.1, C.2, C.3 and C.4 constraints, expressed formally as:⎧⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎨

⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎪⎩

C.1: AvgBminclα

≤ AvgBiclα

≤ AvgBmaxclα

C.1a

C.2:

⎧⎪⎪⎪⎨

⎪⎪⎪⎩

Nclα∑

u=1

(li,�pu

, BWi,alloc,�pu

)≤ BWi,alloc,�

clαC.2a

NCL∑

α=1BWi,alloc,�

clα≤ BWtotal C.2b

C.3: MAP(

qi,�clα

(li,�clα, ri,�

clα), {Business Strategy}

)C.3a

C.4: li,�clα≤ Avgbwi

clα⇔ ti,max

(li,�clα

)≤ τ C.4a

The per-cluster and per-channel optimizations are jointlycalled per-group QoE optimizations, but they have differentformulations. We solve the above-described objective func-tions using the convex relaxation mechanism [12] of onlinedecomposition methods [39], [40]. To improve the decisionswhile avoiding suboptimal solutions, we leverage fast modelpredictive control (fastMPC) [26] (it helps BMS to more accu-rately estimate the network bandwidth variations for the nextfew seconds) and the SDN optimization language (SOL) [25].We note that during the solving of the objective function, ifDRp and SPTp of any client p are unknown, BMS selects theirdefault values which are 240p and basic, respectively. Further,CTp can be easily obtained from the MPD of the requestedvideo content.

V. EXPERIMENTAL EVALUATION

We evaluate the performance and feasibility of BMSthrough trace-driven and emulation-based experiments. We



TABLE IABR AND BANDWIDTH ALLOCATION COMBINATIONS

USED IN THE EXPERIMENTS

have implemented a modified dash.js [30] player calledBMS.js (based on the stable release v2.5), which is fullyintegrated with BMS and has the ability to use the deci-sion recommendations sent by BMA. Its ABR logic usesboth buffer occupancy and throughput estimation heuristics.Our experiments cover a broad range of streaming servicetypes (e.g., VoD) with a different number of heterogeneousHAS clients, various QoE metrics, throughput variability pro-files, CT , SPT , DR, video parameters, and test environments(e.g., mobile and fixed clients, and arbitrary client arrival anddeparture times).

A. Methodology

1) Throughput Variability: We use throughput variabil-ity profiles from [38] including FCC broadband and DASHIndustry Forum datasets as well as synthetic traces to gener-ate a dynamic cross traffic in the network that ranges from 10to 25 Mbps. Synthetic traces follow a Markovian model, inparticular, Poisson and Gaussian distributions.

2) ABR Schemes: We compare BMS.js against BBA [41],ELASTIC [42], QDASH [17], Rate [30] and BOLA [43] thatwe all implemented in dash.js with their original configurationsas described in their respective papers. These schemes wereselected to cover a full spectrum of ABR heuristics.

3) Bandwidth Allocation Schemes: We implement fourvariants of bandwidth allocation schemes:

• Traditional laissez-faire (TLF) represents the baselinescheme used in most forwarding devices nowadays. Itdoes not apply any bandwidth allocation on concurrentstreaming clients, and it works under the assumption ofproviding more bandwidth if needed [3], [44].

• Equal-bandwidth (EB) allocates bandwidth equallyamong all the HAS clients.

• BMS with bandwidth allocation (BMSba) only allo-cates the bandwidth for competing clients based on thecorresponding objective function.

• BMS with representation assistance and bandwidth allo-cation (BMSraba) sends both the recommendations andallocates the bandwidth based on the correspondingobjective function.

Table I describes the set of ABR and bandwidth allocationcombinations used in the experiments. For example, BBA-BA(ELASTIC-BA) indicates that BBA (ELASTIC) is combinedwith BMSba.

4) Video Samples and Parameters: We use five videocontent types along with their MPD and quality manifestfiles from the DASH video dataset [34]: animation (BigBuck Bunny), sports (Red Bull Play Streets), documen-tary (Of Forests And Men), movie (Tears of Steel) and

TABLE IIINPUT PARAMETERS USED IN THE EXPERIMENTS

news. These videos are encoded at 20 bitrate levels Lv ={50, 100, 150, 200, 250, 300, 400, 500, 600, 700, 900, 1200,

1500, 2000, 2100, 2400, 2900, 3300, 3600, 3900} Kbps andsupport five resolutions Rv = {240, 360, 480, 720, 1080}p.The SSIMplus-based perceptual quality manifest files Qv areobtained from [11]. Each video consists of 150 segments witha total duration of 600 seconds (τ = 4 seconds per segment).We selected these five content types for two reasons. First,most of the existing work in the HAS and DASH commu-nity uses the DASH dataset [34]. Thus, the use of a populardataset adds to the credibility of our results. Second, the fivetypes are designed to cover a broad spectrum (i.e., most gen-res) of videos. Such diversity is useful to cover different scenecharacteristics (i.e., high motion in sports, low color noise inanimation, etc.).

5) Experimental Setup and Network Topology: The testbedsetup consists of four Ubuntu 16.04 LTS computers. The firstmachine runs the HAS server with an Apache HTTP server(v2.4); the second one hosts the SDN controller and relatedapplications; the third computer executes 100 heterogeneousdash.js HAS clients on Google’s Chrome browser (v60) withSelenium (http://www.seleniumhq.org/) and Xvfb (to allow aheadless browser); and the fourth machine uses Mininet [45]to implement a virtualized SDN/CMTS infrastructure plane bycreating three Lagopus [33] vSwitches with OpenFlow v1.3,meter tables, and CMTS capabilities support. All machinesare connected to a gigabit Cisco router. We use iPerf3 [46]to generate dynamic cross traffic according to the through-put profiles and tc-NetEm [47] to apply bandwidth throttlingas necessary, with a maximal HAS server capacity limit of170 Mbps. Table II summarizes the input parameters.

6) Performance Metrics: The performance is evaluated bycomparing the average results over all the clients in terms ofQoE metrics (5), video stability, QoE fairness, and utilization.We also obtain the instability, unfairness and underutilizationindexes that are defined in [7] and [8], where the best solutionsachieve an average index value that approaches zero over all



Fig. 4. Experimental results for video stability.

the clients. Each experiment is executed ten times for betteraccuracy.

B. Results and Analysis

We start by first comparing the performance of BMS.jswith existing ABR schemes and then comparing BMS to exist-ing bandwidth allocation schemes, with results depicted inFigs. 4, 5, 6 and 7. Table III summarizes the overall averageresults regarding the performance metrics.

1) Video Stability: We evaluate the video stability in termsof both bitrate and (perceptual) quality changes over time dur-ing a streaming session. Bitrate changes are due to a clientswitching between the available video representations, and thequality changes are due to the quality differences between thesegments that belong to the same or different representations.

In Fig. 4a, the left plot shows the bar plot for the bitratesachieved by each tested scheme over 10 runs where the bottomedge, bar and top edge indicate the 5th percentile, mean and95th percentile values, respectively. The middle and right plotsillustrate how the bitrates vary over time in a typical run forthese schemes. Similarly, in Fig. 4b, the left plot depicts thebox plot for the rendered qualities achieved by each testedscheme over 10 runs, where the central mark indicates themedian, and the bottom and top edges of each box indicatethe 25th and 75th percentiles, respectively, while the red dotsare outliers. The middle and right plots show how the renderedquality varies over time in one of the runs. Figs. 7a and 7dshow the respective instability indexes.

We observe that BMS.js significantly outperforms theexisting ABR schemes. One interesting observation is thatboth BBA-BA and ELASTIC-BA provide the second bestperformance with a significant improvement compared to theirbaseline schemes (BBA and ELASTIC). This is because inboth schemes, BMSba is enabled to allocate and provision theoptimal bandwidth for BBA-BA and ELASTIC-BA clients.Unsurprisingly, the ABR schemes that use EB and TLF band-width schemes perform poorly with many bitrate changes andfrequent quality oscillations, leading to low overall viewerQoE. BMS.js offers a stable bitrate level that stays withina band of 1.4 to 1.6 Mbps, and a very low instability indexof 0.001. Also, it ensures a good video delivery with a highand consistent quality that ranges from 0.91 to 0.92. BMS.jsoutperforms the existing ABR schemes on average in therange between 15% to 70% concerning the video stability.We expected this results because BMS.js leverages BMSraba

that provides full capabilities and support for BMS includingthe optimal joint representation decisions for every client.

Regarding the bandwidth allocation schemes, similar resultsare obtained as depicted in Figs. 4c and 4d, where BMS clientsachieve the best video stability compared to EB and TLFclients. BMSraba outperforms BMSba by 8%. This is because,BMSraba additionally uses representation recommendations toassist the clients. Both BMSraba and BMSba reduce the bitrateand quality changes significantly compared to EB and TLF.

2) QoE and Fairness: The average results related toviewer QoE are calculated using (5) and are illustrated inFigs. 5a, 5b, 5c and 7b for the ABR schemes, and in



Fig. 5. Experimental results for QoE and fairness.

Fig. 6. Experimental results for bandwidth utilization and congestion level.

Figs. 5d and 7e for the bandwidth allocation schemes. Fig. 5cpresents a heat-map view of the normalized QoE of 100BMS.js clients for the entire streaming session. It clearlyshows a fair and high QoE among heterogeneous compet-ing clients with an average value of 4.72, which distinctlyindicates that BMS alleviates unfairness issues by optimizingthe per-session QoE instead of dividing the available band-width equally between HAS clients. This eventually leads toa smooth overall QoE across multiple clients as depicted inFig. 7b, where BMS.js clients achieve the lowest unfair-ness index with an average value of 0.014. Figs. 5a and 5bshow that BMS.js clients achieve the highest average QoE of4.72 followed by BBA-BA and ELASTIC-BA with averagesof 4.23 and 3.92, respectively. BOLA-EB and QDASH-EBclients which use the EB scheme perform almost as poorlyas the other ABR schemes that use the TLF scheme in

terms of viewer QoE. This is because the EB scheme cannotwork effectively in heterogeneous environments and results insignificant variations in viewer QoE as tabulated in Table III.

As for the bandwidth allocation schemes, BMS is able todeliver the highest viewer QoE with a significant improvementas shown in Figs. 5d and 7e. BMSraba outperforms BMSba by7%, which outperforms the EB and TLF schemes by 32% and39%, respectively.

3) QoE Metrics: A major goal for BMS is to minimizethe factors that negatively affect viewer QoE, including thenumber of stalls, the overall stall duration, the frequency andnumber of bitrate and quality changes, and the startup delay,whose results are summarized in Table III. For a streamingsession of 10 minutes, BMS.js clients are able to reduce thenumber of stall events to an average of two, the stall durationto 1.1 seconds, and the number of quality oscillations to three.



Fig. 7. Instability, unfairness and underutilization indexes for different ABR (top) and bandwidth allocation (bottom) schemes.

TABLE IIIAVERAGE RESULTS FOR ABR AND BANDWIDTH ALLOCATION SCHEMES. +X% ≡ THE SCHEME EXCEEDS

THE TOTAL CAPACITY BY X%. METRIC DEFINITIONS CAN BE FOUND IN [4] AND [11]

BMS.js clients achieved an average of two seconds for thestartup delay. On the other hand, BBA-BA and ELASTIC-BAare the runner-ups by obtaining an average number of stallevents of five and three, stall durations of 4 seconds and 6.5seconds, the number of quality oscillations of five and seven,and a startup delay of 2.5 seconds and 4.1 seconds, respec-tively. BBA-BA (ELASTIC-BA) achieves approximately 25%(30%) QoE improvement compared to its baseline schemeBBA (ELASTIC), while BMS.js outperforms both by 10%and 15% in terms of QoE, respectively. Other ABR schemessuch as ELASTIC and BOLA-EB prefer to select the highestbitrate level without considering buffer occupancy or through-put variability, and thus, are more susceptible to frequent andlonger stalls.

4) Utilization and Congestion: We have evaluated the totalutilization of bandwidth by HAS clients in the presenceof dynamic and random cross traffic. Figs. 6a, 6b and 7cshow the total bandwidth utilization including both HASand cross traffic, the congestion level and the underutiliza-tion index, respectively, for different ABR schemes. Similarly,Figs. 6c, 6d and 7f show the total bandwidth utilization,

congestion level and underutilization index, respectively, fordifferent bandwidth allocation schemes.BMS.js clients, and the BMSraba and BMSba schemes,

almost perfectly use the full bandwidth (over 95%) withoutintroducing oversubscription issues during the entire stream-ing session. They ensure the best congestion level, which isnear to one (See Section IV-D), and a low average under-utilization index of 0.015. BBA-BA (ELASTIC-BA) improvesthe bandwidth utilization by 7% (33%) with a reasonable con-gestion level, and reduces the underutilization index by 21%(53%) compared to its baseline scheme BBA (ELASTIC). Incontrast, other ABR schemes and the EB and TLF schemesinefficiently use the total capacity, resulting in poor congestionlevels and higher underutilization index values.

BMS-based clients are able to eliminate bandwidth utiliza-tion related issues, and outperform all the existing ABR andbandwidth allocation schemes by 36% on average because ofthe following reasons:

• The efficient application (BMA) and allocation schemesprovided by BMS select optimal joint representation (andthe respective bandwidth allocation) decisions.



• The comprehensive view of the network and client sta-tuses in BMS helps the PANDA algorithm to accuratelyestimate the total available bandwidth.

• The fastMPC mechanism allows BMS to estimate thenetwork condition dynamics for a few seconds into thefuture. Hence, it avoids any inefficiencies triggered bysudden and long-term bandwidth fluctuations.

C. Discussion

In the following, we present preliminary insights on howBMS performs to address some interesting real-world issues.

1) BMS Scalability: In the presence of multiple CMTSes,HFC networks and heterogeneous clients, scalability for asystem like BMS is a clear concern. We performed severalexperiments with different input parameters such as the num-ber of clients (ranging from 50 to 2,000), cross traffic (rangingfrom 10 to 300 Mbps), and total capacity (ranging from 100to 5,000 Mbps). In all considered scenarios, BMS showedgood efficiency. In future work we will further investigate theperformance of BMS in a large-scale testbed and possibly ina production network. We note that BMS leverages the exist-ing communication interfaces in HAS systems, limiting theoverhead and the impact on the network.

2) Arbitrary Joins and Leaves: At any time, any clientmay start or stop a streaming session. A natural questionis how BMS would perform in such a situation. BMS iscapable of detecting new sessions quickly and identifyingvoluntary or involuntary session terminations after a shortdelay. We performed a small experiment with 20 clientsthat shared a 20 Mbps total capacity, with staggered ses-sion starts at 60-second intervals. BMS.js clients achievedan average of 26%, 22%, 32% and 30% improvement inthe normalized viewer QoE, video stability, QoE fairnessand bandwidth utilization, respectively, compared to BBA,QDASH and dash.js.

3) BMS Deployment: Our solution is designed to be mod-ular to be easily deployed in the existing production networks.BMS integrates many components that are simple to extendand modify. To deploy BMS in the existing networks threeessential requirements need to be met: (i) An SDN-enablednetwork where the forwarding devices support OpenFlowv1.3 capabilities. (ii) Allowing the HFC network entitiesincluding OSS/BSS Subscriber Management, HAS servers,Headend/Video Hub, and HAS clients to interact with BMS.To simplify such interaction, BMS implements different APIsand interfaces to communicate with the existing controllersand components. (iii) Adding quality manifest files for eachvideo to the HAS server(s).

VI. CONCLUSION

In HFC networks, designing a good ABR scheme is not nec-essarily sufficient without a central controller that can assistthe HAS clients in their decisions and allocate suitable band-width slices. To bridge this gap, we proposed a QoE-awareSDN-based bandwidth broker and management solution forHAS flows in an HFC network, called BMS, which dynami-cally selects the optimal joint representation and the respective

bandwidth allocation decisions to meet the per-session andper-group QoE objectives. Since an HFC network is usuallyunder the control of a single corporation, it is very feasibleto implement such an SDN-based approach. BMS successfullydelivers a good QoE with high video stability and equal qualityto all the HAS sessions while also achieving a top network uti-lization. We investigated the efficiency and robustness of theproposed solution through trace-driven and emulation-basedexperiments. The results indicate that our solution outper-forms existing ABR and bandwidth allocation schemes andnicely scales up. As part of our future work, we plan to (i)deploy BMS in a realistic SDN-enabled test network (NCL,https://ncl.sg/), and (ii) extend our solution to support othertypes of broadcast networks.

REFERENCES

[1] “Cisco visual networking index forecast and methodology 2016–2021,”San Jose, CA, USA, Cisco, White Paper, 2016.

[2] A. Begen, T. Akgul, and M. Baugher, “Watching video over the Web:Part 1: Streaming protocols,” IEEE Internet Comput., vol. 15, no. 2,pp. 54–63, Mar./Apr. 2011.

[3] S. Akhshabi, L. Anantakrishnan, A. C. Begen, and C. Dovrolis, “Whathappens when HTTP adaptive streaming players compete for band-width?” in Proc. ACM NOSSDAV, Toronto, ON, Canada, 2012, pp. 9–14.

[4] A. Bentaleb, A. C. Begen, and R. Zimmermann, “SDNDASH: ImprovingQoE of HTTP adaptive streaming using software defined networking,”in Proc. ACM MM, Amsterdam, The Netherlands, 2016, pp. 1296–1305.

[5] S. Akhshabi, L. Anantakrishnan, C. Dovrolis, and A. C. Begen, “Server-based traffic shaping for stabilizing oscillating adaptive streamingplayers,” in Proc. ACM NOSSDAV, Oslo, Norway, 2013, pp. 19–24.

[6] A. C. Begen, “Quality-aware HTTP adaptive streaming,” in Proc. IETInt. Broadcast. Conv. Conf. (IBC), 2015, pp. 1–8.

[7] Z. Li et al., “Probe and adapt: Rate adaptation for HTTP video streamingat scale,” IEEE J. Sel. Areas Commun., vol. 32, no. 4, pp. 719–733,Apr. 2014.

[8] J. Jiang, V. Sekar, and H. Zhang, “Improving fairness, efficiency, andstability in HTTP-based adaptive video streaming with festive,” in Proc.ACM CoNEXT, Nice, France, 2012, pp. 97–108.

[9] M. Seufert et al., “A survey on quality of experience of HTTP adaptivestreaming,” IEEE Commun. Surveys Tuts., vol. 17, no. 1, pp. 469–492,1st Quart., 2015.

[10] D. Kreutz et al., “Software-defined networking: A comprehensivesurvey,” Proc. IEEE, vol. 103, no. 1, pp. 14–76, Jan. 2015.

[11] A. Bentaleb, A. C. Begen, R. Zimmermann, and S. Harous, “SDNHAS:An SDN-enabled architecture to optimize QoE in HTTP adaptivestreaming,” IEEE Trans. Multimedia, vol. 19, no. 10, pp. 2136–2151,Oct. 2017.

[12] D. P. Palomar and M. Chiang, “A tutorial on decomposition methodsfor network utility maximization,” IEEE J. Sel. Areas Commun., vol. 24,no. 8, pp. 1439–1451, Aug. 2006.

[13] R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction.Cambridge, MA, USA: MIT Press, 2011.

[14] D. Tian, H. Yang, and J. M. Boyce, “Statmux method for broadcasting,”U.S. Patent 13/515M 509, Oct. 4, 2012.

[15] M. Rezaei, I. Bouazizi, and M. Gabbouj, “Implementing statisticalmultiplexing in DVB-H,” Int. J. Digit. Multimedia Broadcast., vol. 2009,Apr. 2009, Art. no. 261231.

[16] J. Kua, G. Armitage, and P. Branch, “A survey of rate adaptation tech-niques for dynamic adaptive streaming over HTTP,” IEEE Commun.Surveys Tuts., vol. 19, no. 3, pp. 1842–1866, 3rd Quart., 2017.

[17] R. K. P. Mok, X. Luo, E. W. W. Chan, and R. K. C. Chang, “QDASH:A QoE-aware DASH system,” in Proc. ACM MMSys, Chapel Hill, NC,USA, 2012, pp. 11–22.

[18] J. Jiang, S. Sun, V. Sekar, and H. Zhang, “Pytheas: Enabling data-driven quality of experience optimization using group-based exploration-exploitation,” in Proc. NSDI, Boston, MA, USA, 2017, pp. 393–406.

[19] E. Thomas, M. O. van Deventer, T. Stockhammer, A. C. Begen, andJ. Famaey, “Enhancing MPEG DASH performance via server andnetwork assistance,” IEEE SMPTE Motion Imag. J., vol. 126, no. 1,pp. 22–27, Jan./Feb. 2017.



[20] G. Cofano et al., “Design and performance evaluation of network-assisted control strategies for HTTP adaptive streaming,” ACM Trans.Multimedia Comput. Commun. Appl., vol. 13, no. 3s, p. 42, 2017.

[21] S. Ramakrishnan, X. Zhu, F. Chan, and K. Kambhatla, “SDN based QoEoptimization for HTTP-based adaptive video streaming,” in Proc. IEEEISM, Miami, FL, USA, 2015, pp. 120–123.

[22] H. Nam, K.-H. Kim, J. Y. Kim, and H. Schulzrinne, “Towards QoE-aware video streaming using SDN,” in Proc. IEEE GLOBECOM, Austin,TX, USA, 2014, pp. 1317–1322.

[23] Z. Duanmu, K. Zeng, K. Ma, A. Rehman, and Z. Wang, “A quality-of-experience index for streaming video,” IEEE J. Sel. Topics SignalProcess., vol. 11, no. 1, pp. 154–166, Feb. 2017.

[24] Y. Sun et al., “CS2P: Improving video bitrate selection and adapta-tion with data-driven throughput prediction,” in Proc. ACM SIGCOMM,Florianópolis, Brazil, 2016, pp. 272–285.

[25] V. Heorhiadi, M. K. Reiter, and V. Sekar, “Simplifying software-definednetwork optimization using SOL,” in Proc. NSDI, Santa Clara, CA,USA, 2016, pp. 223–237.

[26] Y. Wang and S. Boyd, “Fast model predictive control using onlineoptimization,” IEEE Trans. Control Syst. Technol., vol. 18, no. 2,pp. 267–278, Mar. 2010.

[27] InMon. (2016). sFlow. [Online]. Available: http://sflow.org/[28] M. Fiedler, T. Hossfeld, and P. Tran-Gia, “A generic quantitative rela-

tionship between quality of experience and quality of service,” IEEENetw., vol. 24, no. 2, pp. 36–41, Mar./Apr. 2010.

[29] Digital Video Broadcasting (DVB), MPEG-DASH Profile for Transportof ISO BMFF Based DVB Services Over IP Based Networks,IEEE Standard ETSI-TS 103 285, p. 72, 2015. [Online]. Available:https://standards.globalspec.com/std/9928399/etsi-ts-103-285

[30] DASH Ref. Player. (2017). [Online]. Available: https://goo.gl/5xM3tV[31] N. McKeown et al., “OpenFlow: Enabling innovation in campus

networks,” ACM SIGCOMM Comput. Commun. Rev., vol. 38, no. 2,pp. 69–74, 2008.

[32] RYU. (2015). RYU SDN Framework. [Online]. Available:https://osrg.github.io/ryu/

[33] H. Kimiyama, M. Kitamura, M. Maruyama, and T. Fujii, “High-resolution video transmission network system using dynamic SDN,” inProc. IEEE APCC, Kyotom, Japan, 2015, pp. 338–342.

[34] S. Lederer, C. Müller, and C. Timmerer, “Dynamic adaptive streamingover HTTP dataset,” in Proc. ACM MMSys, Chapel Hill, NC, USA,2012, pp. 89–94.

[35] M. Cha, P. Rodriguez, J. Crowcroft, S. Moon, and X. Amatriain,“Watching television over an IP network,” in Proc. ACM IMC,Vouliagmeni, Greece, 2008, pp. 71–84.

[36] Z. Li et al., “Streaming video over HTTP with consistent quality,” inProc. ACM MMSys, Singapore, 2014, pp. 248–258.

[37] “Methods for subjective determination of transmission quality,”Int. Telecommun. Union, Geneva, Switzerland, ITU-RecommendationP. 800, 1996.

[38] X. Yin, A. Jindal, V. Sekar, and B. Sinopoli, “A control-theoreticapproach for dynamic adaptive video streaming over HTTP,” in Proc.ACM SIGCOMM, London, U.K., 2015, pp. 325–338.

[39] E. J. Candes and T. Tao, “The power of convex relaxation: Near-optimal matrix completion,” IEEE Trans. Inf. Theory, vol. 56, no. 5,pp. 2053–2080, May 2010.

[40] T. Chen, Q. Ling, and G. B. Giannakis, “Online convex optimizationfor dynamic network resource allocation,” in Proc. EUSIPCO, 2017,pp. 136–140.

[41] T.-Y. Huang, R. Johari, N. McKeown, M. Trunnell, and M. Watson, “Abuffer-based approach to rate adaptation: Evidence from a large videostreaming service,” in Proc. ACM SIGCOMM, Chicago, IL, USA, 2014,pp. 187–198.

[42] L. De Cicco, V. Caldaralo, V. Palmisano, and S. Mascolo, “ELASTIC:A client-side controller for dynamic adaptive streaming over HTTP(DASH),” in Proc. IEEE Packet Video, San Jose, CA, USA, 2013,pp. 1–8.

[43] K. Spiteri, R. Urgaonkar, and R. K. Sitaraman, “BOLA: Near-optimalbitrate adaptation for Online videos,” in Proc. IEEE INFOCOM,San Francisco, CA, USA, 2016, pp. 1–9.

[44] S. Akhshabi, A. C. Begen, and C. Dovrolis, “An experimental evaluationof rate-adaptation algorithms in adaptive streaming over HTTP,” in Proc.ACM MMSys, San Jose, CA, USA, 2011, pp. 157–168.

[45] Mininet Team. (2012). Mininet: An Instant Virtual Network on YourLaptop (or Other PC). [Online]. Available: http://mininet. org

[46] A. Tirumala, F. Qin, J. Dugan, J. Ferguson, and K. Gibbs. (2005).Iperf: The TCP/UDP Bandwidth Measurement Tool. [Online]. Available:https://iperf.fr/

[47] S. Hemminger, Network Emulation With NetEm. Canberra, ACT,Australia: Linux, 2005, pp. 18–23.

Abdelhak Bentaleb received the M.S. degreein computing (network and multimedia) fromMohamed El Bachir El Ibrahimi University,Bordj Bou Arreridj, Algeria, in 2011. He is cur-rently pursuing the Ph.D. degree in computer sciencewith the School of Computing, National Universityof Singapore, Singapore. His research interestsinclude multimedia systems and communication,video streaming architecture, content delivery, dis-tributed computing, computer networks and proto-cols, wireless communication, and mobile network.

Ali C. Begen (S’98–M’07–SM’12) received thePh.D. degree in electrical and computer engi-neering from Georgia Tech. He was a Researchand Development Engineer with Cisco, where hedesigned and developed algorithms, protocols, prod-ucts, and solutions in the service provider andenterprise video domains. Recently he joined theComputer Science Department, Özyegin University,Turkey. Currently, in addition to teaching andresearch, he provides consulting services to indus-trial, legal, and academic institutions through

Networked Media, a company he co-founded. He received a number ofscholarly and industry awards, and he has editorial positions in prestigiousmagazines and journals in the field. He is a senior member of the ACM.In 2016, he was elected distinguished lecturer by the IEEE CommunicationsSociety.

Roger Zimmermann (S’93–M’99–SM’07) receivedthe M.S. and Ph.D. degrees from the Universityof Southern California, Los Angeles, CA, USA,in 1994 and 1998, respectively. He is currentlyan Associate Professor with the Department ofComputer Science, National University of Singapore(NUS), Singapore. He is also a Deputy Directorwith the Smart Systems Institute, and previouslyco-directed the Centre of Social Media Innovationsfor Communities, NUS. He has co-authored a book,seven patents, and over 230 conference publica-

tions, journal articles, and book chapters. His research interests includestreaming media architectures, distributed systems, mobile and geo-referencedvideo management, collaborative environments, spatio-temporal informationmanagement, and mobile location-based services. He is a distinguished mem-ber of the ACM and more information can be found on his website athttp://www.comp.nus.edu.sg/∼rogerz.

Documents

IEEE TRANSACTIONS ON BROADCASTING 1 QoE-Aware …bentaleb/files/papers/journal/BMS.pdf · BENTALEB et al.: QoE-AWARE BANDWIDTH BROKER FOR HTTP ADAPTIVE STREAMING FLOWS IN SDN-ENABLED