Joint Buffering and Rate Control for Video Streaming over ... · Joint Buﬀering and Rate Control for Video Streaming over Heterogeneous Wireless Networks Lei Hua ... 5.2 Adaptive

Joint Buffering and Rate Control for Video Streaming

over Heterogeneous Wireless Networks

by

Lei Hua

A thesis submitted in conformity with the requirementsfor the degree of Master of Applied Science

Graduate Department of Electrical and Computer EngineeringUniversity of Toronto

Copyright c© 2010 by Lei Hua

Abstract

Joint Buffering and Rate Control for Video Streaming over Heterogeneous Wireless

Networks

Lei Hua

Master of Applied Science

Graduate Department of Electrical and Computer Engineering

University of Toronto

2010

The integration of heterogeneous access networks is becoming a possible feature of 4G

wireless networks. It is challenging to deliver the multimedia services over such integrated

networks because of the discrepancy in the bandwidth of different networks. This thesis

presents an adaptive approach that combines source rate adaptation and buffering to

achieve high quality VBR video streaming with less quality variation over an integrated

two-tier network. Statistical information of the residence time in each network or local-

ization information are utilized to anticipate the handoff occurrence. The performance of

this approach is analyzed under the CBR case using a Markov reward model. Simulation

under the CBR and VBR cases is conducted for different types of network models. The

results are compared with a dynamic programming algorithm as well as other naive or

intuitive algorithms, and proved to be promising.

ii

Acknowledgements

I would like to express my sincerest gratitude to my supervisor, Professor Ben Liang,

for this exciting opportunity to work under his supervision at this prestigious institu-

tion. During the whole process he provided me with invaluable guidance, inspiration and

support, without which I couldn’t have completed this work.

I am thankful to the members of my thesis committee, Prof. Elvino S. Sousa, Prof.

Raviraj Adve, and Prof. Jason H. Anderson for the time spent in reviewing my thesis,

and for their helpful feedback and comments on improving its content.

I thank all my current and former colleagues in my research group for their useful

inputs and suggestions on the research work itself and also the presentation of the work.

Special thanks to all of my friends at University of Toronto, Colin Jiang, Eric Yuan,

Junqi Yu, Lilin Zhang, Weiwei Li, Yuan Feng, Yunfeng Lin and others, whose company,

care and encouragement made the two years of Master’s studies much more enjoyable.

Last but never the least, I dedicate this thesis to my family, who are always there for

me in my life.

iii

Contents

1 Introduction 1

1.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.1 Video Streaming . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.1.2 Heterogeneous Wireless Networks . . . . . . . . . . . . . . . . . . 2

1.1.3 Buffering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.1.4 Rate Adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.1.5 Contribution of the Thesis . . . . . . . . . . . . . . . . . . . . . . 4

1.2 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

2 Literature Review 7

2.1 Video Rate Adaptation Techniques . . . . . . . . . . . . . . . . . . . . . 7

2.1.1 Transcoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.1.2 Joint Source/Channel Coding . . . . . . . . . . . . . . . . . . . . 8

2.1.3 Scalable Video Coding . . . . . . . . . . . . . . . . . . . . . . . . 9

2.1.4 Content-Aware Coding Techniques . . . . . . . . . . . . . . . . . 10

2.2 Rate Control in Heterogeneous Wireless Networks . . . . . . . . . . . . . 11

2.3 Buffering in Heterogeneous Wireless Networks . . . . . . . . . . . . . . . 12

3 Problem Statement 14

3.1 Application Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

3.2 Models and Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

iv

3.2.1 Rate Adaptation and Playback . . . . . . . . . . . . . . . . . . . 16

3.2.2 Residence Time and Rate Estimation . . . . . . . . . . . . . . . . 18

3.2.3 Feedback Control Mechanism . . . . . . . . . . . . . . . . . . . . 19

3.3 Problem Formulation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

4 Generic Network Model 22

4.1 Control Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

4.1.1 Adaptive Control Algorithm . . . . . . . . . . . . . . . . . . . . . 22

4.1.2 Simple Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

4.1.3 Mean Residual Life Based Algorithm . . . . . . . . . . . . . . . . 26

4.1.4 Simple Shaping Algorithm . . . . . . . . . . . . . . . . . . . . . . 27

4.2 Analytical Framework and Analytical Results . . . . . . . . . . . . . . . 29

4.2.1 Analytical Results for Generic Model . . . . . . . . . . . . . . . . 30

5 Markov Chain Network Model 36

5.1 Markov Decision Process Model . . . . . . . . . . . . . . . . . . . . . . . 36

5.2 Dynamic Programming Algorithm . . . . . . . . . . . . . . . . . . . . . . 38

5.3 Simulation Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

5.4 More Realistic 3-Zone Network Model . . . . . . . . . . . . . . . . . . . . 43

5.5 PH-Fitting of Residence Times . . . . . . . . . . . . . . . . . . . . . . . 45

5.6 Estimation in Adaptive Control Algorithm . . . . . . . . . . . . . . . . . 47

5.6.1 Utilizing Statistical Information . . . . . . . . . . . . . . . . . . . 47

5.6.2 Utilizing Localization Information . . . . . . . . . . . . . . . . . . 48

5.6.3 Simulation Results for 3-Zone Model . . . . . . . . . . . . . . . . 49

5.7 Simulating with VBR Network and VBR Video Stream . . . . . . . . . . 52

6 Conclusion 57

Bibliography 59

v

List of Tables

3.1 Notations in system model . . . . . . . . . . . . . . . . . . . . . . . . . . 20

4.1 Analysis parameters - 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

4.2 Analysis parameters - 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

5.1 Simulation parameters for 2-zone Markov model . . . . . . . . . . . . . . 40

5.2 Simulation parameters for 3-zone model . . . . . . . . . . . . . . . . . . . 49

5.3 Simulation parameters for VBR network and VBR video . . . . . . . . . 52

vi

List of Figures

3.1 Integrated two-tier network . . . . . . . . . . . . . . . . . . . . . . . . . 15

3.2 Relationship between the transmission sequence in time and the playback

sequence in time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

4.1 Illustration of proportional feedback controller . . . . . . . . . . . . . . . 24

4.2 Distributions of residence times . . . . . . . . . . . . . . . . . . . . . . . 32

4.3 Analysis vs simulation results: generic model, Gamma distribution - 1 . . 34

4.4 Analysis vs simulation results: generic model, Gamma distribution - 2 . . 35

5.1 DP: variation and utilization vs. α . . . . . . . . . . . . . . . . . . . . . 41

5.2 Adaptive algorithm: variation and utilization vs. β . . . . . . . . . . . . 41

5.3 Comparison between algorithms: utilization vs. variation . . . . . . . . . 42

5.4 Integrated two-tier network with 2-zone T2N . . . . . . . . . . . . . . . . 44

5.5 An example of the generated user’s moving trace . . . . . . . . . . . . . . 44

5.6 CDF’s of residence times in different zones . . . . . . . . . . . . . . . . . 46

5.7 PH-fitted Markov chain network model . . . . . . . . . . . . . . . . . . . 47

5.8 DP on 3-zone model: variation and utilization vs. α . . . . . . . . . . . . 50

5.9 Adaptive algorithm on 3-zone model: variation and utilization vs. β . . . 51

5.10 Comparison DP and AA: utilization vs. variation . . . . . . . . . . . . . 51

5.11 VBR simulation: adaptive algorithm with statistical information . . . . . 53

5.12 VBR simulation: adaptive algorithm with localization information . . . . 53

vii

5.13 VBR simulation: simple adaptive algorithm . . . . . . . . . . . . . . . . 54

5.14 Simulating VBR case - variation vs. β . . . . . . . . . . . . . . . . . . . 55

5.15 Simulating VBR case - utilization vs. β . . . . . . . . . . . . . . . . . . . 55

5.16 Simulating VBR case - utilization vs. variation . . . . . . . . . . . . . . . 56

viii

Chapter 1

Introduction

1.1 Overview

1.1.1 Video Streaming

Online video has become a mainstream medium and the single most influential factor

driving the need for increased mobile network capacity [8]. It would take 28 years to

watch the video uploaded to YouTube in the week of April 29th, 2010 [20]; HD(high defi-

nition) movies and television programs are widely available online with the help of CDNs

(Content Distribution Networks) and P2P (Peer-to-Peer) networks; video conferencing

and video phones are not the exclusive rights of large companies any more, but can be

enjoyed by individuals and families. It is then important and interesting to research on

improving video streaming techniques.

There are two types of video streaming applications: live streaming, which captures

real-time events and provides the video to users, and on-demand streaming, which offers

stored video contents. Application scenarios of live streaming include video conference,

video phone and live event broadcasting, which have stringent delay requirement. In this

thesis we consider the transmission of pre-encoded video, which is used for delivering all

kinds of published video contents and user generated contents online and is expected to

1

Chapter 1. Introduction 2

account for sixty-six percent of the world’s mobile data traffic by 2014 [21].

In comparison to other traffic flows such as Web browsing and E-mail, video streaming

has its unique characteristics and therefore may impose certain requirements on the

network. Video streaming traffic is inelastic. Unlike web browsing or file downloading,

where data can be transmitted at any rate, video streaming requires certain amount of

data to be delivered and decoded before the playback deadline. Hence it is sensitive to

variations in both bandwidth and transmission delay. Video streaming applications are

loss-tolerant. Robust coding techniques allow video to be decoded with certain loss of

data. However, this does not mean any level of loss can be tolerated. In high error-

rate networks, it is challenging to develop loss-prevention techniques for robust video

transmission.

1.1.2 Heterogeneous Wireless Networks

With the rapid growth of mobile communication technology, various wireless networking

technologies have evolved and become widely deployed all over the world, allowing people

to access the Internet with all kinds of mobile computing devices, at all times and all

places. The popular access technologies include IEEE802.11 wireless local area networks

(WLAN), WiMAX, GPRS, UMTS, and CDMA2000, etc. These technologies are hetero-

geneous in certain attributes, such as coverage area, protocol, signaling mechanism, data

rate, error rate, etc. However, it is common for the personal mobile devices (laptops,

smart phones, PDAs, digital media players) to support more than one wireless access

technologies simultaneously.

With the coexistence of heterogeneous wireless networks and the devices supporting

multiple access technologies, the integration of heterogeneous wireless networks is be-

coming a trend and is part of the 4G network design [30]. This feature allows user to

seamlessly switch among different wireless network interfaces and enjoy greatly enlarged

coverage and more reliable wireless access on a single device.


However, there are many challenges in deploying such an inter-technology roaming

environment. Active research topics on heterogeneous wireless networks involve admis-

sion control, hand-off mechanism, mobility management, traffic flow assignment, etc.

The heterogeneity of wireless access technologies also imposes great challenges on video

streaming applications running on a mobile device in the integrated network.

In heterogeneous wireless networks, handoffs inside one technology and between tech-

nologies, can cause extra delays, which exaggerates the challenge on the delay require-

ments of video streaming applications. A more substantial problem in the heterogeneous

wireless networks is that, different access networks offer different ranges of bandwidth,

which greatly exacerbates the variations in streamed video quality if we simply match

the video source rate to the available transmission rate. Hence, in this thesis we mainly

focus on reducing the variation in streamed video quality while maintain high average

quality.

1.1.3 Buffering

Two types of video streaming techniques are commonly applied in both wired and wireless

networks to combat the varying network bandwidth and delays: buffering and video rate

adaptation.

Buffering sustains the video playback when available bit rate (ABR) is low, by

prefetching and storing a certain amount of data ahead of (playback) time. With a

finite buffer size, two types of event will happen and may cause detrimental effects to the

streaming process: buffer underflow and buffer overflow. Underflow may happen when

the playback rate (data consumption rate) is higher than the transmission rate, which

leads to playback jitters (stops). Overflow may happen when the transmission rate is

higher than the playback rate, while at the same time the buffer size is small. Buffer

overflow may lead to loss of data and then playback jitters.

Another factor to consider in buffering is the initial buffering delay, i.e. the waiting


time between starting the buffering and starting the playback. There is a trade-off

between the initial buffering delay and the buffer size when we aim to provide satisfiable

video streaming service[19].

In this thesis, we consider the longer-term variations in the transmission rate in

heterogeneous wireless networks, hence we assume an infinite buffer size. Also we set the

initial buffering delay to be minimal. We are primarily interested in how much data to

buffer for the future in every time slot and at what quality should we buffer it.

1.1.4 Rate Adaptation

Rate adaptation techniques match the video source rate to the network transmission

rate, when the transmission rate is low, at the cost of lowering the perceived quality of

decoded video. Various video rate adaptation techniques have been proposed over time,

such as transcoding, joint source/channel coding, multiple file/rate switching, scalable

video coding, and content-aware coding techniques [2]. We present some of them in

Chapter 2.

While theoretically rate adaptation can ensure continuous playback as long as the

ABR is higher than the minimum required rate of the specific adaptation technique, it

introduces fluctuations in the perceived quality of the video, which can be annoying to

users. This problem is exaggerated in heterogeneous wireless networks since the variation

of ABR there is much higher than in homogeneous wireless networks.

1.1.5 Contribution of the Thesis

In this thesis, we consider the problem of streaming pre-encoded video on a moving

mobile terminal (MT) in heterogeneous wireless networks. The video source is stored in

a remote server and transmitted through the backbone network to the local access points

(AP) or base stations (BS) and then to the MT through different wireless networks. The

bottleneck of the connections always lies in the last hop (i.e. the wireless hop.) There is


a receiving buffer on the device, which is used for storing prefetched video contents.

We focus on coping with the variable ABR in heterogeneous wireless networks. The

effects of other network characteristics, such as varying end-to-end delay and high error

rate, are assumed to be resolved using any available technique. Our objectives are con-

tinuous playback, high image quality, and low variation in the perceived image quality

(or constant-quality playback).

To achieve these objectives, we propose to combine buffering and rate adaptation

techniques with prediction of certain attributes of the network. Our scheme predicts

the residence times at each individual network, then dynamically allocates the ABR to

each unit of video sequences being transmitted, hence controlling the buffer and rate

adaptation at the same time, under the constraint of fully utilized network resources.

In order to determine the optimal way to allocate the ABR, we propose an adaptive

video rate control scheme using a linear feedback control technique on a generic network

model for a two-zone network. In designing the scheme, we divide the whole streaming

process within the heterogeneous wireless networks into cycles and try to achieve local

optimality within each cycle.

To show the applicability of our scheme to any arbitrary distribution, the performance

of our proposed scheme in a simplified constant-bit-rate (CBR) network scenario with

CBR video source is evaluated in an analytical framework based on Markov chains, where

the state space is dimensioned by the normalized quality levels and the buffered lengths

of video at the end of each cycle. We associate with each state a cost being the quality

variations within the cycle and calculate the average cost per cycle. Other naive and

intuitive algorithms are also studied within the analytical framework in order to show

the advantage of our adaptive scheme.

Then, for the special case of exponential network residence times, we formulate the

streaming process into a finite-horizon controlled Markov Decision Process (MDP), and

solve the optimization problem using a dynamic programming based optimal control


algorithm. Although this method is assumed to provide the theoretical optimality, it

involves a large amount of computation, cannot deal with increasing dimensionality, and

is not applicable to more generalized residence time distributions. On the other hand,

the aforementioned adaptive algorithm is much simpler than the dynamic programming

algorithm in terms of the amount of computation involved, works with more generalized

distributions, and requires less knowledge of the network. Through simulations we show

that this scheme provides near optimal performance.

Furthermore, we develop a more realistic network model by modeling the movement

of the MT and fitting the actual residence times using Phase-Type distributions. We

also increase the number of network zones to three. We show through simulation that

our scheme also gives near-optimal performance under the new model. Furthermore, we

simulated our scheme with variable-bit-rate (VBR) networks and VBR video sources, and

explored the effects of utilizing different estimations of residence times, i.e. the statistical

information extracted from history mobility traces, and the geographical information

provided by localization service. Our algorithm proves to provide significantly improved

performance with VBR networks and VBR video source than the naive algorithms.

1.2 Thesis Outline

The thesis is organized as follows. The next chapter reviews work in the related areas

of video rate control and buffering techniques in both homogeneous and heterogeneous

wireless networks. The system setup and the problem statement are presented in Chapter

3. Chapter 4 is focused on a generic network model and the design of our adaptive

control algorithm. We also present the analytical performance evaluation framework. In

Chapter 5, we introduce the Markov Chain network model and the dynamic programming

algorithm. We also extend the system models to a 3-zone model with PH-fitted residence

times and simulate the VBR case in Chapter 5. Finally, Chapter 6 concludes the thesis.

Chapter 2

Literature Review

This chapter briefly reviews the existing research progress on video streaming technologies

on both homogeneous and heterogeneous wireless networks and the current challenges,

which motivated our research work. We first discuss some research works on video rate

adaptation in general variable-bit-rate (VBR) networks. Then we present some related

research on buffering and rate control techniques in video streaming over heterogeneous

wireless networks.

2.1 Video Rate Adaptation Techniques

Various video rate control or adaptation techniques have been proposed to combat short-

term variations in homogeneous VBR wireless channels when performing video streaming.

Specifically for streaming pre-encoded video, research focus has been put on transcod-

ing [22, 3, 11], joint source/channel coding [12, 14, 5], scalable video coding [9], and

other techniques such as content-aware or motion-aware coding [18, 7, 25, 27]. While in

commercial systems, the multiple file/rate switching techniques are widely implemented

[2].

These rate adaptation proposals work well in homogeneous wireless network where

average ABR doesn’t vary over time. However, they could not provide satisfactory per-

7

Chapter 2. Literature Review 8

formance in terms of quality variation in heterogeneous wireless networks, since purely

adapting the source bit rate to the channel bit rate will lead to a large variation of video

quality over different sub-networks.

2.1.1 Transcoding

Transcoding is a technique to adapt the video source rate through recompression. [22,

3, 11] are three examples of research studies on video rate adaptation with transcoding.

These techniques dynamically choose the quantizer used in encoding each frame or block,

and try to minimize the total distortion while matching the video source rate with the

network rate. Their focus is mainly on analyzing the specific encoding technique and

extracting the rate-distortion models. The heavy computation involved in transcoding is

its main disadvantage.

2.1.2 Joint Source/Channel Coding

The authors of [12] show that the perceptual source distortion decreases exponentially

with the increasing MPEG-2 source rate, and the perceptual distortion due to data loss

is directly proportional to the number of lost macro blocks. Hence they propose to

use Joint Source/Channel Coding (JSCC) technique, specifically adding FEC (forward

error correction) bits, to protect the data from loss. The optimal channel coding FEC

parameters can be selected according to the aforementioned relationships and the total

rate of received video stream can be controlled to minimize the total distortion.

Similarly, [14] and [5] both consider the Joint Source/Channel Coding problem with

FEC channel coding and focus on how to choose the channel coding parameters. In [14],

the authors translate the Quality of Service (QoS) requirements of the video streaming

applications into a threshold of occupancy of playback buffer. By adapting the JSCC

parameters their scheme tries to maintain a certain level of buffer occupancy to sustain

continuous playback.


In [5], a probabilistic QoS requirement, i.e. the buffer starvation probability has

been proposed. The authors use cycle-based rate control with cycles being successively

alternating between good(non-fading) and bad(fading) period, while guaranteeing an

upper bound on the probability of starvation at the playback buffer. The cycle-based

idea inspired us to divide the streaming process in heterogeneous networks into cycles, but

our “cycle” have a completely different definition from theirs in that our cycle contains

intervals of the MT residing in different sub-networks.

While the Joint Source /Channel Coding technique can adapt the source rate within

certain range, it usually involves cross-layer design with information flows across PHY

/MAC /Network layers, which might be applicable in homogeneous networks but could

become extremely complex in terms of implementation and computation in heterogeneous

networks.

2.1.3 Scalable Video Coding

The layered or scalable video coding techniques are said to be suitable for adapting

to longer-term bandwidth fluctuations [2]. There has been a substantial body of re-

search works developing efficient scalable compression techniques. A scalable extension

of H.264/AVC [31], Scalable Video Coding (SVC) [24] has been standardized, which pro-

vides scalability of temporal, spatial, quality resolution, or a combination of scalability on

these three dimensions, of a decoded video signal through adaptation of the bit stream.

Fine Granularity Scalability (FGS) coding [17] and Fine Granularity Scalability Tempo-

ral (FGST) coding [29] have also been adopted as amendments to the MPEG-4 standard.

Multiple Description Coding (MDC) [13] is another type of scalable video coding where

each description (substream) of the video stream is of equal weight and independent of

each other in contrast to the Base Layer/Enhancement Layer structure of SVC.

The authors of [10] developed a heuristic rate control algorithm for 2-layer FGS coded

video over TCP-friendly “connection”, which can achieve the same level of smoothness


over both TCP and TCP-friendly protocols. Their algorithm works with CBR coded

video and the loss model is simple. The authors of [16] proposed a stochastic dynamic

programming algorithm for VBR scalable coded video with a more realistic loss model.

The authors of [9] studied the problem of minimizing the average distortion of FGS

video under a limited transmission rate. The authors provided a framework which jointly

considers the effects of packet scheduling at the sender and the error concealment at the

receiver.

The authors of [33] explicitly considered the effect of fading in wireless channel and

develops cross-layer rate adaptation algorithm for layered video in fading channels. The

complexity in cross-layer design makes it difficult to implement even in homogeneous

wireless network.

The authors of [23] introduced a novel streaming strategy to improve the probability

of successfully stream a scalable coded video sequence by adaptively selecting the number

of layers according to mobility information in Ad-Hoc wireless networks. This is relevant

to our work in that our proposed algorithm can also utilize the mobility and location

information to predict the MT’s movement and channel status, as described in Chapter

5.

In our control scheme, we can use either the transcoding or scalable coding techniques

to perform rate adaptation. However, we assume a generic rate-quality relationship in

our model and that one cannot change the quality of a video sequence that is already

transmitted.

2.1.4 Content-Aware Coding Techniques

There exists many other video rate adaptation algorithms which try to achieve different

QoS objectives. Content-aware encoding/playout has been an interesting and contro-

versial topic for video rate adaptation, as there exists no generally accepted standard

for perceived quality of motion pictures when we consider the presentation of the actual


content, instead of quantifiable metrics such as resolution, frame rate and PSNR (Peak

Signal-to-Noise Ratio). Representative works of content-aware rate adaptation/playout

control include [18, 7, 25, 27], etc. These works try to analyze the amount of motion

or interested objects in each frame, and allocate the available bit rates unfairly among

different objects / frames to achieve best perceived quality when the network rate is not

high enough to present the full pictures.

2.2 Rate Control in Heterogeneous Wireless Net-

works

Video streaming in heterogeneous wireless networks has been a relatively new topic. Most

of the available works address the issues in architecture design and hand-off handling.

The authors of [26] analyzed the effects of handoffs on rate control and proposed a cross-

layer solution to anticipate the handoff occurrence and to adjust the data rate. They

use transport-layer dummy packets to probe the channel in their solution, while in this

thesis we propose to utilize the statistical information of the residence times and ABR

in each sub-network.

Some researchers consider video streaming in a multiple stream environment with

heterogeneous access technologies and focus on fairness or priority among all users/flows.

In [38] the authors study the rate allocation problem in streaming over wireless networks

with heterogeneous link speeds. The focus of their work is on how to allocate the rate

between multiple video streaming sessions on heterogeneous links to maximize the aver-

age quality among all users, while the quality enjoyed by a single user is not explicitly

considered.

The authors of [1] addressed the problem of flow rate control for different types of

traffic flows and heterogeneous wireless links, and employs an H-infinity optimal rate

controller to achieve efficient utilization of all channels while taking the requirements of


different flow types in to account. Both of them assume that all the access networks in

the integrated network are available all the time, while our assumption is that the user

is moving and the trajectory is not always covered by both sub-networks.

The authors of [35, 36, 37] considered the video streaming problem in an integrated

3G/WLAN network from a monetary cost point of view. Their system setting of the

heterogeneous networks is the most similar one to ours, yet the objective and control

actions are completely different. Different streaming strategies are proposed to decide

how much data to be streamed (i.e. the transmission rate) in each individual network as

well as when to hand off to the other network, so that the monetary cost of streaming the

data is minimized. While in our system model we also consider the cost effects of each

network, we assume that it is always good to fully utilize the ABR and that the MT will

switch to the higher-rate, lower-cost network whenever it is available, as this strategy is

simple and already implemented in commercial systems such as the iPhone.

It is worth mentioning that, some of the techniques mentioned above, such as [33] and

[5] have similar objectives as ours, i.e. minimizing the variations in adapted video rate

caused by variations in network transmission rate. However, in homogeneous wireless

networks, the variation caused by fading and other short-term effects are quite different

from the variation caused by handoff between different access technologies in both time

scale and magnitude. Hence these techniques cannot be directly applied to our problem.

Furthermore, because the time scale of variation in our problem is longer, we may have

more ways to predict the variation, such as utilizing geographical information.

2.3 Buffering in Heterogeneous Wireless Networks

For streaming pre-encoded video, buffering is another technique to overcome the mis-

match between video source rate and channel bit rate. By caching enough data in the

client buffer ahead of time, continuous high-rate playback can be sustained when the


channel throughput is low. Buffering schemes for streaming VBR video over heteroge-

neous wireless networks are studied in [15]. These schemes include fixed/jointly optimized

schemes based on buffering delay, buffered playout data, and playout time. Analysis on

both the jitter frequency and the buffering delay are conducted for these schemes.

However, without rate adaptation buffer underflow happens frequently when the av-

erage channel throughput is lower than the average video source rate, leading to playback

jitters. Hence we propose to combine rate adaptation with buffering for long-term varia-

tions to smooth out the streaming process in heterogeneous networks. We choose only the

playout time based buffering scheme, as when we introduce rate adaptation the buffering

delay become meaningless, and the buffered playout data become highly variable with

the changing video rate.

To the best of our knowledge, this thesis is the first work to propose the combination

of rate adaptation and buffering to address the problem of smooth video playback in

heterogeneous networks. Nevertheless, our work is inspired by and based on the related

works listed here in that the scheme shall utilize a generic rate adaptation techniques

mentioned above to perform the control actions.

Chapter 3

Problem Statement

In this chapter, we explain in details the problem we study in this thesis. We first present

the application scenario, then introduce our way modeling of the system based on this

scenario with some practical assumptions. A mathematical formulation of our problem

is then presented based on these common assumptions.

3.1 Application Scenario

We study video streaming over the heterogeneous wireless networks with the overlapping

of two networks (or two zones), as shown in Figure 3.1. The Tier-1 Network (T1N)

is assumed to provide universal coverage with low bit rate, while Tier-2 Network (T2N)

covers limited areas around the Access Points (APs), with high bit rate. In reality, T1N is

usually more costly than T2N. A proper example of T1N can be the 3G cellular network,

while T2N can be Wireless LAN. A user prefers to access the Internet through T2N due

to its high bandwidth and low cost, thus whenever he/she enters a T2N covered area,

the mobile device switches to T2N for transmission. While our scheme is independent

of lower layer (e.g. PHY/MAC layer) implementations and can actually handle the

simultaneous transmission over both sub-networks, we maintain the assumption of using

only one sub-network at the same time since it is more practical to do so in reality.

14

Chapter 3. Problem Statement 15

T1N

BS

AP

T2N

T2N

Figure 3.1: Integrated two-tier network

Two types of handoffs take place in this network: intra-technology handoff or Hori-

zontal Handoff (HHO) in which the mobile terminal (MT) switches between two Access

Points (AP) or Base Stations (BS) using the same access technology, e.g. from one

WLAN AP to another, and inter-technology handoff or Vertical Handoff (VHO), which

occurs when the MT roams between different access technologies, e.g. switching from 3G

to WLAN when entering a WLAN covered area. VHO affects different system perfor-

mance metrics, such as the signaling load, resource utilization and user perceived QoS. In

particular, the available bit rate (ABR) in our model may vary by one order of magnitude

after any VHO. Both types of handoffs may cause extra delays in the transmission, but

we do not consider the extra delays here and assume that there is a seamless handoff

handling scheme which can eliminate the delays caused by both types of handoffs (which

may be achieved at the cost of ABR). In practice, a handoff handling scheme such as the

one presented in [6] can be employed to satisfy this assumption.

A video streaming session is running on the device while the MT traverses through

this integrated two-tier network. The streaming server lies outside this wireless network,

but the bottleneck of the connection is the last hop - the wireless link between the

MT and the BS/AP. The MT keeps sending control messages to request the server to

adjust the video source rate and transmission rate. The server then makes adjustments

accordingly and transmits the data to the MT. The MT has a buffer, which stores the


received data before they are used for playback. Our goal is then to develop a control

scheme to determine how to choose the video source rate and how much to buffer ahead

of time given some statistical and observed channel information. The streaming session is

assumed to be very long and we analyze the performance of our scheme on a time-average

basis.

3.2 Models and Assumptions

3.2.1 Rate Adaptation and Playback

We model the variation in the wireless channel, the error control scheme, and the handoff

handling effects all into the random ABR of the network, denoted by R(i), which is always

positive. In reality, ABR is the amount of error-free video data received by the MT at

each time slot.

We divide the whole streaming session, which consists of several “in-T1N” and “in-

T2N” intervals, into cycles, and denote each cycle by its sequence number in the whole

streaming session, j, where j = 1 represents the first cycle in this session. Each cycle

starts at a T1N-to-T2N VHO and contains one “in-T2N” interval followed by one “in-

T1N” interval, the lengths of which are denoted as T2 and T1 respectively. (Note that, here

we assume the streaming session always starts in T2N, i.e. the high rate network. This is

reasonable since there is not much we can optimize before the first VHO if it starts within

T1N, and we are considering the average performance in the whole streaming process,

so edge effect at the beginning can be ignored.) We further model the video streaming

process as time-discrete with time slots of equal length, and denote each time slot by its

sequence number in the current cycle, i, where i = 1 represents the first time slot in

the current cycle. The control actions are decided and performed at the beginning of

each time slot.

We assume that we have original video streams with very high quality. The average


rate of video can be higher than the highest network rate, and we can adjust the source

encoding rate to any level at any granularity up to the original rate. This assumption of

rate adaptation at any granularity can be accommodated by quantization in practice. By

making this assumption, we eliminate the possibility of playback stop, since transmission

rate R(i) is assumed to be always positive.

The original source rate of the whole video sequence as well as the rate-quality rela-

tionship of the video are transmitted to the MT before the streaming starts, thus the MT

can utilize this information to adjust the rate of “future” parts of the video which are

being streamed. Here we do not define the specific technique used to adapt the source

rate. The transcoding or SVC technique mentioned in Chapter 2 may be employed in

reality to perform rate adaptation.

It is worth noting that, although control actions are performed at the beginning of

each time slot, it is not necessarily effective for only one time slot in playback time.

This is because control decisions are made at every time slot of transmission time, in

which period more than one time slot of video data in the playback sequence may be

transmitted. We illustrate the relationship between transmission time and playback time

in Figure 3.2.

We use a “quality level” defined as q(k) = f(r(k), r0(k)) to control the source rate

adaptation at time slot k, where f(x, y) is monotonously increasing with x, r0(k) is the

original video rate at time slot k, and r(k) is the adapted video rate at time slot k. Hence

for each time slot, the perceived image quality increases with q(k). While there are several

methods to characterize the quality of the picture, e.g. PSNR, distortion, etc, we do not

choose a specific metric, but assume a generic form of the rate-quality relationship. For

simplicity, we assume the original video stream is constant-quality encoded (whether it

is CBR or VBR), thus the perceived quality only depends on the adjusted bit rate and

the original bit rate, and q(k) = f(r(k), r0(k)) = f(r(k)).

Based on this model, we use the metric V = E(∑

0<i<Tg(|q(i+1)−q(i)|)

T) to evaluate the


Transmission Time

Playback Time

Transmission Rate

Playback Rate

Original Rate

…1

2

2

3

4

4

5

…

1 2 3 4 5m

m

+

1

m

+

1

m+

2

m+

2

2T

m

2 1T T+

2

1

T

T

+

2

1

T

T

+

m+

2

2

1

T

T

+

m

+

1

…

…

Figure 3.2: Relationship between the transmission sequence in time and the playback

sequence in time

performance of our algorithm, where T can be a fixed length of time or the length of one

cycle, and g(·) is a monotonously increasing function. This metric reflects the average

amount of total quality variation of the rate-adapted video stream over time.

3.2.2 Residence Time and Rate Estimation

We assume that statistical information such as mean and variance of the residence time

within each sub-network and also the average transmission rates R2 and R1 throughout

the two intervals can be obtained form the AP. This information can be extracted from

previous data collected by the AP. The MT requests the information at the beginning

of each cycle (i = 0). Another possibility is to utilize the geographical information and

MT’s moving speed/direction to estimate the residence time in each network, which will


be briefly discussed in Chapter 5.

3.2.3 Feedback Control Mechanism

We assume that the ABR of current time slot, R(i), as well as the current connected

network can be estimated with good accuracy and fed back by the MT to the server at

the beginning of this time slot. The server then transmits data at the estimated rate.

This mechanism guarantees that our algorithm runs under the following constraint.

Constraint: the ABR of the network is 100% utilized.

However, this doesn’t mean that the transmitted data are 100% utilized. We introduce

another metric called “data utilization”, calculated as the total data used for playback

up to T divided by the total data transmitted up to T . The difference between these

two values is the amount of data left in the buffer at time slot T . From the design of our

algorithms we will see it is not possible to achieve 100% data utilization, but we still try

to maintain the data utilization above an acceptable level within our observation horizon.

The notations for the system models in this thesis are summarized in Table 3.1.

3.3 Problem Formulation

We can translate the aforementioned objectives, assumptions and constraints into the

following optimization problem:


Notation Description

i Index of time slots (in transmission time).

R(i) ABR in ith slot of current cycle.

k Index of time slots (in playback time).

r0(k) Video source rate (to be played) in kth slot.

r(k) Adapted video rate (to be played) in kth slot.

T1, T1, T1

Residence time in T1N (random variable), its mean and

the estimation value of it used in the rate control scheme.

T2, T2, T2

Residence time in T2N (random variable), its mean and

the estimation value of it used in the rate control scheme.

R1, R2 Average ABR in T1N and T2N.

R1, R2 ABR in T1N and T2N in CBR network.

q(k) Quality of adapted video (to be played) in kth time slot of current cycle.

j Index of cycles.

T1(j) ,T2(j) Residence time in T1N and T2N in jth cycle.

q0(j) Quality of adapted video in the last time slot of the (j − 1)th cycle.

Table 3.1: Notations in system model


min V = E(

∑1<k<T

g(|q(k + 1) − q(k)|)

T)

where q(k) = f(r(k), r0(k))

s.t. 0 < r(k) < r0(k) (3.1)

∑

1<k<t

r(k) ≤∑

1<i<t

R(i), 1 < t < T

ǫ = 1 −

∑1<k<T

r(k)

∑1<i<T

R(i)≤ ǫ0

Here we minimize the expected variation while keeping the data utilization above a

certain level. The output of our algorithm is then the q(k)’s, or equivalently the r(k)’s,

as they are 1-to-1 matched given the r0(k)’s, which are assumed to be known (and fixed

in the CBR case).

Given the randomness of the R(i), this optimization problem is not directly solvable.

In the generic model in Chapter 4, T = T1 + T2 in the above formulation. In the

Markov Chain based channel model in Chapter 5 , we assume T1 and T2 are exponentially

distributed, and T is set to be a fixed length of time instead of the length of each cycle.

Chapter 4

Generic Network Model

Three algorithms are developed for the generic network model with one simple algorithm

as comparison. All of them can work with any type of residence time distributions.

4.1 Control Algorithms

4.1.1 Adaptive Control Algorithm

We first propose an adaptive control algorithm which can use statistical information

or localization information to estimate the residence times, and adaptively adjusts the

quality level based on the estimation at each time slot. The estimation horizon is within

the current cycle. Let Te(i) denote the estimation of residence-time-to-go at time slot

i in the current cycle. If the MT is in T2N at time slot i, Te(i) includes the remain-

ing residence time in T2N and the estimated length of the following “in-T1N” interval:

Te(i) = [T2(i) T1(i)]. Otherwise, T2(i) = 0 and the estimation gives us the remaining

residence time in T1N, i.e. Te(i) = [0 T1(i)].

In one approach, we can use the average residence times, T2 and T1, as the estimations

of residence times. At every time slot i in cycle j, the estimation Te(i) is calculated as

follows:

22

Chapter 4. Generic Network Model 23

If the MT is in T2N at time slot i,

T2(i) = max{T2 − i, 1} (4.1)

T1(i) = T1 (4.2)

If the MT is in T1N at time slot i,

T1(i) = max{T1 − (i − T2(j)), 1} (4.3)

T2(i) = 0 (4.4)

where T2(j) is the actual residence time the MT spent in T2N in the current cycle.

We can also use other forms of estimations in our algorithms. In Chapter 5, we

describe a case where our algorithm can work with estimations generated by localization

service.

Our adaptive control algorithm is as follows: At the beginning of each cycle (time

slot 0), we have the original playback curve (i.e. the cumulated video source rate curve)

starting from the current time slot, denoted as P (t). We may already have some data

buffered from the previous cycle, denoted as TB(0) seconds or B(0) bits in the buffer.

Also, we want to buffer some data for the next cycle, i.e. at the end of this cycle, we

should have TBh seconds of video in the buffer. TBh is set to be 0 in the CBR cases but

to a very small value in the later VBR cases to accommodate short-term variations in

VBR networks.

To simplify the calculation here, we choose a simple form of “quality level”, q(i) =

f(r(i), r0(i)) = r(i)/r0(i), i.e. quality level is proportional to the adjusted video rate

given a fixed original rate. But for any other f(x, y) monotonously increasing with x,

similar calculations can be applied here for all the algorithms mentioned below.

Then we calculate the initial quality level q(0), which is, the average quality level we

can sustain throughout the current cycle under the assumption of perfect estimation:

q(0) =R2T2(0) + R1T1(0)

P (T2(0) + T1(0) + TBh − TB(0))(4.5)


β q-

Estimation

Error Signal Control Signal

Feedback Signal

Desired State

System

Figure 4.1: Illustration of proportional feedback controller

At each subsequent time slot i, the MT keeps estimating Te(i). Also, we have the amount

of buffered data at this time is B(i) bits. Then if we keep using the quality level decided

at the previous time slot, q(i− 1), the estimated amount of buffered data (in bits) at the

end of this cycle would be:

Be(i) = B(i) + R2T2(i) + R1T1(i) − q(i − 1)[P (i + T2(i) + T1(i)) − P (i)] (4.6)

The estimated length of buffered video is TBe(i), which is the solution of the following

equation:

q(i − 1)[P (i + T2(i) + T1(i) + TBe(i)) − P (i + T2(i) + T1(i))] = Be(i) (4.7)

Then, we decide q(i) as follows:

q(i) − q(i − 1) = β · (TBe(i) − TBh) (4.8)

Here, we actually construct a linear negative-feedback controller with a proportional gain

β (See Figure 4.1 for an illustration). The input of our system is q(i − 1), the output

is TBe(i), and our setpoint is TBh. The controller attempts to minimize the “error”

between the given setpoint and the output by adjusting the control inputs according to

the proportional negative feedback law. If the q(i) we get from the above controller is

higher than 1, which means that the new rate will be higher than the original rate (which

is impossible under our system assumptions), we set q(i) = 1. Also, if the new rate cannot

be sustained by the current network transmission rate, i.e. there is not enough buffer

(TB(i) < 1) and the transmission rate is low, we set q(i) to be the highest level that can


be supported for the current time slot:

q(i) =R1

P (1 − TB(i))

Hence,

q(i) =

max{1, q(i − 1) + β · (TBe(i) − TBh)}, if TB(i) ≥ 1

min{ R1

P (1−TB(i)), q(i − 1) + β · (TBe(i) − TBh)}, if TB(i) < 1

(4.9)

Other forms of feedback controllers, such as PID (Proportional-Integral-Derivative)

controller [32], can also be applied here, but since our system is non-linear and stochastic

in general, it is difficult to provide theoretical guidance on choosing the parameters.

Hence we keep the controller simple with only one adjustable parameter, β. Even for the

only parameter, we cannot provide a method to choose the proper β which guarantees

optimality and system stability (which is the case for most PID controllers). Generally,

there might be some empirical rules for choosing parameters in PID control systems,

but the effectiveness of these rules depends highly on the system structure. However,

we discuss the choice of the parameter for the specific set of system parameters in the

analysis of our simulation results in Chapter 5 and provide some intuitions on tuning the

parameters.

During each time slot, the server sets the quality levels of video sequences to be

transmitted according to the adaptive algorithm and transmits them to the MT.

4.1.2 Simple Algorithm

To compare our algorithm with other bandwidth smoothing techniques without long-term

prediction or estimation for the VHOs, we design another simple adaptive algorithm. In

this algorithm, we don not use any information about the network distribution but decide

q(i) only based on the current buffered length, i.e. at every time slot i we decide q(i) as


follows:

q(i) =

max{1, q(i − 1) + β · (TB(i) − TBh)}, if TB(i) ≥ 1

min{ R1

P (1−TB(i)), q(i − 1) + β · (TB(i) − TBh)}, if TB(i) < 1

(4.10)

Where TB(i) is the buffered length of video in seconds at time slot i, which is the solution

for the following equation:

q(i − 1)[P (i + TB(i)) − P (i)] = B(i) (4.11)

We are employing the same negative feedback control technique here, except that our

system does not include the estimation part any more. Instead, the algorithm uses the

observation of the buffer level as the input.

4.1.3 Mean Residual Life Based Algorithm

Assume we know the exact distribution of the residence times. One intuitive algorithm

we can come up with is to calculate the expected remaining residence time (or Mean

Residual Life, MRL) at every time slot given the elapsed residence time the MT spent in

current sub-network, then allocate the expected network resources evenly according to

the MRL.

To be more specific, assume we have the Probability Density Functions (PDFs) for

T1, T2 to be f1(t), f2(t) respectively. At each time slot i in T2N, the algorithm calculates

the MRL in T2N as follows:

ˆMRL2(i) = E(T2 − i|T2 > i) (4.12)

= (−1

∫ ∞i f2(t)dt

∫ ∞

itf2(t)dt) − i

MRL2(i) = max{ ˆMRL2(i), 1} (4.13)

Then, it calculates the quality level to be used according to the MRL (also assuming the

proportional rate-quality relationship here):

q(i) = max{1,R2MRL2(i) + R1T1

P (MRL2(i) + T1 − TB(i))} (4.14)


where TB(i) is the length of video the MT has buffered at time slot i.

If at time slot i the MT is in T1N, the calculation for the MRL is similar, but the

q(i) is calculated as follows:

q(i) =

max{1, R1MRL1(i)P (MRL1(i)−TB(i))

}, if TB(i) ≥ 1

min{ R1

P (1−TB(i)), R1MRL1(i)

P (MRL1(i)−TB(i))}, if TB(i) < 1

(4.15)

For a relatively stable network distribution, the MRLs given different elapsed residence

times can be pre-calculated and stored within a table, hence this algorithm does not

involve large amount of calculation. However, since the transfer function from the resi-

dence times to the time-averaged variation is not linear, the MRL is not necessarily the

optimal estimation for the remaining residence time if we want to achieve the minimum

expected variation.

Furthermore, the performance of the MRL based algorithm depends heavily on the

distributions of residence times. If the residence times are exponentially distributed,

then the MRL is always the mean of the exponential distribution regardless of how long

the MT has stayed in the current sub-network. Thus the MRL based algorithm will

keep increasing the quality level from the beginning of the cycle, until it reaches the

end of T2N, and keep decreasing the quality level from the beginning of the following

T1N interval until the end of the cycle. Such curve introduces quality variations in itself

instead of eliminating them. With certain other light-tail distributions, the MRL based

algorithm gives better performance.

4.1.4 Simple Shaping Algorithm

Another intuitive solution is to explicitly consider the form of our optimization objective:

the time averaged variation. The algorithm then designs a “best shaped” quality level

curve according to the specific form of this metric. Again we consider the quadratic form

of variation we have used all the time: E(∑

0<i<T(|q(i+1)−q(i)|)2

T).


At every time slot, we have an estimation of how much data to receive from current

time slot to the end of current cycle. We also have the record of the quality level set by

the algorithm in the previous time slot. Now assume the estimation is accurate, the best

way to allocate the deliverable data to each time slot of video is to perform a quadratic

programming over the remaining time of the cycle within the constraint of using up all

the deliverable data at the end of the cycle, and the objective of minimum variation

(which is in a quadratic form). We use the expected values of residence times, T1 and T2

as the estimations.

However, if the estimation is inaccurate, which is almost always the case, the output

of quadratic programming is never the optimal result. The problem is not substantial

when the estimation of T2 is inaccurate, as at the end of the T2N period, we always

end up with plenty of buffer to compensate for the low-rate in T1N , which can be used

to guarantee a smooth degrading curve afterwards if the MT enters T1N earlier than

expected. The only case when there is a problem is when T1 > T1. Because when this is

the case, after the (T2 + T1)th time slot, there is nothing in the buffer, and the quality

level of time slot (T2 + T1 + 1) will be forced to set to R1/r0, which will usually cause

a huge difference between q(T2 + T1) and q(T2 + T1 + 1). In order to avoid this, we add

a further constraint to the quadratic programming, that at time slot (T2 + T1 + 1), the

quality level is set to be R1/r0.

So the quadratic programming part in this algorithm becomes:

min∑

T2+2≤i≤T2+T1

(q(i + 1) − q(i))2 + (q(T2) − q(T2 + 1))2 + (R1/r − q(T2 + T1))2

s.t.∑

T2+1≤i≤T2+T1

q(i) = TB1 · R1/r + T1 · R1/r (4.16)

R1/r ≤ q(i) ≤ 1, T2 + 1 ≤ i ≤ T2 + T1

When the MT is in the following T1N network, the algorithm keeps using the “best

shaped” quality curve determined by the quadratic programming until it reaches the end


of this cycle. If T1 > T1, the quality levels for each time slot between [T1, T1] will be set

to R1/r.

4.2 Analytical Framework and Analytical Results

In order to evaluate the performance of these algorithms, we further introduce an ana-

lytical framework based on Markov chains, which can work with any kind of residence

time distribution.

Following the generic network model, we model the streaming process as a discrete-

time Markov chain, with states {TB, q0}, where TB is the playback length of buffered video

(in seconds) at the beginning of each cycle, and q0 is the quality level set at the last time

slot in the previous cycle. We assume T2(j) and T1(j) are i.i.d. random variables (note

that this is an assumption in the analytical framework, we do not make this assumption

when we design the rate control scheme), where j is the index of cycles. Then the values

of {TB(j), q0(j)} depend only on the previous state, {TB(j − 1), q0(j − 1)}. To make the

state space finite, we adopt a uniform quantization scheme on the range of TB and q0

values.

We can see that, since we are always trying to achieve an expected amount of buffered

data of 0 at the end of each cycle, (i.e. high data utilization) the playback length of

buffered video should never exceed the estimated value of T1, T1 (since in T2N period,

the algorithm buffers for no longer than T1(R2 − R1)/R2 to achieve a zero buffer at the

end of the cycle, while in the T1N period, the algorithm uses up the previously buffered

data). Hence this Markov chain is a finite-state Markov chain, with state TB being

{0, 1, 2, ...T1(R2 − R1)/R2}, and q0 being {ql, ql + qs, ql + 2qs, ..., qh}. Where ql is the

lowest quality level allowed by the algorithm, and qh is the highest quality level allowed

by the algorithm.

Between any two states, there is a transition to each other, making this Markov chain a


fully connected graph. To analyze the overall performance of our algorithm, we associate

a cost Je(TB, q0) = E(

∑1<i<T2+T1

g(|q(i+1)−q(i)|)+g(|q0−q(1)|)

T2+T1) on each state, the smaller the

expected total cost is, the better our algorithm performs. Note that, the difference

between this cost and the cost in the later introduced dynamic programming algorithm

is that the later covers a fixed length of time, while the former covers the (random) length

of one cycle. Then we calculate the transition probabilities between states according to

the distribution of residence times and the algorithm being evaluated:

P (TB(j + 1) = TBm, q0(j + 1) = q0n|TB(j) = TBl, q0(j) = q0k)

= P (TBm, q0n|TBl, q0k)

=∑

T2,T1

P (TBm, q0n|TBl, q0k, T2(j) = T2, T1(j) = T1) · P (T2(j) = T2) · P (T1(j) = T1)

(4.17)

Now we can compose the transition matrix PA for our defined Markov chain. Solving

this Markov chain, we can obtain the steady state distribution, π, which satisfies π = πPA.

Then we can calculate the expected cost per cycle:

E(Je) =∑

TB ,q0

πTB ,q0· Je(TB, q0) (4.18)

Note that, in this analytical framework we do not consider the data utilization metric

(while in later studies of the Markov chain network model and dynamic programming

algorithms, we explicitly consider it). This is because, in this model we are observing a

relatively short time horizon (one cycle), the utilization in this time horizon cannot reflect

the long-term utilization. Instead, we set the objective of using up all the transmitted

data at the end of each cycle in the design of our algorithms, which ensures that the

long-term data utilization is near 100%.

4.2.1 Analytical Results for Generic Model

Under the established framework, we analyze the performance of our adaptive control al-

gorithm, the MRL based algorithm and the simple shaping algorithm with the quadratic


Parameter Value

Time slot length 1 s

T1, T2 20 s, 20 s

R1 0.5:0.25:2.75 Mbps

R2 6:-0.25:3.75 Mbps

r0 6 Mbps

a1, b1 6.67, 3

a2, b2 6.67, 3

Table 4.1: Analysis parameters - 1

Parameter Value


T1, T2 20 s, 20 s

R1 0.5:0.25:2.75 Mbps

R2 6:-0.25:3.75 Mbps

r0 6 Mbps

a1, b1 13.33, 1.5

a2, b2 13.33, 1.5

Table 4.2: Analysis parameters - 2

variation metric. The simple adaptive algorithm is also evaluated as a comparison. To

avoid the cases when MRL based algorithm doesn’t perform intuitively (like the expo-

nential distribution case), we choose light-tail distributions of residence times to analyze

the performance. An example of the analytical results with Gamma-distributed residence

times is shown in Figure 4.3 and 4.4 along with the residence time distributions (the same

for T1 and T2) in Figure 4.2. We also simulate the algorithms in the same settings to val-

idate the analytical framework. The parameter β in the adaptive and simple algorithms

are selected based on simulations first to ensure the long-term utilization is above 90%.

Parameters are listed in Tables 4.1 and 4.2 (ai and bi are the two parameters in

the Gamma distribution). Proper values of these parameters (especially the average

residence times) depend on the density and coverage of T2Ns, the MT’s moving speed

and moving pattern. In Section 5.6 when we discuss the network model generated by

mobility modeling, we use practical values for moving speed and T2N coverage, and a

realistic distribution of the T2Ns. The mean residence times we get in that setting are

T1 = 35sandT2 = 25s. So we think the mean residence time of 20s here is also practical

in real scenarios.

As we can see, the adaptive control algorithm gives the best performance all the time,


0 10 20 30 40 500

0.01

0.02

0.03

0.04

0.05

0.06

T1

Pd

f(T

1)

0 10 20 30 400

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

T1

Pd

f(T

1)

Gamma distribution of residence time -1 Gamma distribution of residence time -2

Figure 4.2: Distributions of residence times

even though we do not minimize the variation metric in the adaptive algorithm explic-

itly. In contrast, the MRL based algorithm tries to optimize through selecting the best

estimation, yet it fails because of the non-linear relationship between the residence times

and the variation. The simple shaping algorithm is not optimal either, because although

it tries to minimize the variation by shaping the quality curve under the assumption of

perfect estimation, it fails to deal with the uncertainty of the residence times.

Furthermore, the adaptive algorithm has more advantage if we compare the complex-

ity of the algorithms and the ability to deal with VBR cases. While the adaptive control

algorithm is designed for VBR cases and performs well in VBR cases as we show later,

the MRL based algorithm and the simple shaping algorithm are designed based on CBR

assumptions and would simply fail to deal with more uncertainty in the VBR cases.

We can also see that the simulation results and the analytical results are reason-

ably close to each other and exhibits the same trends, which validates our analytical

framework. However, there is a consistent bias in the analysis shown in both figures in

comparison to the simulation. Through experiments we found that by using finer quan-


tization for TB and q0 we can reduce the gap between analytical results and simulation

results. Hence we believe the bias is caused by quantization errors in the analysis.

On the other hand, comparing between the results with differently shaped distribu-

tions (the first one more dispersed and the second one more centralized), we can also

notice some characteristics about the MRL algorithm: the MRL based algorithm works

better (nearer to the adaptive one) when the distribution of the algorithm is more cen-

tralized, because when the distribution is more centralized, the MRL will be nearer to

the real remaining residence time. The simple shaping algorithm has its own character-

istics as well. However, as they show poor performance in comparison to our adaptive

algorithm, we do not look further into the details of these algorithms.


0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.60

0.002

0.004

0.006

0.008

0.01

0.012

0.014

R1

V

Adaptive(A)

MRL(A)

Shaping(A)

Simple(A)

Adaptive(S)

MRL(S)

Shaping(S)

Simple(S)

Figure 4.3: Analysis vs simulation results: generic model, Gamma distribution - 1


0.6 0.8 1 1.2 1.4 1.6 1.8 2 2.2 2.4 2.60

0.002

0.004

0.006

0.008

0.01

0.012

R1

V

Adaptive(A)

MRL(A)

Shaping(A)

Simple(A)

Adaptive(S)

MRL(S)

Shaping(S)

Simple(S)

Figure 4.4: Analysis vs simulation results: generic model, Gamma distribution - 2

Chapter 5

Markov Chain Network Model

We can also model the heterogeneous wireless network channel as a Markov chain where

we assume exponential network residence time in each sub-network and memoryless tran-

sitions between sub-networks. Based on this Markov chain, we make decisions on video

source rate adaptation, turning the streaming process into a controlled Markov Decision

Process (MDP). We can then apply dynamic programming algorithm on this MDP, which

is assumed to achieve optimal performance.

5.1 Markov Decision Process Model

A common assumption on residence times in separated homogeneous wireless networks,

such as cellular and WLAN networks, is that they are exponentially distributed. Follow-

ing this assumption, we can assume the transitions between the two sub-networks to be

memoryless. The ABR in the channel is then modeled as a Markov chain with two states

{R1, R2} and transition probabilities

P =

p11 p12

p21 p22

Hence a part of the streaming session can be characterized as a Markov Decision

Process (S, T , Φ), where S is the set of possible states in the streaming session, T is

36

Chapter 5. Markov Chain Network Model 37

the transition probability matrix between states, and Φ is the set of possible decisions

(quality levels) we can choose at each state.

We define the system state as s = {R,A,L}, where R is the ABR at the current time

slot, A is the control action (quality level) chosen by the algorithm at the previous time

slot, and L is the buffered length of video stream (in seconds) at the beginning of the

current time slot. Thus the transitions take place as follows:

P (R(i + 1) = Rl|R(i) = Rk) = plk, l, k = 1, 2

A(i + 1) = φ(i) (5.1)

L(i + 1) = L(i) + R(i)/f−1(φ(i)) − 1

Note that the state variables A and L are completely determined given R and the current

control action.

For an arbitrary admissible control action φ at time slot i, we associate a cost with

the transition from i-th slot to (i + 1)th (from state s = {R(i), A(i), L(i)} to state

s′ = {R(i + 1), A(i + 1), L(i + 1)}):

V φi (s) = α · (L(i) + R(i)/f−1(φ) − 1)2 + g(φ(i) − A(i)), if i = N (5.2)

V φi (s) = g(φ − A(i)), i = 1, 2, ..., N − 1 (5.3)

where the g(φ − A(i)) part reflects the variation in the adapted video quality de-

termined by the algorithm, and the α · (L(i) + R(i)/f−1(φ) − 1)2 part reflects another

objective of our algorithms: high utilization of transmitted data, and α is a weight as-

sociated with this part. N is the total number of time slots in the part of the streaming

session we are observing. f−1(φ) is the inverse mapping from control action φ to the

adapted video source rate. The objective of the algorithm is to minimize the expected

cost over all transitions:

JΨi = E[

N∑

l=1

Vl(sΨ)] (5.4)


Thus the system becomes a finite-horizon controlled MDP. The process is not homoge-

neous, because its transition matrix vary from time to time. As a result, there exists no

optimal static control policy.

Now define the cost-to-go at time slot i for an admissible policy Ψ = (φ1, φ2, ...φN):

JΨi = E[

N∑

l=i

Vl(sΨ)] (5.5)

According to Bellman’s Principle of Optimality, the optimal control policy is given by

the following equation:

φi∗(s) = arg min{Vi(s) + E[Ji(s)]} (5.6)

5.2 Dynamic Programming Algorithm

Based on the system model described by Equations (5.1) ∼ (5.6), we use a backward

induction based algorithm to solve for the optimal control action at every state:

Algorithm 1 Find the optimal control policy Φ∗ = (φ1, φ2, ...φN).

Require: T ≥ 1

i ⇐ N

for all states s do

φN(s) = arg min{VN(s)]}

end for

while i ≥ 1 do

for all states s do

φi(s) = arg min{Vi(s) + E[Ji(s)]}

end for

i ⇐ i − 1

end while

After obtaining the optimal policy Φ∗, we can store it into a look-up table at the MT.

At the very beginning, we let the system start within T2N, from state s0 = {R2, q0, 0},


where R2 denotes the ABR in T2N, q0 denotes the quality level corresponding to the

average ABR in the network: q0 = f−1( R2T2+R1T1

T2+T1).

Then at the beginning of each following time slot, the MT chooses the optimal action

according to current system state. Throughout this time slot, the server sets the quality

levels of video sequences to be transmitted as the optimal one and transmits them to the

MT.

A major difference between the dynamic programming algorithm and the adaptive

control algorithm lies in their estimation horizons. The dynamic programming algorithm

has the information about the network throughout the whole length of N , and utilizes

this information in order to achieve global optimality during [0, N ]. Yet the adaptive

algorithm only utilizes the information in the current cycle (though we assume the dis-

tributions of residence times don’t vary from cycle to cycle in the model in simulation

and analysis, it is not an assumption in the algorithm, and is not used by the algorithm.)

Hence the adaptive algorithm only tries to achieve local optimality within the cycle.

5.3 Simulation Results

The dynamic programming algorithm is supposed to provide optimal performance on the

Markov chain network model. While our adaptive algorithm can work with all kinds of

residence time distributions, it also works on the Markov chain model. It is then of our

interest to see how far away is the adaptive algorithm from optimum.

In the simulation, we generate realizations of T2’s and T1’s for a fixed length of

time, within each residence interval the transmission rate is constant. We run our al-

gorithm with different parameters on the generated network rate trace and the (CBR)

video trace, then calculate the expected variation in a fixed length of time, i.e. V =

E(∑

0<i<Tg(|q(i+1)−q(i)|)

T) = E(

∑0<i<T

(q(i+1)−q(i))2

T) as performance metric, where T = N .

Another metric we care about is the utilization of the transmitted data, since one of our


Parameter Value


T 150 s

R1 1 Mbps

R2 6 Mbps

r0 6 Mbps

p11, p22 0.1

T1, T2 10 s

NA 8

NB 17

Table 5.1: Simulation parameters for 2-zone Markov model

objectives is to achieve high data utilization over time.

Some parameters we used in the simulation are listed in Table 5.1. NA denotes the

quantization level of control actions (number of control actions), and NB denotes the

quantization level of buffered video length. Notice that, NB has to be increasing with

NA, otherwise, two control actions on one state may lead to a transition to states with

the same buffered length of video, but different variations, so the control action (quality

level) with the smaller variation will always be selected, and the other one (usually the

largest one) will never be selected by the algorithm.

To ensure a fair comparison, the outputs of the adaptive algorithm and the simple

algorithm are quantized using the same quantization level.

The performance of dynamic programming algorithm as a function of α is shown in

Figure 5.1. As we can see, since α represents the weight of the residual buffer part in the

total cost, as α increases, the data utilization also increases, and the expected variation

increases slightly.

The performance of adaptive algorithm and simple algorithm as a function of β is


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.80.018

0.02

0.022

0.024

0.026

0.028

0.03

0.032

α

V—

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8

0.97

0.975

0.98

0.985

0.99

0.995

1

Uti

liza

tio

n −

−

Figure 5.1: DP: variation and utilization vs. α

0.03 0.032 0.034 0.036 0.038 0.04 0.0420

0.02

0.04

0.06

0.08

0.1

0.12

0.14

β

V—

0.03 0.032 0.034 0.036 0.038 0.04 0.042

0.8

0.82

0.84

0.86

0.88

0.9

0.92

0.94

0.96

0.98

Uti

liza

tio

n −

−

Adaptive

Simple

Figure 5.2: Adaptive algorithm: variation and utilization vs. β

shown in Figure 5.2. β is the proportional gain between the input and the “error” in the

negative feedback system, hence adjusts the amount of variation at each step. From the

figure we can see that, there is an optimal operating point of β for a specific set of network

model parameters, when β is too small, the variation at each step is too small. Although

it offers a small total variation, the utilization can also be low because the cumulative

adjusted video rate cannot track the cumulative transmission rate quite closely.


0 0.01 0.02 0.03 0.04 0.05 0.06 0.07 0.08 0.090.92

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1

V

Uti

liza

tio

n

DP

Adaptive

Simple

Figure 5.3: Comparison between algorithms: utilization vs. variation

While when β is too large, the variation at each step is also too large, thus may cause

a certain amount of oscillation in the adjusted video rate and significantly deteriorates

the performance in terms of both variation and data utilization. The data utilization

becomes lower because if the control output (q(i)) is too high due to the oscillation,

and the network transmission rate cannot sustain that quality level, only the highest

sustainable quality will be chosen, hence the cumulative adjusted video rate cannot track

the cumulative transmission rate closely. Generally, the choice of β is determined by a

lot of parameters, such as the distribution of the network, the MT’s moving speed and

moving pattern, the average video source rate and network transmission rates at different

zones, etc. Hence it is empirical and can be determined through experiments. Since the

distribution of networks is relatively stable, the proper choices of β under different MT

parameter sets can be stored in a look-up table in the coordinating BS or AP which is

providing other network-wise information to the MT and can be regularly updated.

To compare both types of algorithms, we draw the relationship between the time-

averaged variation and the data utilization in Figure 5.3.

Since our objectives are high utilization and low variation, the more upper-left the

curve is, the better the algorithm’s performance is. Dynamic programming algorithm


achieves very high utilization and lowest variation at the same utilization level. The

performance of adaptive control algorithm depends heavily on the choice of parameter,

yet the optimal operating point provide near-optimal variation performance at some

acceptable utilization levels. Also, the performance of adaptive algorithm is much better

than that of the simple algorithm.

5.4 More Realistic 3-Zone Network Model

In the previously used 2-zone Markov chain network model, we assume constant bit-rate

in each sub-network, and assume exponentially distributed residence times. While for

T1N which covers large area, we may make the assumption of constant bit-rate in the

current moving area of the MT, we can hardly make the same assumption for the T2N

network. This is because the T2N we are considering only covers local area, and the

supported transmission rates may vary dramatically with the distance between MT and

AP. Thus we consider a more realistic network model in this chapter, where the T2N is

assumed to support multiple transmission rates at different ranges (zones). We adopt a

simple 2-zone T2N model here, as shown in Figure 5.4. We denote the two zones within

T2N as T2No for outer zone, and T2Ni for inner zone. Hence our network model becomes

a 3-zone model.

Also, it is not generally true that the residence times are exponentially distributed (or

with other commonly used distributions) in a two-tier integrated wireless network such

as the one we are considering. Hence it is of our interest to explore how our proposed

algorithms perform under more realistic residence time distributions. We generate the

actual moving traces for MT in the network with a simple mobility model, and then

obtain the samples of the residence times within each zone along the traces.

One example of the generated map and MT’s moving trace is shown in Figure 5.5.

The map is repetitive so that the MT will reach the other end and continue moving if


T1N

BS

T2N

1r

2r

Low Rate

High Rate

Figure 5.4: Integrated two-tier network with 2-zone T2N

0 50 100 150 200 250 3000

50

100

150

200

250

300

Figure 5.5: An example of the generated user’s moving trace

it passes the border of the map. The moving trace is generated by a simple mobility

model: the MT picks a random point to start first, then at every step, it picks a random

direction and random speed (with in a reasonable range [Vmin, Vmax]), and moves toward

that direction for an exponentially distributed length of time (with mean Ts).


By simulating enough moving traces of the MT on each map, the distributions of

the samples of residence times approximate to the real distribution. As we can see in

Figure 5.6, the distribution for T1 is near exponential, while the distributions for T2i and

T2o are far from exponential. While the distributions of residence times are generally

affected by the mobility models and the coverage map, it is out of our scope to discuss

about the realistic types of distributions in this thesis. So we rely on the simplified model

for generating more realistic residence time distributions than exponential distribution.

Again, we want to claim that our adaptive algorithm does not rely on the structure of

the specific distribution hence can work on any type of distribution in the real world.

5.5 PH-Fitting of Residence Times

While the adaptive algorithm only requires the mean values of residence times and hence

can work directly on the generated distribution, the dynamic programming algorithm

requires a structure of Markov chain and more parameters such as the transition prob-

abilities between any two zones. We adopt Phase-Type (PH) distribution, as suggested

and evaluated by [34], as a modeling tool to convert the generated distributions into a

Markov chain.

In general, a PH distribution is the distribution of the time until absorption in a

Markov chain. The states in this Markov chain are called “phases”, which generally

have no physical meaning. PH distributions are highly versatile and can be used to

approximate any distribution of non-negative random variables. A PH distribution can

be described by (~τ ,Q), where ~τ is the initial distribution vector, and Q is the infinitesimal

generator. We use a fitting tool called EMPHT [4] to fit the PH distributions. An

example of the original and PH-fitted CDF’s for T1, T2o (residence time in outer region

of T2N) and T2i (residence time in inner region of T2N) are shown in Figure 5.6. Each

variable is fitted using a 2-phase PH distribution. As we can see, the CDF for T1 is quite


0 50 100 1500

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1Distribution function

− input, − − fitted PH

0 10 200

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9



0 10 20 30 400

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9



Figure 5.6: CDF’s of residence times in different zones

well approximated. Yet the fitted distributions of T2i and T2o are not that accurate.

While increasing the numbers of phases will increase the accuracy of the model, it will

also increase the state space and the complexity of the dynamic programming algorithm.

On the other hand, our experiments show that there will be no significant improvement

of accuracy in terms of the value of the log-likelihood function in the EMPHT algorithm,

when we increase the number of phases to 12 or more, in comparison to the 2-phase

model. Hence we keep using 2 phases to fit the residence times in each zone.

After fitting the residence time distributions individually, we need to combine them

together to obtain the Markov chain model of the integrated network. Figure 5.7 gives

an illustration of this Markov chain. Denote the transition probability matrix (between

different phases) within each zone and the initial distribution for each zone as {T1, ~τ1},

{T2o, ~τ2o} and {T2i, ~τ2i}, then the transition probability matrix for the Markov chain of

the integrated network would be:

T =

T1 p12to~τo 0

p21t1~τ1 To poiti~τi

0 pioto~τo Ti

where p12 = pio = 1, and the value of p21 and poi can be calculated as p21 =

v21

v21+voi, poi = voi

v21+voi, with v21 and voi representing the numbers of transitions from T2N


11P

12P

2 ,1oP

2 ,2oP

2 ,1iP

2 ,2iP

oip

21p

Figure 5.7: PH-fitted Markov chain network model

to T1N and from T2No to T2Ni in our collected traces, respectively.

Now we have finished constructing the Markov chain on which we can run the dynamic

programming algorithm.

5.6 Estimation in Adaptive Control Algorithm

The dynamic programming algorithm can directly run on the 3-zone model. However,

since the adaptive algorithm is designed for the 2-zone model, when it runs on the 3-zone

model we need some modifications on the residence time and transmission rate estimation

part. There are two types of information we can utilize to estimate the residence times.

5.6.1 Utilizing Statistical Information

In one approach, we can use the statistical information, i.e. the average residence times,

as estimations. To extend the adaptive algorithm to the new model, we still treat the

T2N as a homogeneous zone, but change the way to calculate the average residence time

and the average transmission rate in T2N. Since now we have two zones within T2N, the

estimation of residence time in T2N for the adaptive algorithm is not directly given as

the average residence time. Instead, we may have the average residence times with in

each zone T2i, T2o , as well as the transition probabilities between zones poi = voi

v21+voiand


pio = 1. Thus we can calculate the T2(0) in each cycle as follows:

T2(0) = T2 =∞∑

j=0

[T2o + j · (T2i + T2o)] · pjoi · p21

= p21[T2o ·1

1 − poi

+ (T2i + T2o) ·poi

(1 − poi)2 ]

= p21[T2i ·poi

(1 − poi)2 + T2o ·

1

(1 − poi)2] (5.7)

The estimations at the following time slots is similar to what we described in Chapter 4.

Similarly, we calculate the average transmission rate inside the entire T2N and use it as

the estimation of transmission rate in the adaptive algorithm.

5.6.2 Utilizing Localization Information

With the network model generated by actual mobility trace, we are able to obtain localiza-

tion information of the MT at every time slot, hence we can also utilize the localization

information to estimate the residence time. We make following assumptions when we

simulate the case of utilizing the localization information:

1. The MT’s moving direction and speed, as well as its position at current time slot

are known.

2. The position of each T2N AP and their coverage areas (including both T2No and

T2Ni) are known.

3. The MT moves in straight line most of the time.

When estimating the residence time in each zone, the MT is expected to move toward

current direction with current speed until it performs the next T1N-to-T2N VHO. Thus

we can calculate the residence time in each zone by dividing the distances in each zone

by the moving speed.


Parameter Value


T 150 s

R1 1 Mbps

R2i, R2o 3 Mbps, 6 Mbps

r0 6 Mbps

Vmin, Vmax 3.5 m/s, 5 m/s

r1, r2 30 m, 20 m

Ts 20 s

NA 8

NB 15

Table 5.2: Simulation parameters for 3-zone model

5.6.3 Simulation Results for 3-Zone Model

The parameters we used in 3-zone network model simulation are listed in Table 5.2. The

transition probability matrix we obtain from the above mentioned PH-fitting method

with 2 phases in each zone is as follows:

T =

0.8350 0.1506 0 0.0144 0 0

0 0.8350 0 0.1650 0 0

0.0877 0 0.7662 0 0.0651 0.0810

0.0112 0 0.2041 0.7662 0.0083 0.0102

0 0 0 0.0592 0.9073 0.0335

0 0 0 0.0115 0.0313 0.9572

The performance of each algorithm on the 3-zone model is shown in Figures 5.8- 5.10.

In Figure 5.9, we also draw the performance curve of the adaptive and simple algorithm

running on the actual network model instead of the PH-fitted one. We can see that the


0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.80.012

0.014

0.016

0.018

0.02

0.022

0.024

0.026

0.028

0.030 0.1 0.2 0.3 0.4 0.5

0.92

0.93

0.94

0.95

0.96

0.97

0.98

0.99

1

Uti

liza

tio

n −

−

V—

α

Figure 5.8: DP on 3-zone model: variation and utilization vs. α

two curves for the same algorithm are reasonably near each other and exhibit the same

trend, which shows that the PH-fitting process does not incur a significant amount of

error when comparing the performance of the algorithms. Furthermore, it shows that

the adaptive algorithm performs better on actual model than on the PH-fitted one, thus

it is advantageous in the realistic problem we are considering.

Similar trends exhibited for the 3-zone model as for the 2-zone model. The data

utilization of the adaptive algorithm is significantly lower. This is because we use the ex-

pected values of residence time and transmission rate over the whole T2N as estimations,

the estimations are more inaccurate.

For the dynamic programming algorithm running on the 3-zone model, it was difficult

for us to generate more results when V < 0.013, which is primarily because of the quan-

tization level we were using. The simulation was conducted on a Dell PowerEdge 1950

server with dual dual-core Intel Xeon 3.0 GHz processors and 2GB memory, and further

increasing the dimensionality in this simulation would give us a “OUT OF MEMORY”

error. However, judging from the trend of the curve we can still believe that the dynamic

programming generates less variation than the adaptive and simple algorithms at lower


0.015 0.02 0.025 0.03 0.035 0.04 0.0450

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

0.2

β

V—

,--

0.015 0.02 0.025 0.03 0.035 0.04 0.045

0.82

0.84

0.86

0.88

0.9

0.92

0.94

0.96

0.98

Utilization––

Adaptive(A)

Adaptive(M)

Simple(A)

Simple(M)

Figure 5.9: Adaptive algorithm on 3-zone model: variation and utilization vs. β

0 0.02 0.04 0.06 0.08 0.1 0.120.93

0.94

0.95

0.96

0.97

0.98

0.99

1

V

Uti

liza

tio

n

Adaptive

Simple

DP

Figure 5.10: Comparison DP and AA: utilization vs. variation

data utilization area.


Parameter Value


T 500 s

R1 0.453 Mbps

R2i, R2o 1.520 Mbps, 3.333 Mbps

Table 5.3: Simulation parameters for VBR network and VBR video

5.7 Simulating with VBR Network and VBR Video

Stream

As we mentioned before, our adaptive algorithm can work with VBR network and VBR

video as well. While it is theoretically possible for the dynamic programming based

algorithm to work with VBR cases (as long as we use Markov process to describe the

network ABR and the video source rate), it would be either extremely complex in terms

of computation, if the models are accurate and complex, or far from optimal, if the

models are simple and inaccurate. Hence we only simulate and compare with the simple

adaptive algorithm to show the effect of different types of long-term estimation. We

simulate the VBR cases on the 3-zone network model, where T2N supports two ranges

of transmission rates.

In every simulation, we use a random part of the “Tokyo Olympics” video trace

provided by the authors of [28] to evaluate our algorithm. The average bit rate of this

video trace is 3.71Mbps. The channel rates are generated randomly within each zone,

with different average rates. Parameters used in the simulation are listed in Table 5.3.

Figure 5.11 shows one instance out of many simulations when we run adaptive al-

gorithm and use statistical information to estimate the residence times. In this set of

simulation, β = 0.01.

Figure 5.12 shows the same simulation instance running the algorithm with local-


0 50 100 150 200 250 300 350 400 450 5000

0.5

1

1.5

2

2.5

3

3.5

4

Network Rate (Mbps)

Video Rate (Mbps)

Quality Level

t

Figure 5.11: VBR simulation: adaptive algorithm with statistical information

0 50 100 150 200 250 300 350 400 450 5000

0.5

1

1.5

2

2.5

3

3.5

4

Network Rate (Mbps)

Video Rate (Mbps)

Quality Level

t

Figure 5.12: VBR simulation: adaptive algorithm with localization information

ization information. We use similar map as the one in Figure 5.5 to generate the MT’s

moving traces and the localization information. In this set of simulation, we use the same

parameters as listed above, β = 0.01.

Figure 5.13 shows the result of simple algorithm (without estimation of the residence


0 50 100 150 200 250 300 350 400 450 5000

1

2

3

4

5

6

Network Rate (Mbps)

Video Rate (Mbps)

Quality Level

t

Figure 5.13: VBR simulation: simple adaptive algorithm

times) with same parameter β = 0.01.

Comparing the three results, we can see that the adjusted video source rate curves

are much smoother when we run the adaptive algorithm with either kind of estimation.

Averaging 1500 runs of experiments, we have the average performance with both types

of estimation under different parameters, shown in Figures 5.14 - 5.16. We can see that,

estimation with statistical information (i.e. mean values) leads to lower variations most

of the time, but also with lower data utilization. This is because the amount of buffer

left in the end depends on the accuracy of the estimation. Generally the localization

information provides more accurate estimation, hence utilizing localization information

leads to better data utilization but it also causes large variation when the MT changes

the direction abruptly.

Besides, the results indicate that either kind of information will be helpful in estimat-

ing the residence time and smoothing the video quality to overcome long-term variations

in video adaptation. In our specific setting, the localization information might be more

accurate on an average basis. Yet when the MT’s moving trace is more unpredictable,

utilizing localization information may not have so much advantage over using statistical


0 0.005 0.01 0.015 0.02 0.0251

2

3

4

5

6

7

8

9x 10

−3

β

V

Localization

Simple

Statistical

Figure 5.14: Simulating VBR case - variation vs. β

0 0.005 0.01 0.015 0.02 0.0250.88

0.89

0.9

0.91

0.92

0.93

0.94

0.95

0.96

β

Util

izat

ion

LocalizationSimpleStatistical

Figure 5.15: Simulating VBR case - utilization vs. β

information. With a proper choice of parameter, the adaptive control algorithm can

reduce the variation in quadratic form by 80% or more.


0 0.005 0.01 0.015 0.02 0.0250.88

0.89

0.9

0.91

0.92

0.93

0.94

0.95

0.96

V

Uti

liza

tio

n

Localization

Simple

Statistical

Figure 5.16: Simulating VBR case - utilization vs. variation

Chapter 6

Conclusion

In this thesis, we have proposed an adaptive control algorithm for video streaming over

heterogeneous wireless networks with dramatically different available channel bit rates.

The proposed approach combines source rate adaptation with buffering and employs

certain information of the network, such as the expected residence time in each network

and the location of mobile device and access points.

By modeling the streaming process over an integrated two-tier network using a frame-

work based on Markov chain with associated rewards (costs), the theoretical analysis for

CBR video, CBR channel case shows a significant reduction in video quality variation in

comparison to some intuitive and complex algorithms.

Modeling the network using a Markov chain allows us to run a dynamic programming

based algorithm to adjust the video source rate, the result of which is supposed to be

optimal under our assumptions. In order to model the more complicated and realistic

heterogeneous wireless network, we further introduced Phase-Type fitting of the resi-

dence times and extended the algorithms to a 3-zone network model. Simulations with

both 2-zone and 3-zone models indicate that our adaptive algorithm gives near-optimal

performance with acceptable data utilization. At the same time, it has the advantage of

low computation and requires less information of the network.

57

Chapter 6. Conclusion 58

Further in the simulations of the VBR cases, the result shows that by utilizing statisti-

cal or localization information we are able to reduce the quality variation of the streamed

VBR video by 80% or more under the quadratic form metric.

From these results, we believe that although the proposed approach is simple and

intuitive, it can improve the quality of streamed video significantly in terms of quality

variation, and is promising in achieving constant-quality high-rate video streaming over

heterogeneous wireless networks.

Bibliography

[1] T. Alpcan, J. P. Singh, and T. Basar. Robust rate control for heterogeneous net-

work access in multihomed environments. IEEE Transactions on Mobile Computing,

8(1):41–51, 2009.

[2] J. G. Apostolopoulos, W.-T. Tan, and S. J. Wee. Video streaming: Concepts,

algorithms, and systems. Technical report, HP Laboratories, 2002, 2002.

[3] S. Aramvith, I.-M. Pao, and M.-T. Sun. A rate-control scheme for video trans-

port over wireless channels. IEEE Transactions on Circuits and Systems for Video

Technology, 11(5):569–580, May 2001.

[4] S. Asmussen, O. Nerman, and M. Olsson. Fitting phase-type distribution via the

EM algorithm. Scand.J.Statist., 23:419–441, 1996.

[5] L. Atzori, M. Krunz, and M. Hassan. Cycle-based rate control for one-way and

interactive video communications over wireless channels. IEEE Transactions on

Multimedia, 9(1):176–184, Jan. 2007.

[6] M. Bernaschi, F. Cacace, R. Clementelli, and L. Vollero. Adaptive streaming on

heterogeneous networks. In WMuNeP ’05: Proceedings of the 1st ACM workshop on

Wireless multimedia networking and performance modeling, pages 16–23, New York,

NY, USA, 2005. ACM.

59

Bibliography 60

[7] H.-C. Chuang, C. Huang, and T. Chiang. Content-aware adaptive media playout

controls for wireless video streaming. Multimedia, IEEE Transactions on, 9(6):1273

–1283, oct. 2007.

[8] Allot Communications. The allot mobiletrends report. http://www.allot.com/

mobiletrends.html, 2009.

[9] P. de Cuetos and K. W. Ross. Adaptive rate control for streaming stored fine-grained

scalable video. In NOSSDAV ’02: Proceedings of the 12th international workshop

on Network and operating systems support for digital audio and video, pages 3–12,

New York, NY, USA, 2002. ACM.

[10] P. de Cuetos and K. W. Ross. Optimal streaming of layered video: joint scheduling

and error concealment. In MULTIMEDIA ’03: Proceedings of the eleventh ACM

international conference on Multimedia, pages 55–64, New York, NY, USA, 2003.

ACM.

[11] M.U. Demircin, P. van Beek, and Y. Altunbasak. Delay-constrained and r-d opti-

mized transrating for high-definition video streaming over wlans. Multimedia, IEEE

Transactions on, 10(6):1155 –1168, oct. 2008.

[12] P. Frossard and O. Verscheure. Joint source/fec rate selection for quality-optimal

mpeg-2 video delivery. Image Processing, IEEE Transactions on, 10(12):1815 –1825,

dec 2001.

[13] V.K. Goyal. Multiple description coding: compression meets the network. Signal

Processing Magazine, IEEE, 18(5):74 –93, sep 2001.

[14] M. Hassan and M. Krunz. Video streaming over wireless packet networks: An

occupancy-based rate adaptation perspective. IEEE Transactions on Circuits and

Systems for Video Technology, 17(8):1017–1027, Aug. 2007.

Bibliography 61

[15] G. Ji and B. Liang. Buffer schemes for vbr video streaming over heterogeneous

wireless networks. In IEEE International Conference on Communications (ICC),

Dresden, Germany, June 2009.

[16] G. Ji and B. Liang. Stochastic rate control for scalable vbr video streaming over

wireless networks. In Global Telecommunications Conference, 2009. GLOBECOM

2009. IEEE, pages 1 –6, nov. 2009.

[17] W. Li. Overview of fine granularity scalability in mpeg-4 video standard. IEEE

Transactions on Circuits and Systems for Video Technology, 11(3):301–317, March

2001.

[18] Y. Li, A. Markopoulou, J. Apostolopoulos, and N. Bambos. Content-aware playout

and packet scheduling for video streaming over wireless links. Multimedia, IEEE

Transactions on, 10(5):885 –895, aug. 2008.

[19] G. Liang and B. Liang. Effect of delay and buffering on jitter-free streaming over

random vbr channels. Multimedia, IEEE Transactions on, 10(6):1128 –1141, oct.

2008.

[20] K. McArthur. The future is here: Next is now. http://redboard.rogers.com/

2010/the-future-is-here-next-is-now/, 2010.

[21] Cisco VNI Mobile. Cisco visual networking index: Global mobile data traffic fore-

cast update, 2009-2014. https://www.cisco.com/en/US/solutions/collateral/

ns341/ns525/ns537/ns705/ns827/white_paper_c11-520862.html, 2010.

[22] A. Ortega and M. Khansari. Rate control for video coding over variable bit rate

channels with applications to wireless transmission. In ICIP ’95: Proceedings of the

1995 International Conference on Image Processing (Vol. 3)-Volume 3, page 3388,

Washington, DC, USA, 1995. IEEE Computer Society.

Bibliography 62

[23] M. Qin and R. Zimmermann. Improving mobile ad-hoc streaming performance

through adaptive layer selection with scalable video coding. In MULTIMEDIA ’07:

Proceedings of the 15th international conference on Multimedia, pages 717–726, New

York, NY, USA, 2007. ACM.

[24] H. Schwarz, D. Marpe, and T. Wiegand. Overview of the scalable video coding

extension of the h.264/avc standard. IEEE Transactions on Circuits and Systems

for Video Technology, 17(9):1103–1120, September 2007.

[25] Y. Sun, I. Ahmad, D. Li, and Y.-Q. Zhang. Region-based rate control and bit

allocation for wireless video transmission. Multimedia, IEEE Transactions on, 8(1):1

– 10, feb. 2006.

[26] T. Taleb, K. Kashibuchi, A. Leonardi, S. Palazzo, K. Hashimoto, N. Kato, and

Y. Nemoto. A cross-layer approach for an efficient delivery of tcp/rtp-based mul-

timedia applications in heterogeneous wireless networks. IEEE Transactions on

Vehicular Technology, 57(6):3801–3814, Nov. 2008.

[27] Y.-H. Tseng, E.H.-K. Wu, and G.-H. Chen. Scene-change aware dynamic bandwidth

allocation for real-time vbr video transmission over ieee 802.15.3 wireless home net-

works. Multimedia, IEEE Transactions on, 9(3):642 –654, april 2007.

[28] G. Van der Auwera, P. T. David, M. Reisslein, and L. J. Karam. Traffic and quality

characterization of the h.264/avc scalable video coding extension. Adv. MultiMedia,

2008(2):1–27, 2008.

[29] M. van der Schaar and H. Radha. A hybrid temporal-snr fine-granular scalability

for internet video. IEEE Trans. on CSVT, 11:318–331, 2001.

[30] P. Vidales, J. Baliosian, J. Serrat, G. Mapp, F. Stajano, and A. Hopper. Autonomic

system for mobility support in 4g networks. Selected Areas in Communications,

IEEE Journal on, 23(12):2288 – 2304, dec. 2005.

Bibliography 63

[31] T. Wiegand, G. J. Sullivan, G. Bjontegaard, and A. Luthra. Overview of the

h.264/avc video coding standard. IEEE Transactions on Circuits and Systems for

Video Technology, 13(7):560–576, July 2003.

[32] Wikipedia. PID controller. http://en.wikipedia.org/wiki/PID_controller#

PID_controller_theory.

[33] J. Xu, X. Shen, J.W. Mark, and J. Cai. Adaptive transmission of multi-layered

video over wireless fading channels. Wireless Communications, IEEE Transactions

on, 6(6):2305 –2314, june 2007.

[34] A. H. Zahran and B. Liang. Zone residence time mobility modeling and performance

evaluation framework for heterogeneous multi-tier wireless networks.

[35] A.H. Zahran and C.J. Sreenan. Cost efficient media streaming algorithms for rate-

dependent pricing strategies in heterogeneous wireless networks. In Next Generation

Mobile Applications, Services and Technologies, 2008. NGMAST ’08. The Second

International Conference on, pages 485 –491, 16-19 2008.

[36] A.H. Zahran and C.J. Sreenan. Pgms: Pseudo-optimal greedy media streaming al-

gorithm for heterogeneous wireless networks. In Networking and Communications,

2008. WIMOB ’08. IEEE International Conference on Wireless and Mobile Com-

puting,, pages 390 –396, 12-14 2008.

[37] A.H. Zahran and C.J. Sreenan. Threshold-based media streaming optimization

for heterogeneous wireless networks. Mobile Computing, IEEE Transactions on,

9(6):753 –764, june 2010.

[38] X. Zhu and B. Girod. Distributed rate allocation for video streaming over wireless

networks with heterogeneous link speeds. In IWCMC ’07: Proceedings of the 2007

international conference on Wireless communications and mobile computing, pages

296–301, New York, NY, USA, 2007. ACM.

Documents

Joint Buffering and Rate Control for Video Streaming over ... · Joint Buﬀering and Rate Control for Video Streaming over Heterogeneous Wireless Networks Lei Hua ... 5.2 Adaptive