LAYERED QUALITY ADAPTATION for

INTERNET VIDEO STREAMING

Reza Rejaie, Mark Handley and Deborah EstrinInformation Science Institute (ISI), University of Southern California

by

IEEE Journal On Selected Areas In Communications Vol. 18., NO.12, December 2000

published in

- 2 types of flows: TCP vs. UDP

- TCP : responsive to the congestion : reliable (provide retransmission)

- UDP : non-responsive : non-reliable

Review:

Problem:

- Explosive growth of audio & videostreaming

- Streaming applications: rate-based,delay-sensitive, semi-reliable

- Lack of effective congestioncontrol mechanism in such applications

To support streaming applicationsover the Internet, two conflicting requirements have to be addressed:

- Application Requirement * require relatively constant

BW to deliver a stream with a certain quality

- Network Requirement * end systems are expected

toreact to congestion properly and promptly

To satisfy these two requirementssimultaneously, Internet streamingapplications should be quality adaptivequality adaptive

- the application should adjust thequality of the delivered stream suchthat the required BW matches thecongestion controlled rate-limit

- the main challenge is to minimize the variations in quality, while obeyingthe congestion controlled rate-limit

This paper presents:

- the novel quality adaptation mechanismnovel quality adaptation mechanismwhich can adjust the quality of congestioncontrolled video playback on-the-fly

- the key feature of this mechanism isthe ability to control the level of smoothing (i.e., frequency of changesfrequency of changes) toimprove quality of the delivered stream

Primary Assumption:

- the congestion control protocol is Rate Rate Adaptation Protocol (RAP)Adaptation Protocol (RAP)

- RAP is a rate-based congestion controlmechanism that employs an AdditiveIncrease Multiplicative Decrease (AIMD) algorithm in a manner similar to TCP

RAP Increase/Decrease Algorithm

Base Equation: Si = PacketSize / IPGi

To increase the rate additively, IPGi+1 = (IPGi * C) / (IPGi + C)where C is a constant with the dimensionof time

Upon detecting congestion, the tx rateis decreased multiplicatively, Si+1 = * Si, IPGi+1 = IPGi / where = 0.5

RAP Decision Frequency

- how often to change the rate

- depends on the feedback delayfeedback delay

- feedback delay in ACK-based schemes is equal to one RTT

- it is suggested that rate-based schemes adjust their rates not more than once per RTT

remotevideoserver

INTERNET

Target Environment

Client#1

Client#2

Client#3

Client#4

- heterogeneous clients- Video on demand

- dominant competing traffic is TCP- short startup latency expected

Quality Adaptation Mechanisms

- “Adaptive Encoding”:requantize stored encoding on-the-fly

based on the network feedbackdisadv: CPU-intensive task

- “Switching among multiple pre-encodedversions”:

the server keeps several versions of each stream with different qualities. Asavailable BW changes, the server choosesthe appropriate version of the stream

disadv: need large server’s buffer

- “Hierarchical Encoding”:the server maintains a layered encoded

version of each stream. As more BW isavailable, more layers of the encoding aredelivered. If the available BW decreases,the server may then drop some of the layers being transmitted

*** layered approaches usually have thedecoding constraint that a particular enhancement layer can only be decoded if all the lower quality layers have been received

adv: - less storage at the server- provides an opportunity for selective selective repair of the more important information

*** the design of a layered approach forquality adaptation primarily entails thedesign of an efficient add and drop mechanism that maximizes quality whileminimizing the probability of base-layerbuffer underflow

NOTENOTE ! !

- Hierarchical Encoding provides aneffective way for a video server to coarselyadjust the quality of a video stream without without transcodingtranscoding the stored data

- However, it does not provide thefine-grained controlfine-grained control over BW, that is, BWonly changes at the granularity of a layer

- there needs to be a quality adaptationmechanism with the ability to control thelevel of smoothing

“short-term improvement” vs. “long-term smoothing”

- in the aggressive approachaggressive approach, a new layer is added asa result of minor increase in available BW. However, thisadditional bandwidth does not last long. Thus, theaggressive approach results in short-term improvementshort-term improvement

- in contrast, the conservative approachconservative approach does not adjustthe quality due to the minor changes in BW. Thus, thisresults in long-term smoothinglong-term smoothing

- Hierarchical encoding allows videoquality adjustment over long periods of time,whereas congestion control changes the txrate rapidly over short time intervals

- The mismatch between the two time scales is made up for by buffering databuffering dataat the receiverat the receiver to smooth the rapid variations in available BW and allow a nearconstant number of layers to be played

The Proposed Architecture

- all the streams are layered-encoded and stored at the server - all active layers are multiplexed into a single RAP flow by the server

- at the client side, layers are de-multiplexed and each one goes to its corresponding buffer- the decoder drains data from buffers and feeds the display

- assume that, each layer has the same BW and all buffers are drained with the same constant rate (C) - the congestion control module continuously reports available BW to the quality adaptation module- the quality adaptation module then adjusts # of active layers and allocated share of congestion controlled BW to each active layer

Challenging Questions:

- When is it suitable to add a new layer ?

- When should we drop a layer ?

- What is the optimal allocation of each layer ?

AssumptionAssumption:

- would like to survive a single backoffwith all the layers intact

Adding a LayerAdding a Layer::

The server may add a new layer when:

1.the instantaneous available BW isgreater than the consumption rate of theexisting layers plus the new layer

R > (na + 1) C, and

2. there is sufficient total bufferingat the receiver to survive an immediatebackoff and continue playing all the existing layers plus the new layer

i=0 to na –1 bufi ([na + 1] C – [R/2])2

2*S

where R : the current tx rate na : the number of currently active layer bufi : the amount of buffered data for layer i S : the rate of linear increase in BW

Dropping a LayerDropping a Layer::

- once a backoff occurs, if the totalamount of buffering at the receiver isless than the estimated required bufferingfor recovery (i.e., the area of trianglecde in fig.5), the correct action is to immediately drop the highest layer

- if the buffering is still insufficient, theserver should interactively drop the highest layer until the amount of bufferingis sufficient

while ( naC > R + 2S i=0 to na-1 bufi )

dona = na - 1

Optimal Interlayer Buffer AllocationOptimal Interlayer Buffer Allocation::

- due to the decoding constraint inhierarchical encoding, each additionallayer depends on all the lower layers

- a buffer allocation mechanism shouldprovide higher protection for lower layersby allocating a higher share of total buffering to them

- the minimum number of buffering layersthat are needed for successful recoveryfrom short-term reduction in available BWcan be determined as:

nb = na – [R/(2*C)] , naC > R/2 nb = 0 , naC <= R/2

where nb : # of buffering layers na : # of active layers

- the consumption rate of a layer must besupplied either from the network or fromthe buffer or a combination of the two

-If it is supplied entirely from the buffer,that layer’s buffer is draining at consumption rate C.

The optimal amount of buffering forlayer i is:

if i < nb – 1,

Bufi,opt = C [C (2na – 2i - 1) - R] 2S

if i = nb – 1 (the top layer),

Bufi,opt = C [naC – (R/2) – (i*C)]2

2S

Fine-Grain Bandwidth AllocationFine-Grain Bandwidth Allocation::

- the server can control the filling anddraining pattern of receiver’s buffersby proper fine-grain bandwidth allocationamong active layers

- fine-grain bandwidth allocation isperformed by assigning the next packetto a particular layer

- the main challenge is that the optimalinterlayer buffer allocation depends onthe transmission rate at the time of abackoff (R), which is not known a prioripriori because a backoff may occur at anyrandom time

- to tackle this problem, during thefilling phase, the server utilizes extraBW to progressively fill receiver’s buffersup to an optimal state in a step-wisestep-wisefashionfashion

- the server maintains an image of thereceiver’s buffer state, which is continuously updated based on theplayout information included in ACK pkts

- During the filling phase, the extra BWis allocated among buffering layers on aper-packet basis through the following steps assuming a backoff will occurimmediately:

1. if we keep only one layer (L0), is there sufficientbuffering with optimal distribution to recover ?

* if there is no sufficient buffering, the next packetis assigned to L0 until this condition is met and then thesecond step is started

2. if we keep only two layers (L0, L1), is there sufficientbuffering with optimal distribution to recover ?

* if there is no sufficient buffering, the next packetis assigned to L0 until it reaches its optimal level. Then,the server starts sending packets for L1 until both layershave the optimal level of buffering to survive

- we then start a new step and increase the number ofexpected surviving layers, calculate a new optimal bufferdistribution and sequentially their buffers up to the newoptimal level

- this process is repeated until all layers can survive asingle backoff

-During the draining phase, BW shareplus draining rate for each layer is equalto its consumption rate.

- Thus, maximally efficient bufferingresults in the upper layers being suppliedfrom the network during the draining phase, while the lower layers are suppliedfrom their buffers

If we would like to survive more thanIf we would like to survive more thanone backoff (Kone backoff (Kmaxmax > 1) > 1)::

- During the filling phase:

- During the draining phase:

* due to one or more backoffs * reverse the filling phase

* identify between which two steps we arecurrently located. This determines how many layers should be dropped due to lackof sufficient buffering* then, we traverse through the steps in

the reverse order to determine which buffering layers must be drained and by

how much* the amount and pattern of draining isthen controlled by fine-grain interlayerBW allocation by the server