Streaming Stored Layered Video in the Internet

Streaming Stored Layered Video in the Internet

Keith Ross(joint work with Philippe De Cuetos, Despina Saparilla, Jussi

Kangasharju, Martin Reisslein)

Institut EURECOMSophia Antipolis, France

Streaming Stored Video Streaming stored video is becoming increasingly widespread in the Internet

High-speed access technologies will permit users to stream video at higher rates

User access rates are highly heterogeneous.

The Internet is a best-effort network without QoS guarantees.

2RHDM 2001

Issues to be ExaminedPrefetching

Layered video

Multiple layers or multiple versions?

How to cache layered video

Interactive audio streaming (Wimba startup)

Assume: CBR video; abundant client storage

3RHDM 2001

References

4RHDM 2001

D. Saparilla and K.W. Ross, Optimal Streaming of Layered Encoded Video, Infocom 2000, Tel Aviv.

D. Saparilla and K.W. Rooss, Streaming Stored Continuous Media over Fair-Share Bandwidth, NOSSDAV 2000, Chapel Hill, 2000.

P. Decuetos, D. Saparilla and K.W. Ross, Adaptive Streaming of Stored Video in a TCP-Friendly Context : Multiple Versions or Multiple Layers, International Packet Video Workshop, Korea, May 2001 J.Kangasharju, F. Hartanto, M. Reisslein and K.W. Ross, Distributing Layered Encoded Video through Caches, IEEE Infocom 2001, Anchorage.

Wimba: http://www.wimba.com

Outline of the Talk

I Streaming Stored Layered Video over Fair-Share Bandwidth

II Layers vs. Switching Versions over Fair-Share Bandwidth (Philippe)

III Layered Video through Caches

IV Interactive Audio Streaming (Wimba startup)

5RHDM 2001

Design Principle: TCP BandwidthA TCP connection roughly gets fair-share bandwidth

This bandwidth varies throughout connection.Changes in number of streams.Changes in traffic generated by individual streams Route changes.

Assumption: Our video streaming application gets TCP bandwidth:

sent over HTTP/TCPTCP-friendly connection

6RHDM 2001

TCP Bandwidth

Averaged over 10 secondintervals

Averaged over 100 secondintervals

7RHDM 2001

Streaming Non-Layered CBR Video (1)

Consider three transmission schemes:

Full prefetching: transmit at full available rate

No prefetching: transmit at min{consumption, available rate}

Partial prefetching: prefetch over short intervals

Playback delay of four seconds; video 60 minutes long.

8RHDM 2001

Streaming Non-Layered CBR Video (2)

• Maximum rate for no loss with full prefetching: 70% of average available rate• Maximum buffer for no loss with full prefetching: 15 minutes

Avg availablebandwidth = 1.1 Mbps

9RHDM 2001

What we learned so far:Prefetching is essential with TCP bandwidth.

Need to prefetch minutes into the future.

Rate adaptation should be considered:multiple versionsmultiple layerson-the-fly compression

10RHDM 2001

Optimal Streaming for Layered Video(Infocom 2000)

base and enhancement layer

decoding constraint: enhancement layer information can only be used when the corresponding base layer information is available

Goal: Design streaming policies that maximize the playback quality in a variable bandwidth environment.

11RHDM 2001

Layered Streaming Model

)()( tXtbπ

)()( tXteπ

)(tX

Yb(t)

Ye(t)

rb

re

CM Server Client withprefetch buffers

Base Layer

Enhancement Layer

X(t) = fair-share bandwidth available to the stream at time tstochastic process whose statistical characteristics are

unknown a priorirb, re = encoded rates of base layer and enhancement

layer Yb(t), Ye(t) = prefetch buffer contents at time t

12RHDM 2001

Streaming Policiesπb(t) can depend on the current and past history of the system, including on X(t), Yb(t), and Ye(t)Low-risk policy: πb(t)=1High-risk optimistic policy:

Two fluid queues:)()( tXtbπ

)()( tXteπ

rb

re

Yb(t)

Ye(t)

eb

bb rr

rt+

== απ ˆ)(

13RHDM 2001

Analysis

Initial playback delay ∆ seconds.Delay between server and client is zero. Examine two cases:

Infinite-Length Video, X(t) stationary, λ =E[X(t)]

Finite-Length Video of duration Tseconds.

14RHDM 2001

Infinite Video Duration

Fraction of Traffic LostBase-layer loss:

Enhancement-layer loss:

where H(t) is the consumption rate of enhancement-layer data at time t.

Tr

dttYtXttrP

b

T

tbbb

Tb

∫=

+

∞→

=−= 0

)0)((1)]()([

limπ

π

,)]([

0lim Tr

dttHtrP

e

T

te

Te

∫=

+

∞→

−=π

15RHDM 2001

Partial-loss ModelFraction of enhancement-layer traffic consumed can be as much as the fraction of base-layer traffic consumed:

⎪⎪⎪⎪

⎩

⎪⎪⎪⎪

⎨

⎧

==⎭⎬⎫

⎩⎨⎧

>=

=>>>

=

0)(,0)(when)()(),()(min

0)(,0)(when)()(0)(,0)(when)()(

0)(,0)(when

)(

tYtYr

tXtrtXt

tYtYr

tXtr

tYtYtXttYtYr

tH

ebb

bee

ebb

be

ebe

ebe

ππ

ππ

16RHDM 2001

Set of Feasible Loss Probabilities: Infinite Length

+−−

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

erbrλ

1

erbr +− λ1

⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

+−erbr

λ1+

−⎥⎥⎥

⎦

⎤

⎢⎢⎢

⎣

⎡

brλ1

L

1Pe

eb

b

rrr+

=α̂

Pb1

Theorem: Each point on L can be achieved by some static policy

17RHDM 2001

Finite Video DurationOptimization problem:

Let Tbπ, Te

π be the times at which streaming of each layer is complete

Lemma 1:Lemma 2:

Theorem:

ceb TTT == αα ˆˆ

.policy any for },max{ πTTT ceb ≤ππ

( ) ( )[ ]πee

πbb PdPdJ −+−= 11Emax ππ

. when for optimal is ˆpolicy Theb

e

b

e

eb

b

rr

ddJ

rrr

≥+

= πα

18RHDM 2001

Heuristics for Finite-Length VideoWhen static policies can perform poorly.

Static threshold policy:

Must determine qthres .

⎪⎩

⎪⎨

⎧

−>≥<

=)()(when0

)(whenˆ)(when1

)(tTrtY

qtYqtY

t

bb

thresb

thresb

b απ

b

e

b

e

rr

dd

<

19RHDM 2001

Threshold Policy Results

• trace results: consumption rate = avg. bandwidth

20RHDM 2001

What we have learned:Infinite-length video:

Fixed fraction of bandwidth to base layer is optimal

Finite-length video:static policy performs poorlyneed to dynamically change policy based on prefetchbuffer contents

Note: so far have only considered time-independent thresholds

21RHDM 2001

Dynamic Threshold Policies[NOSSDAV 2000]

Now going to develop a heuristic for setting the thresholds as a function of time.

Heuristic Philosophy:try to always render the base layerrender the enhancement layer for as much as possibleavoid frequent fluctuations in quality

22RHDM 2001

Dynamic Threshold Heuristics (1)

State 1send only base

State 2send both

Add enhancement layer απ ˆ)( =tb

1)( =tbπDrop enhancement layer

Monitor client buffer content in each layerEstimate future average bandwidth (using past observations)Add enhancement layer if base layer can still be reliably delivered and if enhancement layer can be kept for a certain period of time

23RHDM 2001

Dynamic Threshold Heuristic (2)

Threshold at time s for adding enhancement layer is such that base-layer buffer will not starve for the next C seconds:

To avoid rapid quality fluctuations, introduce a threshold for buffered enhancement layer data:

( ))(ˆ)( sXrCsq avgbthres ⋅−⋅= α

( ))()ˆ1()( sXrCsq avgethres ⋅−−⋅′=′ α

24RHDM 2001

Dynamic Threshold Heuristic (3)

State 1 State 2απ ˆ)( =tb

)()()()( tqtYandtqtY thresethresb ′≥≥

)()( tqtY thresb <

1)( =tbπ

25RHDM 2001

Static Threshold vs. Dynamic Threshold Heuristic

static threshold policies dynamic threshold policies

Dynamic threshold heuristic results in fewer quality fluctuations with the same high-quality viewing time.

26RHDM 2001

What we just learned:Measurement-based heuristic performs well:

minimal or no base layer lossfew fluctuations in quality

Open issues: length of prediction interval suitable values for key heuristic parameters

Now that we have a handle on layered video, let’s now consider multiple versions !

27RHDM 2001

Outline of the TalkI Streaming Stored Layered Video over Fair-Share

Bandwidth



IV Asynchronous Interactive Audio Streaming

28RHDM 2001

Switching Versions[Packet Video Conf, 2001]

As an alternative two layered video, server can store multiple versions of the same video.

Multiple versions requires more disk space, but does it provide better viewing quality given the same X(t)?

Comparison approach:Design analogous control policies for layers and for switching among versions.

29RHDM 2001

Switching VersionsDifferent versions of the same video encoded at different bit-rates.Switch among the versions to adapt the transmission rate to the available bandwidth.

7 6 5 4 3 2 1

7 6 5 4 3 2 1

7 6 5 4 3 2 1Version 1

Version 2

Version 3

Selection 2 14 3

7 6 5Transmission

30RHDM 2001

Switching Versions Control PolicyAt any instant, server has to determine which version of the video to send (version 1 or version 2).

Server will estimate Y1(t) and Y2(t) using receiver reports (e.g. RTCP) and make the same computation of Xavg(t).

Y1(t)

)(tX

Y2(t)

r1

r2

Decoder

31RHDM 2001

State transition diagram for switching versions

State 1 State 2

222

2

1

1 )( and))(1()()( rtXr

tXCr

tYr

tYavg

avg≥−≥+

∆<+−<+2

2

1

1

22

2

1

1 )()(or ))(1()()(r

tYr

tYr

tXCr

tYr

tY avg

Stream v1 Stream v2

(1)

(2)

32RHDM 2001

First implementation of switching versions

The server always transmits data from v2 beginning with thefirst video frame that has not yet been buffered in v1.

7 6 5 4

3 2 1

7 6 5 4 3 2 1Decoder

v2

v1Internet

3 2 1

7 6 5

7 6 5

4

4

Decoderv2

v1Internet

33RHDM 2001

Adding/Dropping Layers : Review

Base Layer (BL) and Enhancement Layers (EL).

To decode higher quality layers, all lower quality layers must be available to the decoder. Can add/drop layers to adjust to the available bandwidth.

Enhancement Layer 1Enhancement Layer 2 7

7

7

6

6

6

5

5

5

4

4

4

3

3

3

2

2

2

1

1

1Base Layer

7

7

7

6

6

6

5

5

5 4 3

2

2

1

1Selection Transmission

34RHDM 2001

Adding/Dropping Layers Control Policy

Recall:Yb(t))()( tXtbπ

)()( tXteπ

)(tX

Ye(t)

rb

re

Decoder

⎪⎩

⎪⎨⎧

==

=

+ eb

bb

b

rrrt

t

απ

π

ˆ)(

1)( => Stream BL only

=> Stream BL & EL in proportion to their consumption rate

35RHDM 2001

State transition diagram for adding/dropping layers

State 1 State 2απ ˆ)( =tb

eavgb

avg

b

b rtXr

tXCr

tY≥−≥ )().ˆ-(1 and))(.ˆ1(

)(αα

1)( =tbπ

∆<−<b

b

b

avg

b

b

rtY

rtXC

rtY )(

or ))(.ˆ1()(

α

(1)

(2)

36RHDM 2001

First implementation for adding/dropping layers

Transmit the enhancement layer data with the same playbackdeadline as the BL currently being transmitted.

7

7

6

6

5

5

4

4

3

3

2

2

1

1BL

EL DecoderInternet

7

7

6

6

5

5

4

4 3 2Decoder

1BL

ELInternet

37RHDM 2001

Immediate Enhancement for adding/dropping layers

Transmit the enhancement layer data with the earliest playbackdeadline. => synchronization is more complex

7

7

6

6

5

5

4

4

3

3

2

2

1

1BL

EL DecoderInternet

7

7

6

6

5

5

4

4 3 2Decoder

1BL

EL

1

23Internet

38RHDM 2001

Immediate Enhancement for switching versions

Transmit data from v2 beginning with the frame which has theearliest playback deadline

7 6 5 4 3 2

7 6 5 4 3 2 1Decoder

v2

v1Internet

1

1

3 2 1

7 6 5

7 6 5

4

4Decoder

v2

v1

3 2Internet

39RHDM 2001

Comparison of Rates

In order to make a fair comparison :BL and v1 have the same perceptual quality.BL+EL and v2 have the same perceptual quality.

Layering has a coding penalty H percent (between 1% and 10%).Assume that all the coding overhead is associated with the EL:

1

2).1(

⎩⎨⎧

=+=+

rrrHrr

b

eb

7

7

6

6

5

5

4

4

3

3

2

2

1

1BL

ELrb

re7 6 5 4 3 2 1

7 6 5 4 3 2 1Version 1

Version 2 r2

r1

40RHDM 2001

Simulations1-hour throughput traces from TCP connections on the Internet, averaged over 1 sec=> TCP-friendly bandwidth conditions

3 different performance metrics to compare performance:

Fraction of high-quality viewing time (th).Fraction of time the decoder can not display the video (td).Quality fluctuations (S).

Study behavior of our heuristics under different bandwidth conditions.

41RHDM 2001

ResultsFirst implementation:

Performance of adding/dropping layers deteriorates as layering overhead increases.

Immediate enhancement:Inefficient for switching versions (waste of bandwidth).Under certain bandwidth conditions, adding/dropping layers attains higher th than switching versions for H up to 5% r2/X =0.7 r2/X =1.0 r2/X =1.3

th td S th td S th td S

Versions 94% 0% 5 58% 0% 5 44% 0.4% 5

Layers-immediate H = 1 % 95% 0% 11 62% 0% 21 44% 0.4% 21

Layers-immediate H = 5 % 92% 0% 17 60% 0% 25 41% 0.4% 25

42RHDM 2001

What we just learned:

Simple implementation :Switching versions always performs better than adding/dropping layers because of layering overhead

Immediate enhancement implementation:layering’s enhanced flexibility can compensate the loss

in high quality viewing time due to layering overhead

Neither scheme seems to dominate:adding/dropping layers is probably better when streams pass through caches

43RHDM 2001

Outline of the Talk




IV Asynchronous Interactive Audio Streaming

44RHDM 2001

Layered Video Through Caches (Infocom 2001)

45RHDM 2001

Local Access Network(abundant bandwidth)

Wide AreaNetwork

G

C Caching proxy

Origin server

Clients

Model for Layered CachingM videos, each has L layersT(m) = duration of video mEach layer has rate bl(m)Revenue R(j, m) = j layers of mRequests for “j layers of m”cm = number of layers cached for video mCaching Policy: c = (c1,…,cM)

46RHDM 2001

Stream Delivery

Caching proxyLink (Stochastic Knapsack)

Requestj of m

Is j of mcached?

Yes

No

Capacity CML classes, one per request typeRequest holds rj(m) of bandwidth for T(m)

Admit if enough capacity, otherwise reject request Calculate expected blocking probability

∑+=

=j

cllj

m

mbmr1

)()(

Calculate long run total rate of revenue

47RHDM 2001

Optimization Problem

Maximize revenue

subject to∑∑

= =

−=M

m

L

jmjBmjpmjRR

1 1)),(1)(,(),()(max cc c λ

GmTmbM

m

c

ll

m

≤∑∑= =

)()(1 1

48RHDM 2001

Optimal Caching

Analytically intractable

Exhaustive searches not feasibleM = 50, L = 2, G = 20 streams2.9 * 1016 possibilities

Need heuristics

49RHDM 2001

Heuristics

Assign utility to each layerCache layers in decreasing utilityThree utility definitions:

Popularity: Popularity of layer + higher layersRevenue: Popularity * revenueRevenue density: Revenue / size

Layer is cached only if all lower layers are cached

50RHDM 2001

Definitions of Heuristics

Popularity heuristic

Revenue heuristic

Revenue density heuristic

∑=

=L

ljml mjpu ),(,

∑=

=L

ljml mjpmjRu ),(),(,

∑=

=L

lj jml mTmb

mjpmjRu)()(

),(),(,

51RHDM 2001

Evaluation Methodology

1000 videos, each 2 layersRates uniformly distributed: 0.1 to 3 MbpsRevenue uniformly distributed: 1 to 10Length exponential, mean 1 hourZipf-popularity (parameter 1.0)G = 12-560 GB (0.9-42 % of video bytes)C = 10-150 Mbit/s

52RHDM 2001

Evaluation - Link Capacity

53RHDM 2001

Evaluation - Cache Size

54RHDM 2001

What we just learned:

Interesting stochastic knapsack model for 2-resource layered video caching problem3 heuristics: best is revenue densityWork in progress:

Compare versions and layering through caches

55RHDM 2001

Outline of the Talk




IV Asynchronous Interactive Audio Streaming (Wimba startup)

56RHDM 2001

Wimbahttp://www.wimba.comAsynchronous Interactive VoiceeLearning applicationsbringing together the telephone and Internet worlds

57RHDM 2001

Documents

Streaming Stored Layered Video in the Internet