Video StreamingVideo Streaming

Ali Saman TosunComputer Science

Department

2

Broadcast to True Media-on-Broadcast to True Media-on-DemandDemand

Broadcast (No-VoD) Traditional, no control

Pay-per-view (PPV) Paid specialized service

Near Video On Demand (N-VoD) Same media distributed in regular time intervals Simulated forward / backward

True Video On Demand (T-VoD) Full control for the presentation, VCR capabilities Bi-directional connection

3

Streaming Stored Video

Streaming media stored at source transmitted to client streaming: client playout begins

before all data has arrived

timing constraint for still-to-be transmitted data: in time for playout

4

constant bit rate videotransmission

Cum

ula

tive

data

time

variablenetwork

delay

client videoreception

constant bit rate video playout at client

client playoutdelay

bu

ffere

dvid

eo

Streaming VideoStreaming Video

Client-side buffering, playout delay compensate for network-added delay, delay jitter

5

Smoothing Stored VideoSmoothing Stored Video

For prerecorded video streams: All video frames stored in advance at server Prior knowledge of all frame sizes (fi, i=1,2,..,n) Prior knowledge of client buffer size (b) workahead transmission into client buffer

12n b bytes

Server Client

6

Smoothing ConstraintsSmoothing Constraints

Given frame sizes {fi} and buffer size b Buffer underflow constraint (Lk = f1 + f2 + … +

fk) Buffer overflow constraint (Uk = min(Lk + b, Ln)) Find a schedule Sk between the constraints

Algorithm minimizes peak and variability

L

U

Stime (in frames)

num

ber

of b

ytes

rate changes

7

Proxy-based Video Proxy-based Video DistributionDistribution

Server

Proxy

Client

Client

Proxy adapts videoProxy caches video

8

Proxy OperationsProxy Operations Drop frames

Drop B,P frames if not enough bandwidth Quality Adaptation

Transcoding Change quantization value Most of current systems don’t support

Video staging, caching, patching Staging: store partial frames in proxy Prefix caching: store first few minutes of movie Patching: multiple users use same video

9

Online SmoothingOnline Smoothing

Larger window w reduces burstiness, but… Larger buffer at the source/proxy Larger processing load to compute schedule Larger playback delay at the client

b bytes

Source/Proxy Client

streamingvideo

Source or proxy can delay the stream by w time units:

stream withdelay w

10

Online Smoothing ModelOnline Smoothing Model

Arrival of Ai bits to proxy by time i in frames Smoothing buffer of B bits at proxy Smoothing window (playout delay) of w frames Playout of Di-w bits by client by time i Playout buffer of b bits at client

Transmission of Si bits by proxy by time i

B bDi-wAi Si

proxy client

11

Online Online Smoothing Must send enough to avoid underflow at client

Si must be at least Di-w

Cannot send more than the client can store Si must be at most Di-w + b

Cannot send more than the data that has arrived Si must be at most Ai

Must send enough to avoid overflow at proxy Si must be at least Ai - B

max{Di-w, Ai - B} Si <= min{Di-w + b, Ai}

12

Online Smoothing Online Smoothing ConstraintsConstraints

Source/proxy has w frames ahead of current time t:

LU

time (in frames)

num

ber

of b

ytes

t t+w-1

don’t know the future

Modified smoothing constraints as more frames arrive...

?

13

Smoothing Smoothing Star WarsStar Wars

GOP averages 2-second window 30-second window

MPEG-1 Star Wars,12-frame group-of-pictures Max frame 23160 bytes, mean frame 1950 bytes Client buffer b=512 kbytes

14

Prefix Caching to Avoid Start-Up Prefix Caching to Avoid Start-Up DelayDelay

Avoid start-up delay for prerecorded streams Proxy caches initial part of popular video streams Proxy starts satisfying client request more quickly Proxy requests remainder of the stream from server

smooth over large window without large delay

• Use prefix caching to hide other Internet delays – TCP connection from browser to server– TCP connection from player to server– Dejitter buffer at the client to tolerate jitter– Retransmission of lost packets

apply to “point-and-click” Web video streams

15

Changes to Smoothing Changes to Smoothing ModelModel

Separate parameter s for client start-up delay Prefix cache stores the first w-s frames Arrival vector Ai includes cached frames Prefix buffer does not empty after transmission Send entire prefix before overflow of bs Frame sizes may be known in advance (cached)

bs

bp

bc

Ai Di-sSi

16

Scalable codingScalable coding

Typically used asLayered coding

A base layer Provides basic quality Must always be

transferred One or more

enhancement layers Improve quality Transferred if possible Sending rate

Qu

alit

yBest p

ossible quality

at possi

ble sending ra

te

Base layer

Enhancement layer

17

Temporal ScalabilityTemporal Scalability Frames can be dropped

In a controlled manner Frame dropping does not violate dependancies Low gain example: B-frame dropping in MPEG-1

18

Spatial ScalabilitySpatial Scalability Base layer

Downsample the original image Send like a lower resolution version

Enhancement layer Subtract base layer pixels from all pixels Send like a normal resolution version

If enhancement layer arrives at client Decode both layers Add layers

72

75 83

6173

-1 -12

2 10

Base layer

Enhancement layerBetter compression due to low values

Less data to code

19

SNR ScalabilitySNR Scalability SNR – signal-to-noise ratio Idea

Base layer Is regularly DCT encoded A lot of data is removed using quantization

Enhancement layer is regularly DCT encoded Run Inverse DCT on quantized base layer Subtract from original DCT encode the result

If enhancement layer arrives at client Add base and enhancement layer before running Inverse

DCT

20

Multiple Description Multiple Description CodingCoding Idea

Encode data in two streams Each stream has acceptable quality Both streams combined have good quality The redundancy between both streams is low

Problem The same relevant information must exist in both

streams Old problem: started for audio coding in telephony Currently a hot topic

21

Delivery Systems Delivery Systems DevelopmentsDevelopments

Network

Saving network resources:

Stream scheduling

Several Programs or

Timelines

22

PatchingPatching

Server resource optimization is possible

multicast

Unicast patch stream

Central server

1st client 2nd client

Join !

cyclicbuffer

23

Proxy Prefix CachingProxy Prefix Caching

Split movie Prefix Suffix

Operation Store prefix in prefix cache

Coordination necessary! On demand

Deliver prefix immediately Prefetch suffix from central

server

Goal Reduce startup latency Hide bandwidth limitations,

delay and/or jitter in backbone Reduce load in backbone

Client

Unicast

Unicast

Central server

Prefix cache

24

Interval Caching (IC)Interval Caching (IC) caches data between requests following requests are thus served from the cache sort intervals on length

Video clip 1

S11

Video clip 1

S11S12

Video clip 1

S12 S11S13

Video clip 2

S22 S21

Video clip 3

S33 S31S32S34

I11I12

I21

I31I32I33

I32 I33 I21I11I31I12

25

Receiver-driven Layered Multicast Receiver-driven Layered Multicast (RLM)(RLM)

Requires IP multicast layered video codec

(preferably exponential thickness) Operation

Each video layer is one IP multicast group Receivers join the base layer and extension layers If they experience loss, they drop layers (leave IP multicast

groups) To add layers, they perform "join experiments“

Advantages Receiver-only decision Congestion affects only sub-tree quality Multicast trees are pruned, sub-trees have only necessary traffic

26

Receiver-driven Layered Multicast Receiver-driven Layered Multicast (RLM)(RLM)