Video Streaming ConceptsVideo Streaming Concepts

Reading:John G. Apostolopoulos, Wai- tian Tan, Susie J. Wee,“Video Streaming: Concepts, Algorithms, and Systems”, HP Laboratories Palo Alto, 2002.

1

Classification of video appsClassification of video apps

Point2point vs multicast vs broadcast Is there a “reverse channel”? Pros and cons of reverse channel

Real-time vs pre-encorded(stored) video What about “almost real-time”?

Interactive vs non-interactive video Static vs dynamic channels

Bandwidth, delay, and loss are static or dynamic

CBR vs VBR channels Do not confuse with CBR/VBR encoding

Packet-switched vs circuit-switched channels QoS support?

Video compression standardsVideo compression standards What do the standards specify?

Encoder and decoder implementations? Bit stream syntax? Decoding process?

Video streaming challengesVideo streaming challenges

Video delivery via file download vs streaming ? Characteristics of Internet are unknown or time-

varying Bandwidth rate control Delay jitter playout buffer Loss error control

Transport and Rate Control Transport and Rate Control for overcoming Time-varying BWfor overcoming Time-varying BW Objectives

How to estimate the appropriate transmission rates dynamically at the time of streaming ? – available BW estimation

How media coding has evolved to support such dynamic changes in transmission rates ? – coding rate control

Rate control for video streaming over TCP TCP is a window (not rate) based transport protocol TCP adjusts the window size based on AIMD congestion-control

algorithm varying throughput (saw-tooth pattern) Retransmission enlarge delay jitter Nevertheless, TCP is often used in video streaming, especially

when receiver window is appropriately sized Stable and scalable TCP rate control TCP guarantees delivery How would you calculate the receiver window size to set the

streaming rate at a certain value?

Transport and Rate Control Transport and Rate Control for overcoming Time-varying BWfor overcoming Time-varying BW Rate control for video streaming over UDP

Separation of error control and rate control TCP-friendly rate control

Mimic average throughput of TCP = f(RTT, p) Smooth the instantaneous fluctuations of TCP’s AIMD algorithm

Rate 1/RTT : video streaming 에 적합한가 ?

Some media streaming system do not perform rate control Multicasting: Identical stream is transmitted to all recipients via

channels of different levels of congestion No feedback channel

Receiver-driven rate control Layered multicast of scalable or layered compressed video

Meeting Transmission BW ConstraintsMeeting Transmission BW Constraints If channel transmission rate < media bit rate, then ?? Transcoding

Decode and re-encode to the desired bit rate Include bit rate reduction, spatial downsampling, frame rate

reduction, changing compression formats Requires application-layer gateways

Multiple file switching Multiple copies of the same content at different bit-rates. Client

chooses the appropriate media rate. Multi-rate switching: enables dynamic switching between

different rates within a session No recompression penalty, but BW waste for multiple copies of

the same media

Scalable compression (layered coding)

7

Evolving Approaches in InternetEvolving Approaches in Internet

IntServ model QoS guarantees BW, packet loss rate, delay on per

flow basis Explicit resource reservation via RSVP

DiffServ model Classify and differentiate among classes based on a

tag(code-point) in each packet

8

Playout Buffer for Overcoming JitterPlayout Buffer for Overcoming Jitter

Solution: Playback buffering Key question: how long

should the playback buffer (or playback delay) be?

Playback buffers have additional advantages: Error recovery through ReTx Error resilience through interleaving Smooth throughput variations

(e.g., due to TCP)

Adaptive Media Playout

Error Control for Overcoming Channel Error Control for Overcoming Channel LossesLosses Bit error vs packet loss

In wired network, almost packet loss due to congestion In wireless network, bit errors or burst errors may cause

To be passed up to the appl. Layer Or, discarded (packet loss)

Solutions? Retransmissions

Commonly used, but constrained by delay budget and playback delay Requires back-channel

Forward Error Correction Send N packets (K data packets + K-N redundant packets). If received

packets >= K correctlry, data packets are recovered. Introduce bandwidth overhead (and potentially delay) Or, cannot recover.

Loss/error concealment Loss/error resilient video coding

Loss/Error ConcealmentLoss/Error Concealment

Pixels, MBs, slices, and entire frame may be lost.

Exploits the correlation along Spatial interpolation Temporal extrapolation (freeze frame) Motion-compensated temporal extrapolation

Motion-compensated block + estimation of lost MV from ??

Error concealment is performed at the decoder

11

Error Resilience Video Coding: Error Resilience Video Coding: Overcoming Loss of Bitstream Synch.Overcoming Loss of Bitstream Synch. Resynch Marker

Put the Resync markers (placed where?)

Reversible VLCs (MPEG-4) Less efficient than VLS

Data Partitioning (MPEG-4) Place most important data just after markers

Application Level Framing (ALF) Design the packet(frame) payload to minimize the effect of loss

12

Error Resilience Video Coding: Error Resilience Video Coding: Overcoming Incorrect State and Error PropagationOvercoming Incorrect State and Error Propagation Overcome error propagation

Use periodic I-pictures (GOP) Use periodic intra-coding of MBs

In point-to-point comm. with back channel (short RTT) Both the encoder and decoder store multiple previously coded

frames Decoder notifies the packet error to the encoder The encode tells which picture should be used as the reference

for the next prediction (Reference Picture Selection in H.263) 13

Scalable (Layered) Video Coding Scalable (Layered) Video Coding for Lossy Networksfor Lossy Networks SVC implicitly assumes broadcast or multicast of

video steams (including overlay multicasting) Base layer and several enhancement layers

need different treatments unequal error protection (UEP), prioritized transmission

Useful if network provides several classes of service or priorities (DiffServ, IntServ)

Under the best effort Internet ?

Multiple Description Video CodingMultiple Description Video Coding Multiple description coding (MDC) video

Several “descriptions” of the same video The more descriptions you receive, the better Significant redundancy among descriptions What if all descriptions are subject to simultaneous losses?

MDC video with path diversity

Media Streaming Protocols and StandardsMedia Streaming Protocols and Standards Protocols for video streaming over the Internet

Media delivery: RTP/RCTP RTP does not provide real-time delivery, neither support any

QoS Only provides time stamps, sequence numbering, video

source identification, payload specification RTCP provides QoS feedback in terms of # of los packets,

delay, inter-arrival jitter, etc.

Media Session Control RTSP (Realtime Streaming Protocol) SIP (Session Initiation Protocol)

Media Description and Announcement SDP (Session Description Protocol) SAP (Session Announcement Protocol)

16

Additional Video Streaming TopicsAdditional Video Streaming Topics

Multicast Should solve heterogeneity problem

Network heterogeneity: different channel conditions (BW, error protection)

Receiver heterogeneity Approaches

Different multicasts for different ranges of intended bit-rates: e.g) simulcast

Different multicasts can contain incremental information (SVC): e.g) layered multicast

End-to-end security and Transcoding Streaming over wired and wireless Links

Loss differentialtion: Loss due to congestion or noise ?

Streaming media CDN17

Streaming Media CDNStreaming Media CDN

Conventional CDN vs streaming media CDN Issues

Application-layer multicast (overlay multicast) Seamless hand-off

MD-CDN Achieve path diversity from the infrastruture of CDN

18