Streaming Video Over the Internet

Streaming Video Over The Internet1

Streaming Video Over the Internet

Andreas Panteli

03/05/2012

ECE 654 Advanced Computer Networks

03 May 2012


Introduction Video principles

When an image appears on the retina it is retained for some millisecond before decaying

If a sequence of images is drawn line by line at 50 images/sec the eye does not notice that it is looking at discrete images

Streaming Video Video content need not be downloaded in full It is being played out while parts of the content are being received

and decoded Real-time multimedia have timing constraints

Audio and video data must be played out continuously Bandwidth, delay and loss requirements

No QoS guarantees to streaming video, from the current best-effort Internet!

03 May 2012


An Architecture for Video Streaming

03 May 2012

Raw Video

CompressedAudio

Application-layerQoS ControlCompressed

Video

TransportProtocol

VideoDecoder

AudioDecoder

Raw Audio

VideoCompression

AudioCompression

Application-layerQoS Control

TransportProtocol

MediaSynchronization

Storage Device

Streaming Server Client/Receiver

Internet(Continuous media distribution services)


Key Areas

Video Compression

Application-Layer QoS Control

Continuous Media Distribution Services

Media Synchronization Mechanisms

Protocols for Streaming Media

03 May 2012


Key Areas

Video Compression JPEG standard MPEG standard





03 May 2012


Digital Video Digital video is represented as a sequent of frames, each

consisting of a rectangular grid of picture elements (pixels) For color video, 8 bits for each of the RGB colors are being

used 24 bits/pixel => 16 million colors

To produce smooth motion, digital video, we must display at least 25 frames/sec, but to avoid flickering we need to repaint each frame on the screen twice

Smoothness and flickering affects the bandwidth requirements Consider the lower video configuration for computer monitor

1024x768 with 24 bits/pixel and 25 frames/sec Need to be fed at 472 Mbps

We need video compression!03 May 2012


Video Compression Two algorithms are needed

Encoding and Decoding Asymmetries of the algorithms

Acceptable for the encoding algorithm to be slow and require expensive HW, if the decoding algorithm is fast Only for video on demand For real-time multimedia like video conferencing, slow encoding

is unacceptable Acceptable to have a slightly different video signal than the original,

after encoding and decoding (Lossy system) Lossy systems are important

Accepting a small amount of information loss can give a huge payoff in terms of the compression ratio possible

03 May 2012


The JPEG Standard Compression algorithm for still images

Could be applied to each image in succession to achieve video compression

03 May 2012

Compute the luminance Y and the two chrominances I and Q Y = 0.30R + 0.59G + 0.11B I = 0.60R – 0.28G - 0.32B Q = 0.21R – 0.52G + 0.31B

Compress the two chrominances twice than the luminance Lossy reduction, but the eye barely notices it since it responds to

luminance more than to chrominance JPEG often produces 20:1 compression or better Decoding a JPEG image requires running the algorithm backwards


The MPEG Standard (1/2) Main algorithms for video compression and international

standards since 1993 Can compress both audio and video

03 May 2012

Three parts in MPEG-1 Audio Video System (integrates the other two)


The MPEG Standard (2/2) Two kinds of redundancies in videos

Spatial redundancy Coding each frame separately with JPEG

Temporal redundancy Consecutive frames are often almost identical

MPEG-1 output consists of four kinds of frames I (Intracoded) frames: Self-contained JPEG-encoded still pictures P (Predictive) frames: Block-by-Block difference with the last frame B (Bidirectional) frames: Differences between the last and the next

frame D (DC-coded) frames: Block averages used for fast-forward

MPEG-1 can achieve a compression factor of 40 Decoding has about the same complexity as encoding

03 May 2012


Key Areas

Video Compression

Application-Layer QoS Control Congestion control Error control




03 May 2012



Congestion Control Rate Control

Source-Based Rate Control

Receiver-Based Rate Control

Hybrid-Rate Control Rate Shaping

Frame-dropping filter Layer-dropping filter Frequency filter Re quantization filter

Error Control FEC Delay-Constrained

Retransmission Error-Resilient Encoding Error-Concealment

03 May 2012

Avoids congestion and maximizes video quality in the presence to packet loss


Rate Control (1/2) Technique to determine the sending rate of video traffic

based on the estimated available bandwidth of the network Source-based rate control

The sender is responsible for adapting the video transmission rate based on feedback from the receivers

Probe-based approach Sending rate adjustment to maintain the packet loss ratio below a

certain threshold Model-based approach

03 May 2012


Rate Control (2/2) Receiver-based rate control

The receivers regulate the receiving rate of video streams by adding/dropping channels

Probe-based approach When no congestion is detected, join a channel

=> increase of it’s receiving rate When congestion is detected, drop a channel

=> reduction of it’s receiving rate Model-based approach

Explicit estimation for the available network bandwidth Also based on Equation (1)

Hybrid rate Control Combination of the above mentioned techniques

03 May 2012


Rate Shaping Matches the rate of a pre-compressed video bit stream to

the target rate constraint A rate shaper (or filter) is required for source based rate

control

03 May 2012

Types of filters Codec filter

Performs transcoding between different compression schemes

Frame-dropping filter I, P ,B , D frames in MPEG

Frequency filter Operates in the frequency domain (DCT

coefficients) Re-quantization filter

Re-quantizes the DCT coefficients with a larger quantization step


Error Control (1/2) Forward Error Correction (FEC)

Add redundant information so that original message can be reconstructed in the presence of packet loss

Channel coding Stream is chopped into segments Each segment is packetized into k packets A block code generates an n-packet block, where n>k User only needs to receive any k packets out of n of the block

Source coding N-th group of blocks contains redundant information of (n-1)th

group of blocks Joint source/channel coding

Optimal rate allocation between Channel and Source coding

03 May 2012


Error Control (2/2) Delay-constraint Retransmission

03 May 2012

Error-resilient Encoding Enhances robustness of compressed video to packet loss A video is compressed into multiple streams (descriptions) Each description provides acceptable visual quality Combined descriptions provide a better visual quality

Error Concealment Performed by the receiver when packet loss has already occurred Spatial interpolation

Missing pixel values are reconstructed using neighboring pixels Temporal interpolation

Loss data is reconstructed using data from the previous frame

When the receiver detects the loss of packet N

if (Tc + RTT + Ds < Td(N))send the request for packet N to

the sender

WhereTc current timeRTT estimated round-trip timeDs a slack term (e.g receiver’s decoding delay)Td(N) time when packet N is scheduled for display


Key Areas

Video Compression


Continuous Media Distribution Services Network Filtering Application-level Multicast Content Replication



03 May 2012


Network Filtering Adequate support from the network is critical in order to

provide quality multimedia presentations

03 May 2012

Routers have no knowledge of the format of the media streams Randomly discard packets

Filters receive the client’s requests and adapt the stream sent by the server accordingly

Typically, frame-dropping filters are used Drop packets in a way that gracefully

degrades the stream’s quality Increase bandwidth efficiency by discarding

late frames

R: Router


Application-level Multicast

Internet’s original design fails to effectively support streaming-media multicast IP multicast has barriers

Scalability, network management, support for higher layer functionality (e.g. error, flow and congestion control)

Application-level multicast to the rescue Builds a multicast service on top of the Internet

Media bridge Routing in the application layer Each Media Bridge is interconnected with one or more neighboring Media-Bridges

through explicit configuration Collectively employ a distributed application-level multicast routing algorthm

03 May 2012

Unicast Multicast


Content Replication

Mirroring Scatter copies of the

original multimedia files in different locations around the Internet

Clients retrieve data from nearest duplicate server

Caching Exploits temporal locality Cache retrieves data from

server Clients retrieve data from

cache if available Cache sharing and cache

hierarchies03 May 2012

Advantages Increased network’s bandwidth efficiency Reduced load on streaming servers Reduced latency for clients Increased availability


Key Areas

Video Compression



Media Synchronization Mechanisms Intra-stream Synchronization Inter-stream Synchronization Inter-object Synchronization


03 May 2012


Media Synchronization Multimedia applications

Various integrated media streams that must be presented in a synchronized fashion

Media streams may lose synchronization after moving from the server to the client

Three layers of synchronization Intra-stream synchronization

Maintain continuity of logical data units Inter-stream synchronization

Maintain temporal relationships among different continuous media e.g. movements of the lips of a speaker do not correspond to the

presented audio Inter-object synchronization

Relation with time-independent data, such as text, still image and streams e.g. during a presentation, audio is commenting one slide while another

slide is presented

03 May 2012


Media Synchronization Mechanisms Delay in the network is unpredictable due to the best-effort

nature of the Internet Axes-based specifications, or time-stamping

At the source, a stream is time-stamped to keep temporal information within the stream and with respect to other streams

Mechanisms implemented on the end systems Preventive mechanisms

Minimize latencies and jitters i.e. disk-reading scheduling algorithms, network transport

protocols, operating systems and synchronization schedulers Corrective mechanisms

Recover synchronization in the presence of synchronization errors i.e. Stream synchronization protocol (SSP)

Concept of ”intentional delay”

03 May 2012


Key Areas

Video Compression




Protocols for Streaming Media Transport Protocols Session Control Protocols

03 May 2012


Protocols for Streaming Video Data Plane

Compressed video/audio is packetized at the real-time transport protocol (RTP) layer

RTP packetized streams are then passed to the UDP/TCP layer and IP layer

IP packets are transported over the Internet

03 May 2012

Control Plane Real-time control protocol (RTCP) and real-time streaming protocol

(RTSP) packets are multiplexed at the UDP/TCP layer IP packets are transported over the Internet


Transport Protocols UDP and TCP provide basic transport functions

UDP and TCP Multiplexing, Error control (e.g. checksum)

TCP only: Retransmission, Congestion control, Flow control

UDP is typically employed as the transport protocol for video streams No packet delivery guarrantees Receiver needs to rely in upper layer to detect packet loss

RTP and RTCP run on top of UDP/TCP

03 May 2012

RTP : does not guarantee QoS or reliable delivery Time stamping Sequence numbering Payload type identification Source identification

RTCP : designed to work in conjunction with RTP QoS feedback Participant identification Control packets scaling Inter-media synchronization Minimal session control

information


Session Control Protocols RTSP (Real-Time Streaming Protocol)

Support VCR-like control operations (stop, pause/resume, fast forward, fast backward)

Choosing delivery channel (UDT, multicast UDT, TCP) Establish and control streams of continuous audio and video media

between the media servers and the clients Media retrieval: ask server to setup a session to send the

requested media data Adding media to an existing session: server or client can

notify each other about any additional media becoming available to the established session

SIP (Session Initiation Protocol) Similar to RTSP Support user mobility by proxying and redirecting request

03 May 2012


References A. S. Tanenbaum, Computer Networks, 4th ed. New Jersey:

Prentice Hall, 2002. Dapeng Wu, Hou, Y.T., Wenwu Zhu, Ya-Qin Zhang, Peha,

J.M., "Streaming video over the Internet: approaches and directions“, Circuits and Systems for Video Technology, IEEE Transactions on , vol.11, no.3, pp.282-300, Mar 2001

Dapeng Wu, Yiwei Thoms Hou, Ya-Qin Zhang, "Transporting real-time video over the Internet: challenges and approaches," Proceedings of the IEEE , vol.88, no.12, pp.1855-1877, Dec 2000

Dapeng Wu and Y. Thomas Hou and Jason Yao and Y. Thomas and Hou Jason Yao and H. Jonathan Chao, “Real-time Video over the Internet: A Big Picture”, IEEE NetWorld+Interop, 2000

03 May 2012


Questions?

Thank you!

03 May 2012

Documents

Streaming Video Over the Internet