View
47
Download
0
Category
Tags:
Preview:
DESCRIPTION
Streaming Video Over the Internet. Andreas Panteli 03/05/2012 ECE 654 Advanced Computer Networks. Introduction. Video principles When an image appears on the retina it is retained for some millisecond before decaying - PowerPoint PPT Presentation
Citation preview
Streaming Video Over The Internet1
Streaming Video Over the Internet
Andreas Panteli
03/05/2012
ECE 654 Advanced Computer Networks
03 May 2012
Streaming Video Over The Internet2
Introduction Video principles
When an image appears on the retina it is retained for some millisecond before decaying
If a sequence of images is drawn line by line at 50 images/sec the eye does not notice that it is looking at discrete images
Streaming Video Video content need not be downloaded in full It is being played out while parts of the content are being received
and decoded Real-time multimedia have timing constraints
Audio and video data must be played out continuously Bandwidth, delay and loss requirements
No QoS guarantees to streaming video, from the current best-effort Internet!
03 May 2012
Streaming Video Over The Internet3
An Architecture for Video Streaming
03 May 2012
Raw Video
CompressedAudio
Application-layerQoS ControlCompressed
Video
TransportProtocol
VideoDecoder
AudioDecoder
Raw Audio
VideoCompression
AudioCompression
Application-layerQoS Control
TransportProtocol
MediaSynchronization
Storage Device
Streaming Server Client/Receiver
Internet(Continuous media distribution services)
Streaming Video Over The Internet4
Key Areas
Video Compression
Application-Layer QoS Control
Continuous Media Distribution Services
Media Synchronization Mechanisms
Protocols for Streaming Media
03 May 2012
Streaming Video Over The Internet5
Key Areas
Video Compression JPEG standard MPEG standard
Application-Layer QoS Control
Continuous Media Distribution Services
Media Synchronization Mechanisms
Protocols for Streaming Media
03 May 2012
Streaming Video Over The Internet6
Digital Video Digital video is represented as a sequent of frames, each
consisting of a rectangular grid of picture elements (pixels) For color video, 8 bits for each of the RGB colors are being
used 24 bits/pixel => 16 million colors
To produce smooth motion, digital video, we must display at least 25 frames/sec, but to avoid flickering we need to repaint each frame on the screen twice
Smoothness and flickering affects the bandwidth requirements Consider the lower video configuration for computer monitor
1024x768 with 24 bits/pixel and 25 frames/sec Need to be fed at 472 Mbps
We need video compression!03 May 2012
Streaming Video Over The Internet7
Video Compression Two algorithms are needed
Encoding and Decoding Asymmetries of the algorithms
Acceptable for the encoding algorithm to be slow and require expensive HW, if the decoding algorithm is fast Only for video on demand For real-time multimedia like video conferencing, slow encoding
is unacceptable Acceptable to have a slightly different video signal than the original,
after encoding and decoding (Lossy system) Lossy systems are important
Accepting a small amount of information loss can give a huge payoff in terms of the compression ratio possible
03 May 2012
Streaming Video Over The Internet8
The JPEG Standard Compression algorithm for still images
Could be applied to each image in succession to achieve video compression
03 May 2012
Compute the luminance Y and the two chrominances I and Q Y = 0.30R + 0.59G + 0.11B I = 0.60R – 0.28G - 0.32B Q = 0.21R – 0.52G + 0.31B
Compress the two chrominances twice than the luminance Lossy reduction, but the eye barely notices it since it responds to
luminance more than to chrominance JPEG often produces 20:1 compression or better Decoding a JPEG image requires running the algorithm backwards
Streaming Video Over The Internet9
The MPEG Standard (1/2) Main algorithms for video compression and international
standards since 1993 Can compress both audio and video
03 May 2012
Three parts in MPEG-1 Audio Video System (integrates the other two)
Streaming Video Over The Internet10
The MPEG Standard (2/2) Two kinds of redundancies in videos
Spatial redundancy Coding each frame separately with JPEG
Temporal redundancy Consecutive frames are often almost identical
MPEG-1 output consists of four kinds of frames I (Intracoded) frames: Self-contained JPEG-encoded still pictures P (Predictive) frames: Block-by-Block difference with the last frame B (Bidirectional) frames: Differences between the last and the next
frame D (DC-coded) frames: Block averages used for fast-forward
MPEG-1 can achieve a compression factor of 40 Decoding has about the same complexity as encoding
03 May 2012
Streaming Video Over The Internet11
Key Areas
Video Compression
Application-Layer QoS Control Congestion control Error control
Continuous Media Distribution Services
Media Synchronization Mechanisms
Protocols for Streaming Media
03 May 2012
Streaming Video Over The Internet12
Application-Layer QoS Control
Congestion Control Rate Control
Source-Based Rate Control
Receiver-Based Rate Control
Hybrid-Rate Control Rate Shaping
Frame-dropping filter Layer-dropping filter Frequency filter Re quantization filter
Error Control FEC Delay-Constrained
Retransmission Error-Resilient Encoding Error-Concealment
03 May 2012
Avoids congestion and maximizes video quality in the presence to packet loss
Streaming Video Over The Internet13
Rate Control (1/2) Technique to determine the sending rate of video traffic
based on the estimated available bandwidth of the network Source-based rate control
The sender is responsible for adapting the video transmission rate based on feedback from the receivers
Probe-based approach Sending rate adjustment to maintain the packet loss ratio below a
certain threshold Model-based approach
03 May 2012
Streaming Video Over The Internet14
Rate Control (2/2) Receiver-based rate control
The receivers regulate the receiving rate of video streams by adding/dropping channels
Probe-based approach When no congestion is detected, join a channel
=> increase of it’s receiving rate When congestion is detected, drop a channel
=> reduction of it’s receiving rate Model-based approach
Explicit estimation for the available network bandwidth Also based on Equation (1)
Hybrid rate Control Combination of the above mentioned techniques
03 May 2012
Streaming Video Over The Internet15
Rate Shaping Matches the rate of a pre-compressed video bit stream to
the target rate constraint A rate shaper (or filter) is required for source based rate
control
03 May 2012
Types of filters Codec filter
Performs transcoding between different compression schemes
Frame-dropping filter I, P ,B , D frames in MPEG
Frequency filter Operates in the frequency domain (DCT
coefficients) Re-quantization filter
Re-quantizes the DCT coefficients with a larger quantization step
Streaming Video Over The Internet16
Error Control (1/2) Forward Error Correction (FEC)
Add redundant information so that original message can be reconstructed in the presence of packet loss
Channel coding Stream is chopped into segments Each segment is packetized into k packets A block code generates an n-packet block, where n>k User only needs to receive any k packets out of n of the block
Source coding N-th group of blocks contains redundant information of (n-1)th
group of blocks Joint source/channel coding
Optimal rate allocation between Channel and Source coding
03 May 2012
Streaming Video Over The Internet17
Error Control (2/2) Delay-constraint Retransmission
03 May 2012
Error-resilient Encoding Enhances robustness of compressed video to packet loss A video is compressed into multiple streams (descriptions) Each description provides acceptable visual quality Combined descriptions provide a better visual quality
Error Concealment Performed by the receiver when packet loss has already occurred Spatial interpolation
Missing pixel values are reconstructed using neighboring pixels Temporal interpolation
Loss data is reconstructed using data from the previous frame
When the receiver detects the loss of packet N
if (Tc + RTT + Ds < Td(N))send the request for packet N to
the sender
WhereTc current timeRTT estimated round-trip timeDs a slack term (e.g receiver’s decoding delay)Td(N) time when packet N is scheduled for display
Streaming Video Over The Internet18
Key Areas
Video Compression
Application-Layer QoS Control
Continuous Media Distribution Services Network Filtering Application-level Multicast Content Replication
Media Synchronization Mechanisms
Protocols for Streaming Media
03 May 2012
Streaming Video Over The Internet19
Network Filtering Adequate support from the network is critical in order to
provide quality multimedia presentations
03 May 2012
Routers have no knowledge of the format of the media streams Randomly discard packets
Filters receive the client’s requests and adapt the stream sent by the server accordingly
Typically, frame-dropping filters are used Drop packets in a way that gracefully
degrades the stream’s quality Increase bandwidth efficiency by discarding
late frames
R: Router
Streaming Video Over The Internet20
Application-level Multicast
Internet’s original design fails to effectively support streaming-media multicast IP multicast has barriers
Scalability, network management, support for higher layer functionality (e.g. error, flow and congestion control)
Application-level multicast to the rescue Builds a multicast service on top of the Internet
Media bridge Routing in the application layer Each Media Bridge is interconnected with one or more neighboring Media-Bridges
through explicit configuration Collectively employ a distributed application-level multicast routing algorthm
03 May 2012
Unicast Multicast
Streaming Video Over The Internet21
Content Replication
Mirroring Scatter copies of the
original multimedia files in different locations around the Internet
Clients retrieve data from nearest duplicate server
Caching Exploits temporal locality Cache retrieves data from
server Clients retrieve data from
cache if available Cache sharing and cache
hierarchies03 May 2012
Advantages Increased network’s bandwidth efficiency Reduced load on streaming servers Reduced latency for clients Increased availability
Streaming Video Over The Internet22
Key Areas
Video Compression
Application-Layer QoS Control
Continuous Media Distribution Services
Media Synchronization Mechanisms Intra-stream Synchronization Inter-stream Synchronization Inter-object Synchronization
Protocols for Streaming Media
03 May 2012
Streaming Video Over The Internet23
Media Synchronization Multimedia applications
Various integrated media streams that must be presented in a synchronized fashion
Media streams may lose synchronization after moving from the server to the client
Three layers of synchronization Intra-stream synchronization
Maintain continuity of logical data units Inter-stream synchronization
Maintain temporal relationships among different continuous media e.g. movements of the lips of a speaker do not correspond to the
presented audio Inter-object synchronization
Relation with time-independent data, such as text, still image and streams e.g. during a presentation, audio is commenting one slide while another
slide is presented
03 May 2012
Streaming Video Over The Internet24
Media Synchronization Mechanisms Delay in the network is unpredictable due to the best-effort
nature of the Internet Axes-based specifications, or time-stamping
At the source, a stream is time-stamped to keep temporal information within the stream and with respect to other streams
Mechanisms implemented on the end systems Preventive mechanisms
Minimize latencies and jitters i.e. disk-reading scheduling algorithms, network transport
protocols, operating systems and synchronization schedulers Corrective mechanisms
Recover synchronization in the presence of synchronization errors i.e. Stream synchronization protocol (SSP)
Concept of ”intentional delay”
03 May 2012
Streaming Video Over The Internet25
Key Areas
Video Compression
Application-Layer QoS Control
Continuous Media Distribution Services
Media Synchronization Mechanisms
Protocols for Streaming Media Transport Protocols Session Control Protocols
03 May 2012
Streaming Video Over The Internet26
Protocols for Streaming Video Data Plane
Compressed video/audio is packetized at the real-time transport protocol (RTP) layer
RTP packetized streams are then passed to the UDP/TCP layer and IP layer
IP packets are transported over the Internet
03 May 2012
Control Plane Real-time control protocol (RTCP) and real-time streaming protocol
(RTSP) packets are multiplexed at the UDP/TCP layer IP packets are transported over the Internet
Streaming Video Over The Internet27
Transport Protocols UDP and TCP provide basic transport functions
UDP and TCP Multiplexing, Error control (e.g. checksum)
TCP only: Retransmission, Congestion control, Flow control
UDP is typically employed as the transport protocol for video streams No packet delivery guarrantees Receiver needs to rely in upper layer to detect packet loss
RTP and RTCP run on top of UDP/TCP
03 May 2012
RTP : does not guarantee QoS or reliable delivery Time stamping Sequence numbering Payload type identification Source identification
RTCP : designed to work in conjunction with RTP QoS feedback Participant identification Control packets scaling Inter-media synchronization Minimal session control
information
Streaming Video Over The Internet28
Session Control Protocols RTSP (Real-Time Streaming Protocol)
Support VCR-like control operations (stop, pause/resume, fast forward, fast backward)
Choosing delivery channel (UDT, multicast UDT, TCP) Establish and control streams of continuous audio and video media
between the media servers and the clients Media retrieval: ask server to setup a session to send the
requested media data Adding media to an existing session: server or client can
notify each other about any additional media becoming available to the established session
SIP (Session Initiation Protocol) Similar to RTSP Support user mobility by proxying and redirecting request
03 May 2012
Streaming Video Over The Internet29
References A. S. Tanenbaum, Computer Networks, 4th ed. New Jersey:
Prentice Hall, 2002. Dapeng Wu, Hou, Y.T., Wenwu Zhu, Ya-Qin Zhang, Peha,
J.M., "Streaming video over the Internet: approaches and directions“, Circuits and Systems for Video Technology, IEEE Transactions on , vol.11, no.3, pp.282-300, Mar 2001
Dapeng Wu, Yiwei Thoms Hou, Ya-Qin Zhang, "Transporting real-time video over the Internet: challenges and approaches," Proceedings of the IEEE , vol.88, no.12, pp.1855-1877, Dec 2000
Dapeng Wu and Y. Thomas Hou and Jason Yao and Y. Thomas and Hou Jason Yao and H. Jonathan Chao, “Real-time Video over the Internet: A Big Picture”, IEEE NetWorld+Interop, 2000
03 May 2012
Streaming Video Over The Internet30
Questions?
Thank you!
03 May 2012
Recommended