Upload
duane-oliver
View
214
Download
0
Embed Size (px)
Citation preview
Multimedia ApplicationsMultimedia Applications
Ali Saman TosunComputer Science
Department
2
Compression - NecessityCompression - Necessity
E.g., video sequence 25 images/sec.
PAL standard 3 byte/pixel
YUV (luminance + 2 chrominance values) RGB (red-green-blue values)
Image resolution 640 * 480 pixel Data rate = 640 * 480 * 3 Byte * 25/s = 23040000 byte/s
~ 22 MByte/s Approx. 1/16 stream over Ethernet Approx. 1/2 stream over Fast Ethernet
Compression is necessary
3
Multimedia Networking Multimedia Networking Applications Applications
Fundamental characteristics:
Typically delay sensitive
end-to-end delay delay jitter
But loss tolerant: infrequent losses cause minor glitches
Antithesis of data, which are loss intolerant but delay tolerant.
Classes of MM applications1) Streaming stored audio and
video2) Streaming live audio and
video3) Real-time interactive audio
and video
Jitter is the variability of packet delays within the same packet stream
4
From Broadcast to True From Broadcast to True Media-on-DemandMedia-on-Demand
Broadcast (No-VoD) Traditional, no control
Pay-per-view (PPV) Paid specialized service
Quasi Video On Demand (Q-VoD) Distinction into interest groups Temporal control by group change
Near Video On Demand (N-VoD) Same media distributed in regular time intervals Simulated forward / backward
True Video On Demand (T-VoD) Full control for the presentation, VCR capabilities Bi-directional connection
5
Streaming Stored Video Streaming Stored Video
Streaming media stored at source transmitted to client streaming: client playout begins
before all data has arrived
timing constraint for still-to-be transmitted data: in time for playout
6
constant bit rate videotransmission
Cum
ula
tive
data
time
variablenetwork
delay
client videoreception
constant bit rate video playout at client
client playoutdelay
bu
ffere
dvid
eo
Streaming MultimediaStreaming Multimedia
Client-side buffering, playout delay compensate for network-added delay, delay jitter
7
Proxy-based Video Proxy-based Video DistributionDistribution
Server
Proxy
Client
Client
Proxy adapts videoProxy caches video
8
Proxy OperationsProxy Operations Drop frames
Drop B,P frames if not enough bandwidth Quality Adaptation
Transcoding Change quantization value Most of current systems don’t support
Video staging, caching, patching Staging: store partial frames in proxy Prefix caching: store first few minutes of movie Patching: multiple users use same video
9
Basic Encoding StepsBasic Encoding Steps
10
H.261 (px64)H.261 (px64) International Standard
Video codec for video conferences at p x 64kbit/s (ISDN): Real-time encoding/decoding, max. signal delay of 150ms Constant data rate
Intraframe coding DCT as in JPEG baseline mode
Interframe coding, motion estimation
Search of similar macroblock in previous image and compare Position of this macroblock defines motion vector Difference between similar macroblocks
11
MPEG (Moving Pictures Expert MPEG (Moving Pictures Expert Group)Group)
International Standard: Compression of audio and video for playback (1.5 Mbit/s): Real-time decoding
Sequence of I-, P-, and B-Frames:
Random access at I-frames at P-frames: i.e. decode previous I-frame first at B-frame: i.e. decode I and P-frames first
12
MPEG-2MPEG-2
Beyond MPEG-1: Higher quality encoding Higher data rates Interleaved modes
Higher data rates MPEG-1: about 1.5 MBit/s MPEG-2: 2-100 MBit/s
Scaling: Signal to Noise Ration (SNR) scalin
progressive compression Spatial scaling
several pixel resolutions Temporal scaling
frame dropping
13
Scalable codingScalable coding
Typically used asLayered coding
A base layer Provides basic quality Must always be
transferred One or more
enhancement layers Improve quality Transferred if possible Sending rate
Qu
alit
yBest p
ossible quality
at possi
ble sending ra
te
Base layer
Enhancement layer
14
Temporal ScalabilityTemporal Scalability Frames can be dropped
In a controlled manner Frame dropping does not violate dependancies Low gain example: B-frame dropping in MPEG-1
15
Spatial ScalabilitySpatial Scalability Base layer
Downsample the original image Send like a lower resolution version
Enhancement layer Subtract base layer pixels from all pixels Send like a normal resolution version
If enhancement layer arrives at client Decode both layers Add layers
72
75 83
6173
-1 -12
2 10
Base layer
Enhancement layerBetter compression due to low values
Less data to code
16
SNR ScalabilitySNR Scalability SNR – signal-to-noise ratio Idea
Base layer Is regularly DCT encoded A lot of data is removed using quantization
Enhancement layer is regularly DCT encoded Run Inverse DCT on quantized base layer Subtract from original DCT encode the result
If enhancement layer arrives at client Add base and enhancement layer before running Inverse
DCT
17
Multiple Description Multiple Description CodingCoding Idea
Encode data in two streams Each stream has acceptable quality Both streams combined have good quality The redundancy between both streams is low
Problem The same relevant information must exist in both
streams Old problem: started for audio coding in telephony Currently a hot topic
18
Delivery Systems Delivery Systems DevelopmentsDevelopments
Network
Saving network resources:
Stream scheduling
Several Programs or
Timelines
19
PatchingPatching
Server resource optimization is possible
multicast
Unicast patch stream
Central server
1st client 2nd client
Join !
cyclicbuffer
20
Proxy Prefix CachingProxy Prefix Caching
Split movie Prefix Suffix
Operation Store prefix in prefix cache
Coordination necessary! On demand
Deliver prefix immediately Prefetch suffix from central
server
Goal Reduce startup latency Hide bandwidth limitations,
delay and/or jitter in backbone Reduce load in backbone
Client
Unicast
Unicast
Central server
Prefix cache
21
Interval Caching (IC)Interval Caching (IC) caches data between requests following requests are thus served from the cache sort intervals on length
Video clip 1
S11
Video clip 1
S11S12
Video clip 1
S12 S11S13
Video clip 2
S22 S21
Video clip 3
S33 S31S32S34
I11I12
I21
I31I32I33
I32 I33 I21I11I31I12
22
Receiver-driven Layered Multicast Receiver-driven Layered Multicast (RLM)(RLM)
Requires IP multicast layered video codec
(preferably exponential thickness) Operation
Each video layer is one IP multicast group Receivers join the base layer and extension layers If they experience loss, they drop layers (leave IP multicast
groups) To add layers, they perform "join experiments“
Advantages Receiver-only decision Congestion affects only sub-tree quality Multicast trees are pruned, sub-trees have only necessary traffic
23
Receiver-driven Layered Multicast Receiver-driven Layered Multicast (RLM)(RLM)
24
Multimedia Disk Multimedia Disk SchedulingScheduling Suitability of classical algorithms
minimal disk arm movement (short seek times) no provision of time or deadlines generally not suitable
Continuous media server requirements serve both periodic and aperiodic requests never miss deadline due to aperiodic requests aperiodic requests must not starve support multiple streams balance buffer space and efficiency tradeoff
25
Group Sweep Scheduling Group Sweep Scheduling (GSS)(GSS)GSS combines Round-Robin (RR) and SCAN requests are serviced in rounds (cycles) principle:
divide S active streams into G groups service the G groups in RR order service each stream in a group in C-SCAN order playout can start at the end of the group
special cases: G = S: RR scheduling G = 1: SCAN scheduling
tradeoff between buffer space and disk arm movement try different values for G giving minimum buffer requirement – select
minimum a large G smaller groups, more arm movements, smaller buffers
(reuse) a small G larger groups, less arm movements, larger buffers
26
Power ManagementPower Management Based on functionality
Server Proxy Client
Based on component Disk Processor