Adaptive Content-Aware Scaling for Improved Video Streaming

Adaptive Content-Aware Scaling for Improved Video Streaming.

Avanish Tripathi Advisor: Mark ClaypoolReader: Bob Kinicki

Outline

Introduction Motivation Related Work

MethodologyExperimentsResultsConclusions and Future Work

Motivation

Internet disseminates enormous amounts of informationTCP is the de facto standard… but TCP is not ideal for multimedia and…77% of all Web traffic is Multimedia, of this about 33% is streaming content.

[Chandra, Ellis ’99]

Multimedia Flows…

…tend to use UDP with no congestion controlOther network protocols are being developed: TFRC: smooth reduction in rates as against abrupt

drops in TCP [Floyd et. al. ’00] RAP: Architecture for delivery of layered encoded

streams. [Rejaie et. al. ’99] MPEG-TFRCP: Mapping MPEG to TFRC Protocol

[Miyabayashi et. al. ’00]

Idea-rate based with smooth increase and decrease

Multimedia issues

Generally very high bandwidth requirements Random packet drop by routers during congestion is detrimental to perceptual quality due to interdependencies between packetsNeed application level solution…

…Media Scaling

Need media scaling: Application level data-rate reductionScaling types:

Temporal Quality Spatial

“Content of the stream should influence the choice of scaling mechanism”To the best of our knowledge this idea has not yet been employed

Related Work

Quality Scaling: Receiver-driven Layered Multicast [McCanne ’96]Temporal Scaling:

Player for adaptive MPEG Streaming [Walpole et. al. ‘97] Better Behaved Better Performing MM networking

[Chung, Claypool ‘00]

Content based forwarding for differentiated networks: use priorities based on MPEG characteristics [Shin et. al. ’00]Filtering System: used for media scaling of MPEG streams. [Yeadon ’96]

Outline

Introduction Motivation Related Work

MethodologyExperimentsResultsConclusions and Future Work

Methodology: Content-Aware Scaling

Develop and verify motion measurement mechanismDefine temporal and quality scaling levelsEvaluate the potential impact of content-aware scalingBuild system to do content-aware scaling adaptivelyEvaluate the practical impact of the full system

MPEG Overview

Three kinds of pictures I- Intra encoding P- Predictive encoding B- Bi-directional predictive encoding

Subdivided into Macroblocks Intra, predictive, interpolated macroblocks

Motion vectors are used for motion compensation

Motion Measurement

Higher percentage of interpolated macroblocks means low motionLower percentage of interpolated macroblocks means high motionConducted a pilot study to verify our hypothesis

Divide frame into 16 sub-blocks Count the number of blocks that have motion Correlate that with the percentage of

Interpolated macroblocks.

Pilot Study Result: Motion Measurement

Motion ComputationKeep latency low so that the system is sufficiently reactive

Methodology: Content-Aware Scaling

Develop and verify motion measurement mechanismDefine temporal and quality scaling levelsEvaluate the potential impact of content-aware scalingBuild system to do content-aware scaling adaptivelyEvaluate the practical impact of the full system

Filtering

We extend the system developed at Lancaster university

Frame dropping filter (Temporal Scaling) Requantization filter (Quality Scaling)

User Study Details

22 graduate and undergraduate students in the departmentPlatform: 3 Pentium III machines with 128MB RAM

running Linux Clips were on local hard drives

Four ~10 second clips (2 high motion, 2 low motion)Users rated the clips with numbers from 0 -100

User Study DetailsFive versions of each clip:

Perfect, Temporal Level 1, Temporal Level 2, Quality Level 1, Quality Level 2

Methodology: Content-Aware Scaling

Develop and verify motion measurement mechanismDefine temporal and quality scaling levelsEvaluate the potential impact of content-aware scalingBuild system to do content-aware scaling adaptivelyEvaluate the practical impact of the full system

Results Four men sitting at a bar

Low Motion ( 70 % interpolated macroblocks)

ResultsA girl walks across a room while talking on the phone

Low Motion (57% interpolated Macroblocks)

ResultsRodeo scene where a man on horseback tries to rope a bull

High Motion (27% interpolated macroblocks)

Results Car commerical

High Motion (20% interpolated macroblocks)

Methodology: Content-Aware Scaling

Develop and verify motion measurement mechanismDefine temporal and quality scaling levelsEvaluate the potential impact of content-aware scalingBuild system to do content-aware scaling adaptivelyEvaluate the practical impact of the full system

Full System Architecture

MPEG Server

Input

MotionMeasurement

High

Low

Temporal Filter

Quality Filter

Internet Feedback Generator Client

System Functionality

Server is capable of quantifying motion as the movie playsThe filtering system has five scale levels for finer granularity The system is adaptive and scales movies in real-time depending on the loss pattern as received from the feedback module

User Study

Four clips (2 or more scene) ~30 seconds Four versions of each

Perfect Quality Temporal scaling Quality scaling Adaptive scaling

Bandwidth distribution functions: how often the rate changes

Every 3 seconds Every 200ms

Fit the scale values(1 through 5) on a normal curve [Floyd ‘00]

Future Work

Accurately determine the threshold below which temporal scaling is unacceptableMore accurate bandwidth distribution functionHybrid scaling methods (Quality + Temporal)Audio Scaling

Conclusions

Application level solution to the problem of congestion due to unresponsive video streamsDeveloped a mechanism to quantify the amount of change in a video streamShown that content aware scaling can improve user perceived quality by as much as 50%Developed a system to do adaptive content-aware scaling and are in the process of determining it impact on user perceived quality