Upload
elma
View
42
Download
2
Tags:
Embed Size (px)
DESCRIPTION
Modeling and Evaluating Feedback-Based Error Control for Video Transfer. PhD Candidate: Yubing Wang - Computer Science, WPI, EMC Corp. Committee: Prof. Mark Claypool - Computer Science, WPI Prof. Robert Kinicki - Computer Science, WPI Prof. Dan Dougherty - Computer Science, WPI - PowerPoint PPT Presentation
Citation preview
Ph.D. Dissertation Defense
Modeling and Evaluating Feedback-Based Error Control for Video
Transfer PhD Candidate:
Yubing Wang - Computer Science, WPI, EMC Corp.
Committee:
Prof. Mark Claypool - Computer Science, WPI
Prof. Robert Kinicki - Computer Science, WPI
Prof. Dan Dougherty - Computer Science, WPI
Prof. Ketan Mayer-Patel – Computer Science, UNC at Chapel Hill
2Ph.D. Dissertation Defense
Video TransferVideo Transfer
5
Video Frames
InternetInternet
Client
4 3 2 1
Frame Loss
Capacity Constraint
Server
5 3 1
Delay Constraint
Too Late
Error Propagation
4 3 2 15
3Ph.D. Dissertation Defense
Error ControlError Control
5
Video Frames
InternetInternet
Client
4 3 2 1
Server
Retransmission
NACK
3
Change Coding Parameter
Local Concealment
3
4Ph.D. Dissertation Defense
MotivationMotivation
Frame loss degrades video qualityFeedback-based error control techniques use information from decoder to repair
Feedback indicates damage location.Encoder and decoder cooperate in error control process.Better than error control techniques where no interaction between encoder and decoder Major techniques: RPS, Intra Update, RetransmissionChoice and Effectiveness depends on packet loss, RTT, video content and GOP size
No systematic exploration and comparison of impact of video and network conditions on the performance of feedback-based error control techniques
5Ph.D. Dissertation Defense
The DissertationThe Dissertation
Analyze video quality with feedback based error controlDevelop analytical models to predict quality of videos streamed with RPS NACK, RPS ACK, Intra Update or RetransmissionConduct systematic study of effects of reference distance on video qualityValidate analytical models through simulationsAnalysis of loss rate, round-trip time, video content, Group Of Pictures (GOP)Determine choice between RPS NACK, RPS ACK, Intra Update or RetransmissionPublications
“Impact of Reference Distance for Motion Compensation Prediction on Video Quality”, MMCN07“An Analytic Comparison of RPS Video Repair”, MMCN08“Modeling RPS and Evaluating Video Repair with VQM”, IEEE Transactions on Multimedia, 2009, (to appear)
6Ph.D. Dissertation Defense
OutlineOutline
Introduction
Background RPS ACKRPS NACKIntra UpdateRetransmission
Impact of Reference Distance on Video QualityAnalytical Models and ResultsModel ValidationsConclusions
7Ph.D. Dissertation Defense
Reference Picture Selection (ACK)Reference Picture Selection (ACK)
The decoder acknowledges all correctly received frames Only the acknowledged frames are used as a reference Error propagation is avoided entirely Distance from reference frame is reference distance Reference distance increases with round-trip delay Coding efficiency decreases as reference distance increases Video quality degrades as coding efficiency decreases
1 2 3 4 5 6 7
ACK(1) ACK(2) ACK(3)
8Ph.D. Dissertation Defense
Reference Picture Selection (NACK)Reference Picture Selection (NACK)
The previous frame is used as a reference for encoding during the error-free transmission. Reference distance is always 1 regardless of RTT
The decoder sends a NACK for the erroneous frame along with a reference frame number Error propagation Impact of loss increases with RTT
NACK(3)
1 2 3 4 5 6 7 8
9Ph.D. Dissertation Defense
Intra UpdateIntra Update
Upon receiving a NACK from the decoder, encodes the current frame with intra mode
Frame is independently encoded without using any information from previous frames
Coding efficiency is reduced because of intra coding
1 2 3 4 5 6 7 8 9
NACK(4)
Intra-coded
10Ph.D. Dissertation Defense
RetransmissionRetransmission
Retransmission of lost frames needs extra bandwidth Packets arriving after their display times are not discarded but
instead are used to reduce error propagation
1 2 3 4 5 6 7 8 9
1 2 3 4 5 6 7 8 9
Encoder
DecoderNACK(3)
3
11Ph.D. Dissertation Defense
OutlineOutline
IntroductionBackgroundImpact of Reference Distance on Video Quality
HypothesisMethodologyResults and Analysis
Analytical Models and ResultsModel ValidationsConclusions
12Ph.D. Dissertation Defense
Impact of Reference Distance on Video QualityImpact of Reference Distance on Video Quality
RPS selects one of several previous frames as a reference frame during compression
Distance from selected frame is reference distance
Higher reference distance, lower quality and vice versa
How reference distance affects video quality has not been quantified
A systematic study of the effects of reference distance on video quality
Data is needed for modeling RPS
13Ph.D. Dissertation Defense
HypothesisHypothesis
The y-intersect is determined by motion and scene complexity.
High-motion video sequences starts with low quality, degrade slower.
Low-motion video sequence starts with high quality, degrade faster.
Low Motion:
The similarities among frames are high;
More macro-blocks are inter-coded;
High motion:
The similarities among frames are low;
More macro-blocks are intra-coded;
14Ph.D. Dissertation Defense
MethodologyMethodology
Select a set of non-compressed video clips with a variety of motion content.
All in YUV 4:2:2, CIF (352x288)Each video sequence contains 300 video frames with a frame rate of 30 fps.
Change reference distances for each selected video sequenceEncode the video clips using H.264 Measure video quality using
Peak-Signal-to-Noise-Ratio (PSNR) Video Quality Metric (VQM)
Analyze the results.
15Ph.D. Dissertation Defense
PSNR vs. Reference DistancePSNR vs. Reference Distance
Video Clips a b R-Squared
Akiyo -2.0116 47.965 0.9953
Container -1.9023 44.838 0.9948
News -1.8556 43.295 0.9984
Silent -1.5283 41.41 0.9929
Mom & Daughter
-1.4581 41.442 0.9904
Foreman -1.1681 38.511 0.9265
Mobile -1.1553 26.663 0.9754
Coastguard -0.8626 35.582 0.9975
The relationship between PSNR and reference distance can be characterized using a logarithmic function: bxay )ln(
y = -0.8626Ln(x) + 35.582
y = -1.4581Ln(x) + 41.442
y = -2.0116Ln(x) + 47.965
30
35
40
45
50
1 2 3 4 5 6 7 8Reference Distance
PS
NR
(db)
Akiyo
mom_daut
coastguard
16Ph.D. Dissertation Defense
VQM vs. Reference DistanceVQM vs. Reference Distance
Video Clips a b R-Squared
Akiyo -0.0113 0.9847 0.9869
Container -0.0114 0.9766 0.9848
News -0.0115 0.9732 0.9931
Silent -0.0124 0.9606 0.9937
Mom & Daughter
-0.0085 0.9217 0.9821
Foreman -0.0068 0.9059 0.9779
Mobile -0.0022 0.8055 0.9076
Coastguard -0.0014 0.8423 0.9671
The relationship between VQM and reference distance can be characterized using a linear function: baxy
y = -0.0113x + 0.9847
y = -0.0085x + 0.9217
y = -0.0014x + 0.84230.82
0.84
0.86
0.88
0.9
0.92
0.94
0.96
0.98
1
1 2 3 4 5 6 7 8
Reference Distance
1-VQ
M
akiyo
mom_daut
coastguard
17Ph.D. Dissertation Defense
OutlineOutline
IntroductionBackgroundImpact of Ref. Distance on Video QualityAnalytical Models and Results
AssumptionsRPS ACKRPS NACKIntra UpdateRetransmissionResult & Analysis
Model ValidationsConclusions
18Ph.D. Dissertation Defense
AssumptionsAssumptionsEach GOB is independent from other GOBs in the same frame.An independent video sub-sequence is referred to as a reference chain.Each GOB is carried in a single network packet.Reliable transmission of feedback messages are assumed. Erroneously-decoded GOBs are repaired by local concealment.Make no assumption on specific local concealment techniques.
1 2 3 4 5 6 7
Assume independent packet loss with a random loss distribution.
In this talk, GOB and Frame is exchangeable.
19Ph.D. Dissertation Defense
Model ParametersModel Parameters
20Ph.D. Dissertation Defense
Modeling of RPS ACKModeling of RPS ACK
The probability of decoding GOB (n) correctly using GOB (n-δ-i) as a reference:
The probability of GOB (n) being successfully decoded is:
p Packet loss probability
Probability of GOB (n-δ-i) being successfully decoded
Round-trip time
Time-interval between two frames
inq
RTTt
INTt
10,)1( niqpp ini
INT
RTT
t
t
pqn 1
ACK(1)
1 2 3 4 5
ACK(2)
21Ph.D. Dissertation Defense
RPS ACK Modeling (cont.)RPS ACK Modeling (cont.)
The expected video quality for n-th GOB:
Average video quality for a GOB encoded using the GOB that is r GOBs backward.
Average video quality for a Intra-Coded GOB
Average PSNR value for a GOB that is repaired using local concealment
nUpUp
nUpqppUQ
n
iin
ii
n
,*)1(
,*)1(
'0
'1
0
rU
0U'U
22Ph.D. Dissertation Defense
RPS NACK -- ModelRPS NACK -- Model The probability of GOB (n) being successfully decoded:
np
nqqq
n
n
iinn
n
,)1(
,1
0,1, inq , --- the probability of decoding GOB (n) correctly
using GOB (n- δ -i) as a reference
1- p
1- p
1- p
1- p1- p1- p1- p
1- p
p
p p
p pp
[1]
[1](1)
[2](1)(1)
[1]
(2)
(2)(1)(1) (3) [3]
GOB 1
GOB 2
GOB 3
GOB 4
(2)
p
p
root
[1]
A
B
C
D
NACK(1)
1 2 3 4 5
NACK(2)
GOB Dependency Tree
23Ph.D. Dissertation Defense
Intra Update -- ModelIntra Update -- Model The probability of GOB (n) being successfully decoded:
np
nqqq
n
INTRAnn
n,)1(
,,1, INTRAnq , -- the probability of decoding GOB (n) correctly using Intra coding
1 2 3 4 5
NACK
Intra-coded
1- p
1- p
1- p
1- p1- p1- p1- p
1- p
p
p p
p pp
GOB 1
GOB 2
GOB 3
GOB 4
p
p
A E
B
root
C
D
F
GOB Dependency Tree
24Ph.D. Dissertation Defense
RetransmissionRetransmission
pNN
pNCC
CNp
pNNC
N
NpppC
RRG
GE
G
RRGE
G
RRE
)1(*
,)1(
)(*)]1(*)(1[* 32
2),1(
11),1(1'1
1
'1
RRNN
RRnn
nNnqUqU
NnqUqUQ
RRRR
Capacity constraint:
The n-th GOB in the reference chain being successfully decoded:
11, RRn
n Nnqq
2,1 RR
Nn Nnqq RR
The expected video quality for GOB (n):
25Ph.D. Dissertation Defense
OutlineOutline
IntroductionBackgroundImpact of Ref. Distance on Video QualityAnalytical Models and Results
AssumptionsRPS ACKRPS NACKIntra UpdateRetransmissionResult & Analysis
Model ValidationsConclusions
26Ph.D. Dissertation Defense
Analytic ExperimentsAnalytic Experiments
Our analytical models consider a number of factors that may affect feedback-based repair performance:
Reference distance change
Loss probability
Round-trip time
Bitrate constraint
Video content
GOP Size
Select a set of video clips
with a variety of motion content
27Ph.D. Dissertation Defense
Quality versus Round-Trip TimeQuality versus Round-Trip Time
30
32
34
36
38
40
42
44
40 80 120 160 200 240 280 320 360 400
Round Trip Time (ms)
PS
NR
(db)
p=0.01
p=0.05
p=0.1
p=0.2
25
27
29
31
33
35
37
39
41
43
45
40 80 120 160 200 240 280 320 360 400
Round Trip Time (ms)
PS
NR
(db
)
P=0.01
p=0.05
p=0.1
p=0.2
RPS ACK RPS NACK
Quality degrades with round-trip time increase
NACK resistant to degradation with round-trip time for low loss
ACK degrades uniformly with round-trip time
28Ph.D. Dissertation Defense
Quality versus Loss RateQuality versus Loss Rate
30
32
34
36
38
40
42
44
46
0 0.05 0.1 0.15 0.2
Loss (fraction)
PS
NR
(db
)
RTT=80ms
RTT=160ms
RTT=240ms
RTT=320ms
30
32
34
36
38
40
42
44
0 0.05 0.1 0.15 0.2
Loss (fraction)
PS
NR
(db
)
RTT=80ms
RTT=160ms
RTT=240ms
RTT=320ms
RPS ACK RPS NACK
Quality degrades with loss rate increase
NACK degrades faster with high round trip times
ACK uniform degradation
29Ph.D. Dissertation Defense
RPS NACK vs. RPS ACKRPS NACK vs. RPS ACK
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0.1
80 120 160 200 240 280 320 360 400
Round Trip Time (ms)
Lo
ss C
ross-o
ver
Container News Silent Mom-Daughter Foreman Mobile
Above trend line, ACK better. Below trend line, NACK better
Crossover points for low-motion are higher than for high-motion
Error propagation more harmful to quality than reference distance
30Ph.D. Dissertation Defense
ComparisonComparison
27
29
31
33
35
37
39
41
43
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2
Loss Rate
PS
NR
(db)
RPS NACK Intra Update
RPS ACK Retransmission
32
34
36
38
40
42
44
0 0.02 0.04 0.06 0.08 0.1 0.12 0.14 0.16 0.18 0.2
Loss Rate
PS
NR
(db
)
RPS NACK Intra Update
RPS ACK Retransmission
RPS NACK performs best in low loss
RPS ACK performs best in high loss
RPS ACK performs worst in low loss
Retransmission performs worst in high loss
Intra Update performs as well as RPS NACK as RTT increases
RTT=80 ms
RTT=240 ms
31Ph.D. Dissertation Defense
OutlineOutline
IntroductionBackgroundImpact of Ref. Distance on Video QualityAnalytical Models and ResultsModel Validations
MethodologyResults
Conclusions
32Ph.D. Dissertation Defense
Validation -- MethodologyValidation -- Methodology
Randomly drop controllable number of frames in input sequence based on given loss probabilityBased on given round-trip time and randomly selected lost frames, regenerate video sequenceEncode video sequence generated in step 2 using H.264Measure average PSNR and VQM for encoded H.264 video sequenceCalculate average PSNR and VQM based upon video quality measured in step 4
1(I) 2(P) 5(P) 6(P) 7(P)
RPS NACK, round-trip time = 2 frames, frame 3 is lost
33Ph.D. Dissertation Defense
Validation – RPS NACKValidation – RPS NACK
25
27
29
31
33
35
37
39
41
43
45
0 0.05 0.1 0.15 0.2 0.25 0.3
Loss (Fraction)
PS
NR
(db
)
RTT=80 ms(simulation) RTT=240 ms(simulation)
RTT=80 ms(model) RTT=240 ms(model)
Error bar represents 95% confidence interval
As loss probability or round-trip time increases, the variance is increased
Simulation results are consistent with values predicted by analytical model for both PSNR and VQM
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1
0 0.05 0.1 0.15 0.2
Loss (Fraction)
VQ
M
RTT=80ms (experiment) RTT=240ms (experiment)
RTT=80ms (model) RTT=240ms (model)
34Ph.D. Dissertation Defense
OutlineOutline
IntroductionBackgroundImpact of Ref. Distance on Video QualityAnalytical Models and ResultsModel ValidationsConclusions
35Ph.D. Dissertation Defense
Major ContributionsMajor Contributions
1. Systematic study of effects of reference distance on video quality for a range of video coding conditions
2. Two utility functions that characterize impact of reference distance on video quality based upon study
3. Modeling prediction dependency among GOBs for RPS NACK and Intra Update using binary tree
4. Analytical models for feedback-based error control techniques including Full Retransmission, Partial Retransmission, RPS ACK, RPS NACK and Intra Update
5. Simulations that verify accuracy of our analytical models
6. Analytic experiments over a range of loss rates, round-trip times and video content using our models
36Ph.D. Dissertation Defense
Future WorkFuture Work
Explore and incorporate other existing video quality metrics or develop a new quality metricInvestigate how local concealment may affect the choice of feedback-based repair techniquesInvestigate the impact of the extra bandwidth consumed by feedback messages on performanceBuild a videoconference system that automatically adapts to the best repair techniques
37Ph.D. Dissertation Defense
ConclusionsConclusionsDegree of video quality degradation is affected by video content
High-motion video sequences starts with lower quality, degrade slower.Low-motion video sequences starts with higher quality, degrade more rapidly.Mathematical Characterization of the relationship between video quality and reference distance:
PSNR:
VQM:
Analytical models reveal: RPS NACK performs best in low lossRPS ACK performs best in high loss, worst in low lossRPS NACK outperforms RPS ACK over a wider range for low motion videos than for high motion videosRetransmission performs worst in high loss Intra Update performs as well as RPS NACK as RTT increases
bxay )ln(baxy
38Ph.D. Dissertation Defense
AcknowledgeAcknowledge
Prof. Claypool and Prof. Kinicki
Prof. Dougherty
Prof. Mayer-Patel from UNC at Chapel Hill
Faculty/Staff of Computer Science Dept., WPI
Huahui Wu, Mingze Li, Feng Li, and everyone from PEDS and CC groups
Attendees today
My Family