Embed Size (px)
Citation preview
Maria Grazia Albanesi, Riccardo AmadeoUniversity of Pavia, Faculty of Engineering, Computer Department
Impact of Fixation Time on Subjective Video Quality Metric: a New Proposal for Lossy
Compression Impairment Assessment
ICMVIPPA 2011 : International Conference on Machine Vision, Image Processing, and Pattern
AnalysisVenezia (Mestre), November 28, 2011
• The addressed problem:– subjective video quality assessment for lossy
compression impairment• The tools and the experiments
– eye tracking and subjective experiments• The goals
– Comparison to literature• The results and their interpretation• A possible application: a new protocol for
no-reference video quality assessment• Future developments
Outline
2 ICMVIPPA 2011 - Venezia (Mestre) - November 28, 2011
How I can measure the loss of quality due to compression?
• Field of applications: TV, video services on Internet, video for mobile applications, test of emerging compression algorithms…..Evaluation of multimedia quality user experience
• Two approaches: objective and subjective metrics
• Our goal: find objective parameters coming form subjective experiments which reflect the subjective video quality, as perceived by a human observer.
The problem
3 ICMVIPPA 2011 - Venezia (Mestre) - November 28, 2011
• Eye tracker: it records the point and the duration of fixation of the eye, when the observer looks at a monitor.
• Data are subsequently analyzed from a statistical point of view (mean, std. dev….)
The tools: eye tracker and subjective QA experiments
4 ICMVIPPA 2011 - Venezia (Mestre) - November 28, 2011
The set of videos
5
• A set of 19 videos downloaded from available online public libraries
– http://trace.eas.asu.edu/yuv/ (Video trace library of Arizona State University)
– ftp://ftp.tnt.uni-hannover.de/pub/svc/testsequences/ (Hannover Liebnitz University video library)
– http://media.xiph.org/video/derf/
• The original files: YUV sequences, 4:2:0, in CIF resolution (352x288) at 30 fps are converted in avi sequences and compressed by a H.264 at 2 bitrates: 450 bps and 150 bps
ICMVIPPA 2011 - Venezia (Mestre) - November 28, 2011
Examples:
6 ICMVIPPA 2011 - Venezia (Mestre) - November 28, 2011
Visual behavior and impairment
7
Visual path for original «best» video
Visual path for compressed (br150 bps)
video
• Protocol ACR5-HR (absolutely category ranking – hidden reference– MOS scale with five levels: – Only one observation for each video– The observer has no information about the
unimpaired version of the video.• The subjects: 8 females and 10 males, of age
varying from 22 to 27 years old.– Their vision was normal or corrected-to-normal – They had no experience in subjective video quality
assessment.– They had normal or good experience in using IT
interfaces to watch videos both online and offline.
Methodology
8 ICMVIPPA 2011 - Venezia (Mestre) - November 28, 2011
• Our parameter are not related to fixation points, but to the duration of the fixation.
• Videos are classified according to color content relevance and and movement relevance to create semantic filters
• Parameters:– Duration of fixation time– MOS, five point scale– Subjective Color Score, three point scale– Subjective Movement Score (SCS e SMS), three point scale.
– Removal of «Memory effect» due to the conditioning of ocular motion activity by the visual attention of preceding scenes.
Novelties and comparison to literature
9
Starting point: O. Le Meur, A. Ninassi, P. Le Callet, D. Barba, Overt visual attention for free-viewing and quality assessment tasks: Impact of the regions of interest on a video quality
metric, Signal Processing Image Communication, 2010, vo. 25, pp- 547-548.
• 18 tester, 6 in each playlist
• Each video has three version: reference br450 br150 (57 videos)
• Each observer looks at only one version of each video. No repetitions are allowed in each playlists.
Numero video utilizzati 19N. ID Playlist A Playlist B Playlist C
1 Foreman ref Br150 Br4502 Silent Br150 ref Br4503 Flower Br450 Br150 ref4 Bus Br450 ref Br1505 Tempete ref Br450 Br150
6Bridge_close ref Br150 Br450
7 Ice Br150 ref Br450
8Coastguard Br450 ref Br150
9Mother_daug Br150 Br450 ref
10 Football ref Br150 Br45011 Crew Br450 ref Br15012 Paris Br450 Br150 ref13 Container ref Br450 Br15014 Highway Br150 Br450 ref15 Waterfall Br150 ref Br45016 Hall Br450 Br150 ref17 Stefan ref Br450 Br15018 News ref Br150 Br45019 Mobile Br150 Br450 ref
tot:ref 7 6 6
Br450 6 6 7Br150 6 7 6
Playlists to remove memory effect
10 ICMVIPPA 2011 - Venezia (Mestre) - November 28, 2011
The MOS really reflect the progressive loss of quality due to compression.
Mean Opinion ScoreFo
rem
an
Silent
Flow
er
Bus
Tem
pete
Bri
dge_c
lose Ice
Coast
guard
Moth
er_
daughte
r
Footb
all
Cre
w
Pari
s
Conta
iner
Hig
hw
ay
Wate
rfall
Hall
Ste
fan
New
s
Mobile
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
0.000.501.001.502.002.503.003.504.004.505.00
MOS
RefBr 450Br 150
MO
S A
CR
5-H
R
11
• Color and movement are considered relevant if the score is > 2
• «Highly animated video»: 2, 4, 7, 8, 10, 12, 14, 17, 19
• «Highly coloured video» »: 3, 5, 7, 10, 15, 19
N. Playlist SCS SMS
1 Foreman 1,67 1,50
2 Silent 1,28 2,28
3 Flower 2,33 1,61
4 Bus 1,67 2,33
5 Tempete 2,50 1,89
6 Bridge_close 1,44 1,50
7 Ice 2,11 2,33
8 Coastguard 1,72 2,33
9Mother_daughter 1,78 1,28
10 Football 2,39 2,78
11 Crew 1,83 1,94
12 Paris 1,94 2,11
13 Container 1,83 1,67
14 Highway 1,39 2,56
15 Waterfall 2,72 1,72
16 Hall 1,72 1,72
17 Stefan 1,67 2,50
18 News 1,78 1,83
19 Mobile 2,56 2,22
SCS e SMS
12
• The mean fixation time does not seem to be related to the video quality!
Analysis of Mean fixation time (MFT)
Fore
man
Silent
Flow
er
Bus
Tem
pete
Bri
dge_c
lose Ice
Coast
guard
Moth
er_
daughte
r
Footb
all
Cre
w
Pari
s
Conta
iner
Hig
hw
ay
Wate
rfall
Hall
Ste
fan
New
s
Mobile
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
200
250
300
350
400
450
500
550
600
Durata media fissazioni, MFT
RefBr450Br150
ms
13
MFT, semantic filtering
Silent
Bus
Ice
Coast
guard
Footb
all
Pari
s
Hig
hw
ay
Ste
fan
Mobile
2 4 7 8 10 12 14 17 19
200
250
300
350
400
450
500
550
600MFT, SMS>2
RefBr 450 Br 150m
s
Flow
er
Tem
pete Ice
Footb
all
Wate
rfall
Mobile
3 5 7 10 15 19
200
250
300
350
400
450
500
550
600 MFT, SCS>2
RefBr 450 Br 150
ms
• Even by filtering by movement or colour, there is not a clear relation between MFT and MOS
14
• Even the standard deviation of fixation time does not seem to be related to the video quality!
Analysis of Standard deviation of FT
Fore
man
Silent
Flow
er
Bus
Tem
pete
Bri
dge_c
lose Ice
Coast
guard
Moth
er_
daughte
r
Footb
all
Cre
w
Pari
s
Conta
iner
Hig
hw
ay
Wate
rfall
Hall
Ste
fan
New
s
Mobile
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
0
50
100
150
200
250
300
350
400
450
Deviazione Standard, SDoFT
RefBr450Br150
ms
15
Ref Br 450 Br 150
Average SDoFT, SMS>2155,475
2170,623
3225,570
9
Standard deviation of SDoFT, SMS>2 69,7361 85,7656100,306
2
Ref Br 450 Br 150
Average SDoFT149,388
8174,228
9173,837
8Standard deviation of SDoFT 60,6719 88,2987 90,7023
The solution: third order statistics!
Ref Br 450 Br 150
Average SDoFT, SCS>2145,433
5169,090
0178,957
9
Standard deviation of SDoFT, SCS>2 69,4838122,821
3 87,1634
• Standard deviation of FT is on the average less for videos of high quality
• The semantic filtering shows that this behaviour is stressened for highly animated videos.
16
• The duration of fixation time seems to have a more predictable behavior when the observer watches to a high quality video.
• If we compute the third order statistics on the fixation time, we can guess a rank of a collection of video which reflects the perceptive visual quality
• The experiments confirm this behavior for degradation due to lossy compression.
• The rank according third order statistic reflect the loss of quality and subjective MOS especially for highly animated videos.
Future researches: • Test on a greater level of quality impairments• Test on other kinds of quality impairments• Finding a more efficient semantic filtering about color or
other criteria.
Conclusions and future researches
17 ICMVIPPA 2011 - Venezia (Mestre) - November 28, 2011
