QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE

QUALITY OF EXPERIENCE PREDICTIONFOR STREAMING VIDEO

Zhengfang Duanmu

Joint work with Abdul Rehman, Kai Zeng, and Zhou Wang

July 13, 2016

Video Streaming and Quality of ExperienceStreaming Quality Index (SQI)

Conclusions

Outline

1 Video Streaming and Quality of ExperienceVideo StreamingQuality of ExperienceObjective QoE Prediction

2 Streaming Quality Index (SQI)Objective QoE Prediction ModelExperimental Results

3 Conclusions

2 / 26


Conclusions

Video StreamingQuality of ExperienceObjective QoE Prediction

Outline



3 Conclusions

3 / 26


Conclusions


The Age of Streaming

4 / 26


Conclusions


The Age of Streaming

Factors of Picking Streaming ServiceQuality of Experience;

Content;

Price;

Advertisement.

5 / 26


Conclusions


Quality of Experience

DefinitionThe degree of delight or annoyance of the user of an application orservice. [Callet, 2013]

6 / 26


Conclusions



Influencing FactorsPlayback smoothness

7 / 26


Conclusions



Influencing FactorsDuration of initial buffering

8 / 26


Conclusions



Influencing FactorsVideo presentation quality

9 / 26


Conclusions


Interaction between Presentation Quality and StallingExperience

0 20 40 60 80 1000

5

10

15

20

25

30

35

SSIMplus of stalling frames

MO

S dr

op

initial bufferingfitted initial bufferingstallingfitted stalling

Figure: SRCC = 0.79

10 / 26


Conclusions


Subjective QoE Prediction

Subjective QoE Assessment

Enormous video space

11 / 26


Conclusions


Existing QoE models: Quality of Service-based

PhilosophyThere exists a causal relationship between generic QoS problems andgeneric QoE problems.

FactorsThroughput -> Delivered video quality

Stalling duration -> Waiting experience

Existing ModelsLinear mapping [Mok, 2011];

Exponential mapping [Hoßfeld, 2012];

Logrithmic mapping [Rodriguez, 2012];

12 / 26


Conclusions


Existing QoE models: Signal Fidelity-based

PhilosophyQoE can be measured by the distance from test video to the prestinevideo in the video space.

Existing ModelsPSNR;

SSIM [Wang, 2004];

MS-SSIM [Wang, 2003];

VQM [Pinson, 2004];

SSIMplus [Rehman, 2015];

13 / 26


Conclusions


Our method: Hybrid

MotivationQoS-based: not directly related to human perception;

Signal fidelity-based: only work for static videos;

No modeling on the interaction between video presentation quality andstalling.

14 / 26


Conclusions

Objective QoE Prediction ModelExperimental Results

Outline



3 Conclusions

15 / 26


Conclusions


Presentation Quality

Source

Bitrate 1

Bitrate 2

Bitrate 3

V

Q

A

S

E

G

M

E

N

T

E

R

Seg1 Seg2 Seg3

Seg1 Seg2 Seg3

Seg3Seg1 Seg2

T

R

A

N

S

C

O

D

I

N

G

16 / 26


Conclusions



0 500 1000 150075

80

85

90

95

100

Frame #

Stat

ic v

ideo

qua

lity

Static video qualityStalling position

0 500 1000 150075

80

85

90

95

100

Frame # with stallings

Stre

amin

g vi

deo

qual

ity (

P)

PStalling positionRecovery position

17 / 26


Conclusions


Stalling Experience Quantification


Sk(t) =

Pik−1

(−1 + exp

{−(

tf − ikT0

)})ikf ≤ t ≤ ik+lk

f

Pik−1

(−1 + exp

{−(

lkT0

)})·(

exp{−(

tf − ik − lkT1

)})t > ik+lk

f

0 otherwise

(1)

18 / 26


Conclusions



Parameters

Parameter DescriptionT0 rate of dissatisfaction in stalling eventT1 strength of memory in stalling event

T init0 rate of dissatisfaction in initial buffering event

T init1 strength of memory in initial buffering eventP0 expectation on initial quality of the video

19 / 26


Conclusions



0 500 1000 150075

80

85

90

95

100

Frame #

Stat

ic v

ideo

qua

lity

Static video qualityStalling position

0 500 1000 150075

80

85

90

95

100


Stre

amin

g vi

deo

qual

ity (

P)

PStalling positionRecovery position

0 500 1000 1500−100

−80

−60

−40

−20

0


Stal

ling

expe

rien

ce (

S nk )

Sn1

Sn2

Stalling positionRecovery position

0 500 1000 15000

20

40

60

80

100


QoE

(Q

)

QStalling positionRecovery position

20 / 26


Conclusions


Experimental Results

Performance comparison of QoE models on streaming video QoEdatabase.

PLCC MAE SRCC KRCCFTW [Hoßfeld, 2012] 0.3313 14.9455 0.3154 0.2583

PSNR 0.6663 10.7254 0.6715 0.4697SSIM [Wang, 2004] 0.8432 7.6039 0.8177 0.6070

SSIMplus [Rehman, 2015] 0.8350 7.6934 0.8024 0.5924SQI-PSNR 0.7391 9.8445 0.7492 0.5434SQI-SSIM 0.9015 5.8941 0.9009 0.7238

SQI-SSIMplus 0.9026 5.8330 0.9007 0.7213

21 / 26


Conclusions


Experimental Results

40 60 80 1000

20

40

60

80

100

FTW

MO

S

no stallinginitial bufferingstalling

0.85 0.9 0.95 10

20

40

60

80

100

SSIM

MO

S


20 40 60 80 1000

20

40

60

80

100

SSIMplus

MO

S


0.5 0.6 0.7 0.8 0.9 10

20

40

60

80

100

VsQM

MO

S


0.8 0.85 0.9 0.95 10

20

40

60

80

100

SQI−SSIM

MO

S


0 20 40 60 80 1000

20

40

60

80

100

SQI−SSIMplusM

OS


22 / 26


Conclusions

Outline



3 Conclusions

23 / 26


Conclusions

Conclusions

ContributionProposed an objective QoE model for video streaming that considerspresentation quality and its interaction with stalling;

Achieved the best performance in predicting subject opinions.

Future WorkConstruct comprehensive database;

Investigate other QoE-related factors;

Improve the QoE model.

24 / 26


Conclusions

References

P. L. Callet, S. MÃuller, and A. Perkis “Qualinet White Paper on Definitions of Quality ofExperience”, European Network on Quality of Experience in Multimedia Systems and Services,2013.

R. Mok, E. Chan, and R. Chang, “Measureing the quality of experience of HTTP video streaming,”IFIP/IEEE Int. Sym. Integrated Network Management, 2011.

T. Hoßfeld, S. Egger, R. Schatz, K. Masuch, and C. Lorentzen, “Initial delay vs. interruption:between the devil and the deep blue sea,” IEEE QoMEX, 2012.

D. Rodriguez, J. Abrahao, D. Begazo, R. Rosa, G. Bressan, “Quality metric to assess videostreaming service over TCP considering temporal location of pauses,” IEEE TCE, 2012.

Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: From errorvisibility to structural similarity,” IEEE TIP, 2004.

Z. Wang, E. P. Simoncelli, and A. C. Bovik, “Multiscale structural similarity for image qualityassessment,” IEEE Asilomar, 2003.

M. H. Pinson, S. Wolf, “A new standardized method for objectively measuring video quality,” IEEETrans. Broadcasting, 2004.

A. Rehman, K. Zeng, and Z. Wang, “Display device-adapted video quality-of-experienceassessment,” SPIE Electronic Imaging, 2015.

25 / 26


Conclusions

Thank you

26 / 26

Documents

QUALITY OF EXPERIENCE PREDICTION FOR STREAMING VIDEOzduanmu/icme16qoe/paper/... · Video Streaming and Quality of Experience Streaming Quality Index (SQI) Conclusions Objective QoE