The Path from QoS- to QoE-centric Managementof Modern Communication Systems
Jiri Hosek, Ph.D.
Youth School-Seminar, DCCN 2016
RUDN University, Moscow, November 24, 2016
*These slides are intended for educational purposes and include materialpublished by WISLAB group as well as available openly on the Internet.
Lecture’s Content
• Definition of network quality
• Quality of Service (QoS)
• Quality of Experience (QoE)
• Subjective evaluation of modern mobile applications
Evaluation of Service Quality
• Two main approaches how to measure the network service quality
• Objective
• View from the network perspective
• Measuring of traffic parameters and their comparison with pre-defined values
• Different services (applications) have different requirements
• Subjective (Relative)
• View of the end user
• Difficult and time-consuming
• MOS (Mean Opinion Score)
• Substitution of subjective evaluation by mathematical model
MOS Scale
• Used especially for multimedia transmissions
• Numerical expression of the human user’s view on the quality of the network service
Key Network Traffic Parameters
• Dominant measurable parameters of network traffic
• Packet delay
• How long the transmission takes
• Packet delay variation (jitter)
• Difference between the delay of two consequent packets
• Packet loss
• How many packets are lost during the transmission
• Throughput
• Transmission speed (b/s)
• Key Performance Indicators (KPIs)
Uniform Network Traffic Treatment
• As default option in all older network technologies
• Seemingly fair approach
• Each data unit is of the same priority
• No possibility to prefer or defer any type of traffic
• “Best-effort” treatment
• No guarantees, just try
Differentiated Network Traffic Treatment
• Characteristic for modern network technologies
• Traffic classes
• Different requirements on transmission parameters
• Efficient operation of network services
• Identification of data units of different services/flows
• Guarantee of adequate treatment
• 2 basic QoS support mechanisms
• Integrated Service – IntServ
• Differentiated Services – DiffServ
Differentiated Services – DiffServ
• Offers different transmission parameters for different service types
• Every data packet is processed individually
• Fixed network resource sharing
• QoS functions implemented in each router
• Service guarantees
• Service-class based
DiffServ – Traffic Classification and Policing
• Traffic classification into an appropriate category
• Usually based on fields of IP and TCP/UDP header
• Differentiated Service CodePoint (DSCP) field in the IP packet header
• Relative priorities
• Identifies service classes
• Traffic policing
• Ensures, that the incoming traffic satisfies the declared parameters
• Service Level Agreement – SLA
• Network operator / end-user obligations
DiffServ – Packet Scheduling Techniques
• Main packet scheduling mechanisms
• First In First Out – FIFO queuing
• No QoS prioritization
• Priority Queuing – PQ
• Fair Queuing – FQ
• Weighted Fair Queuing – WFQ
FIFO
Priority Queuing
Application of DiffServ Mechanism
• Each network node (router) is performing QoSfunctions individually
• DiffServ routers
• Edge router
• Core router
Subjective Evaluation of Network Services• Customers are demanding constantly increasing quality
of mobile services
• Modern mobile (multimedia) applications are highlydiverse
• Specific requirements and user expectations
• QoS-based evaluation and network control is not enough anymore
• New investments for increasing network quality at all levels.
• Quality of (user) Experience (QoE)
• User eXperience (UX)
• Influenced by many factors
• Technical, socioeconomical, etc.
QoE Key Questions and Goals(from network operator point of view)• How the key network parameters affect the
quality of mobile service perceived by an end user?
• How the type of application affects user‘sevaluation?
• How long are end users willing to wait for specific mobile service and being still satisfied?
• What are the “premium quality” and saturation thresholds for the specific service?
• How to setup and control the mobile network to avoid the over-provisioning?
Experimental Study of QoE in Mobile Networks
• The impact of following variables were investigated:
• End-user device:
• Smartphone Samsung Galaxy Nexus
• Tablet Samsung Galaxy Tab 10.1 inch
• Applications
• Web browsing, file download (DL), file upload (UL)
• YouTube video streaming
• Network and content parameters
• Bit rates (BR) in the range of 32kb/s to 16 Mb/s
• Initial loading delay (connection establishment) in the range of 0 to 11seconds
• YouTube video resolution in the range of 320x180 to 1280x720
• Stalling effects with different duration and repetition
• More than 200 test scenarios evaluated by almost 300 test participants
Experimental Study of QoE in Mobile Networks– Methodology• Laboratory environment
• Completely controllable system
• Test Methods: (rec. ITU-T P.800)
• Absolute Category Rating (ACR):
• Scaling: 5 grade MOS scale:
• 1: Bad
• 5: Excellent
• Acceptability rating:
• Question: Were you satisfied with tested quality?
• Binary answer: YES or NO
Mobile QoE – Automatic Assessment Tool – UserInterface
Questionnaire:Demographic data:Briefing phase:
Web: Sample Results – Quality Rating
• Web browsing, tablet
Web: Sample Results – Acceptability Rating
• Web browsing, smartphone
0
10
20
30
40
50
60
70
80
90
100
0 200 400 600 800 1000 1200
Acc
ep
tab
ility
[%
]
BR [kbps]
0s 1s 3s 5s 7s 11s
Mobile Web QoE – Quality Thresholds
• We specified 3.7 MOS as a quality threshold for “premium quality” service.
• The saturation threshold refers to BR with zero initial loading delay and where quality rating is not more increasing or achieved maximum quality rating or 4.5 MOS.
• This is very practical output in order to avoid quality overprovisioning.
Scenario Notebook Smartphone
Web 256 kbps 256 kbps
File DL 4 Mbps 4 Mbps
File UL 1 Mbps 1 Mbps
Scenario Notebook Smartphone
Web 512 kbps 256 kbps
File DL 4 Mbps 8 Mbps
File UL 4 Mbps 1 Mbps
Youtube: Sample Results (1)
• Quality rating vs. Resolution
• The subjective premium quality threshold (3.7 MOS)
• Resolution [pixels]: 640 x 360
• Saturation threshold (4.5 MOS)
• Resolution [pixels]: 854 x 480
Overprovisioning area
Youtube: Sample Results (2)
• CDF of task’s leaving for infinity loading delay
50 % users wait
more than
104 / 44
seconds
Youtube: Sample Results (3)
• Acceptability rating results vs. single stalling duration – smartphone
1st level
quality
reduction
2nd level quality reduction
Youtube: Sample Results (4)
• Quality rating results vs. multiple stalling – smartphone
1st level
quality
reduction
2nd level quality reduction
Web: Analytical User Experience Prediction
• We need an adequate QoE prediction models based on given results of extensive QoE assessment.
• Analytical tools (regression analysis)
• Calculation of QoE value (MOS) for mobile data services based on input parameters:
• Initial loading delay
• Throughput (bit rate)
ac
cBRc
abMOS
D
2
10
~)(~1
)(
YouTube: Analytical User Experience Prediction
MOS 𝐷, 𝑅 =2,7
1+9,5𝑒0,1𝐷+
10,7
(0,01𝑅)2
𝑒−𝑁0,6𝑆0,2+1
Summary
• Future-generation mobile services are more user- / service-oriented
• QoS-centric evaluation and control of modern wireless networks is not enough
• User-centric / application-centric approach needs to be implemented
• Complex evaluation from network- and user-perspective as well
• QoE modelling is very demanded research topic nowadays
• Not easy task due to high heterogeneity of modern applications and their constant development
• Models need to be updated as users’ expectations are growing
• Standardization in QoE assessment for new mobile services is highly required
Thank you for your attention!
Questions?