Monitoring QoS/QoE of mobile networks-novel approach

Kampala, Uganda, 25 June 2014

Monitoring QoS/QoE of mobile networks-novel approach

Tahitii ObiohaNetwork Planning and Optimisation Engineer

Planet Network International, France.

ITU Regional Standardization Forum for Africa

(Kampala, Uganda, 23-25 June 2014)

Why Monitor QoS/ QoE of mobile networks

Monitoring = observing, checking and keeping a continuous record of the progress or quality of something.It is the use of any available technical tool to assess permanently or for a given period of time a particular QoS parameter, e.g. call set up time. (ETSI EG 202 009-2)

For end-users : Satisfaction, get the best value for their money amidst others

For operators : end-user Satisfaction, Increase revenue, achieve customer loyalty, maintain competitive

edge amidst others. Surveys have shown that about 90% of customers will not complain before defecting.For Regulators: protect consumers, ensure end-user satisfaction, create a level playing field for operators to compete amidst others.

They all share a common Objective CUSTOMER SATISFACTIONKampala, Uganda, 25 June 2014 2

Reasons

QoE Measurement

Quality of Experience(QoE) has become an important concept since the growth of mobile services.QoE is very SUBJECTIVE in nature as it involves human dimensions.Measuring and improving QoE though challenging, should be undertaken in order to assess the most accurate and complete vision of the value offered by the provider to end-users. If the QoE is high, the user is happy, satisfied and loyal while poor QoE will result in dissatisfied customers.

Kampala, Uganda, 25 June 2014 3

QoE Measurement

USER EXPECTATIONS CAN BE GROUPED UNDER TWO MAIN CATEGORIES 1. RELIABILITY (Service quality of Accessibility and Retainability KPIs) 2. COMFORT (Service quality of Integrity KPIs)


QoE KPIs

QoE KPI Most important measuresService availability(anywhere)

Local and global coverage includedHow seamless it is for the user?

Service accessibility(anytime)

The success rate of user connections for any service

Service access time The delays in setting up any service connection

Continuity of service connection-interruption ratio

The retainability of the service connection and its performance overtime

Reliability QoE KPIs

QoE Measurement


QoE KPI Most important measures

Quality of Session Jitter (delay variation %)Average throughput(kbits/s) towards mobile Bearer stabilityApplication layer packet loss ratioAverage end-to-end delay (ms)

Ease of use How easy is it to use service offered by the network

Level of support How quick and easy is it to get customer support

Comfort QoE KPIs

QoE Measurement

Service level approach using Statistical Samples •Approach relies on a statistical sample of the overall network users to measure the QoE for all the users in the network. (Mean Opinion Score (MOS))

Network Management system approach using QoS parameters•Approach relies on using NMS to collect QoS KPIs from network elements and comparing them with predefined target

Knowing full well that the holy grail of subjective measurements is to predict it from

the objective measurements i.e. predict QoE from a given set of QoS parameters.

NMS approach provides high accuracy than Service level because of its deterministicand objective nature. Using both methods nevertheless provides a better and more accurate picture of the users’ experience


Two pratical approaches to measuring QoE

Relationship between QoS and QoE

QoE = f(AQoS, NQoS) + other subjective factors (expectations, mood, etc.)

Strong Correlation between QoS and QoE

Relationship Between QoE and QoS may be near-linear for a voice call

but non-linear for a video call


Relationship between QoS and QoE

There is an exponential relationship between QoS and QoE

End-to-end QoS is an important enabler for QoE in otherwords a better network QoS in many cases will result in better QoE

QoE can be predicted from a given set of QoS parameters and vice versa (ETSI TS 102 250-1)

QoE may be improved with the right QoS mechanisms.

Relationship Between QoE and QoS may be near-linear for a voice call

but non-linear for a video callKampala, Uganda, 25 June 2014 8

QoS Measurement

Quality of Service(QoS) arrived with 2nd Generation NetworksAbility of a Network to provide a service with an assured service level.It is intrisincally a technical concept, measured and understood in terms of networks and network elements.An objective metric that refers to the ability of the network to achieve a more deterministic behaviour.

Service Qualification is based on six 6 primary components as defined

in ITU-T Rec E-800:

Operability Accessibilty

Retainability Integrity Security

Support

QoS can be measured using KPIs and KQIs


Performance

Assurance

85%

15%

Key Performance Indicators(KPIs)

Key Quality (Assurance) Indicators(KQIs)

Relationship between NP and QoS

Strong Correlation between NP and QoS

In otherwords Poor Network Performance ultimately results to Poor QoS


QoS aspects of service usage(ETSI TS 102 250-1)

Network related QoS

Relationship between NP and QoS


Non network related QoS criteria

Network related QoS criteria85%

Mapping

Network PerformanceIndicators/Parameters Target - range or limit

CALL DROP RATECALL SETUP SUCCESS RATE……PARAMETER N

< 2%< 90%……<= x%

Qos Criteria

15%

QoSQoS

ObjectiveObjective

NP Non-NP

IntrusiveIntrusiveNon-IntrusiveNon-Intrusive

Walk/Drive Test Around test

Walk/Drive Test Around test

OMC-R Counters using NMS

OMC-R Counters using NMS

Active Passive

Surveys, churn-rate, etc…Surveys, churn-rate, etc…

Ex. DT Tools: Nemo, XCAL, TEMS …

Ex.NMS: RPM System, Net Act,imanager M2000 …

SubjectiveSubjective

QoS Measurement

Relationship between NP,QoS and QoE

NP QoS & QoS QoENP is a subset of QoS which in turn, is a subset of the overall QoE scope

NP parameters ultimately determine the QoS

QoE can be predicted from a given set of QoS parameters and vice versa (ETSI TS 102 250-1)


QoS/QoE Assesement

TWO DIFFERENT AND COMPLEMENTARY APPROACHES TO MOBILE QoS (ETSI EG 202 057-3)

Walk/Drive- round tests

Measurements based on network element counters using NMS

Monitoring :use of any available technical tool to assess permanently or for a given period of time a particular QoS parameter. (ETSI EG 202 009-2)


QoS Assesment Target Best Suitable QoS Approach(es)

One-time snap shot DT

Acceptance Procedure DT/ NMS

Continuous Monitoring NMS

Optimisation Cycle DT + NMS

Difference Between NMS and DT


Network Management System Drive Test (DT)QoS experienced by users are taken into account as the calls are made with the real terminals by the real users.

The QoS experienced is not taken into account by users depends to some extent on the design of their terminals which may differ somewhat from those used for formal tests.

Representativity is assured as real traffic provides measurements from the entire network 24h/7

Drive test does not provide a representative of the performance of the whole network because of variation in user density and the complexity of providing service in different areas. (60% of calls made in a network are made indoor).

Track performance down to the cell level Performance stops at Base station level

Provides comparabilty of Congestion and network failures

Misses out on planned Site outages and network failures.

Follow up on Operators’ network expansion, type of equipment deployed and mobile evolution

Impossible to track Network Expansion using DT

NMS reports are actionable as it tracks the availability of network elements (Ex. BSC/RNC)

DT reports are simply informative

Indepedent of enviromental factors Depedent on environmental factors such as weather conditions, seasons, location, timing etc

Traffic carried over the network and associated grade of service (GoS) are obtained using this approach

DT has no information on Traffic carried over a given network.

KPI Build flexibility from numerous counters of NEs Limited KPI observation as it measures the network from an external point.

Relatively no OPEX involved High OPEX as well as time and Labor intensive

Passive and non- intrusive, best approach in continious QoS monitoring.

Active and intrusive, best approach for QoS (coverage) benchmarking purposes.

Draw backs of DT illustrated






Quest for a Monitoring Tool

Monitor simultaneously 24H/7 the state of all network elements/acess technologies in an

operator network passively without compromising their daily operations

Put a check on service degradation and sometimes outages to the benefit of the

government and the masses.

Quest for a system that can monitor the performance of mobile operators independent

of the vendor and technology

Flexible Benchmark Audit; weekly, monthly or quarterly.

Track the evolution and expansion of the mobile operators network.

Reduce churn rate of end-users directly or indirectly

A robust system that is easily upgraded to meet the rapid evolution of mobile

technology.

A platform that will facilitate actionable decision making to the benefits of all parties

involved with little or no OPEX.

Create a level playing field for all operators and protect the interest of all consumers not

just those in the urban areas.


Network Management Systems

EXAMPLES OF NMS solutions in the market today

For REGULATORS (Vendor-Independent)

Regulators (QoS)Performance Management (RPM)system- compatible with all major network vendors (Huawei, ZTE, Ericcsson, Alcatel and NSN)

FOR OPERATORS (Vendor –dependent)

imanager M2000 for Huawei vendor

NetNumen U31 for ZTE vendor

NEtAct for Nokia Siemens Vendor

NetOp EMS for Ericsson and Omnivista 2500 for Alcatel vendor


RPM SYSTEM OVERVIEW

• QoS monitoring and reporting 24/7 all year round

• Automatic Reporting and 3D GIS display(compatible with google earth

• Unlimited number of Users/ Single License

• Multi vendor Multi access technology

• Friendly and intuitive web user interface

Regulators’ (QoS) Performance Management System is the novel NMS solution that handles the task of interfacing all the operators’/service providers’ network monitoring systems, collect perfomance data records and create KPI reports that renders a given network performance against published benchmarks.

• No running costs and upgrades easily as mobile access technology evolves

Main Features

• Built in compliance with ETSI TS 102 250-4&7/ ETSI EG 202 057-3 and ETSI EG 202 009-2


Conclusion

The assessment of the QoS/QoE is expected to be evaluated in checking criteria against reference values. These criteria are measured either objectively via technical means or subjectively via surveys amongst the users. Experts agree that a mix of objective(intrusive and non-intrusive) and subjective measurements remains the best means to get the whole QoS picture.

Today, four out of five Regulators only employ Drive Test(intrusive) both for one-time snap shot, benchmarking and continuous QoS monitoring(reactive sense). Given DTs drawbacks and inaccuracy with respect to continuous QoS monitoring, its high-time Regulators, gaining from operators’ perspective, adopted the novel, practical and more efficient approach – the use of Network Management Systems (NMS) for proactive continuous QoS monitoring.


Recommendations

Regulators, as recommended in ETSI EG 202 009-1 clause 7, in the quest for effective continuous QoS monitoring and assessment should perform the measurements themselves and make the results available publicly rather than entrust in mobile operators to provide the QoS information

Thus Regulators should procure RPM system or any other NMS that facilitates the complex task of enforcing mobile operators to adhere to agreed KPIs, put a check on service degradations /outright outages and even penalities for repeated failure to achieve minimum QoS/ QoE targets.


THANK YOU FOR YOUR ATTENTION

For more information on RPM system or any other NMS

Feel free to contact me or email [email protected]


mailto:[email protected]

Documents

Monitoring QoS/QoE of mobile networks-novel approach