Traffic optimization through active testing/media/White Papers/Mobile/Traffic... · In 2G and 3G networks, ... QoE metrics originate in the mobile device, ... White paper Traffic

Traffic optimization through active testing

The evolution in mobile network testing

By Monica Paolini, Senza Fili

Sponsored by

White paper Traffic optimization through active testing

© 2016 Senza Fili Consulting • www.senzafiliconsulting.com |2|

1. Introduction Evolution in usage and technology drive a new testing approach

Wireless networks have expanded their reach and overtaken

wireline communications at a rapid pace, and operators have to

deal with a seemingly unstoppable growth in traffic volumes.

Today, mobile devices already outnumber people. With the

emergence of IoT, we can expect to see a sharp increase in the

number of connected devices. Along with the growth in traffic

volumes, expansion in coverage, and improvements in capacity,

the composition, distribution and use of data services is changing.

These changes force operators to manage traffic differently, to

utilize network resources more effectively, and to meet ever-rising

QoE expectations. New technologies and new tools are now

available that enable mobile operators to manage mobile traffic in

real time, with greater flexibility and control over network assets.

As mobile operators transition to a more active approach to traffic

management, they will have to learn to monitor and control more dynamic and complex

networks, which are also more vulnerable to disruption or degradation of service. Performance

issues result in steep costs: lost revenues, increased churn, brand damage, more complex

maintenance, and new customer support demands. As a result, operators have to step up their

efforts to gain visibility into network performance, and to address service issues as they arise and,

where possible, to prevent them.

In this context, network testing, monitoring and troubleshooting acquire a higher profile. They

become ongoing activities that help operators continuously fine-tune networks as they evolve, in

order to accommodate new and fluctuating usage patterns. Network testing is moving from

passive to active, becoming a crucial element within a more proactive approach to optimizing

network performance and resource utilization.



2. Making the subscriber experience count Network optimization shifts toward QoE

The emergence of QoE as one of the hottest topics among operators may seem odd:

wasn’t a great subscriber experience always the ultimate goal for mobile operators,

even before we started to talk incessantly about QoE? Subscriber experience has

always been paramount for keeping churn down, attracting new subscribers and

supporting new services, but, as data services have gained prominence and wireless

networks have moved to IP, mobile operators have faced new challenges in

optimizing the use of network resources to maximize QoE explicitly.

To do that, they have to be able to measure QoE reliably and accurately – and that is

difficult. On the right we list some of the challenges that operators face in measuring

QoE, which is inherently subjective and impossible to reduce to a few KPIs. Even

more complex is the task of relating QoE degradation to specific problems in the

network and identifying the appropriate corrective action.

In a switched-voice network, KPIs such as dropped calls or MOS measure voice

performance. In LTE networks, voice performance is more variable, and KPIs like

MOS and dropped calls – while still relevant – are no longer sufficient. Requirements

for data traffic vary even more. Streaming video, for instance, requires a lower

latency or jitter and more bandwidth than messaging or web browsing.

Conversational video such as ViLTE, in turn, needs symmetric real-time connectivity

and hence has even more stringent requirements.

In this context, an averaged measure of network latency is not sufficient to capture

the impact of latency on QoE – and the same holds for other KPIs. Low network

latency may hide a relatively high latency for voice or video through significant

packet loss. Conversely, a somewhat higher average latency induced by giving voice

and video traffic priority over other applications may result in an overall superior

QoE, because the latency is kept low only for the applications that are more sensitive

to it – i.e., voice and video.

Why is it difficult to measure and assess QoE?

QoE is subjective. What counts as a good experience varies across subscribers. Some may be happy to have enough basic coverage in their home to make a voice call; others expect no stalls in an HD video inside a fast-moving train during rush hour.

Multiple QoE metrics for data services. Because the measurement of QoE is still in its early days, there are no metrics that are used consistently – e.g., there is no equivalent of voice MOS for video. QoE metrics vary for different types of traffic, application, service, network, or device. For instance, voice QoE is affected by call setup time and call stability, which go beyond the standard voice KPIs. The variability is bound to increase with the introduction of IoT services, each of which have different capabilities and requirements.

QoE metrics are specific to the device. Operators can measure QoE from simulated traffic data, or indirectly from the network or from crowdsourcing data, but ultimately the measurements reflect the experience on the device. In addition, the device itself – not the network – may be the cause of bad QoE. Measuring QoE directly from the device would be ideal, but it raises privacy issues and implementation is complex, because operators would need control over the devices.

Identifying the source of low QoE is complex. QoE degradation may depend on the specific state of a device, but when QoE issues are widespread, they typically depend on a device type, an application, a security issue, or a network performance issue – or a combination of these factors. It is difficult to pinpoint the cause, especially fast enough to take action that will contain disruption. When the network causes QoE issues, the operator needs to combine high-level, end-to-end visibility into the network with the ability to focus in on individual network elements, functions, or sources (e.g., operator or OTT application).



Furthermore, using the same example, a high-latency issue may have different root

causes inside or outside the network and, depending on the causes, its impact varies

across applications. In 2G and 3G networks, error coding and protection for large-

packet transmissions makes the dropping of individual, small packets, possibly due to

faulty equipment, go unnoticed. With the increased reliance on transmission of small

packets, LTE networks are more sensitive to dropped packets when using equipment

similar to that in 2G and 3G networks. To detect and resolve these issues, operators

need new tools and techniques for determining latency, jitter and packet loss.

Finally, what counts as a good or acceptable QoE depends on context to some

extent. No operator wants to deliver bad service, but inevitably, there is some

variability in the quality of service. The threshold for acceptability varies on factors

such as network type (e.g., 3G, 4G or Wi-Fi), location (e.g., indoor/outdoor;

rural/metropolitan), subscriber tier or policy rules (e.g., subscribers may choose low

video resolution to get a cheaper plan), congestion level – and the interaction among

these factors. For some subscribers, service availability is more important than

quality – e.g., they would tolerate suboptimal call quality in an indoor environment,

rather than not being able to make the call at all. Others may accept nothing short of

toll quality, and may prefer not to make a call if coverage is tenuous.

All this points to a mobile environment in which performance measurement, QoS

assessment, and management of network degradation and traffic have become

more complex, less deterministic and open to choices. There is no longer a best way

to run a mobile network. Instead, there are different ways to approach the

optimization of network assets, which depend on operators’ specific QoE targets. It is

more work, but this new flexibility gives mobile operators the opportunity to

differentiate themselves from competitors in a way that goes beyond coverage and

capacity; it is no longer just RF optimization.

Operators have started to embrace this new approach only recently, spurred by the

increasing availability of solutions that enable them to manage their networks

proactively, in ways that are affordable and effective. But there is still much to learn

before QoE can become the main driver to optimization. Testing and monitoring

methods must undergo deep changes to enable the shift to a QoE-driven approach –

and to gain operators’ confidence that this approach is effective in boosting QoE.

Requirements for QoE-based network optimization

End-to-end visibility. QoE metrics originate in the mobile device, but they provide a measure of performance of the end-to-end network. To relate QoE metrics to network performance, operators need end-to-end visibility into the network. They have to take a high-level view of the network initially, and then narrow down the focus to different network areas or functions in order to pinpoint the root causes of QoE degradation. The required depth of analysis varies depending on what is creating the problem at hand. Operators need flexible tools that give them the right level of granularity – not too much, to avoid data overload; not too little, as that would prevent them from fully solving the problem.

Real-time data. Historical data is and will continue to be useful for assessing overall network performance and understanding trends, but it is not sufficient for addressing performance issues as they arise or for preventing them. Operators need real-time data to alert them to unpredictable changes in network performance caused by events such as changes in demand, network malfunctions, or security breaches. Depending on the issue, the appropriate time resolution changes. It might be one day, or it might be a millisecond. Sometimes knowledge about the immediate past is crucial – for instance when identifying a network element that is malfunctioning and causing degradation in voice QoE or an increase in setup time; in other cases, a warning on an impending disruption – for instance, a sudden growth in traffic in one location – enables preventive action.

Automation. Managing real-time data is challenging: the volumes of data are large, and data has to be monitored continuously. Automation is needed so operators can leverage real-time data without being overwhelmed by it. Operators are eager to use automation, but the mobile ecosystem is still learning how to automate processes, so the transition requires time.

Analytics. To benefit from the automated real-time data collection, operators need analytics to extract the relevant information from the data, and take preventive or corrective actions.



3. In a virtualized network, testing never stops Testing solutions will become virtualized, too

Mobile operators have started to virtualize their networks – in efforts that extend all

the way from the core to the RAN – with NFV, SDN and C-RAN. Virtualized networks

promise much more than cost savings; they enable (and push) operators to embrace

a more flexible, agile and dynamic approach to deploying, expanding and operating

their networks. With the increase in data volumes and in the number and variety of

devices, both from human subscribers and the IoT, operators need to find a more

efficient way to use their network resources. Virtualization gives them the

opportunity to do that.

The transition to virtualized networks requires a deep transformation in the way

operators manage their networks, going well beyond the shift from dedicated

hardware, to bare metal or cloud infrastructures. Specifically, operators will have to

rethink the way they test and validate their networks. The traditional model of a

network being tested and validated ahead of commercial launch is no longer

adequate, because the mobile network continues to evolve, grow, and be fine-

tuned, and because network resources are allocated in real time in response to

variable network conditions and subscriber demand. The mobile network becomes a

moving target that is more vulnerable to performance variability.

On the right we list some of the changes that virtualization brings. Testing and

validation rise in prominence and shift toward a more proactive approach, geared to

capturing fluctuations in performance. Eventually we expect network testing and

monitoring to shift from identification to prediction of performance issues, once the

wireless ecosystem learns how to recognize the symptoms of disruption, and not just

its effects.

In a virtualized network, active testing ensures that software-based solutions

perform reliably in the COTS hardware environment, by directing the testing activity

to specific scenarios that might identify potentially weak areas. Ahead of full

How virtualization changes testing and monitoring

Lab-based and probe-based passive testing is still necessary, but it is not sufficient to assess performance in highly dynamic networks, in which the parameters, the physical instantiation of functions in the core, and the allocation of network resources change continuously.

In a virtualized environment, the hardware implementation that supports a function changes all the time, and the operator has to ensure that this does not affect performance. As a result, testing and monitoring have to be constantly active to capture the network KPIs and QoE metrics, and to identify any emerging issues early enough to preempt performance degradation.

Active testing is well suited to capturing variations in performance in virtualized mobile networks, which may operate under intense pressure from congestion and require constant fine-tuning. With active testing, the operator can collect performance data from a large number of use cases, and repeat those tests as the network parameters and resource allocation change.

Continuous testing and monitoring of the network requires a real-time platform to collect and analyze data. A high degree of automation and an effective analytics platform are necessary to manage the higher and more complex volume of performance data in a cost-effective, efficient way.

As the core functions are virtualized, testing and monitoring will also be under pressure to virtualize. Vendors have started to offer virtualized solutions, and this trend will continue as operators request virtualized solutions in order to reduce their costs, reduce the number of engineers in the field, and make network testing and validation faster and more flexible. Virtualized testing solutions can also be deployed in legacy networks.



virtualization, operators can already use active testing to explore the impact that

new devices, services or applications will have, before making them available to

subscribers.

Testing and monitoring will take up a role in the orchestration of network functions,

because they provide operators with the visibility into the performance and

capabilities of their network resources.



4. Real-time predictive testing and monitoring take center stage The focus on QoE and virtualization are key drivers of a new testing approach

Both the way we use mobile networks and the way mobile operators run their

networks have changed and continue to change. The move to a QoE-based approach

to network optimization and the transformation induced by virtualization are two

powerful drivers to a fundamental change in the way we test, validate, monitor,

troubleshoot and optimize mobile networks.

One of the most prominent changes is the transition from a primarily passive

approach in testing to an active one. Passive testing measures performance in key

network locations – often with physical probes – by recording KPIs. In most cases,

operators still use these KPIs retrospectively, as historical data. Passive testing

continues to be valuable for characterizing the level of network performance,

managing the customer experience, and understanding historical performance

trends.

Yet in a highly dynamic network that has to be optimized in real time, passive testing

is no longer sufficient, as we discussed in the previous pages. Operators are starting

to adopt a hands-on, proactive approach, tracking network performance at different

levels of granularity to identify the root causes of performance issues as or before

they happen, and to resolve them in real time. By actively simulating network

conditions that are likely to occur, operators gain insight into strengths and

vulnerabilities of their networks and can prepare for them.

Every time a change is introduced in a network – and changes are ubiquitous in

today’s networks – there is a potential for disruption. Active testing helps operators

explore the impact on the network, the devices and, ultimately, QoE. For instance,

ahead of a software update on a mobile device or network element, the operator

Advantages of a proactive approach in testing

Identification of indicators of disruption, to prevent negative impact on QoE

Operators can proactively identify use cases, simulate specific use scenarios, or stress-test the network to predict which conditions may trigger QoE degradation or service disruption. Early detection of indicators enables operators to take preventive, rather than corrective, action.

Resolution of performance issues in real time

When prevention is not possible, operators still have to identify performance issues as quickly as possible, and find resolutions in real time.

Reduced operational costs

The ability to quickly and effectively isolate and address problems reduces opex. A network less affected by disruption can be managed with fewer staff resources. Customer support costs are also curtailed, because operators can contain the effects of disruption.

Fewer engineers in the field

Active testing expands the role of emulated data, and this reduces and potentially eliminates the need for expensive and staff-intensive field testing.

Faster introduction of network changes

Software upgrades to network elements are increasingly frequent. By testing upgrades proactively, mobile operators can validate them and test the impact across devices, locations, technologies, and services before making them live.



may simulate its impact on the device itself; also it may ensure that the update does

not affect network performance or compromise specific network functions or

services. The operator can test different scenarios involving different devices,

applications, and services, and see if any is disrupted. When this is the case, the

operator can troubleshoot the problem, make changes to fix the problem, and rerun

the simulation until the problem is resolved.

Because of the high frequency of changes in the network, it has become more

difficult – but also less relevant – to draw a line between testing and monitoring:

both are ongoing activities geared to optimizing the allocation of network resources.

At the same time, the shift toward more proactive testing and monitoring runs

parallel to an equally new and proactive approach to network optimization and

traffic management. In this context, testing and monitoring cease to be lateral

supportive functions that validate new deployments or that ensure everything works

as expected. Instead, testing and monitoring take a central role in the orchestration

of mobile networks. They provide the visibility and insight into the automated

orchestration systems and the end-to-end network – the intelligence that operators

need in order to make the best use of their network assets.

Emerging challenges in testing and monitoring

Measure QoE metrics at the aggregated level, and along specific dimensions, such as traffic type, application, service, device, location, air interface.

Assess the performance of voice services using switched voice, VoLTE and Wi-Fi Calling, with a possible extension to OTT voice apps.

Understand the video experience for both streaming video and conversational video (e.g., ViLTE) from both QoE and network perspectives, so operators can assess the impact and benefits of video optimization solutions, as well as the quality of the video experience across devices, locations, applications, etc.

Ensure that best-effort data services (e.g., web browsing, texting, email) retain a good QoE even in networks where real-time data (voice and video) receive preferential treatment.

Test RAN and EPC performance across multiple cellular interfaces (2G, 3G and 4G), Wi-Fi, and, in the future, LTE unlicensed and eventually 5G.

Include testing of multilayer HetNets, to provide visibility into how different elements interact both with each other (e.g., macro- and small-cell interference) and to new RAN topologies, such as C-RAN and vRAN.

Verify that backhaul and fronthaul have the high capacity and low latency, jitter and packet loss needed to support RAN and EPC transmission.

Conduct network testing and monitoring on both the control plane and the user plane, to make sure that signaling does not become a bottleneck that limits RAN and EPC performance.

Extend testing and monitoring to the physical and virtualized packet core, while continuing support for the coexisting legacy core.

As the network becomes virtualized, and as testing and monitoring become more deeply embedded in the mobile network, make testing and monitoring solutions virtualized and less dependent on field-based activities.



5. Implications Testing and monitoring become more closely embedded in mobile networks

Testing and monitoring have started to take a larger, more active role in helping

mobile operators optimize the use of network resources, and meet the increasing

demand for data access from subscribers and IoT devices. Providing connectivity is

no longer sufficient. Mobile operators have to be able to do so in a cost-effective

way and provide reliably good service, as traffic volume and complexity grow. As a

result, operators are moving toward managing traffic and networks to maximize

QoE, and adding flexibility and agility to their networks with virtualization.

To meet their performance goals, mobile operators have to take a new approach so

they can identify and address any performance issues immediately. Specifically,

operators are moving to testing and monitoring platforms that give them access to

performance data in real time, at multiple granularity levels, for each element and

function in the network.

This more active approach widens the role of testing and monitoring within mobile

networks (see list on the right); it also makes testing and monitoring more complex,

because operators need to collect and analyze huge data sets from multiple sources.

To be effective, testing and monitoring solutions have to be fully automated and

supported by a robust analytics platform that enables operators to gain insight from

their data and use it to address performance issues or optimize the network.

Although adding complexity, the ability to leverage network performance data in

real time can deliver substantial benefits:

Per-bit and per-connection cost reduction through a more effective use of

network assets, which facilitates the management of video and IoT traffic

Reduction of opex though automation and a reduction of field testing

Faster and smoother introduction of network and software updates.

A widened scope for testing and monitoring

View Provide end-to-end visibility in the mobile network in real time, and relate performance to QoE, enabling operators to optimize resource utilization.

Emulate Identify relevant scenarios that network changes may induce and that may cause disruption; simulate those scenarios to find potential network vulnerabilities and address them.

Detect Identify performance and QoE issues before subscribers start calling in.

Identify Provide the information with the appropriate granularity to enable the mobile operator to identify the impact and the root cause of network disruption or service degradation.

Warn Alert the operator when unusual patterns of activity emerge, before they affect service or performance.

Recreate Once a performance issue arises, simulate a reoccurrence of the event to get a deeper understanding of its cause and its implications beyond the initial occurrence.

Suggest Provide mobile operators with suggestions on how to take protective or preventive action.

Predict

Predict potential factors that may degrade or block service so operators can take action before a performance issue arises, avoiding the high costs and lost revenues of network disruption.



Glossary

2G Second generation

3G Third generation

4G Fourth generation

5G Fifth generation

COTS Commercial off-the-shelf

C-RAN Cloud RAN

EPC Evolved Packet Core

HD High definition

HetNet Heterogeneous network

HSS Home subscriber server

IMS IP multimedia subsystem

IoT Internet of things

IP Internet Protocol

KPI Key performance indicator

LTE Long Term Evolution

MME Mobility management entity

MOS Mean opinion score

NFV Network Functions Virtualization

OTT Over the top

PCRF Policy and charging rules function

QoE Quality of experience

RAN Radio access network

RF Radio frequency

SDN Software-defined networking

SGW Serving gateway

SON Self-organizing network

ViLTE Video over LTE

VoLTE Voice over LTE

vRAN Virtual RAN



About Spirent The mobile infrastructure ecosystem is vast and moving quickly. Spirent Landslide™ encompasses it from end to end, providing our

customers with the vision to help them minimize risk and optimize performance. Landslide is (re)defining Network Operations by

delivering solutions for the lab and live network including: Test standards and Methodologies, DevOps and Conformance Testing,

Performance Testing, POC and Network Design, Turn-up Testing, SLA Active Testing, Fault Isolation, Big Data Analytics. Landslide uses

highly configurable end-points that emulate real-world control and data plane traffic of millions of subscribers moving through the LTE,

GSM, UMTS, eHRPD and Wi-Fi networks while consuming IMS and Over-The-Top services. Landslide in the operational network enables a

smarter approach for network performance management and minimizes network outages. For additional infomration, visit

http://www.spirent.com/Products/Landslide or contact Jim Kinnebrew at [email protected] or +1 817 271 7711.

About Senza Fili Senza Fili provides advisory support on wireless data technologies and services. At Senza Fili we have in-depth expertise in financial

modelling, market forecasts and research, white paper preparation, business plan support, RFP preparation and management, due

diligence, and training. Our client base is international and spans the entire value chain: clients include wireline, fixed wireless, and

mobile operators, enterprises and other vertical players, vendors, system integrators, investors, regulators, and industry associations. We

provide a bridge between technologies and services, helping our clients assess established and emerging technologies, leverage these

technologies to support new or existing services, and build solid, profitable business models. Independent advice, a strong quantitative

orientation, and an international perspective are the hallmarks of our work. For additional information, visit

www.senzafiliconsulting.com, or contact us at [email protected] or +1 425 657 4991.

About the author Monica Paolini, PhD, is the founder and president of Senza Fili. She is an expert in wireless technologies and has helped clients worldwide

to understand technology and customer requirements, evaluate business plan opportunities, market their services and products, and

estimate the market size and revenue opportunity of new and established wireless technologies. She has frequently been invited to give

presentations at conferences and has written several reports and articles on wireless broadband technologies. She has a PhD in cognitive

science from the University of California, San Diego (US), an MBA from the University of Oxford (UK), and a BA/MA in philosophy from

the University of Bologna (Italy). You can contact Monica at [email protected].

© 2016 Senza Fili. All rights reserved. This white paper was prepared on behalf of Spirent. The views and statements expressed in the white paper are those of Senza Fili, and they should not be inferred to reflect the position of Spirent. The document can be distributed only in its integral form and acknowledging the source. No selection of this material may be copied, photocopied, or duplicated in any form or by any means, or redistributed without express written permission from Senza Fili. While the document is based upon information that we consider accurate and reliable, Senza Fili makes no warranty, express or implied, as to the accuracy of the information in this document. Senza Fili assumes no liability for any damage or loss arising from reliance on this information. Trademarks mentioned in this document are property of their respective owners. Cover page photo by emEF/Shutterstock.

http://www.spacecurve.com/

http://www.senzafiliconsulting.com/

mailto:[email protected]

http://www.senzafiliconsulting.com/

Documents

Traffic optimization through active testing/media/White Papers/Mobile/Traffic... · In 2G and 3G networks, ... QoE metrics originate in the mobile device, ... White paper Traffic