Modular conception of software and hardware components for … · 2019. 5. 20. · Check KPIs Root cause analysis with internal tools (OSS, Actix) possible Apply solution Low KPIs

Leonid Semakov

ROHDE & SCHWARZ Product Engineer

Valentin Kozharskii

ROHDE & SCHWARZ MNT expert

Saint Petersburg, 21.05.2019

Modular conception of software and hardware

components for service quality data acquisition and

analytics in evolving mobile communication networks

ı The Future

ı Future Earth concept

ı Evolving Technologies

ı The Present

ı Artificial Intelligence and Machine Learning

for a new level of insights

ı New quality control reality

ı The Past

ı Workshop

Agenda

Network quality data acquisition and analysis

The Future

5G, OTT, edge computing and AI

will change lives in 2019

*IHS Markit white paper

Imagine that deforestation is

tracked in real time, and AI takes

steps to prevent it

*Global Forest Watch (GFW)

Tree cover loss map


Safe Earth

ı Real-time segment monitoring

ı Aggregate data from Satellites, Drones, Smart Sensor Systems

ı Fast processing by AI

ı Start prevention processes

ı Immediate response

ı Termination of illegal actions

ı Additional benefits

ı Track forest fires

ı Earliest fire suppression


How we can change the future?Fully Digital and Connected World


Smart Earth

ı Smart Cities

ı Full Earth Monitoring

ı Antarctic and Arctic Sea Ice Monitoring and Tracking (5G/6G Sat segment + LP sensors)

ı Pollution monitoring

ı Mammals monitoring

ı Plants monitoring


What’s next for humanity?

Planetary scale monitoring and communications

Planetary scale monitoring architecture

Advances in the science and technology

2019 2030

ı 5G

ı GHz/ mmW

ı Silicon

ı 3d transistor

ı Multi-core

ı Key encryption

ı Folding display

ı Interaction sensors

ı Brain sensors

ı Robotic hand

ı Human-machine interface (HMI)

ı 6G

ı THz/ Light

ı Carbon Nanotubes

ı Spin qubits

ı Quantum Computing

ı Quantum Cryptography

ı LCD/Sensor Fabric

ı New physical principles sensors

ı Neural interfaces

ı Artificial organs

ı Invisible support system


R&D Evolution

ı Source of new technologies

ı Sub-ms delays, mmW range, AI, ML: today we are on the verge of a new leap

ı Sub-THz range, quantum computing: research and development is currently at an exciting

stage

ı Research and development centers need high specification, precision instruments

ı Demand in terahertz (THz) products and systems


Overcoming technologies

ı 2019: Influence on precision instruments demand

ı We need more advanced technologies for R&D

ı R&S: the company 16% of the companies annual

net revenue is invested into R&D (*Wikipedia)


The A&D Component Foodchain

Design

Build

MeasureEvaluate

Integrate

The Problem:

Compare physical measurements with expected (design) data

Verify

Qualify


Benchmarking for evolving systems

ı Sub-ms delay test system

ı Sub-THz band test system

ı Terabit-per-second traffic test, DPI & benchmark systems

ı New KPI for OTT, Telephony, Neural interfaces, Avatar transmission, etc.


Data consumption is exponentially growingAlready 10 times in 5 years

Source: Ericsson Mobility report – 2018, Ericsson

How can you make sure

network support the data

without affecting services?

Need a complete suite of

tests to verify both!

Near-future technical challenges requires future-proof hardware

TSMA6

TSME6

Size andweight

reduction, easy setup

Spectrumrequirementup to 6 GHz

Paving theway for newtechnlogies

(LAA, 5GNR)

Future proof(control

additional HW

instances)

New 3GPP Bands…

• 5GNR below 6 GHz

• Need for spectrum clearance

@ 3,5…6 GHz spectrum

• Spectrum refarming

• 3GPP 5GNR definition phase

• Capacity enhancements…

(MIMO 4x4 deployments…)

• „Above 6 GHz“ spectrum with

first field tests @ 28 GHz,

Beamforming

• Non-Stand alone mode in

5GNR: 5GNR + LTE

• Carrier Bandwidth of 100 Mhz

and more


5G devices: currently still prototype boards / MTP

Source: Ericsson Mobility report – 2018, Ericsson

One software architecture, many hardware possibilities

Qualipoc Android

phones

BD, AI, ML


One software architecture, many hardware possibilities

QualiPoc 6G DRONES

BCD, AAI, FML

QualiPoc NeuralSmartCity sensors

QualiPoc QubeSat


*2030-2040 Possibility (art)

HW – TSME4xDC – The spectrum challengeEurope

700 MHz

3.4 - 3.8 GHz

24.25 - 27.5 GHz

China

3.3 - 3.6 GHz

4.8 - 5.0 GHz

24.75 - 27.5GHz (study)

37 - 43.5 GHz (study)

US

[CBRS band (3.5GHz)]

27.5 - 28.35 GHz

37.0 - 40 GHz

64 - 71 GHz (unlicensed)

Australia

3.6 GHz

26 GHz

Korea

3.5 GHz

28 GHz

Japan

4.4 - 4.9 GHz

28 GHz

NR frequency range 1 (450 MHz – 6 GHz)

reserved numbers 65-256

Downlink Uplink

… … …

n77 3.3 – 4.2 GHz 3.3 – 4.2 GHz

n78 3.3 – 3.8 GHz 3.3 – 3.8 GHz

n79 4.4 – 5.0 GHz 4.4 – 5.0 GHz

… … …

NR frequency range 2 (24.25 – 52.6GHz)

Reserved numbers 257-512

Downlink Uplink

n257 26.5 – 29.5 GHz 26.5 – 29.5 GHz

n258 24.25 – 27.5 GHz 24.25 – 27.5 GHz

n259 n/a n/a

n260 37 – 40 GHz 37 – 40 GHz


HW – TSME4xDC – The spectrum challengeEurope

700 MHz

3.4 - 3.8 GHz

24.25 - 27.5 GHz

China

3.3 - 3.6 GHz

4.8 - 5.0 GHz

24.75 - 27.5GHz (study)

37 - 43.5 GHz (study)

US

[CBRS band (3.5GHz)]

27.5 - 28.35 GHz

37.0 - 40 GHz

64 - 71 GHz (unlicensed)

Australia

3.6 GHz

26 GHz

Korea

3.5 GHz

28 GHz

Japan

4.4 - 4.9 GHz

28 GHz

NR frequency range 1 (450 MHz – 6 GHz)

reserved numbers 65-256

Downlink Uplink

… … …

n77 3.3 – 4.2 GHz 3.3 – 4.2 GHz

n78 3.3 – 3.8 GHz 3.3 – 3.8 GHz

n79 4.4 – 5.0 GHz 4.4 – 5.0 GHz

… … …

NR frequency range 2 (24.25 – 52.6GHz)

Reserved numbers 257-512

Downlink Uplink

n257 26.5 – 29.5 GHz 26.5 – 29.5 GHz

n258 24.25 – 27.5 GHz 24.25 – 27.5 GHz

n259 n/a n/a

n260 37 – 40 GHz 37 – 40 GHz


HW –TSMx-NR


Software 5G-NR (TSMx6-K50) – we are there


QualiPoc Android

Smartphone-based test tool for measuring the service quality and

network performance of a mobile network

ı Intuitive User Interface The intuitive and customizable user interface is based on the latest features of today's smartphone

technology and offers an easy operation.

ı Extensive set of service tests Including call tests, voice quality (including POLQA, PESQ, and SQuad08) data tests, video

streaming and video quality as well as app service tests

ı Customizable GUI QualiPoc offers a high grade of customization features offering to create own monitors out of over

200 parameters

ı Advanced RF optimization feature Advanced channel and cell locking are crucial for RF optimization feature to control the quality and

coverage of a wireless networks.


Optimization Team

Check KPIsRoot cause analysis

with internal tools

(OSS, Actix) possible

Apply

solution

Low KPIs

OSS

Quality/

bechmarking3rd level

trouble ticket

Solution not found

solution found

Drive test and

analysis

Service

monitoring

solution found

Prepare for

Connect/Chip test

Regulatory complaint

synergies by having

compatible benchmarking,

monitoring and optimization

solutions (in data collection

and post-processing)


There are two approaches

A. Optimization engineer goes to the field himself (not driving the car),

analysing the data in realtime, working with network operations

directly to check improvements on site

B. Unskilled staff are sent to collect to collect measurement data while

analysis happens in the office

Trend

ı optimization engineers have increasingly less skills: requires the tool to

do the analysis and assist in solution finding analysis modules in post-

processing!

Optimization team and drive testingDrive test is used for complex problems


Centralize expertize and reduce costs in the fieldCentralized campaign management with real time assistance to field technicians

SmartMonitor

• Centralized test campaign configurationDistribute test

campaigns to

users in the field

• Real time feedback from the systems in

the field

Report position information,

test results or selective failed

session data


Fix the problem during the first visitReal time monitoring and troubleshooting

QualiPoc Android

Mobile

Network

IP

Network

Control & Monitoring Analysis & Reporting

Freerider with QualiPoc slaves

Expert in

the office

Engineer

in the fieldEngineer

in the fieldQualiPoc

Android Probe

Remote* TSMA scanner

Remote control and

monitoring

Direct link between real time

results (SmartMonitor) and for

real time troubleshooting (SmartAnalytics)


SmartAnalytics – common analytics solution for all probes


Probes

Analytics and

Reporting

Scene

SmartAnalytics

Anywhere

Any device

SmartAnalyticsWhat it is – Web Frontend

ı The “Smart” one

Single portal to all our tools “Smart”

Web Frontend


SmartAnalyticsWhat it is – Detail Analysis and Mapping

ı Drill down analysis

L3 message decoding

Single result displays (line chart, table view, data specific view e.g. scanner,…)

ı Feature rich mapping

Plot any point value on the map

Line to Cell

Cell coverage


SmartAnalytics

What it is – NPA* Analysis modules

BTS evaluation

LTE Carrier aggregation

Delta / comparison modes

2G / 3G / 4G data

LTE MIMO / DLAA

VoLTE

Neighborhoods / Handover Spectrum analysis

Voice call extension Coverage / Interference


SmartAnalyticsWhat it is – Web reporting

ı Dashboard based

ı Ad-hoc report

ı WYSIWYG configuration

ı Rich set of predefined dashboards

ı Scalable and powerful


SmartAnalytics – common analytics solution for all probes


Scene

SmartAnalytics

Scene

Extract more value out of the collected

data

Use case driven, example Network Performance Score

Statistical Analysis; specialized interface

Drill-down & root cause analysis

Customizable workspaces

Intelligent filtering, cross-correlate different dimensions

Insights on competitors’ QoE

Creating more insights leveraging

Machine Learning

Vision

• Call Setup/Success Rate are two fundamental KPIs to measure performance

• A traditionally binary result, the call either drops or not

• We find levels of CSR/CSSR of 1% (even less)

• Large number of calls to make the result statistically significant

• Calls may fail or drop at different locations

Creating more insights through Machine Learning

Vision – The problem


Binary test scoring

Objective: Getting more value out of each test


Vision – The proposition


• Provide a way to predict or calculate the risk for a call to drop, by

assigning every call a “Call Stability Score”

• Calculate this score by using deep learning

• Advantage: instead of just being able to look at binary results (e.g the

1% dropped or failed calls), every call gets a score which indicates the

“stability” and therefore the potential for optimization. This leads to a

dramatic efficiency improvement

• Applications

• Score-based benchmarking (by operator, area, over time, etc.)

• Outlier detection (candidate samples for root cause analysis)

• Prediction of test results


Vision – The proposition



Vision – Implementation

Data• CHIP Test 2017 Database

• >20’000 VoLTE calls, ~1.5% dropped, ~2.5% failed

• Filtering network issues

• Performing per-test analysis

Features• LTE Measurements close to the drop or failure

• RSSI, RSRQ, RSRP, SINR

• Voice Quality

• Signaling

• Time sequence

• …54 aggregated features

Machine Learning methods• Traditional: Naive Bayes, Logistic Regression

• Deep Learning: testing different techniques and architectures

Test

Test

Test

Test

Data

Training

Process

Training

DataMachine

Learning

Model

Training Stage

Test

ML Model

Test

Data

Sample Score or

Prediction

Inference Stage


Multilayer Operator Map widgets

Aggregated call stability scores

Low stability scores markers


Visualization Example

Operator A Operator B

CDR: 0.20%

CSFR: 0.35%

Call Stability Score: 71

CDR: 0.25%

CSFR: 0.28%

Call Stability Score: 85

Machine learning for increased efficiencySmarter analytics with machine learning introduction

Smart platform

MNT – Creating more insights through Machine Learning

Summary of the benefits

• Automatic calculation of the call stability score for every call. Every result becomes more meaningful.

We maximize the number of usable/actionable test results by a dramatic factor

• Obtain meaningful results also from places where the “traditional” test result would not indicate any

problem

• Easier to measure optimization improvements. Proactive identification of risky areas

• Less test data needed

Extract more value out of the collected

dataEfficiency


1. Understand at a glance best Operator and/or areas, technologies CSS wise

2. Next step: statistical analysis (what, where and when) to proactive identification of risky areas

Call Stability Score – use case integrated in SmartAnalytics Scene


1. Understand at a glance best Operator and/or areas, technologies CSS wise

2. Next step: statistical analysis (what, where and when) to proactive identification of risky areas

Call Stability Score – use case integrated in SmartAnalytics Scene

We click on

one red bar

(“when”) to

focus our

analysis


1. Drilling-down, after click on a poor performing contribution in the trend chart, we can identify a poor CSS area and a

culprit session in which we obtain a poor CSS, 0.17, despite the call did complete successfully

2. Next step: straightforward session analysis

Call Stability Score – proactive identification of risky areas


1. Drilling-down, after click on a poor performing contribution in the trend chart, we can identify a poor CSS area and a culprit

session in which we obtain a poor CSS, 0.17, despite the call did complete successfully

2. Next step: straightforward session analysis

Call Stability Score – proactive identification of risky areas

During the call session, there is a

sudden drop of both signal level

and SINR.

That has led into the machine

learning model providing a poor

score.

The model has identified calls that

have dropped with this behavior.

This approach contributes to

proactively identifying risky

areas, obtaining more value out

of the collected data and

exposing previously hidden data


Thank you! Questions?

Time for demonstration


LIVE DEMO SESSION

Workshop by Valentin Kozharskii

ı Performed by ROHDE & SCHWARZ MNT expert (Mobile Network Testing)

ı Walk & drive test results: May 2019, Saint-Petersburg / ONLINE

ı 5G Networks, R&S Germany results / Stored Data

ı Smart Platform Demonstration (Smart Analytics online server)

ı Freerider III

ı Freerider 4


Mobile operators spending Smart (platform)

Source: The Mobile World Economy – 2017, GSMA

Smart (analytics)

Mobile networks complexity and quality control tasksStandards:

- 1990s – 1G-2G

- 2000s – 1G-2G-3G (IMT2000 concept)

- 2010s – 2G-3G-4G-5G, NB-IoT

Number of radio network elements to be controlled:

- Dramatically increased since beginning of 90s till the present time (230order more that at the

beginning).

Other circumstances:

- Cost reductions

- Operators staff headcount decreasing

- Work volume and responsibility area widening

- As a result of previous three: radio engineers qualification level at operators and regulators

sides is seriously decreased.

NEW QUALITY CONTROL REALITY!


Mobile networks complexity and quality control tasks

New quality control reality needs vendors to follow next ways:

1. Maximum automation implementation in network management processes.

2. Pursuing the highest level of standardization in the delivered service quality

management business processes.

3. Pursuing the maximum simplification in new standards network structures.

4. Simplification of quality objective data collection means and,

consequently implementation of big data, machine learning and artificial

intelligence approaches in analyzing of collected data.


Service quality data acquisition tools modular concept.

The modular concept is used for service

quality data acquisition tools as this

approach is in outline with current trends

in the whole industry.

This approach increases efficiency of

tools utilization as well as saves costs

and decreases TCO for end users.

On the other hand this concept allows to

seriously simplify the data acquisition

processes as well as setup processes.

This fully covers all types of service

quality data acquisitions which are:

- Troubleshooting

- Benchmarking

- Monitoring

The same fleet of testing devices

(probes) is used for all types.

The concept layout


Data analysis conceptFor collected data analysis following tendencies are actual:

1. Avoid the usage of dedicated analyzing tools for each data acquisition setup.

2. Using the database engines as a SW backend to build up data analysis tools.

3. Making the analyzing tools as big as possible to collecting all collected data in the same point.

4. As a consequence of item 3: tight usage of big data engine possibilities.

5. Using machine learning and artificial intelligence achievements in working with big data

arrays.

6. Shifting from desktop user applications to web user applications with user defined

workspaces.


R&S®FR4 Freerider 4

Common platform for R&S walk test products

ROMES SmartBenchmarker NESTOR

Cellular Network

Analysis

BenchmarkingEngineering /

Optimization

Freerider 4


R&S®FR4-CORERuggedized, flexible & future proof

ı Walk & drive test

ı Multi-channel

ı QoS/QoE testing

ı RF trace

ı Customizable GUI

Active ventilation

Multi-band, multi-

technology RF scanner;

MIMO-capable

Tablet controlled,

smartphone based

Hot swappable

batteries for high

autonomy

Custom-made, light-

weight backpack

Future proof for 5G


R&S®FR4-CORE

5G Backpack Setupsı 5GNR sub 6 GHz

Supported by TSME6/TSMA6

Can be enhanced with further TSME6 for

parallel technology scanning / LTE-MIMO

ı 5GNR mm wave with FR4-5G-ANT

Supported by TSME6/TSMA6/TSME30DC

Can be enhanced with further TSME6 for

parallel technology scanning / LTE-MIMO


Smart platform: the perfect intersection

Edge of technology

measurements to provide

trustworthy data from network

to quality of experience

Easy to use, remote

managed and automatic

from data collection to

data validation

Use case driven analytics

Automated insights

Ne

two

rk e

ng

inee

ring

Mo

nito

ring

Optimization

Benchmarking

SmartEfficient and reliable operation

Modularity and reusability

Unique insights


MNT testing in Saint-Petersburg, Russia

MNT testing in Saint-Petersburg, Russia, 20 May 2019SmartAnalytics Live demo


5G testing in Kista, Stockholm


Ooredoo 5G Measuremnets

Same backgroundcolor= same PCI

Bar color: beam index based

First bar: SS-RSRP

Second bar: SS-SINR

Beam @ PCI

SyncSignal measurement values


All 5G Views

MAP

VIEW

CHART

VIEW

HISTORY

VIEW

PERFORMANCE

VIEW

CELL

SPECIFIC

TOP N

VIEW


Top N Views : PCI Color Map


Thank you


Links / materials

ı Rohde & Schwarz

https://en.wikipedia.org/wiki/Rohde_%26_Schwarz

https://www.rohde-schwarz.com/solutions/broadcast-and-media/always-on-blog/posts/01-19-

transformative_250573.html

ı IHS Markit White Paper: The Top Technology Trends of 2019

https://news.ihsmarkit.com/press-release/technology/ihs-markit-white-paper-top-technology-

trends-2019

ı Outer space (regions)

https://en.wikipedia.org/wiki/Outer_space

ı Eahison products

https://www.eahison.com/6g-will-use-millimeter-wave.html

ı "Arctic and Antarctic Research Institute“ materials

http://www.aari.ru/main.php?lg=1&id=62

ı Slide 10: Logic system art by IamUman


https://en.wikipedia.org/wiki/Rohde_%26_Schwarz

https://www.rohde-schwarz.com/solutions/broadcast-and-media/always-on-blog/posts/01-19-transformative_250573.html

https://news.ihsmarkit.com/press-release/technology/ihs-markit-white-paper-top-technology-trends-2019

https://en.wikipedia.org/wiki/Outer_space

https://www.eahison.com/6g-will-use-millimeter-wave.html

http://www.aari.ru/main.php?lg=1&id=62

Links / materials

ı Moscow smart city concept model

https://www.mos.ru/upload/alerts/files/1_Prezentaciya.pdf

https://www.mos.ru/upload/alerts/files/3_Tekststrategii.pdf

ı ISO 37122:2019(en) Sustainable cities and communities — Indicators for smart cities

https://www.iso.org/obp/ui/#iso:std:iso:37122:ed-1:v1:en

ı Global Forest Watch (GFW)

https://www.globalforestwatch.org/

ı Bloomberg

https://www.bloomberg.com/opinion/articles/2019-05-16/huawei-s-woes-change-the-game-for-

cisco-ericsson-nokia

ı ITU-T

https://www.itu.int/en/ITU-T/Workshops-and-Seminars/201905/Pages/programme.aspx

https://www.itu.int/en/ITU-T/Workshops-and-Seminars/20190311/Pages/Programme.aspx


https://www.mos.ru/upload/alerts/files/1_Prezentaciya.pdf

https://www.mos.ru/upload/alerts/files/3_Tekststrategii.pdf

https://www.iso.org/obp/ui/#iso:std:iso:37122:ed-1:v1:en

https://www.globalforestwatch.org/

https://www.bloomberg.com/opinion/articles/2019-05-16/huawei-s-woes-change-the-game-for-cisco-ericsson-nokia

https://www.itu.int/en/ITU-T/Workshops-and-Seminars/201905/Pages/programme.aspx

https://www.itu.int/en/ITU-T/Workshops-and-Seminars/20190311/Pages/Programme.aspx

Documents

Modular conception of software and hardware components for … · 2019. 5. 20. · Check KPIs Root cause analysis with internal tools (OSS, Actix) possible Apply solution Low KPIs