Mobile Network Testing ENSURING QOE IN 5G NETWORKS · INTERACTIVITY SCORE –THE QOE MODEL 27 For emulating ‘real’ applications we will apply a generic QoE model The QoE model

Mobile Network Testing

Arnd Sibila

Technology Marketing MNT

(Dr. Jens Berger)

December 2019

ENSURING QOE IN 5G NETWORKS

EVOLVING APPLICATIONS AND MEASUREMENTS

AGENDA► Mobile Network Testing

► Technical improvements and who uses 5G

► Traditional and new applications and how

they will change

► Interactivity and how to test

► New QoE concepts and QoE models

► Conclusion

Rohde & Schwarz

5G TECHNICAL IMPROVEMENTSWHAT DO PEOPLE EXPECT AND WHAT ARE 5G PROMISES?

► More data-speed! Technically it means more transport capacity

► More interactivity! Technically it means very short transport latency and seamless transport

► More flexibility! Technically addressed by ‘network slices’ serving different needs and QoS requirements

► New applications and use cases! (Finally)

Dec 2019 Ensuring QoE in 5G3

Rohde & Schwarz

5G DEPLOYMENT – FIRST STEP IS DONE IN FIELD

► First 5G real field deployments are done!

► What does it mean in a first step?

5G EN-DC 5G non-standalone mode

Device is connected by LTE (4G), one or more 5G carriers are added for supporting transfer capacity

Requires dual connectivity (4G + 5G)

► What is next?

5G standalone access (no dual connectivity required)

URLLC

(mMTC)

It is not as much benefit for a normal customer…

Download is going a bit faster,

but none of the 5G promises, yet.


eMBB

mMTC URLLC

Rohde & Schwarz

WHO USES 5G AND HOW?

► Todays networks are optimized for humans using smartphones

► In 5G new classes of users will take advantage of the infrastructure

Humans (smartphone use case)

Automotive (connected, autonomous driving, Vehicle-to-X)

Industry 4.0 (Smart Manufacturing, private 5G networks)

IoT, mMTC (Smart City, Connected Energy,...)

Each class causes individual traffic patterns and has individual requirements to the network!

A network optimized for human users may not deliver best performance for cars or industry

Requires

5G standalone


Rohde & Schwarz

WHAT TO EXPECT FROM 5G?

► Human users will be one user class amongst others

► Main popular applications will remain, but evolve

► New application areas and use cases will be

launched and used

► It is not a ‘switch-on’ rather a ‘phase-in’

Applications will use what is made available

There will be a transition phase

6

Source:

Dec 2019 Ensuring QoE in 5G

Rohde & Schwarz

WHAT HAPPENS BEHIND THE SCENES TECHNICALLY? APART FROM 5G

► Simple Video Download is replaced by DASH

► HTTP almost disappeared and is replaced by HTTPs

► FTP almost disappeared

► TCP is more and more replaced by QUIC

► Today’s remaining UDP traffic could be replaced by a QUIC derivate to make it reliable

► New applications will create and use new types of protocols (e.g. AR, VR, XR)

► Far most data traffic is handled by proprietary and encrypted protocols


DASH: Dynamic Adaptive Streaming over HTTP

QUIC: Quick UDP Internet Connections

Rohde & Schwarz

WHAT DOES 3GPP SAY ABOUT QOS?


3GPP TS23.501: System Architecture for the 5G System (5GS); Stage 2 (Rel16, draft 09/19)

5G QoS characteristics (chap 5.7.3.) ► Priority Level

► Packet Delay Budget (including Core Network Packet Delay Budget)

► Packet Error Rate

► Maximum Data Burst Volume (for Delay-critical GBR resource type only)

► Averaging window (for GBR and Delay-critical GBR resource type only)

Non-GBR




they will change



► Conclusion

Rohde & Schwarz

LET’S START SIMPLE: HOW TELEPHONY WILL EVOLVE IN 5G?

► 3GPP Telephony (telephony as primary service)

QoE makes no significant difference to 4G VoLTE

Very short call setup time, EVS 24.4 high quality coding

Accessibility and sustainability will further increase

Video Telephony may increase but strong competition by OTT

► OTT Telephony

Today (4G) OTT telephony applications are still behind VoLTE

Lower accessibility and sustainability (more failed and dropped calls)

Voice quality can be lower due to time-warping and packet loss (unreliable channels), not by coding

Significant improvement in 5G (more reliable channels, short latency)

OTT services will become fully equivalent or even superior to VoLTE (because of wider functionality)


Rohde & Schwarz

DATA APPLICATIONS IN 5GTHE VAST MAJORITY

► Existing applications and use cases

Today’s use cases will remain popular

(HTTP-Browsing, Video streaming, OTT Messaging, posting to social media,…)

► Evolving applications and use cases (based on existing use cases)

4K/8K video, HDR Similar to today, ‘just’ more throughput

360° video, Virtual Reality Similar to today, ‘just’ more throughput

Rendering will move from device to core (in interaction with device)

Live video feed, video upstream Similar to today, more throughput, more reliable

Real time gaming More throughput, shorter latency, more reliable

Rendering will move from device to core (cloud gaming)

► New applications and use cases

Applications: Augmented reality, Virtual Reality vs. Use Cases: AR gaming, remote control, VR retail shopping,…

Highly adaptive to network conditions by Machine Learning and AI techniques


Rohde & Schwarz

WILL WE HAVE AN AR OR VR ‘SERVICE’?

► “Service” is UDP (or QUIC)

► Individual use cases will use

same underlying libraries and techniques

and may lead to similar data patterns

in the network

► Differences will be in the objectives and

the requirements in performance

► Individual QoE models for individual

use cases

(an AR retail shopping has different

objectives than an AR assisted surgery)

Source: Dec 2019 Ensuring QoE in 5G12

Application Use Cases

Rohde & Schwarz

► Let’s go back to the integral QoE concept and its dimensions

HOW TO MEASURE QOE – WHAT WILL HAPPEN?

Connectivity will be

‘always’ given, how to

measure very rare ‘fails’?

Instead chasing individual

‘failed access’,

a new concept could be a

‘network stability score’

(‘probability to fail’)

This is the idea of

‘ultra reliable’ in URLLC


5G

Rohde & Schwarz



Access time will become

very short

Only minor influence on QoE

in the future


5G

Rohde & Schwarz



This is the key and the

challenge at the same time!

How to measure? What are the key parameters?


5G




they will change



► Conclusion

Rohde & Schwarz

TESTING INTERACTIVITY – THE DOOR INTO 5G


Interactivity

► The future will be interactive and real-time

► There will be tons of new interactive applications

► What does it mean: Interactivity?

More than Bitrate

Latency and Continuity are key

Latency stands for transport and also response time

Continuity covers interruptions, undercuts

of required data rate and indirectly packet loss

Rohde & Schwarz

HOW TO TEST INTERACTIVITY?

18

► Interactivity is a perceptual metric, depends on application and use case

► Let’s start with: How to measure

Bitrate

Round-trip latency / response time

Continuity

of a data connection between a device in the field

and a server in the network


Rohde & Schwarz

THE NEW INTERACTIVITY TEST

19

► The device sends a stream of unique packets to an (active) server

► The server sends back packets as responses

Data-rate controlled by frequency and size of packets

Round-trip latency, packet delay variation and packet error rate / corruption ratio measured

► Based on UDP (…is the future anyway)

► Packet rates of 100 to 1500 per second.

This is a continuous packet flow!


Rohde & Schwarz

THE NEW INTERACTIVITY TEST – THE TECHNIQUE BEHIND

20

► The protocol and basic implementation is taken from TWAMP

► TWAMP: Two-Way Active Measurement Protocol – specified by IETF, open source

► Implemented under Android native (R&S MNT) to minimize OS influence

Ready for URLLC

► Server location and implementation:

Industry target: TWAMP responders in firewalls, routers, infrastructure

R&S MNT will deploy own server installations

Server package available by R&S MNT for private installations


Rohde & Schwarz

THE NEW INTERACTIVITY TEST – PARAMETERS & RESULTS

21

► Control parameters (on technical level) are:

Packet rate

Packet size

Delay budget (…if exceeded,

packet is counted

as dropped)


► Results:

Data rate TX and RX

Round-trip latency and

packet delay variation

Packet error rate

low

high

small

large

accepted delay

Rohde & Schwarz

THE NEW INTERACTIVITY TEST – IN PRACTICE

22

► Pre-defined ‘Traffic Pattern’ to emulate a certain application, e.g. e-Gaming

► Constant packet flow (low, medium, high)

► Composed patterns (e.g. emulating a 10s ‘e-Gaming’ phase)

Initial phase

Highly interactive phase

Sustainable phase

Trailing phase

are emulated by different packet flow rates

► Achieved bitrate, round-trip latency and packet

delay variation for each phase and for ‘overall’

duration

► Interactivity score ‘e-Gaming’


Rohde & Schwarz

► e-Gaming emulation in 4G / LTE


23 Dec 2019 Ensuring QoE in 5G

Round-trip latency and

packet delay variationPacket error rate

Interactivity Score Per packet

round-trip latency

Achieved bitrate

Round-trip latency

(Median and 10th percentile)

► Detailed measurement results available (even per packet)

Rohde & Schwarz


24

► e-Gaming emulation in 4G / LTE Bad channels


Delay variation

Packet loss

Rohde & Schwarz

► e-Gaming emulation in 4G / LTE

INTERACTIVITY TEST SMART ANALYTICS

25 Dec 2019

Round-trip latency

Packet delay variation

Packet error rate

Achieved bitrate

Ensuring QoE in 5G

Interactivity Score




they will change



► Conclusion

Rohde & Schwarz

INTERACTIVITY SCORE – THE QOE MODEL

27

► For emulating ‘real’ applications we will apply a generic QoE model

► The QoE model produces a ‘synthetic’ MOS based on QoS and technical KPIs

► Interactivity Score (~ ‘synthetic MOS’)

Interactivity Score = f1 (latency, time) * f2 (delay variation) * f3 (packet loss)


Rohde & Schwarz

INTERACTIVITY SCORE – E-GAMING

28

► Interactivity Score

𝐼𝑛𝑡𝑒𝑟𝑎𝑐𝑡𝑖𝑣𝑖𝑡𝑦 𝑆𝑐𝑜𝑟𝑒 = 𝑆𝑐𝑜𝑟𝑒𝐿𝐴𝑇𝐸𝑁𝐶𝑌 ∗ 𝑆𝑐𝑜𝑟𝑒𝑃𝐷𝑉 ∗ 𝑆𝑐𝑜𝑟𝑒𝑃𝐿 ∗ 100%


Factors scaling linearly

the Interactivity Score

Rohde & Schwarz

► This sort of generic QoE model is fully scalable

► Based on a common set of parameters

INTERACTIVITY TEST CONCEPT – THE QOE MODEL


‘Drone control’

► Different QoE models for different

Interactivity use cases

Rohde & Schwarz

SUMMARY

30

► 5G allows for ‘always available’ services and very short access times

focus more on quality of a running use case

► Interactivity and continuity become crucial for real-time applications under 5G

► Interactivity Test: combines testing round-trip latency, packet delay variation,

packet error rate and proofing bitrate in one single test

► Full set of detailed results (down to per-packet)

► Smart integrative scoring: Interactivity Score for given use cases (e.g. e-Gaming)

► Interactivity is key for future applications!

Interactivity Test released in R&S network test solutions

► We are 5G URLLC ready!


THANK YOU!

…AND KEEP BEING REAL AND INTERACTIVE!

Documents

Mobile Network Testing ENSURING QOE IN 5G NETWORKS · INTERACTIVITY SCORE –THE QOE MODEL 27 For emulating ‘real’ applications we will apply a generic QoE model The QoE model