TEMS™ Monitor Master v10.4 - mantica-qm.ant …mantica-qm.ant-cumulus.co.uk/qmweb/Documentation/testing_volte.pdf · (The solution for the S7 modems has a simpler architecture,

Testing VoLTE

© InfoVista Sweden AB, 2000-2017. All rights reserved. Page 1 of 38 21 July 2017

TEMS™ Monitor Master v10.4.4

Testing VoLTE

Document: MM-0073-Q

Created by: EWJP

Status: Released

Testing VoLTE


Copyright © InfoVista Sweden AB, 2000-2017. All rights reserved. Unauthorized copying, distribution and use of the

document or its content is strictly prohibited. All registered trademarks, quotes, products or service names are

acknowledged.

Note: In this guide the term "Monitor Master" refers to the InfoVista product "TEMS Monitor Master".

InfoVista Sweden AB

Laboratorgränd 3, 931 62

Skelleftea, Sweden

TEMS Monitor Master is part of the TEMS Portfolio, from InfoVista.

www.infovista.com

Version Overview

Version Date Author Modification

1.0 21/07/2017 EWJP Initial release

Information in this document is subject to change without notice. For the latest version, please visit

https://customerportal.tems.infovista.com

Document Management

File name: testing_volte.pdf

Pages: 38

Released on: 21/07/2017

Approved by: GJ

http://www.infovista.com/

https://customerportal.tems.infovista.com/

Testing VoLTE


Table of Contents CHAPTER 1 - INTRODUCTION .................................................................................................................................... 4

VOICE OVER LTE (VOLTE) .............................................................................................................................................. 4

USING THIS GUIDE ............................................................................................................................................................. 6

OTHER GUIDES .................................................................................................................................................................. 6

CHAPTER 2 - INSTALLING VOLTE TESTING ......................................................................................................... 7

INSTALLING VOLTE HARDWARE ...................................................................................................................................... 7

INSTALLING VOLTE SOFTWARE ....................................................................................................................................... 8

CHECKING OPERATION OF THE VOLTE SOLUTION ......................................................................................................... 10

CHAPTER 3 - VOLTE TESTING WITH THE GALAXY S5 .................................................................................... 11

USING THE VOLTE SCRIPT FOR THE GALAXY S5 ........................................................................................................... 11

THE VOLTE SCRIPT FOR THE GALAXY S5 ...................................................................................................................... 16

CHAPTER 4 - VOLTE TESTING WITH THE GALAXY S7 .................................................................................... 19

USING THE VOLTE SCRIPT FOR THE GALAXY S7 ........................................................................................................... 19

THE VOLTE SCRIPT FOR THE GALAXY S7 ...................................................................................................................... 24

CHAPTER 5 - VOLTE TESTING RESULTS .............................................................................................................. 25

USING BOX OFFICE REPORTS .......................................................................................................................................... 25

USING TRACE FILES ......................................................................................................................................................... 25

METRICS AND MESSAGES SUMMARY .............................................................................................................................. 26

VOLTE METRICS DETAILS ............................................................................................................................................. 28

INDEX ............................................................................................................................................................................... 37


Chapter 1 - Introduction

WARNING: This guide provides advance notice of new features that are still in development at the time of

writing. This information is subject to change without notice, do not rely on it. It will be replaced by an official

version as soon as possible.

Monitor Master is a suite of software products for measuring the quality and user-experience of wireless and wired

voice and data services.

This guide describes how to set up and use Monitor Master to test and monitor Voice over LTE (VoLTE), based on

the following major components:

VoLTE Modems:

the Samsung Galaxy S5 SM-G900A EMU

the Samsung Galaxy S5 SM-G900V EMU

the Samsung Galaxy Note 4 SM-N910G EMU

the Samsung Galaxy S7 SM-G930A EMU

the Samsung Galaxy S7 SM-G930R4 EMU

the appropriate VoLTE Script

Radio Trace (TRP) - to capture low-level radio trace information from the session

IP Trace (PCAP) - to capture low-level IP trace information from the session

for the S5 and Note 4: the VoLTE Post-processing Package - to analyze the TRP Trace file to produce

detailed VoLTE metrics

This guide is for:

System installers (Chapters 1 and 2)

Personnel who test or monitor the VoLTE service (Chapters 1, 3, 4)

Personnel who compile and interpret results from testing and monitoring (Chapters 1 and 5)

Voice over LTE (VoLTE) This section provides some background information on Voice over LTE, including key terms and concepts you will

need to understand to test this functionality with Monitor Master.

LTE is a data-only packet switched system, with no direct support for the other legacy circuit switched services

that customers require, such as Voice and SMS. In 2010, the GSMA agreed that while systems such as CSFB and

VoLGA might suffice for transitional stages, the final goal was Voice over LTE (or VoLTE) using the IP Multimedia

Systems, or IMS.

Monitor Master v10.1.0 introduced support for VoLTE testing and monitoring on specific networks in 2013. The

solution was redesigned (to remove the requirement for post-processing) and extended to support additional

modems for Monitor Master 10.4.2.

Key components

Support for VoLTE involves elements of the user's mobile device (cellphone), the operator's Radio Access Network

and IMS Servers, as shown by following diagram, adapted from the GSMA document:

Testing VoLTE


The VoLTE Solution (S5 modems)

The Monitor Master VoLTE Testing Solution for the S5 modems uses hardware and software organized like this:

(The solution for the S7 modems has a simpler architecture, with no need for post-processing by TEMS Discovery.)

Notice that the metrics in the Projector database come through two different routes:

From the Monitor Master Agent running the Monitor Master script - Standard Metrics

From the Projector running the VoLTE Post-processing Package - VoLTE Metrics

Chapter 1 - Introduction


Further information

For more information see the GSMA document IMS Profile for Voice and SMS, Version 7.0, 03 March 2013,

available from http://www.gsma.com/newsroom/wp-content/uploads/2013/04/IR.92-v7.0.pdf.

Using this guide This guide includes the following chapters:

1. Introduction

2. Installing VoLTE Testing

3. VoLTE Testing with the Galaxy S5

4. VoLTE Testing with the Galaxy S7

5. VoLTE Testing Results

Other guides You may also wish to consult the following:

The Installing & Managing Monitor Master guide - for information on setting up, configuring and managing

other parts of your Monitor Master system

The Samsung Galaxy S5 SM-G900A EMU SPD - for technical information on this Samsung S5 modem

The Samsung Galaxy S5 SM-G900V EMU SPD - for technical information on this Samsung S5 modem

The Samsung Galaxy S7 SM-G930A EMU SPD - for technical information on this Samsung S7 modem

The Samsung Galaxy S7 SM-G930R4 EMU SPD - for technical information on this Samsung S7 modem

The Samsung Galaxy Note 4 SM-N910G EMU SPD - for technical information on this Samsung Note 4 modem

The Installing & Managing an EMU guide - for information on setting up, configuring and managing EMUs

The QM User Guide - for information on scheduling tests and running reports

The GSMA document IMS Profile for Voice and SMS, Version 7.0, 03 March 2013, available from

http://www.gsma.com/newsroom/wp-content/uploads/2013/04/IR.92-v7.0.pdf.

http://www.gsma.com/newsroom/wp-content/uploads/2013/04/IR.92-v7.0.pdf

http://www.gsma.com/newsroom/wp-content/uploads/2013/04/IR.92-v7.0.pdf

Testing VoLTE


Chapter 2 - Installing VoLTE Testing

This Chapter describes how to add VoLTE Testing to your Monitor Master system, in three main stages:

1. Install the hardware

2. Install the software

3. Check operation

These steps will usually be carried out by your System Administrator or TEMS Field Services personnel. They

should only need to be done once.

Installing VoLTE Hardware Hint: Monitor Master hardware is usually installed and configured by your System Administrator's staff or TEMS

Field Services.

Modems

The VoLTE Testing Solution requires two suitable modems for the scripts to use. You can use:

two Samsung Galaxy S5 SM-G900A EMUs

two Samsung Galaxy S5 SM-G900V EMUs

two Samsung Galaxy S7 SM-G930A EMUs

two Samsung Galaxy S7 SM-G930R4 EMUs

two Samsung Galaxy Note 4 SM-N910G EMUs

These must be used in combination, you can only use two modems of the same type.

The EMUs may be housed within a Smart1 or Smart2 enclosure if required.

Each EMU must be installed by connecting it to a suitable probe, as described in the Installing & Managing an

EMU guide. They can be installed on separate probes (to allow testing across the network) or on the same probe

(for testing within a single cell of the network).

Once they've been installed on a probe, add the EMUs to the QM database so tests can use them - see Adding a

new Terminal in the QM Admin Guide or the Help.

SIMs

For VoLTE testing, each EMU requires a SIM that supports the host LTE network. While SIM Server SIMs can be

used for VoLTE testing, we recommend that a Local SIM is also installed in each EMU, for flexibility. (See Installing

VoLTE Software for SIM Server setup.)

Install the Local SIM in the SIM Slot on the EMU's rear panel. See Installing a local SIM in the Installing

& Managing an EMU Guide or the Help.

Once they've been installed in an EMU, add the Local SIMs to the QM database so tests can use them - see Adding

a new SIM in the QM Admin Guide or the Help.



Probes

The EMUs can be installed on the following probes:

Agent RTU-5 with the software as described in Installing VoLTE Software

The RTU-5 must have a free USB port to connect to, and may be dedicated to VoLTE testing or may run other tests

in parallel on its other modems. It may be housed within a Smart1 enclosure if required.

QM System

The VoLTE Solution requires a fully operational Monitor Master QM system.

Installing VoLTE Software Hint: Monitor Master software is usually installed and configured by your System Administrator or their deputies.

VoLTE Testing requires a fully operational Monitor Master QM system. It has the following specific software

requirements that you may need to install:

Monitor Master 10.4.1 SP1 or later

Drivers for the EMUs on the probes

SIM Server support (optional)

Proprietary TEMS software installed on the VoLTE handsets

The appropriate version of the VoLTE Script on QM Server

The VoLTE Post-processing Package on Projector (S5 modems only)

System FTP Server on the Projector (to upload audio and trace files)

Radio Trace for Type 2 Modems (optional for the S7 modems)

Optional: IP Trace for Type 2 Modems (license) on the probes

Optional: The latest version of the BIOS on the probes (recommended)

Each of these requirements will be described briefly in turn.

Monitor Master version

The VoLTE solution requires v10.4.1 SP1 (or later) of the TCC software on the probes used to control the VoLTE

Modems.

This may be installed already if you have purchased new probes or have upgraded your system to Monitor Master

10.4.1 SP1 or later. If not, upgrade the probes that will use the VoLTE modems - see Upgrading an Agent

appliance in the Installing & Managing Monitor Master guide, or in the Help.

Note that you do not need to upgrade other components of your system (such as Projector and QM Server).

Drivers

If you are connecting EMUs to a probe for the first time and the probe has a pre-9.2.0 image on it, you will need to

install the EMU drivers on it. (Later probes have the EMU drivers built in.)

If you are connecting Samsung EMUs to a probe that has never used a Samsung EMU, you will need to install the

Samsung handset drivers on the probe. (All current Samsung EMUs use the same handset drivers.)

For details, see Installing EMU drivers and Installing handset drivers in the Installing & Managing an EMU guide,

or in the Help.

Testing VoLTE


SIM Server Support

To use SIM Server SIMs with the EMUs:

Add the SIMs in QM - see Adding a SIM Server SIM in the QM Admin Guide, or in the Help

Set the terminal (modem) access type to Server in QM - see Adding a new Terminal in the QM Admin

Guide, or in the Help

Edit the modem powerup-report-delay on the probe - see the Using the xxx EMU section for this modem in

Chapter 7 of the Story Boarder User Guide, or in the Help

Handset Software

The VoLTE solution uses TEMS software that has been installed on the handsets during production of the

Extended Measurement Units, to provide the low level access required. This software should not require any

maintenance - simply make sure that you do not delete it.

Radio Trace for Type 2 Modems

For the S5 modems, VoLTE testing requires a Radio Trace for Type 2 Modems license. Trace can be licensed per

probe or per modem. For details see the Monitor Master Radio Trace for Type 2 Modems SPD. Note that the Radio

Trace license is required even if you do not want to collect Radio Trace files, by using the trace option.

For the S7 modems, a Radio Trace license is optional. The VoLTE Script includes an option to capture Radio Trace

(TRP). If you want to use this option, install a Radio Trace for Type 2 Modems license on the probe. Radio Trace

can be licensed per probe or per modem. For details see the Monitor Master Radio Trace for Type 2 Modems SPD.

The VoLTE Scripts

The VoLTE Scripts are included with the Monitor Master example scripts, for download. There is a separate

version for each type of VoLTE Modem. This should be uploaded to QM Server as follows:

1. Download the appropriate VoLTE script to your PC

For the Samsung S5, download QM VoLTE Availability.mmts

For the Samsung S7, download QM VoLTE Voice Quality.mmts

2. Login to QM as Administrator

3. Upload the script so users can see and run it from the QM Scheduler:

a. Select Manage Test Files from the Tests menu

b. Click the (New) button on the Manage Test Files screen and select the Test Script from File option

from the dropdown menu

c. Browse to the script file, select it and click the Upload button

d. This will add the VoLTE Script to the FileUpload folder on QM Server - you will probably want to

move it to a more suitable folder (e.g. by loading it into Story Boarder and saving it to the preferred

location)

4. At this stage all users will be able to run, copy, delete or edit the VoLTE Script. We recommend you control

access by using QM's File Permissions:

a. Select Manage Test Files from the Tests menu

b. Select the VoLTE Script

c. Click the Permissions button to display the user permissions for this file



d. Select the Everyone user group and click the Change button

e. Untick the Change resource and Change permissions checkboxes to stop users editing these scripts

and changing their Permissions

f. Click OK to save the changes

g. If you have set up other User Groups, you can use them to control which users can run the scripts.

For details, see Adding Permissions to a Test File in the QM Admin guide or in the Help

System FTP Server

To capture and view Radio Trace, IP Trace and audio files, a System FTP Server must be set up on Projector, to

receive the files from the probes. See Installing a System FTP Server in the Installing & Managing Monitor Master

guide or the Help.

VoLTE Post-processing Package

For the S5 modems, VoLTE testing relies on a custom package of TEMS components, which will be installed for

you on your Projector by TEMS Field Services. This package analyzes the Radio Trace files it receives (produced by

the VoLTE Script) to produce additional detailed VoLTE Metrics. These metrics are merged with the testrun

metrics in the Projector results database and can be used just like them, once the post-processing and merge have

completed.

IP Trace for Type 2 Modems (optional)

The VoLTE Script includes an option to collect IP Trace (PCAP). If you want to use this option, install an IP Trace

for Type 2 Modems license on the probe. IP Trace can be licensed per probe or per modem. For details see the

Monitor Master IP Trace for Type 2 Modems SPD.

BIOS Upgrade (optional)

If you are using an Agent RTU-5 appliance purchased before June 2013 for your VoLTE testing, we recommend

that you upgrade it to the latest BIOS (D119 or later) as this should deliver improved stability. Please contact our

support department for details.

Checking Operation of the VoLTE Solution Once you have installed the hardware and software elements of the solution, you should check operation before

allowing use in production.

Login to QM as an ordinary user (not an Administrator)

Launch the QM Scheduler and check that you can see the appropriate VoLTE Script, VoLTE Modems and

SIMs

Schedule a VoLTE test using each VoLTE modem at least once, with the upload and trace options enabled

Launch QM Task Control to monitor the tests and check that the scripts ran OK: without system errors and

with service availability

Launch Projector reports and check that the full set of VoLTE Metrics is being reported (as described in

Chapter 6)

If you've restricted access to specific users (by using Test File Permissions), check that the right users have

access - and others do not.

Once you're happy with operation, inform the end users that the VoLTE Solution is ready for use.

Testing VoLTE


Chapter 3 - VoLTE Testing with the Galaxy S5

Hint: For background information see Voice over LTE in Chapter 1.

This chapter describes VoLTE testing with the Galaxy S5 SM-G900A, Galaxy S5 SM-G900V and Galaxy Note 4 SM-

N910G modems (S5 modems). Make sure you use the version of the VoLTE Script for these modems: QM VoLTE

Availability.mmts.

This Chapter describe how to perform VoLTE testing with these modems and the VoLTE Script under these

headings:

Using the VoLTE Script for the Galaxy S5 - an introduction and overview of how to use it including runtime

options, for users unfamiliar with Monitor Master testing

The VoLTE Script for the Galaxy S5 - a detailed description for background information

Using the VoLTE Script for the Galaxy S5 The Monitor Master VoLTE Solution is based on a range of VoLTE Modems with matching VoLTE Scripts. This

section describes how to use the VoLTE Script for:

the Samsung Galaxy S5 SM-G900A (for Verizon networks)

the Samsung Galaxy S5 SM-G900V (for Verizon networks)

the Samsung Galaxy Note 4 SM-N910G (for Asia and South America)

This script is described in detail in the next section, The VoLTE Script for the Galaxy S5.

WARNING: Make sure you use the correct version of this script!

This section describes how to use this VoLTE Script, including:

General principles

Requirements

Scheduling a test

Runtime options

Monitoring execution

Viewing results

Trace files

Audio files

Temporary files

Modem power and error recovery

General principles

The VoLTE Script is designed to be run from QM as supplied, adjusting operation to their specific needs by setting

runtime options. Customers may adapt the script to their specific needs, but this is a skilled activity and may be

better handled by trained TEMS personnel.

It uses two Galaxy S5 VoLTE modems, one to make the call and one to receive it. These modems can be controlled

the same Agent or by different Agents, as required.



This script uses standard Scenes and Resources, that are fully described in the Story Boarder Reference and a

number of Advanced Script scenes.

Requirements

To test VoLTE you will need:

VoLTE hardware and software installed on your Monitor Master system as described in Chapter 2

A QM user id with permissions to use the VoLTE Script, the VoLTE modems and their Local SIMs

Each test will take a few minutes to set up and a few minutes to run, and will require access to two Galaxy S5

VoLTE Modems.

Scheduling a test

The VoLTE Script must be launched from QM (Central Mode), usually from the QM Scheduler. You can schedule

an immediate test, a single test for execution later, or multiple runs of the same test at intervals. (You can also

include the VoLTE Script in your own Sequences.)

The QM Scheduler is fully described in Chapter 4 of the QM User Guide (or search the Help for "the QM schedule

tests option" including the quotes).

In brief, here's how to schedule an immediate testrun:

1. Login to QM

2. Select the Schedule Tests option from the Tests menu

3. Open the VoLTE folder to list the scripts

4. Select the test script (or workflow) to run from the list and click Load

5. Assign a Galaxy S5 VoLTE modem to each party (Originator and Recipient)

6. Assign a suitable Subscriber (SIM) to each modem

7. Select the Parameters tab to list the runtime parameters

8. Type an appropriate value for each runtime parameter. See Runtime options, below.

Hint: To save you time in future, once you've set the parameters, store your choices as a Workflow: click the

Save Workflow button, select a folder and type a name for the workflow. From now on you can run the

same test again (with the same resources and parameters) by loading this Workflow in step 4 - and skip

steps 5-8.

9. Click the Run Once button to run the test

Runtime options

The VoLTE Script supports the following runtime options (parameters):

OriginatorOptions - options for this party, separated by commas. Default: pcap, trace, upload. The

supported options are:

pcap : to enable IP Trace

trace : to enable Radio Trace

upload to upload the trace and audio files at the end of the test

disableanalysis : to skip the Audio Analysis scene. For example: this should be used if the Rscript

application is not installed (as D:\APS\tools\audio\R-Software\bin\Rscript.exe) on the probe

Testing VoLTE


standardupload to use the standard Upload file to Projector scene to upload the files using FTP. By

default, files are saved locally by copying them to the D:\FTPfiles folder on the probe and then

uploaded by running the FTPTestFiles.mmts script.

OriginatorUploadTraceSuffix - suffix to add to the TRP trace file name, to distinguish it from non-VoLTE

trace files. This will be inserted before the .trp extension. Default: blank

PlaybackVolume - the volume level to use to play the audio (on the originator handset): a number between

0 (low) and 100 or blank for the default setting. Default: blank

RecipientOptions - options for this party, separated by commas. The supported options are: pcap (enable

IP Trace): trace (enable Radio Trace) and upload (upload the trace and audio files at the end of the test).

Default: blank.

RecipientUploadTraceSuffix - suffix to add to the TRP trace file name, to distinguish it from non-VoLTE


RecordVolume - the volume level to use to record the audio (on the recipient handset): 0-100 or blank for

the default setting. Default: blank

AudioFileToPlay - the pathname of the wav file to play during the call. Default:

D:\APS\tools\audio\1004hz_120s.8a.wav

TypeRanPrefix - optional prefix to add to the ServiceTypeRan message to help categorize the results when

reporting. Default: blank

Recommendations:

For a simple test of VoLTE function and availability - no trace files, audio files, or VoLTE Metrics - set

OriginatorOptions and RecipientOptions to blank.

For a simple test of VoLTE function and availability plus the degraded audio file - but no trace files or

VoLTE Metrics - set OriginatorOptions to blank and set RecipientOptions to upload

For VoLTE Metrics and access to the diagnostic Radio Trace files, set both OriginatorOptions and

RecipientOptions to trace,upload and set OriginatorUploadTraceSuffix and OriginatorUploadTraceSuffix

to ".volte".

For access to the diagnostic IP Trace file from either party, add pcap to its list of options

Note: If you run the script without the trace and upload options, the only VoLTE metric you will get is service

availability.

Script properties

The script is optimized for use with a SIM Server with 200 ms latency. If necessary, the following script properties

can be adjusted to suit other scenarios:

Originator::IMSRegistrationMaxWait: The maximum time the originator party waits for a successful IMS

registration

Recipient::IMSRegistrationMaxWait: The maximum time the recipient party waits for a successful IMS

registration

Originator::VAPIRunningMaxWait: The maximum time the originator party waits for the OnDevice VApi

service to be running

Recipient::VAPIRunningMaxWait: The maximum time the recipient party waits for the OnDevice VApi





To watch your tests run, use QM Task Control. This is fully described in the QM User Guide - look for the Task

Control option in Chapter 4 or in the Help.

In brief here's how to check your current and recent tests:

1. Login to QM

2. Select the Task Control option from the Tests menu

3. This displays the Running Tasks tab, which lists all tests that are running on your system now, including

their user, start time and current status. To display just your own tests, type your username in the filter box.

4. To see tests that have finished, select the Completed Tasks tab. This lists all tests that have finished recently,

including their user, start and end time, and result. To see just your own tests, type your username in the

filter box.

5. The result for each test should be green. If a result is orange, the test probably reported a service error. If a

result is red, the test probably reported a system error.

6. To check a test's results, select it and click the Testrun Report button. For more information on this report,

see Testrun Summary Dashboard in the Using Box Office Guide or in the Help.

Viewing results

The results (service availability, metrics, timings, messages, etc) from a single testrun can be examined later by

using Box Office reports, or results from multiple tests can be combined to create graphs, maps or tables to help

summarize and analyze them - reveal the bigger picture, trends and hotspots. See Chapter 4, Results, for more

information on the reporting options.

Trace files

You can choose to create detailed trace files for analysis after the testrun as follows:

To produce a Radio Trace file, use the trace runtime option

To produce an IP Trace file (for the data session in an Active test), use the pcap runtime option

And to save the trace (and audio) files, use the upload runtime option

Creating and uploading trace files imposes a considerable extra load on the system, so these options should only be

used when you need this additional low-level information.

Notes:

If you use the trace or pcap option, remember to use the upload option too, to copy the trace files to your

system's FTP server (on Projector); the files are deleted automatically from the probe when the script ends,

to stop its disk filling up. The testrun results include a link to the files on Projector so you can find them.

Audio files

The Originator party plays the following 44kHz standard audio file over the VoLTE connection:

%TCC_HOME%\AudioReferences\pesq3_dtx_44k.wav (stored on the probe)

This is the Reference Audio File. The Recipient party records the call as a wav file and stores it on the probe; this is

the Degraded Audio File. The degraded audio file is deleted automatically before the script ends - if you wish to

keep it, use the upload option to store it on the System FTP Server.

Note: The degraded audio file is not suitable for use in call quality (e.g. PESQ) analysis.

Testing VoLTE


Temporary files

These scripts can create large temporary files on the probe's disk:

A Radio Trace file - if requested by the trace option

An IP Trace file - if requested by the pcap option

Setup and Call audio files

These files are deleted automatically when the script ends, to stop the probe's disk filling up. (If you wish to access

them, use the upload option to copy them to your Projector.)


The VoLTE Script uses a Power Down Modem scene and Check outcome scene near the end of the Originator and

Recipient parties, like this:

These are used for modem error recovery, as follows.

When a modem uses a Local SIM (not a SIM Server SIM), Monitor Master turns it on the first time it is used, and

then leaves it turned on. This runs scripts more efficiently, as they don't have to wait for the modem to power up at

the start of each test. (When a modem uses a SIM Server SIM, Monitor Master turns it on at the start of the test and

off at the end.)

If a script executes the Power Down Modem scene, it turns the modem off - and the modem will turn on again the

next time a script uses it. This "power cycle" resets the modem, a good way to restore normal behavior for the next

test run after many of the transient hardware problems that affect modems. In the VoLTE Script, the Can disable

trace scene's error routes (orange arrows) are connected directly to the Power Down Modem, to recover the

modem if it encounters a problem. If other scenes encounter a problem that warrants a modem reset, they record



the issue by setting the My Sync Outcome prop to a value such as "Initialization failure". The Check outcome

scene (a Match Values scene) checks this prop and executes the Power Down Modem scene if its value indicates

that a problem has been encountered.

The VoLTE Script for the Galaxy S5 Note: This topic is for background information only.

This section briefly describes the VoLTE Script and how it works. If you want to know how to use it, see the

previous topic, Using the VoLTE Script for the Galaxy S5.

This script (qm_volte_availability.mmts) is a three-party Story Boarder script designed to test VoLTE availability,

capture the received audio, and produce detailed Radio Trace data to analyze.

It script must be installed on QM Server as described in Chapter 2 before you can use it. (If it is installed but you

can't find it, you may not have the right permissions - ask your System Administrator.)

It can be used as supplied, without any changes, as customers can adjust behavior at runtime, by configuring the

modems, SIMs, etc and the runtime options, described above. It may also be adapted by suitably qualified

personnel (such as TEMS Field Services) to suit specific customer requirements.

Party 1: The Originator

This party runs in parallel with the Recipient party. It places the VoLTE call to the Recipient party. In detail it:

checks the environment and gets the runtime parameters

starts IP Trace (PCAP) if requested

starts Radio Trace if requested

checks the handset is IMS registered

sets the record and playback volume

synchronizes with the other party (initialization complete)

starts recording call setup

places a VoLTE call to the other party

synchronizes with the other party (call answered)

stops recording call setup

Testing VoLTE


plays outgoing audio

synchronizes with the other party (play finished)

starts recording the incoming audio

synchronizes with the other party (play finished (1))

stops recording the incoming audio

drops the call

synchronizes with the other party (call dropped)

performs audio analysis (if requested)

stops IP Trace (if it was requested)

gets SIP Messages, SEER analysis, RTP Addresses and IP Addresses

stops Radio Trace (if it was requested)

uploads the Setup Audio, Audio, IP and Radio Trace files to Projector (if requested)

powers down the modem if required

deletes local audio files

Party 2: The Recipient

This party runs in parallel with the Originator party. It receives the VoLTE call from the Originator party:

checks the environment and gets the runtime parameters

starts IP Trace (PCAP) if requested

starts Radio Trace if requested

checks the handset is IMS registered

sets the record and playback volume level

synchronizes with the other party (initialization complete)

answers the VoLTE call

synchronizes with the other party (call answered)

starts recording the incoming audio

synchronizes with the other party (play finished)

stops recording the incoming audio

synchronizes with the other party (ready to play)

plays the outgoing audio

synchronizes with the other party (play finished (1))

performs audio analysis (if requested)

stops IP Trace (if it was requested)

gets SIP Messages, SEER analysis, RTP addresses and IP addresses

stops Radio Trace (if it was requested)



uploads the Audio, IP and Radio Trace files to Projector (if requested)

powers down the modem if required

deletes the local audio files

(Ignoring error handling for simplicity.)

Party 3: The Collector

This party runs after the Originator and Recipient parties have finished. It performs some simple post-processing

of the results from the other parties to produce combined results:

records the originator MSISDN

records the recipient MSISDN

records key originator messages

records key recipient messages

records the combined, receive and send availabilities

(Ignoring error handling for simplicity.)

Debugging - DPL Dropped Segments

The Galaxy S5 VoLTE script relies on DPL Messages from the handset for call control and KPI calculations.

Messaging is not 100% reliable and a missing DPL segment in the IP capture may cause a testrun to fail, or produce

incorrect KPIs.

To help you detect when this happens, if this script detects a missing DPL segment, it reports an extra metric,

DPLDroppedSegment with its value set to 1. The metric's timestamp is set to the time the missing DPL segment

was detected.

Note that dropped segments don't always mean that the testrun or metrics are invalid - they may be dropped from

non-critical parts of the messaging.

Testing VoLTE



WARNING: This script is still in development. All information about it is still subject to change without notice.

Hint: For background information see Voice over LTE in Chapter 1.

This chapter describes VoLTE testing with the Galaxy S7 SM-G930A and Galaxy S7 SM-G930R4 modems (S7

modems). Make sure you use the version of the VoLTE Script for these modems: QM VoLTE Voice Quality.mmts.

This Chapter describe how to perform VoLTE testing with these modems under these headings:

Using the VoLTE Script for the Galaxy S7 - an introduction and overview of how to use it including runtime

options, for users unfamiliar with Monitor Master testing

The VoLTE Script for the Galaxy S7 - a detailed description of the script, for background information

Using the VoLTE Script for the Galaxy S7 WARNING: This script is still in development. All information about it is still subject to change without notice.

The Monitor Master VoLTE Solution is based on a range of VoLTE Modems with matching VoLTE Scripts. This

section describes how to use the VoLTE Script for:

the Samsung Galaxy S7 SM-G930A (for AT&T networks)

the Samsung Galaxy S7 SM-G930R4 (for US Cellular networks)

This script is described in detail in the next section, The VoLTE Script for the Galaxy S7.

WARNING: Make sure you use the correct version of this script.

This section describes how to use this VoLTE Script, including:

General principles

Requirements

Scheduling a test

Runtime options


Viewing results

Trace files

Audio files

Temporary files


General principles

The VoLTE Script is designed to be run from QM as supplied, adjusting operation to their specific needs by setting

runtime options. Customers may adapt the script to their specific needs, but this is a skilled activity and may be

better handled by trained TEMS personnel.

It uses two Galaxy S5 VoLTE modems, one to make the call and one to receive it. These modems can be controlled

the same Agent or by different Agents, as required.



This script uses standard Scenes and Resources, that are fully described in the Story Boarder Reference and a

number of Advanced Script scenes.

Requirements

To test VoLTE you will need:

VoLTE hardware and software installed on your Monitor Master system as described in Chapter 2

A QM user id with permissions to use the VoLTE Script, the VoLTE modems and their Local SIMs

Each test will take a few minutes to set up and a few minutes to run, and will require access to two Galaxy S5

VoLTE Modems.

Scheduling a test

The VoLTE Script must be launched from QM (Central Mode), usually from the QM Scheduler. You can schedule

an immediate test, a single test for execution later, or multiple runs of the same test at intervals. (You can also

include the VoLTE Script in your own Sequences.)

The QM Scheduler is fully described in Chapter 4 of the QM User Guide (or search the Help for "the QM schedule

tests option" including the quotes).

In brief, here's how to schedule an immediate testrun:

1. Login to QM

2. Select the Schedule Tests option from the Tests menu

3. Open the VoLTE folder to list the scripts

4. Select the test script (or workflow) to run from the list and click Load

5. Assign a Galaxy S5 VoLTE modem to each party (Originator and Recipient)

6. Assign a suitable Subscriber (SIM) to each modem

7. Select the Parameters tab to list the runtime parameters

8. Type an appropriate value for each runtime parameter. See Runtime options, below.

Hint: To save you time in future, once you've set the parameters, store your choices as a Workflow: click the

Save Workflow button, select a folder and type a name for the workflow. From now on you can run the

same test again (with the same resources and parameters) by loading this Workflow in step 4 - and skip

steps 5-8.

9. Click the Run Once button to run the test

Runtime options

The VoLTE Script supports the following runtime options (parameters):

OriginatorOptions - options for this party, separated by commas. Default: pcap, trace, upload. The

supported options are:

pcap : to enable IP Trace

trace : to enable Radio Trace

upload to upload the trace and audio files at the end of the test

disableanalysis : to skip the Audio Analysis scene. For example: this should be used if the Rscript

application is not installed (as D:\APS\tools\audio\R-Software\bin\Rscript.exe) on the probe

Testing VoLTE


standardupload to use the standard Upload file to Projector scene to upload the files using FTP. By

default, files are saved locally by copying them to the D:\FTPfiles folder on the probe and then

uploaded by running the FTPTestFiles.mmts script.

OriginatorUploadTraceSuffix - suffix to add to the TRP trace file name, to distinguish it from non-VoLTE


PlaybackVolume - the volume level to use to play the audio (on the originator handset): a number between

0 (low) and 100 or blank for the default setting. Default: blank

RecipientOptions - options for this party, separated by commas. The supported options are: pcap (enable

IP Trace): trace (enable Radio Trace) and upload (upload the trace and audio files at the end of the test).

Default: blank.

RecipientUploadTraceSuffix - suffix to add to the TRP trace file name, to distinguish it from non-VoLTE


RecordVolume - the volume level to use to record the audio (on the recipient handset): 0-100 or blank for

the default setting. Default: blank

AudioFileToPlay - the pathname of the wav file to play during the call. Default:

D:\APS\tools\audio\1004hz_120s.8a.wav

TypeRanPrefix - optional prefix to add to the ServiceTypeRan message to help categorize the results when

reporting. Default: blank

Recommendations:

For a simple test of VoLTE function and availability - no trace files, audio files, or VoLTE Metrics - set

OriginatorOptions and RecipientOptions to blank.

For a simple test of VoLTE function and availability plus the degraded audio file - but no trace files or

VoLTE Metrics - set OriginatorOptions to blank and set RecipientOptions to upload

For VoLTE Metrics and access to the diagnostic Radio Trace files, set both OriginatorOptions and

RecipientOptions to trace,upload and set OriginatorUploadTraceSuffix and OriginatorUploadTraceSuffix

to ".volte".

For access to the diagnostic IP Trace file from either party, add pcap to its list of options

Note: If you run the script without the trace and upload options, the only VoLTE metric you will get is service

availability.

Script properties

The script is optimized for use with a SIM Server with 200 ms latency. If necessary, the following script properties

can be adjusted to suit other scenarios:

Originator::IMSRegistrationMaxWait: The maximum time the originator party waits for a successful IMS

registration

Recipient::IMSRegistrationMaxWait: The maximum time the recipient party waits for a successful IMS

registration

Originator::VAPIRunningMaxWait: The maximum time the originator party waits for the OnDevice VApi


Recipient::VAPIRunningMaxWait: The maximum time the recipient party waits for the OnDevice VApi





To watch your tests run, use QM Task Control. This is fully described in the QM User Guide - look for the Task

Control option in Chapter 4 or in the Help.

In brief here's how to check your current and recent tests:

1. Login to QM

2. Select the Task Control option from the Tests menu

3. This displays the Running Tasks tab, which lists all tests that are running on your system now, including

their user, start time and current status. To display just your own tests, type your username in the filter box.

4. To see tests that have finished, select the Completed Tasks tab. This lists all tests that have finished recently,

including their user, start and end time, and result. To see just your own tests, type your username in the

filter box.

5. The result for each test should be green. If a result is orange, the test probably reported a service error. If a

result is red, the test probably reported a system error.

6. To check a test's results, select it and click the Testrun Report button. For more information on this report,

see Testrun Summary Dashboard in the Using Box Office Guide or in the Help.

Viewing results

The results (service availability, metrics, timings, messages, etc) from a single testrun can be examined later by

using Box Office reports, or results from multiple tests can be combined to create graphs, maps or tables to help

summarize and analyze them - reveal the bigger picture, trends and hotspots. See Chapter 4, Results, for more

information on the reporting options.

Trace files

You can choose to create detailed trace files for analysis after the testrun as follows:

To produce a Radio Trace file, use the trace runtime option

To produce an IP Trace file (for the data session in an Active test), use the pcap runtime option

And to save the trace (and audio) files, use the upload runtime option

Creating and uploading trace files imposes a considerable extra load on the system, so these options should only be

used when you need this additional low-level information.

Notes:

If you use the trace or pcap option, remember to use the upload option too, to copy the trace files to your

system's FTP server (on Projector); the files are deleted automatically from the probe when the script ends,

to stop its disk filling up. The testrun results include a link to the files on Projector so you can find them.

Audio files

The Originator party plays the following 44kHz standard audio file over the VoLTE connection:

%TCC_HOME%\AudioReferences\pesq3_dtx_44k.wav (stored on the probe)

This is the Reference Audio File. The Recipient party records the call as a wav file and stores it on the probe; this is

the Degraded Audio File. The degraded audio file is deleted automatically before the script ends - if you wish to

keep it, use the upload option to store it on the System FTP Server.

Note: The degraded audio file is not suitable for use in call quality (e.g. PESQ) analysis.

Testing VoLTE


Temporary files

These scripts can create large temporary files on the probe's disk:

A Radio Trace file - if requested by the trace option

An IP Trace file - if requested by the pcap option

Setup and Call audio files

These files are deleted automatically when the script ends, to stop the probe's disk filling up. (If you wish to access

them, use the upload option to copy them to your Projector.)


The VoLTE Script uses a Power Down Modem scene and Check outcome scene near the end of the Originator and

Recipient parties, like this:

These are used for modem error recovery, as follows.

When a modem uses a Local SIM (not a SIM Server SIM), Monitor Master turns it on the first time it is used, and

then leaves it turned on. This runs scripts more efficiently, as they don't have to wait for the modem to power up at

the start of each test. (When a modem uses a SIM Server SIM, Monitor Master turns it on at the start of the test and

off at the end.)

If a script executes the Power Down Modem scene, it turns the modem off - and the modem will turn on again the

next time a script uses it. This "power cycle" resets the modem, a good way to restore normal behavior for the next

test run after many of the transient hardware problems that affect modems. In the VoLTE Script, the Can disable

trace scene's error routes (orange arrows) are connected directly to the Power Down Modem, to recover the

modem if it encounters a problem. If other scenes encounter a problem that warrants a modem reset, they record



the issue by setting the My Sync Outcome prop to a value such as "Initialization failure". The Check outcome

scene (a Match Values scene) checks this prop and executes the Power Down Modem scene if its value indicates

that a problem has been encountered.

The VoLTE Script for the Galaxy S7 WARNING: This script is still in development. All information about it is still subject to change without notice.

This section will describe the QM VoLTE Voice Quality.mmts script once it is stable and fully tested, but

information is not available at this time.

If you want to know how to use the script, see the previous topic, Using the VoLTE Script for Galaxy S7.

Testing VoLTE


Chapter 5 - VoLTE Testing Results

Hint: For background information on VoLTE, see Voice over LTE in Chapter 1.

When you run the VoLTE Script, it can produce three broad types of output:

Testrun Metrics and Messages, which describe the test system and how well it is performing - what we

usually call Meta information

IP Trace and Radio Trace Files, which indicate how the network is behaving, in great detail at a low level -

what we usually call Diagnostics

VoLTE Metrics, which indicate how well the VoLTE Service is performing, from a user's point of view -

what we usually call Results

This Chapter describes the results and diagnostics produced from your VoLTE testing (described in Chapters 3-5),

including how to access and interpret them, under these headings:

Using Box Office Reports

Using Trace Files

Metrics & Messages (summary)

Metrics Details

(For information on monitoring tests while they are running, see Chapters 3-4.)

Using Box Office reports The results (service availability, metrics, timings, messages, etc) from VoLTE testing are stored on your system's

Projector and can be used just like other testrun metrics.

Results from a single testrun can be examined later by using Box Office reports, or results from multiple tests can

be combined to create graphs, maps or tables to help summarize and analyze them - reveal the bigger picture,

trends and hotspots.

A range of standard reports is provided with Monitor Master, as described in the Using Box Office Reports guide.

See for example:

The Screening Room Dashboard - to list testruns and provide drill-down into messages, metrics and timings

The Metric Graphs Dashboard - to graph how any metric changes over time

The Map of the World dashboard - to show service availability by location

Customers can create their own VoLTE reporting solution by using the Box Office developer facilities (see the

Developing Reports with Box Office guide) or by exporting their results to another reporting platform.

Using trace files If VoLTE testing is configured to capture and upload trace files (by using the script's trace, pcap and upload

options as described earlier), the trace files will be stored by your System FTP Server, on your Projector.

You will need additional tools to view and analyze these files, such as TEMS Discovery (for Radio Trace) and the

Open Source Ethereal (for IP Trace).

To access a trace file on Projector, you will need to know:



the root directory used for the upload to FTP Server - this is configured by the datarepository path attribute

in your system's SiteConfig.xml file. This defaults to "\" but will typically be reconfigured to another

location by your system administrator.

the filename used for the file is unique - the filename is output in the metrics for each testrun

Metrics and Messages Summary This section lists the most useful Metrics and Messages output by the VoLTE Solution. Metrics provide

information about the testrun and service under test, for use in reports; Messages provide information of more

immediate interest, for monitoring or debugging a testrun.

Script Metrics

Script Messages

VoLTE Metrics

VoLTE Messages

Script Metrics

These Metrics provide information about the test and test environment, for the Originator (MO suffix) and

Recipient (MT suffix) parties:

Metric Name Notes

Service availability 0%=call failed; 100%=call succeeded

System availability 0%=test failed; 100%=test succeeded

Trace Upload File The pathname of the TRP Trace file

MSISDN Example: 6148969999

Hardware Status of the EMU e.g. "Device LGVS870#11 powered up"

MCC Example: 311

MNC Example: 480

TAC Example: 51777

CID Example: 356

Bearer Technology Example: LTE

SID Example: 275

NID Example: 130

IMEI Example: 990001501157945

IMSI Example: 311480059650796

Script Messages

These Messages include a range of status and progress messages such as:

Modem Manufacturer

Modem Model

Modem SW Version

IMS Registration Status

Testing VoLTE


If a step fails, you will also see diagnostic messages such as:

Diagnostic: Failed to query device information

Diagnostic: Error starting Adb server

Diagnostic: Error obtaining status of VApi Call Control service

Diagnostic: OnDevice VApi Call Control service is not running

Diagnostic: Unable to check the status of airplane mode

Diagnostic: Timed out waiting for IMS registration

Diagnostic: Failed to retrieve IMS registration status

The presence of a diagnostic message generally indicates a failure, which should be investigated and resolved if it

persists.

VoLTE Metrics

The VoLTE Metrics measure how well the VoLTE service is operating and include:

Metric Name Units Notes

ActivateDedicatedEpsBearerContextRequest 1 = Success; 0 = Failure

BearerResourceAllocationReject 0 = Success; 1 = Failure

ConversationStartDelay ms

DeactivateEpsBearerContextRequest

DPLDroppedSegment 1 = dropped segments

EpsBearerDrop

InterarrivalJitter ms mean for the test

MaxGuaranteedBitRateDownlink kbps

MaxGuaranteedBitRateUplink kbps

MinGuaranteedBitRateDownlink kbps

MinGuaranteedBitRateUplink kbps

PacketLossRatio %

PresenceSubscriptionDelay ms

PresenceSubscriptionFailure 0 = Success; 1 = Failure

PresenceSubscriptionSuccess 1 = Success; 0 = Failure

PublishRequestDelay ms

PublishRequestFailure 0 = Success; 1 = Failure

PublishRequestSuccess 1 = Success; 0 = Failure

SessionCompletionFailure 0 = Success; 1 = Failure

SessionCompletionSuccess 1 = Success; 0 = Failure

SessionDisconnectDelay ms

SessionDurationTime s

SessionSetupDelay s

SessionSetupFailure 0 = Success; 1 = Failure



Metric Name Units Notes

SessionSetupSuccess 1 = Success; 0 = Failure

SipDeregistrationSuccess 1 = Success; 0 = Failure

SipHandshakeTime ms

SipRegistrationDelay ms

SipRegistrationDelayC1 ms

SipRegistrationDelayC2 ms

SipRegistrationFailure 0 = Success; 1 = Failure

SipRegistrationSuccess 1 = Success; 0 = Failure

For more detailed information see VoLTE Metrics Details.

VoLTE Messages

VoLTE Messages may be provided to provide additional information about failures, as follows:

Message Name [TD Name] Notes

SessionCompletionFailureDiagnostic

[SIPRegistrationFailureCause]

The reason why the SIP session failed to

complete, if applicable. For example:

Timeout, BYE ACK Not Received, (TD)

SessionSetupFailureDiagnostic

[SessionSetupFailureCause]

The reason for the failure to set up the VoLTE

call (SIP session), if applicable. For the originator

party, the SIP INVITE will be outgoing (from

phone to SIP server), for the recipient party, the

direction is reversed.

For example:

Busy Here

Service Unavailable - service based local

policy function not available

Session Setup Timed Out, (TD), after 180

RINGING, before UPDATE

Session Setup Timed Out, (TD), before 180

RINGING, before UPDATE

SipRegistrationFailureDiagnostic

[SIPRegistrationFailureCause]

The reason for the SIP registration failure, if

applicable.

See your TEMS Discovery user guides for more information on any Message.

VoLTE Metrics Details This section describes the main VoLTE Metrics reported by the Monitor Master VoLTE Solution in more detail.

(For a summary, see Metrics Summary, above.)

Testing VoLTE


The VoLTE Solution generates VoLTE KPIs that map to those seen in TEMS Discovery – but with some minor

changes. (TD KPI names are included below for ease of cross-reference.)

ActivateDedicatedEpsBearerContextRequest

TD name: Activate Dedicated EPS Bearer Context Request

Units: none

Number of Activate Dedicated EPS Bearer Context Request events. This event is triggered by an LTE NAS

Signaling message of the same name. Part of the procedure to activate the bearer for the VoLTE call (which also

provides a guaranteed bit rate).

0 = EPS bearer context activation did not take place

1 = EPS bearer context activation did take place

Higher values should not be see

BearerResourceAllocationReject

TD name: Bearer Resource Allocation Reject

Units: none

Number of Bearer Resource Allocation Reject events.

0 = Bearer Resource Allocation was accepted

1 = Bearer Resource Allocation was rejected; EPS Bearer was not established, the test will fail

If the bearer resource allocation request cannot be accepted by the network, then the MME sends a BEARER

RESOURCE ALLOCATION REJECT message to the UE.

ConversationStartDelay

TD name: ConversationStartDelay

Units: ms

Time from the start of the VoLTE call (from the timestamp of the 200 OK response to SIP INVITE) to the first

incoming RTP packet (not necessarily the first RTP packet seen in the capture).

For the Originator, the 200 OK response is received from the SIP Server; for the Recipient it is sent to the SIP Server.

Typical value: 900 ms

DeactivateEpsBearerContextRequest

TD name: Deactivate EPS Bearer Context Request

Units: none

Number of Deactivate EPS Bearer Context Request events. This event is triggered by receipt of the appropriate LTE

NAS Signaling message. Part of the procedure to de-activate the bearer for the VoLTE call.

0 = no events detected

1 = event detected

Higher values should not be seen

Typical value: 0 (rarely detected)



DPLDroppedSegment

Note: Only reported by the Galaxy S5 VoLTE script.

TD name: none

Units: none

The Galaxy S5 VoLTE script relies on DPL Messages from the handset for call control and KPI calculations.

Messaging is not 100% reliable and a missing DPL segment in the IP capture may cause a testrun to fail, or produce

incorrect KPIs.

To help you detect when this happens, if this script detects a missing DPL segment, it reports an extra metric,

DPLDroppedSegment with its value set to 1. The metric's timestamp is set to the time the missing DPL segment

was detected.

Note that dropped segments don't always mean that the testrun or metrics are invalid - they may be dropped from

non-critical parts of the messaging.

1 = missing segments detected

Typical value: not reported. The metric is only output if missing segments are detected, so it is never output with a

value of 0.

EpsBearerDrop

TD name: EPS Bearer Drop

Units: none

Number of EPS Bearer Drop events. This event is triggered if we detect a RTP drop and EPS Bearer is already

deactivated, or if no EPS Bearer deactivation was recorded before a new session attempt.

0 = no events detected

1 = event detected


Typical value: 0 (not detected)

InterarrivalJitter

TD name: InterarrivalJitter

Units: ms

The mean of the inter-arrival jitter experienced during the test.


MaxGuaranteedBitRateDownlink

TD name: GuaranteedBitRateDL

Units: kbps

The Maximum Guaranteed Bit Rate for the downlink. This value is read from NAS signaling messages.

Typical value: 38 kbps

MaxGuaranteedBitRateUplink

TD name: GuaranteedBitRateUL

Testing VoLTE


Units: kbps

The Maximum Guaranteed Bit Rate for the uplink. This value is read from NAS signaling messages.


MinGuaranteedBitRateDownlink

TD name: GuaranteedBitRateDL

Units: kbps

The Minimum Guaranteed Bit Rate for the downlink. This value is read from NAS signaling messages.


MinGuaranteedBitRateUplink

TD name: GuaranteedBitRateUL

Units: kbps

The Minimum Guaranteed Bit Rate for the uplink. This value is read from NAS signaling messages.


PacketLossRatio

TD name: PacketLossRatio

Units: %

The number of RTP packages lost divided by the number of RTP packets expected. TEMS Discovery calculates a

PacketLossRatio metric once per second. Monitor Master reports the average of all these values as a single metric.

Typical value: 3%

PresenceSubscriptionDelay

TD name: PresenceSubscriptionDelay

Units: ms

The time from the (SIP) PUBLISH Request to the 200 OK (on PUBLISH) Response.

Typical value: not reported by Monitor Master

PresenceSubscriptionFailure

TD name: Presence Subscription Failure

Units: none

The number of Presence Subscription Failure events. This event is triggered by exceeding the Presence Registration

Timeout.

0 = no events seen

1 = one event seen

Typical value: Monitor Master always reports 0



PresenceSubscriptionSuccess

TD name: Presence Subscription Success

Units: none

The number of Presence Subscription Success events. This event is triggered by reception of the 200 OK Response

to a NOTIFY Request followed by successful SUBSCRIBE.

0 = no events seen

1 = one event seen

Typical value: Monitor Master always reports 0

PublishRequestDelay

TD name: PublishRequestDelay

Units: ms

The time from the (SIP) PUBLISH Request to the 200 OK (on PUBLISH) Response.

Typical value: 200 – 4000 ms

PublishRequestFailure

TD name: Publish Request Failure

Units: none

The number of Publish Request Failure events. This event is triggered in several ways, including:

Exceeding the Publish Timeout

489 error with PUBLISH in any field

0 = no event seen

1 = one event seen, etc

Typical value: 0

PublishRequestSuccess

TD name: Publish Request Success

Units: none

The number of Publish Request Success events. This event is triggered by reception of the 200 OK Response to a

PUBLISH Request.

0 = no event seen

1 = one event seen

Typical value: 1

SessionCompletionFailure

TD name: Session Completion Failure

Units: none

Testing VoLTE


The number of Session Completion Failure events. The number of times a VoLTE call (SIP session) encountered an

error state before completing. Error states include:

BYE received after a Session Hangup Timeout

No RTP packets received since Session Drop Timeout

No RTP packets within one second of INVITE

Reason field of BYE is a 500 error

600 error occurs in any field of a BYE

0 = no event seen (session did not fail)

1 = one event seen (session failed)


Typical value: 0

SessionCompletionSuccess

TD name: Session Completion Success

Units: none

The number of Session Completion Success events. The number of VoLTE calls (SIP sessions) completed during the

test run. SIP session completion is determined by reception of a 200 OK response to the SIP BYE. If the party is

hanging up the call, the SIP BYE will be outgoing, otherwise the SIP BYE will be received from the SIP Server.

0 = no event seen (no successful call)

1 = one event seen (successful call)


Typical value: 1

SessionDisconnectDelay

TD name: SessionDisconnectDelay

Units: ms

The time taken to terminate the call: the time from the SIP BYE request and the 200 OK response. If the party is

hanging up the call, the SIP BYE will be outgoing, otherwise the SIP BYE will be received from the SIP Server.


SessionDurationTime

TD name: SessionDurationTime

Units: s

The total duration of the call (SIP session): the time from the 200 OK response to the SIP INVITE and the SIP BYE

request. If the party is hanging up the call, the SIP BYE will be outgoing, otherwise the SIP BYE will be received

from the SIP Server.

Typical value: depends on call length

SessionSetupDelay

TD name: SessionSetupDelay180



Units: s

The time taken to set up the VoLTE call: the time from the SIP INVITE to the 180 (Ringing) Response. For the

Originator, the SIP INVITE will be outgoing (from phone to SIP Server), and the 180 Ringing response will be

incoming (from SIP Server to phone). For the Recipient, these directions are reversed.

Typical value: 0.75 – 5 seconds

SessionSetupFailure

TD name: Session Setup Failure

Units: none

Number of times a VoLTE call (SIP session) setup encountered an error state. Causes include:

Session setup timeout

A CANCEL request was received between an INVITE or 180 RINGING response and a 200 INVITE

A 400, 500 or 600 error (with a few exceptions) occurred after an INVITE or 180 response

successful SIP Session setup events. The count of the number of times a VoLTE call (SIP session) was successfully

established. This is determined by receipt of a 200 OK response to a SIP INVITE. For the Originator, the SIP

INVITE will be outgoing (from phone to SIP server), and the 200 OK response will be incoming (from SIP server to

phone). For the Recipient, these directions are reversed.

0 = a single VoLTE call was successfully set up.

1 = no call was set up.

SessionSetupSuccess

TD name: Session Setup Success

Units: none

Number of successful SIP Session setup events. The count of the number of times a VoLTE call (SIP session) was

successfully established. This is determined by receipt of a 200 OK response to a SIP INVITE. For the Originator,

the SIP INVITE will be outgoing (from phone to SIP server), and the 200 OK response will be incoming (from SIP

server to phone). For the Recipient, these directions are reversed.

1 = a single VoLTE call was successfully set up.

0 = no call was set up.

SipDeregistrationSuccess

TD name: SIP Deregistration

Units: none

Number of SIP de-registration events that occur during the duration of the test. De-registration is triggered by the

'Expires' message header field of SIP REGISTER message being set to 0

Valid values are 0 or 1.

Typical value: 0

A value of 0 is always reported in our tests, indicating that SIP de-registration did not occur.

Testing VoLTE


SipHandshakeTime

TD name: SipHandshakeTime

Units: ms

The time between the SIP PRACK (Provisional acknowledgement) request and the 200 OK Response. The

Originator sends a SIP PRACK message to the server, the Recipient receives one.

Typical value: around 300ms

SipRegistrationDelay

TD name: SipRegistrationDelay

Units: ms

The time between the outgoing SIP REGISTER request and the 200 OK response.

Typical value: around 3000 ms

SipRegistrationDelayC1

TD name: SipRegistrationDelayC1

Units: ms

The time between the outgoing SIP REGISTER request and the 200 OK response. Generated if a 401 (authentication

required) status code is received from the server during registration.

The values reported are the same as for SipRegistrationDelay

SipRegistrationDelayC2

TD name: SipRegistrationDelayC2

Units: ms

The time between the outgoing SIP REGISTER request and the 200 OK response. Generated if a SIP NOTIFY is

received during registration.

Typical value: not seen in our testing

SipRegistrationFailure

TD name: SIP Registration Failure

Units: none

Number of SIP Registration Failure events. The number of errors encountered during SIP registration. Causes

include:

600 error with INVITE in any field

400 or non-200 response received after SIP Registration Timeout

0 = SIP registration success

1 = SIP registration failed




SipRegistrationSuccess

TD name: SIP Registration Success

Units: none

Number of SIP Registration Success events. The number of successful SIP registrations that have taken place

during the test. You should not see multiple SIP registration during a single test.

Valid values are 0 or 1.

Testing VoLTE


Index

D

DPL Dropped Segments, 18

E

Ethereal, 25

Example

VoLTE scripts, 11, 19

Example Scripts

VoLTE, 11, 16, 19, 24

I

Installing

SIM Server support, 8

VoLTE hardware, 7

VoLTE modems, 7

VoLTE reports, 8

VoLTE scripts, 8

VoLTE software, 8

IP Trace

VoLTE, 8

L

L3 Trace. See Radio Trace

Layer 3 Trace. See Radio Trace

LG Lucid2 VS870 Modem, 4

LTE

VoLTE, 4, 11, 19

Lucid2 Modem. See LG Lucid2 VS870 Modem

M

Messages

VoLTE, 26

Metrics

VoLTE, 26, 28

MME, 4

MSC, 4

P

Parameters. See also Runtime Parameters

PCAP. See IP Trace

Post-processing Package, 8

Q

QM VoLTE Availability script

for Galaxy S5, 8, 11, 16, 19, 24

for Lucid2, 8

installing, 8

versions, 8

R

Radio Trace

VoLTE, 8

Reports

VoLTE, 25

Results

overview, 25

Runtime Parameters, 12, 20

S

Samsung Galaxy Note 4 SM-N910G Modem, 11, 19

Samsung Galaxy S5 SM-G900A Modem, 11

Samsung Galaxy S5 SM-G900V Modem, 4, 11, 16, 19,

24

Samsung Galaxy S7 SM-G930A Modem, 4

Samsung Galaxy S7 SM-G930R4 Modem, 4

SiteConfig.xml File

VoLTE, 25

T

Testing

VoLTE with Galaxy Note 4, 11, 19

VoLTE with Galaxy S5, 19

VoLTE with Galaxy S5 SM-G900A, 11

VoLTE with Galaxy S5 SM-G900V, 11

Timings

Testing VoLTE


VoLTE, 26, 28

Trace files

VoLTE, 25

U

Using

VoLTE, 4, 11, 19

VoLTE results, 25

V

Voice over LTE. See VoLTE

VoLTE, 4

diagrams, 4

hardware, 7

installing, 7, 8

IP Trace, 8, 25

licensing, 8

messages, 26

metrics, 26, 28

modems, 7

probes, 7

Radio Trace, 8, 25

reports, 8, 25

results, 25

script for Galaxy S5 modems, 8, 16

script for Galaxy S7 modems, 8, 24

scripts, 11, 19

SIMs, 8

software, 8

software requirements, 8

TCC version, 8

testing with Galaxy Note 4, 11

testing with Galaxy S5 SM-G900A, 11

testing with Galaxy S5 SM-G900V, 11

testing with Galaxy S7 SM-G930A, 19

testing with Galaxy S7 SM-G930R4, 19

timings description, 28

timings summary, 26

trace files, 25

Documents

TEMS™ Monitor Master v10.4 - mantica-qm.ant …mantica-qm.ant-cumulus.co.uk/qmweb/Documentation/testing_volte.pdf · (The solution for the S7 modems has a simpler architecture,