Umetrix Voice LM...• A PH12 power USB hub, which controls, collects data from, and charges up to 12 smartphones. The PH12 can also interface with a GPS puck and an RF scanner. •

Umetrix® Voice LM Release 3.12 Getting Started Guide

Umetrix Voice LM Release 3.12 – Getting Started Guide

www.spirent.com 2

Copyright © 2019 Spirent Communications, Inc. All Rights Reserved.

All of the company names and/or brand names and/or product names referred to in this document, in particular, the name “Spirent” and its logo device, are either registered trademarks or trademarks of Spirent plc and its subsidiaries, pending registration in accordance with relevant national laws. All other registered trademarks or trademarks are the property of their respective owners. The information contained in this document is subject to change without notice and does not represent a commitment on the part of Spirent Communications. The information in this document is believed to be accurate and reliable; however, Spirent Communications assumes no responsibility or liability for any errors or inaccuracies that may appear in the document. Page Part Number: 71-008740, Version A1


www.spirent.com 3

Table of Contents 1. Introduction ........................................................................................ 5

Overview .............................................................................................................. 5

How to Contact Us ................................................................................................ 6

Access the Latest Documentation ........................................................................ 7

Document Conventions ........................................................................................ 7

2. Installing and Setting Up Umetrix Voice LM ....................................... 8

Overview .............................................................................................................. 8

Prerequisites......................................................................................................... 8

Step 1: Download the Software ............................................................................ 9

Step 2: Install the Software ................................................................................. 10

Step 3: Configure the Smartphones ................................................................... 18

Step 4: Connect the Equipment .......................................................................... 19

Step 5: Configure the Software........................................................................... 22

Step 6: Configure a VoLTE Voice Call Test ........................................................ 30

Step 7: Run the VoLTE Voice Call Test .............................................................. 35

3. Using Audio Tunes .......................................................................... 44

Overview ............................................................................................................ 44

Prerequisites....................................................................................................... 44

Procedure ........................................................................................................... 45

4. Integrating Umetrix Voice LM with Umetrix Data .............................. 57

Overview ............................................................................................................ 57

Prerequisites....................................................................................................... 57

Procedure ........................................................................................................... 58

5. Using the KPI Report Tool ............................................................... 62

Overview ............................................................................................................ 62

Prerequisites....................................................................................................... 63

Generate a Report .............................................................................................. 64

Sample Analysis Sheets ..................................................................................... 68

Report Dashboard ................................................................................... 68

MOS Analysis .......................................................................................... 69

RTP Analysis ........................................................................................... 70

Call Performance Analysis ....................................................................... 71


www.spirent.com 4

RF Analysis .............................................................................................. 72

Handover Analysis ................................................................................... 73

MRAB (MultiRAB) Data Analysis ............................................................. 74

Serving Network Analysis ........................................................................ 75

CA Utilization Analysis ............................................................................. 76

MIMO Utilization Analysis ........................................................................ 77

Call Performance Details ......................................................................... 78


www.spirent.com 5

1. Introduction

Overview This guide describes how to install, set up, and start testing phones with Spirent Umetrix® Voice LM. The Umetrix Voice LM solution consists of the following components:

• A PH12 power USB hub, which controls, collects data from, and charges up to 12 smartphones. The PH12 can also interface with a GPS puck and an RF scanner.

• Either 4, 8, or 12 Audio Interface Modules (AIM), which are used to connect smartphones to the PH12 and provide audio quality MOS and Bluetooth control. The AIM allows you to evaluate voice audio quality on smartphones through either the 3.5 mm ear jack connector or Bluetooth. Bluetooth audio quality evaluation is provided for smartphones that do not have a wired ear jack connector. The AIM provides simultaneous measurement of both the audio quality and diagnostic data, which allows correlation of audio problems to the specific cause.

• A PC running Umetrix Voice LM software, which controls the AIM units and the smartphones connected to them.

The following figure shows a PH12 and 2 AIMs.

Figure 1. A PH12 (bottom) and 2 AIMs.


www.spirent.com 6

How to Contact Us To obtain technical support for any Spirent Communications product, please contact our Support Services department using any of the following methods: Americas E-mail: [email protected] Web: http://support.spirent.com Toll Free: +1 800-SPIRENT (+1 800-774-7368) (North America) Hours: Monday through Friday, 05:30 to 18:00 Pacific Time Europe, Africa, Middle East E-mail: [email protected] Web: http://support.spirent.com EMEA Phone: +33 (1) 6137 2270 UK Phone: +44 1803 546333 Toll Free Phone: +1 818-676-2616 Hours: Monday through Thursday, 09:00 to 18:00, 9:00 to 17:00 Friday, Paris Time Asia Pacific E-mail: [email protected] Web: http://support.spirent.com In China Mainland Phone: +86 (400) 810-9529 (toll-free) Out of China Mainland Phone: +86 (10) 8233 0033 India: 1800-419-2111 Operating Hours: Monday through Friday, 09:00 to 18:00 Beijing Time The Spirent Knowledge Base (http://support.spirent.com) is designed to serve your technical information needs. The Knowledge Base gives you access to tens of thousands of documents that help answer your network analysis and measurement questions. New content is added daily by Spirent’s communications and networking experts. Sign in with your user ID and password to gain access to additional content that is available only to customers – user manuals, Help files, release notes, Tech Bulletins, and more. When you sign in, you can also use the Knowledge Base to download software and firmware, and to manage your SRs. Information about Spirent Communications and its products and services can be found on the main company website at http://www.spirent.com. Company Address Spirent Communications, Inc. 26750 Agoura Road Calabasas, CA 91302 USA


www.spirent.com 7

Access the Latest Documentation To access the latest version of this document, perform the following steps: 1. Log into the Spirent Customer Service Center website

(https://support.spirent.com) using the email address and password assigned to you by Spirent.

2. In the Search Knowledge Base box, enter DOC11601 and press the ENTER key. The results of the search are displayed.

3. Click on the link Spirent Umetrix Voice LM Documentation The Spirent Umetrix Voice LM Documentation page appears.

4. In the Documentation section, click on the link for the document in which you are interested. The page for the selected document appears.

5. Click on the link in the Attachment area to view the corresponding PDF.

Document Conventions This document uses the following conventions:

• Text you type appears in this type style

• Keyboard keys are displayed IN THIS TYPE STYLE


www.spirent.com 8

2. Installing and Setting Up Umetrix Voice LM

Overview This section describes how to install, set up, and run tests with Umetrix Voice LM. In this section, you will perform the following steps: 1. Download the software from the Spirent Customer Service Center (CSC). 2. Install the software on your PC. 3. Configure the smartphones. 4. Connect the equipment. 5. Configure the software. 6. Configure a VoLTE voice call test. 7. Run the VoLTE voice call test.

Prerequisites Before performing the steps in this guide, make sure you have:

• a PC running Microsoft® Windows® 10 with: An i7 CPU 16GB or higher of RAM A display supporting 1024 x 768 resolution A minimum of 2 USB ports (if you are using a GPS) A hard disk that has 30GB of space available (for log file storage)

• a Umetrix Voice LM PH12 hub

• either 4, 8, or 12 Umetrix Voice LM AIM units (depending on the package you purchased). Each AIM ships with the following cables: 1 micro USB-C cable 1 USB to micro USB-C cable 1 audio cable

• the Umetrix Voice LM software license provided by Spirent


www.spirent.com 9

Step 1: Download the Software Perform the following steps to download the Umetrix Voice LM software: 1. Using a web browser, log into the Spirent Customer Service Center website at

http://support.spirent.com using the e-mail address and password assigned to you by Spirent.

2. On the Customer Service Center page, click the Downloads tab. The Downloads page appears.

3. From the Release Type box, select General Release. 4. From the Product Line box, select Umetrix. 5. Download the following software: Umetrix Voice LM 32-bit Installer, v3.12 for Windows Umetrix Voice LM KPI Report Installer, v3.12 for Windows

6. Go to the next section to install the Umetrix Voice LM software.


www.spirent.com 10

Step 2: Install the Software Perform the following steps: 1. Using Windows Explorer, access the folder containing the Umetrix Voice LM

software, and double-click on the file UmetrixVoiceLM_3.12.xxxx_Setup.exe. The UmetrixVoiceLM Setup window appears, and the License Agreement screen appears.

Figure 2. License Agreement screen.


www.spirent.com 11

2. Read the license agreement. 3. If you agree to the terms of the license agreement, click the I Agree button.

The Choose Install Location screen appears.

Figure 3. Choose Install Location screen.


www.spirent.com 12

4. Click the Next button. The Choose Start Menu Folder screen appears.

Figure 4. Choose Start Menu Folder screen.


www.spirent.com 13

5. Click the Install button. A message box appears showing the status of the installation. When the installation is complete, the Installation Complete screen appears.

Figure 5. Installation Complete screen.

6. Click the Close button.


www.spirent.com 14

7. Double-click on the file UmetrixVoiceLM_KPI_Report_3.12.xxxx_Setup.exe. The Spirent Communications Setup window appears, and the License Agreement screen appears.

Figure 6. License Agreement screen.


www.spirent.com 15

8. Read the license agreement. 9. If you agree to the terms of the license agreement, click the I Agree button.

The Choose Install Location screen appears.

Figure 7. Choose Install Location screen.


www.spirent.com 16

10. Click the Next button. The Choose Start Menu Folder screen appears.

Figure 8. Choose Start Menu Folder screen.


www.spirent.com 17

11. Click the Install button. A message box appears showing the status of the installation. When the installation is complete, the Installation Complete screen appears.

Figure 9. Installation Complete screen.

12. Click the Close button. 13. Go to the next section to configure the smartphones.


www.spirent.com 18

Step 3: Configure the Smartphones For each smartphone you want to use, make sure:

• USB debugging mode is enabled.

• Bluetooth is enabled.

• USB mode is set to MTP.

• The DM port is enabled.

• The driver for the smartphone is installed on the PC that has Umetrix Voice LM software installed.

• You know the appropriate engineering code for the phone. (If you do not know this code, contact the phone manufacturer to obtain it.)

Perform the following steps: 1. On the smartphone, dial the appropriate engineering code for that phone. If you

do not know this code, contact the phone manufacturer to obtain it. Hidden Menu appears.

2. Tap on SVC Menu. 3. Tap on Port Check Test. 4. Tap on Port Check Test to enable QXDM. 5. On the smartphone, select Settings > About device and tap on Build number 7

times to enable Developer mode. 6. Select Settings > Developer options. 7. Set USB Debugging to ON. 8. Set Verify apps over USB to OFF. 9. Set USB Mode/Configuration to MTP. 10. Repeat Steps 1 through 9 for each smartphone you want to use. 11. When finished configuring the smartphones, go to the next section to connect the

equipment.


www.spirent.com 19

Step 4: Connect the Equipment Perform the following steps: 1. Connect a USB cable to the PC port on the front panel of the PH12.

Figure 10. Front panel of the PH12.

2. Connect the other end of the USB cable to a USB port on the PC. 3. Connect a USB cable to the PC port on the AIM.

Figure 11. Location of the PC port on an AIM.


www.spirent.com 20

4. Connect the other end of the USB cable to an available port (1 through 12) on the PH12.

5. Connect a USB cable to the Mobile port on the AIM.

Figure 12. Location of the Mobile port and Audio port on an AIM.

6. Connect other end of the USB cable to USB port on the smartphone. 7. If the smartphone has an ear jack:

a. Connect an audio cable to the Audio port on the AIM. b. Connect the other end of the audio cable to the ear jack on the smartphone.

8. Repeat Steps 3 through 7 for each AIM. The following figure shows Umetrix Voice LM configuration containing 4 AIMs and 4 smartphones.


www.spirent.com 21

Figure 13. Umetrix Voice LM configuration containing 4 AIMs.

9. When finished connecting the equipment, go to the next section to configure the software.


www.spirent.com 22

Step 5: Configure the Software Perform the following steps: 1. On the PC desktop, double-click on the Umetrix Voice LM icon as shown in the

following figure.

Figure 14. Spirent Umetrix Voice LM desktop short cut icon.

The User Account Control dialog box appears prompting you to allow this app to make changes to your device.

2. Click the Yes button. The Port Setting dialog box appears.

Figure 15. Port Setting dialog box.


www.spirent.com 23

The Port Setting dialog box enables you to specify the phones you have connected to the Umetrix Voice LM system. A phone that is connected to an AIM is referred to as a “channel.”

3. Click the Device Manager button. The Device Manager window appears.

4. Make sure that the ADB interface and the USB serial port for each smartphone you connected to the AIMs appear under Android Device and Ports (COM & LPT). The following figure shows the ADB (Android Debug) interface and USB serial port for 2 LG smartphones that were connected to AIMs.

Figure 16. Location of ADB interface and USB serial port for each smartphone in

the Device Manager window.


www.spirent.com 24

5. Close the Device Manager window. 6. In the Mobiles area of the Port Setting dialog box, check the CH1 check box. 7. From the Mobile Chip box, select the mobile chip used by the smartphone

identified as channel 1. 8. From the Phone box, select General. 9. From the DM Port box, select the diagnostic mode (DM) port for the smartphone

identified as channel 1. (This is the USB serial port of the smartphone.) The software displays the ports that are physically associated with the phone that is connected to the AIM that is plugged into port 1 on the hub. The DM port provides information coming out of the RF chip.

10. Click the Change Log Masks button, which is located next to the Log masks box. The Logmask Set window appears.

Figure 17. Logmask Set window.

The Logmask Set window enables you to specify which messages coming out of the RF chip you want to log.


www.spirent.com 25

11. Click the Select All button to log all available messages coming out of the chip. 12. Click the OK button. 13. Make sure the ADB box displays the ADB identifier for the smartphone. This

information appears automatically if recognized by the PC running Umetrix Voice LM.

14. From the Use RMA box, select Use(ADB). RMA (Remote Monitoring App) is an application that enables Umetrix Voice LM to capture specific Android information from the smartphone. When you select Use (ADB), Umetrix Voice LM uses ADB to communicate with RMA.

15. From the Use Bluetooth box, select Use (Android). This sets Umetrix Voice LM to use Bluetooth for call control (that is, dialing and answering the smartphone). If the smartphone does not have an ear jack, Umetrix Voice LM will use also use the Bluetooth link to send voice clips on the smartphone.

16. If you want to perform VoLTE testing, check the Receive and Save TCP/IP payload check box. This enables information for the SIP layers and the RTP layer to come into Umetrix Voice LM from the smartphone.

17. Perform one of the following steps: If you have 4 AIMs, repeat Steps 6 through 16 for channels 2 through 4. If you have 8 AIMS, repeat Steps 6 through 16 for channels 2 through 8. If you have 12 AIMs, repeat Steps 6 through 16 for channels 2 through 12.

18. If you are using a GPS device, perform the following steps: a. Click the GPS check box.

The GPS settings appear.

Figure 18. GPS settings.

b. From the GPS source box, select the appropriate setting. c. From the GPS interface box, select the appropriate setting.


www.spirent.com 26

d. From the Port box, select the port. e. From the Baud rate, select the baud rate.

19. When you are finished configuring the channels and GPS, click the OK button. The Umetrix Voice LM window appears. On each smartphone, you will be prompted to allow USB debugging.

20. On each smartphone, select the Allow option for USB debugging. The connection status for each smartphone connected to an AIM appears in the status bar at the bottom of the Umetrix Voice LM window. The following figure shows the status of 2 smartphones.

Figure 19. Status bar showing the status of 2 smartphones.


www.spirent.com 27

The status bar provides the following information for each smartphone (channel): M#: Mobile Chip (for example, M1). Shows the status of the connection

between Umetrix Voice LM and the RF chip for that smartphone. Green indicates that Umetrix Voice LM is communicating properly with the RF chip for that mobile.

A: Android. Shows the status of RMA. Green indicates that RMA is running on the smartphone.

B: Bluetooth. Shows the status of the Bluetooth connection between Umetrix Voice LM and the smartphone. Green indicates that Bluetooth connection is established.

Network in use (for example, LTE). Shows the network that the smartphone is currently using,

21. From the Control menu, select RMA Setting. The RMA Install & Execute dialog box appears.

Figure 20. RMA Install & Execute dialog box.


www.spirent.com 28

22. Check the check box for each channel you are using. 23. Click the Install RMA button to install RMA on each smartphone.

The Critical Warning! window appears.

Figure 21. Critical Warning! window for a configuration with 2 smartphones.

24. Click the Close button. 25. Click the Close button in the RMA Install & Execute dialog box.

The A indicator in the status bar of the Umetrix Voice LM window turns green for the smartphones, indicating that RMA is running on each smartphone.


www.spirent.com 29

26. From the Control menu, select Bluetooth Control. The BlueTooth Control dialog box appears.

Figure 22. BlueTooth Control dialog box.

27. Check the check box for each channel you are using. 28. Click the Refresh UE BT MAC Address button. 29. Click the Connect button.

A pairing request appears on each smartphone. 30. Accept the pairing request on each smartphone.

As you accept the pairing request on a smartphone, the B indicator in the status bar of the Umetrix Voice LM window turns green for the associated smartphone. When you are finished accepting the pairing requests, all of the indicators in the status bar for all the smartphones should turn green. At this point, the M, A, and B indicators for all the smartphones should be green.

31. In the BlueTooth Control dialog box, click the Close button. 32. Go to the next section to configure a VoLTE voice call test.


www.spirent.com 30

Step 6: Configure a VoLTE Voice Call Test In this section, you will configure Umetrix Voice LM to run a VoLTE voice call test. Perform the following steps: 1. Click the Plan Setting button located at the top of the Umetrix Voice LM window

as shown in the following figure.

Figure 23. Plan Setting button.

The Plan Config dialog box appears and displays the Idle tab. The Plan Config dialog box enables you to configure the application layer test you want to run.

2. Check the CH1 check box in the Channel area. 3. Click on the Voice call tab.

The Voice call tab appears.


www.spirent.com 31

Figure 24. Sample Voice call tab for a setup with 4 channels.

4. From the Call type box, select Pair call with other CH. This setting will have CH1 call CH2, and each of these smartphones will exchange voice clips back and forth. When these voice clips are exchanged, Umetrix Voice LM will measure the MOS voice quality for each voice clip.

5. From the Call pattern box, select O to set CH1 to be the originating phone. 6. Click the MIN button next to the Dial number box, and select CH2.

The telephone number for CH2 appears in the Dial number box. 7. From the Service options box, select 103. Bluetooth – Voice with Dial Number

to set the phone (CH1) to use Bluetooth to dial the phone number for CH2. 8. Check the Detect UE-UE Conversation check box. 9. From the Pair channel box in the Pair channel plan for <Sync. Call with other

UE> area, select CH2 to pair CH1 with CH2. The phone number for CH1 appears in the Dial number box. In the Channels area, the check box for CH2 is now checked, and >> appears next to CH1 and CH2 indicating that these channels are now paired together.


www.spirent.com 32

10. From the Service options box, select 103. Bluetooth – Voice with Dial Number to set CH2 to dial the phone number for CH1 using Bluetooth. By default, the Call count is set to 50, indicating that 50 calls you want made during this test. The Traffic time is set to 65, which specifies that the call will be active for 65 seconds.

11. In the Channels area, click the CH3 check box and repeat Steps 4 through 10 to pair CH3 with CH4. If you have more than 4 channels, repeat Steps 4 through 10 for the remaining pairs you want to configure.

12. After you have paired the other channels, select CH1 in the Channels area to set up the voice quality testing.

13. In the QoE measurement area, check the Voice quality check box and then click the Config button as shown in the following figure.

Figure 25. Location of Voice quality check box and Config button.

The Voice Quality Measurement Configuration dialog box appears.


www.spirent.com 33

Figure 26. Voice Quality Measurement Configuration dialog box.

14. From the Standard box in the Measurement Method area, select P.863 (POLQA 2.4 – SWB).

15. From the Audio path box, perform one of the following steps: If an audio cable is connected from the ear jack of the smartphone to the

AIM, select Ear-Jack. If an audio cable is NOT connected to the ear jack of the smartphone (that is,

the smartphone has no ear jack), select Air. (When this option is selected, Bluetooth is used to send and receive the audio clips.)

The Audio path box defines how the voice will be moved from the phone into the AIM, which the software uses to record the audio.

16. From the Rx Speech box in the Reference speech area, select swb_ref_48kHz.wav. This is the 7-second audio clip that will be sent. If you want to listen to the selected wav file, click the Play sound button located next to the Rx Speech box.

17. From the Tx Speech box in the Reference speech area, select swb_ref_48kHz.wav. This is the 7-second audio clip that will be sent. If you want to listen to the selected wav file, click the Play sound button located next to the Tx Speech box.

18. From the Clip Record box in the Wave File Save (Rx Clip) area, select Don’t Save to prevent the software from saving recordings of the 7-second audio clips. The other options allow you to save recordings of either all the audio clips during the test or only those audio clips that have a MOS that is lower than the threshold you specify.


www.spirent.com 34

19. Check the Secure measurement count check box in the Measurement Count Control area to define how many of the 7-second audio clips you want to measure. By default, 4 audio clips will be measured when the Secure measurement count check box is selected. You can specify the number of audio clips to be measured. For this test, the Traffic time is set to 65 seconds. This means there are 65 seconds for the entire call. During the test, a 7-second audio clip will be sent back and forth between the two phones, which adds up to 14-seconds of total audio. For this test, the software is set to measure 4 audio clips. This means that the software will measure 56 seconds of audio. Since the Traffic time for the entire call is set to 65 seconds, there will be 9 seconds of silence in the test after the 4 audio clips are measured. If you wanted to have longer calls (for example, a Traffic time of 600 seconds), and you enabled the Secure measurement count check box with 4 audio clips, the software would only measure the first 4 audio clips (which is 56 seconds), and then there would be 544 seconds of silence. If you are doing a long call, you may want to uncheck the Secure measurement count check box so that the software will keep measuring the audio clips throughout the duration of the call.

20. In the BQ & CBQ Condition area, check the Check BQ & CBQ check box. This check box enables you to classify certain calls as bad quality (BQ) and continuous bad quality (CBQ). By default, a call is classified as bad quality if it has a MOS of less than 2.20. You can change this MOS threshold using the BQ MOS threshold box. By default, continuous bad quality occurs when 4 or more calls have a continuous bad quality MOS of less than 1.90. You can change number of calls from 4 using the CBQ count box. You can change this MOS threshold using the CBQ MOS threshold box.

21. Check the Copy to all channels check box to apply these settings to all the channels.

22. Click the OK button. 23. In the Channels area, select CH2, and make sure the Voice Quality check box is

checked. 24. Repeat Step 23 for each channel you want to use. 25. Click the OK button. 26. Go to the next section to run the VoLTE voice call test.


www.spirent.com 35

Step 7: Run the VoLTE Voice Call Test In this section, you will run the VoLTE voice call test that you configured. Perform the following steps. 1. From the Control menu, select Mobile Control.

The Mobile Control window appears.

Figure 27. Mobile Control window.

Using the Mobile Control window, you will put the smartphones into Airplane mode and then turn off Airplane mode to restart the registration process. Several of the VoLTE networks use encapsulated security payload (ESP). To decode the SIP and RTP packets, Umetrix Voice LM must capture some information off the smartphone through the registration process. Enabling and then disabling Airplane mode forces a registration, allowing Umetrix Voice LM to capture the registration information needed to decode the SIP and RTP packets.

2. Under Attach/Detach in the left pane of the Mobile Control window, click on Airplane mode on.

3. From the Method box, select AT for each channel. 4. Click the Run button.

The AT command to put the phones in Airplane mode is issued to the selected channels. The status bar at the bottom of the window shows all of the phones go into No Service mode (NoSVC).

5. Under Attach/Detach in the left pane of the Mobile Control window, click on Airplane mode off.

6. From the Method box, select AT for each channel.


www.spirent.com 36

7. Click the Run button. The AT command to turn off Airplane mode is issued to the selected channels. The phones go back into network mode (for example, LTE). If the key has been successfully captured, K appears in the status bar.

8. Click the Close button. 9. In the Umetrix Voice LM window, click on All Channel on the View tab as shown

in the following figure.

Figure 28. Location of the All Channel option on the View tab.


www.spirent.com 37

10. In the left pane, click on Voice Call Monitor as shown in the following figure.

Figure 29. Location of the Voice Call Monitor option.

A Voice Call Monitor window appears for each channel.


www.spirent.com 38

Figure 30. Voice Call Monitor windows for 4 channels.

NOTE: If you only want to view the Voice Call Monitor for a specific channel, click on that channel on the View tab.

The Voice Call Monitor window provides a real-time view of the test data and allows you to see the status of the dialing and answering processes during the test.


www.spirent.com 39

11. From the Information menu, select Voice Quality Monitor. The Voice Quality Monitor window appears.

Figure 31. Voice Quality Monitor window for 4 channels.

The Voice Quality Monitor window displays the MOS and the status of the voice quality settings of the test.

12. Click the Start measurement button as shown in the following figure.

Figure 32. Location of the Start measurement button.

The Select folder & Input log file name dialog box appears.

Figure 33. Select folder & Input log file name dialog box.


www.spirent.com 40

The Folder box shows the location where the log files will be stored. The Log file name box allows you to name the log file for each channel. (Each channel has its own log file.) The application prepopulates the following information for each channel: phone make phone model phone IMEI The Role box enables you to specify whether the associated phone is a reference (REF) or a device under test (DUT). The Postfix box enables you to add an identifier to the end of each log file name. By default, each log file name has the time/date stamp when the test started, the channel number, and the make, model, IMEI, and role of the phone.

13. Click the OK button to start the test. The originating channels start dialing their pair channels, and the phones waiting to receive the calls are in a paging state. The Voice Call Monitor windows show the call status of the selected channels.


www.spirent.com 41

Figure 34. Voice Call Monitor displaying call data.

As soon as the calls become active, the phones start sending and receiving the audio clips. The following figure shows audio clips in the Voice Quality Monitor window.


www.spirent.com 42

Figure 35. Voice Quality Monitor showing audio.

The blue lines represent the received voice samples. The white lines represent the sent voice samples. The blue and white lines should be roughly on top of one another. For each audio clip, the following information is displayed: MOS attenuation (volume of the received sample) mouth-to-ear piece delay (which represents how long it takes a single word to

move from one phone to the other phone) The following figure shows the MOS, attenuation, and mouth-to-ear piece delay of an audio clip for one channel.

Figure 36. MOS, Attenuation, and mouth-to earpiece delay of an audio clip for

one channel.


www.spirent.com 43

You can click on the other tabs in the Voice Quality Monitor window to see different views of the test data. At this point, the calls will continue until the specified number of calls is reached. This test was configured to make 50 calls. All information from the test is saved to the log file for analysis at a later time.


www.spirent.com 44

3. Using Audio Tunes

Overview This section describes how to use Audio Tunes, which is a standalone application that is used with Umetrix Voice LM for audio quality tests. Audio Tunes helps you set the volume setting on the smartphones to achieve the best MOS for audio tests. Umetrix Voice LM will use these settings when you run audio quality tests on the smartphones.

NOTE: Spirent recommends that you run Audio Tunes before performing audio quality tests with Umetrix Voice LM.

Prerequisites Before using Audio Tunes, make sure:

• The Umetrix Voice LM components (including the smartphones you want to test) are installed, configured, and powered on.

• USB debugging mode is enabled on the smartphones.


www.spirent.com 45

Procedure Perform the following steps: 1. Make sure the Umetrix Voice LM application is NOT running. 1. From the Start menu, select UmetrixVoiceLM > Audio Tunes.

The User Account Control dialog box appears prompting you to allow this app to make changes to your device.

2. Click the Yes button. The Audio Tunes window appears and displays the Setting dialog box.

Figure 37. Settings dialog box.

3. Perform one of the following steps: If the smartphone is connected to the AIM via the audio cable, select

AIM (Earjack) from the PM Device box located at the top of the Settings dialog box.

If an audio cable is NOT connected between the smartphone and the AIM, select AIM (BT) from the PM Device box located at the top of the Settings dialog box.


www.spirent.com 46

4. Click the Refresh button. A dialog box appears prompting you to change the settings.

. Figure 38. Change Settings dialog box.

5. Click the Yes button. The Settings dialog box appears.

Figure 39. Settings dialog box showing 2 channels.


www.spirent.com 47

The Settings dialog box displays the following information for each channel (smartphone): Phone number model ADB serial number Speaker setting (the volume level sent from the smartphone) Line setting (the volume level coming into the smartphone) If this is the first time you are using one of the connected smartphones with Audio Tunes, a red icon appears at the top of the channel for that phone, indicating that the monitor volume app is not installed on the phone. Audio Tunes requires an app that runs on the smartphone and monitors the volume on that phone The following figure shows the location of the red icon.

Figure 40. Location of the audio app install button for a channel.

6. If the red icon appears at the top of a channel, click on the icon to install the monitor volume app on the phone. The smartphone will prompt you to allow permissions to install the app.

7. Allow the app to install on the smartphone. After the app is installed, a message box appears, stating that the installation was successful.

8. Click the OK button to close the message box. 9. Repeat Steps 7 through 9 for each channel that has the red icon displayed. 10. Click the Refresh button. 11. Make sure the Auto Mode button is selected at the top of the window.


www.spirent.com 48

12. Click the OK button. The Settings dialog box closes. The Audio Tunes window displays the telephone number for each channel.

Figure 41. Audio Tunes window.


www.spirent.com 49

13. Click the Play button located at the top of the Audio Tunes window. The Test Start dialog box appears.

Figure 42. Test Start dialog box.


www.spirent.com 50

14. Make sure the ON button is selected for Mobile Volume. By default, the Tune Key Step settings are configured to use the maximum volume on the smartphone (5 in the UP box) and then decrease the maximum volume by 1 (DOWN box).

15. If you want to change the Tune Key Step settings, specify the appropriate values in the UP and DOWN boxes. The Mos Test area enables you to tune for best MOS. The Mos Test Per Combination box enables you to specify the number of recording samples the Audio Tunes application will perform. During the test, the application is looking for combinations of play and record that give the device an attenuation between -1 and 1. (A MOS value of -1 and 1 is based on the POLQA specification.) When the application finds a combination of play and record that has an attenuation range of -1 and 1, Audio Tunes does 5 recording samples by default. After the 5 recording samples are complete, the application: drops the highest MOS value of the 5 recording samples drops the lowest MOS value of the 5 recording samples averages the remaining 3 MOS values The Play Volume Range boxes set the volume going out of the phone. The Record Volume Range boxes set the volume coming into the phone. The Volume Change Step box changes the record volume range. When the test first starts, Audio Tunes uses the specified Play Volume Range value and the Record Volume Range value. It will adjust the Play Volume but not the Record Volume until the -1 to 1 range is met. At that point, Audio Tunes will take the 5 recording samples. After the 5 recording samples are complete, Audio Tunes will adjust the Record Volume (using the Volume Change Step value) and then adjust the Play Volume to get to the range of -1 to 1. Audio Tunes continues this process until it completes all of the different combinations it can perform. To have Audio Tunes run more combinations, you must decrease the Volume Change Step. The Time Limit box specifies the amount of time you want the test to run.

16. In the Module area, click on the module (for example, POLQA 2.4) you want to use.

17. From the Rx Speech box, select the audio file you want the phone to receive. 18. From the Tx Speech box, select the audio file you want the phone to send.

Typically, you would select the same audio file you selected for Rx Speech.


www.spirent.com 51

19. Click the OK button. One phone makes a call to the other phone. After the call is established, the phones start playing the audio file to one another. The Audio Tunes window displays the attenuation, Volume, and MOS for the current play volume and record volume. The following figure shows the values for a test.

Figure 43. Values displayed during a test.


www.spirent.com 52

If you want to view the values of the current test, click the Report icon as shown in the following figure.

Figure 44. Location of the Report icon.

The Audio Tunes window displays a table view of the data for each channel. This table shows: Play volumes used Record volumes used Attenuation


www.spirent.com 53

The following figure shows a sample table view.

Figure 45. Sample table view for a test.

NOTE: The yellow rows indicate that Audio Tunes is scoring the 5 recording samples. It will drop the high MOS sample and the low MOS sample and then average the remaining 3 samples.

To view the data for a different channel, click on the CH icon at the top of the window, and then select the channel in which you are interested. If you select a row of data, the wave form appears in the Draw All area. If you want to listen to the selected wave form, click the Headset icon located on the right-side of the Draw All area.


www.spirent.com 54

Figure 46. Selected wave form displayed in the Draw All area.

When the test is complete (that is, either Audio Tunes has run all of the combinations within the ranges you specified, or the time limit you specified has been reached), Audio Tunes reruns the settings that had the best MOS during the test 3 more times. This appears in blue.

Figure 47. Blue rows indicating Audio Tunes is rerunning the settings for best

MOS.

When Audio Tunes is finished running the test, the Test Finished! dialog box appears.


www.spirent.com 55

Figure 48. Test Finished! dialog box.

The Test Finished! dialog box displays the settings where Audio Tunes achieved the best MOS value.

20. Click the Save To Default button to save these settings to the file MOS_MGR.ini. This is the file that Umetrix Voice LM will use when you run voice quality tests. A Warning dialog box appears, prompting you to confirm that you want to save the settings to the file MOS_MGR.ini.

21. Click the Yes button. The volume settings from the test are stored in the file MOS_MGR.ini. All previous volume settings in the file MOS_MGR.ini are overwritten with these new volume settings.


www.spirent.com 56

22. If you want to save these settings to a different file that you can use later: a. Click the Save To File button.

The Save to INI dialog box appears. b. In the File name box, type the name for the file. c. Click the Save button.

23. Click the Cancel button to close the Test Finished! dialog box. If you want to review the values that were run during the test, you can export that information to a Microsoft Excel file.

24. To export the test data to an Excel file: a. Click on the Excel icon at the top of the Audio Tunes window.

The Save to Excel dialog box appears. b. In the File name box, type the name for the file. c. Click the Save button.

The Complete!! Dialog box appears. d. Click on the Open Excel icon to open the Excel file you just created.

The Excel file appears and displays the information in the same format as the table view in Audio Tunes.

Figure 49. Test values displayed in an Excel file.

When you are ready to use these values with the selected channels in an audio test in Umetrix Voice LM, close Audio Tunes and then start Umetrix Voice LM. When Umetrix Voice LM starts, it automatically uses these values for the channels.


www.spirent.com 57

4. Integrating Umetrix Voice LM with Umetrix Data

Overview This section describes how to integrate Umetrix Voice LM with Umetrix Data. When the integration is complete, Umetrix Voice LM will automatically run the Umetrix Data test set you specify on the Umetrix Data Server.

Prerequisites To integrate Umetrix Voice LM with Umetrix Data, you must have:

• Umetrix Data Mobile Client for Android installed on the smartphone

• the Umetrix Data Mobile Client activation code you received from Spirent

• the configuration ID for the smartphone on the Umetrix Server (for example, the IMEI)

• the name of the project on the Umetrix Data Server

• the name of the Umetrix Data test set you want to run


www.spirent.com 58

Procedure Perform the following steps: 1. On the PC desktop, double-click on the Umetrix Voice LM icon.

The Umetrix Voice LM window appears. 2. From the Control menu, select Plan Setting.

The Plan Config dialog box appears. 3. Click on the Idle tab.

The Idle tab appears..

Figure 50. Idle tab in the Plan Config dialog box.


www.spirent.com 59

4. In the Channels area, select the channel you want to use. 5. From the Action Command box on the Idle tab, select Umetrix Data.

The Umetrix Data settings boxes appear.

Figure 51. Umetrix Data settings.

6. In the Activation box, type the Umetrix Data Mobile Client activation code you received from Spirent.

7. In the Configuration ID box, type the ID for the smartphone (for example, the IMEI).

8. In the Project Name box, type the name of the project on the Umetrix Data Server.

9. In the TestSet Name box, type the name of the test set you want to run. 10. Click the OK button.

The Umetrix Data Mobile Client app starts automatically and runs the test you specified.


www.spirent.com 60

To view the Umetrix Data results from Umetrix Voice LM, perform the following steps: 1. In the Umetrix Voice LM window, click on the View tab.

The View tab appears. 2. On the View tab, expand Common. 3. Under Measurement, select Umetrix Data Monitor as shown in the following

figure.

Figure 52. Umetrix Data Monitor option on the View tab.


www.spirent.com 61

The Umetrix Data Monitor window appears and shows the Umetrix Data results.

Figure 53. Umetrix Data Monitor window.


www.spirent.com 62

5. Using the KPI Report Tool

Overview This section describes how to use the Umetrix Voice LM KPI Report tool, which is a standalone application that imports the Umetrix Voice LM log files you specify, processes the data, and then puts the processed data into a Microsoft Excel template file. This Excel file provides MOS, relevant RF and IP metrics, and various high-level measurements that were made during the Umetrix Voice LM tests. The following figure shows a sample KPI report.

Figure 54. Sample Umetrix FIT report.

By default, the Device Overview tab is displayed for the report. The Device Overview tab provides the following information for each device:

• Make

• Model

• Position

• IMEI

• MSISDN

• Android version

• Start date (UTC)

• Start time (UTC)

• Test duration (hours)


www.spirent.com 63

Using the links provided at the bottom of the Device Overview tab, you can access the following analysis sheets in the KPI report:

• Report Dashboard, which provides a summary of KPIs from each channel

• MOS Analysis, which provides uplink and downlink results

• RTP Analysis, which provides RTP delay, jitter, and packet failure rate

• Call Performance Analysis, which provides initiation failures, setup time, and drops

• RF Analysis, which provides RSRP, RSRQ, SINR, Tx power, and BLER

• Handover Analysis, which provides RRC events and handover success rate

• MRAB Data Analysis, which provides call performance statistics during MRAB

• Serving Network Analysis, which provides the serving network in use

• CA Utilization Analysis, which provides carrier aggregation bands in use

• MIMO Utilization Analysis, which provides the MIMO scheme in use

• Call Performance Details, which provides the raw data for each call.

Prerequisites Before using the KPI Report tool, make sure Microsoft Excel is installed on the PC.


www.spirent.com 64

Generate a Report Perform the following steps: 1. From the Start menu, select Spirent Communications > Umetrix Voice LM KPI

Report. The User Account Control dialog box appears prompting you to allow this app to make changes to your device.

2. Click the Yes button. The Umetrix FIT Report window appears.

Figure 55. Umetrix FIT Report window.


www.spirent.com 65

3. Click the Add Files button. The Open dialog box appears.

Figure 56. Open dialog box.

4. Select the log files (.dml files) in which you are interested, and then click the Open button. The KPI Report tool determines the channel on which the data was collected for each log file and displays that information in the Umetrix FIT Report window.

Figure 57. Selected log files displayed in the Umetrix FIT Report window.


www.spirent.com 66

5. Click the Make FIT Report button. The Save As dialog box appears.

Figure 58. Save As dialog box.

6. Specify the name for the report and the location where you want to save it. 7. Click the Save button.

The Importing log files status box appears showing the status of the report generation process. The KPI Report tool imports the log files, processes the data, and then puts the processed data into a Microsoft Excel template file.

NOTE: Depending on the size and number of log files being processed, it could take a long time to generate the report. For example, if you are processing 4 hours of data for 12 channels, this process could take up to 30 minutes.

After the report is generated, OK appears next to the processed logs in the Umetrix FIT Report window as shown in the following figure.


www.spirent.com 67

Figure 59. OK message displayed in the Umetrix FIT Report window.

8. Using Windows Explorer, go to the folder where the report is located, and then double-click on the report file. Microsoft Excel displays the selected report. The Device Overview tab is displayed.

Figure 60. Sample Umetrix FIT report.

9. Click on the links provided at the bottom of the Device Overview tab to access the analysis sheets in the KPI report.


www.spirent.com 68

Sample Analysis Sheets This section provides sample analysis sheets for a KPI report.

Report Dashboard The Report Dashboard provides a summary of KPIs from each channel.

Figure 61. Sample Report Dashboard.


www.spirent.com 69

MOS Analysis The MOS Analysis sheet provides uplink and downlink results.

Figure 62. Sample MOS Analysis sheet.


www.spirent.com 70

RTP Analysis The RTP Analysis sheet provides RTP delay, jitter, and packet failure rate.

Figure 63. Sample RTP Analysis sheet.


www.spirent.com 71

Call Performance Analysis The Call Performance Analysis sheet provides initiation failures, setup time, and drops.

Figure 64. Sample Call Performance Analysis sheet.


www.spirent.com 72

RF Analysis The RF Analysis sheet provides RSRP, RSRQ, SINR, Tx power, and BLER.

Figure 65. Sample RF Analysis sheet.


www.spirent.com 73

Handover Analysis The Handover Analysis sheet provides RRC events and handover success rate.

Figure 66. Sample Handover Analysis sheet.


www.spirent.com 74

MRAB (MultiRAB) Data Analysis The MRAB (MultiRAB) Data Analysis sheet provides call performance statistics during MRAB.

Figure 67. Sample MRAB Data Analysis sheet.


www.spirent.com 75

Serving Network Analysis The Serving Network Analysis sheet provides the serving network in use.

Figure 68. Sample Serving Network Analysis sheet.


www.spirent.com 76

CA Utilization Analysis The CA (Carrier Aggregation) Utilization Analysis sheet provides carrier aggregation bands in use.

Figure 69. Sample CA Utilization Analysis sheet.


www.spirent.com 77

MIMO Utilization Analysis The MIMO Utilization Analysis sheet provides the MIMO scheme in use.

Figure 70. Sample MIMO Utilization Analysis sheet.


www.spirent.com 78

Call Performance Details The Call Performance Details sheet provides the raw data for each call.

Figure 71. Sample Call Performance Details sheet.

Documents

Umetrix Voice LM...• A PH12 power USB hub, which controls, collects data from, and charges up to 12 smartphones. The PH12 can also interface with a GPS puck and an RF scanner. •