Real Time Test Metric (RTTM) Getting Started Guide · Web viewThese are examples of what you would see in the case of the operations current multi-Radio Access Bearer (RAB) test set

Spirent

Communications Nomad

Real Time Test Metric (RTTM) Getting Started Guide

Spirent Communications5280 Corporate Drive, Frederick, MD. 21703

Document Version: 1.0

Contents1 Introduction........................................................................................................................................................................ 3

2 Setting up Nomad and Datum to collect Real Time Test Metric (RTTM) data.......................................3

2.1 Pre-Start Requirements........................................................................................................................................3

2.2 Administrative users configuring Datum test sets...................................................................................3

3 Test Set Configuration Recommendations............................................................................................................4

4 Collecting RTTM data......................................................................................................................................................4

FiguresFigure 4-1 - Entering your Datum Device Code into Nomad...................................................................................4Figure 4-2 - Device information...........................................................................................................................................5Figure 4-3 – Phone number settings..................................................................................................................................5Figure 4-4 – Complete phone number settings.............................................................................................................6Figure 4-5 – Bluetooth Device Control..............................................................................................................................6Figure 4-6 – Test Setup Tab................................................................................................................................................... 7Figure 4-7 – Current Channel Review................................................................................................................................7Figure 4-8 – Start logging Session Wizard.......................................................................................................................8Figure 4-9 – File Path................................................................................................................................................................ 8Figure 4-10 – Session Duration............................................................................................................................................9Figure 4-11 – Incomplete........................................................................................................................................................9Figure 4-12 – Finished Test.................................................................................................................................................10Figure 4-13 – Generate Reports........................................................................................................................................10

Copyright © Spirent Communications, Inc. 2014 2

1 IntroductionThis publication describes configuring Nomad and, your Datum client, as well as collecting RTTM data, along with analyzing and troubleshooting the test results.

2 Configuring Nomad and Datum to collect Real Time Test Metric (RTTM) data

2.1 Pre-Configuration RequirementsBefore starting a project, verify:

You received your Spirent license files were copied to your program directory C:/ProgramFiles/SpirentCommunications/Nomad.

Your Datum activation code has both RTTM logging and extended RTTM logging enabled. To verify this, create a test, run it with an activation code, view the first RTTM message and view the value codes. Note that content does not matter in this test. If there are 32 values you have the proper activation code with extended RTTM.

Your Nomad license has the RTTM logging right enabled. To verify this, launch Nomad, create a task, and open the device information page to verify the system mentions a device code. If the device code field displays, you have the RTTM logging rights enabled.

Your Nomad license file is pointing to the correct instance of Datum base. With the release of RTTM integration, Nomad talks to an assigned Datum base instance. Currently, only one Datum base instance is allowed per license. To verify that you are pointing to the correct Datum base instance perform the following actions. Locate a device code from the Datum base instance you wish to test against and enter it into the device code field in Nomad.

You received the latest version of the Datum Android Client. Obtain this from the Google Playstore.

Contact technical support if any of these procedures fail.

2.2 Administrative users configuring Datum test sets The user site administrator will configure the test set and the field engineers will collect Extended RTTM while running any supported Datum Android task. Standard RTTM data can be collected while running any supported Datum Android, iPhone, or Windows Phone task.

Datum now contains a Diagnostic Logging test set, which incorporates the MDL standalone features into the Datum Android client. This test set provides a UI display of data the system collects during a test; but, a user can also collect extended RTTM data while running other Datum Android tests.

WARNING: When a test set is running on the device and it is between two tasks, there may be a gap of more than one second between RTTM. Accordingly if you configure the test set to run for multiple cycles or upload, the system will not collect RTTM data between those cycles.

The following are examples of how the RTTM data collection results appear in the Datum log file and Nomad exports. These are examples of what you would see in the case of the operations current multi-Radio Access Bearer (RAB) test set. The test set contains:


A 20 second wait HTTP download 60 second wait

The system typically collects RTTM as

During these 20 seconds wait task the system will log RTTM data. Between the 20 second wait task and the start of the http download test, the system does not log RTTM data. This is typically one to two seconds in length. When the http download task runs, the system collects RTTM data again. Between the http download and the next wait task, the system will not log RTTM data. This is typically one to two seconds in length.

After the wait task ends, the system uploads the test set results log file to Datum base. During this time the system collects no RTTM data. Typically this takes from several seconds to more than a minute if a user is attempting to upload in poor network conditions.

3 Test Set Configuration Recommendations In a future Datum release, Spirent will enable you to collect RTTM data between tasks and when uploading a test results log file. Until this enhancement is implemented, however, the following suggestions are offered for administrators:

Datum Android contains a feature that disables uploading test result log files until testing is complete. This is an activation configuration change; please contact support to get your activation configured to turn on this feature. If this feature is turned on, you can turn off automatic uploads and decrease the delay between cycles as you do not need to wait for the file to upload.

For only the RTTM data, create a test set that includes only the new Diagnostic logging task and set it to either stop manually or set a time of 300 seconds or more. This limits the number of uploads performed during the test, so you get the largest number of RTTM samples.

For both RTTM data and a Datum specific task, create a test set with multiple tests instead of just two wait tasks and one http download task. Create a test set containing ten waits and five http download tasks to reduce the number of uploads that take place.

4 Collecting RTTM data Use Datum to collect Real Time Test Metric (RTTM) data directly from the device during Nomad testing. After the test is executed, the system merges the RTTM data collected by Datum with the Nomad test results. To collect RTTM information with a Nomad test, perform the following steps:

1. Verify Datum version 2.7.0.40 or above is installed on the Datum handset being tested.


Figure 4-1 - Entering your Datum Device Code into Nomad

Warning: Configure Nomad as you previously configured it. Failure to do so stops the system from running. Once configured, from Nomad open the device information page and a screen like the following displays.

Figure 4-2 - Device information

2. Enter the Device Code displayed in the Datum User Interface (on the handset) into the Device Code text box on the Nomad configuration screen. Note that Nomad now fills in the hardware data and Datum interface data after you enter a correct device code.

3. Return to the Datum handset and start the test by clicking the start bar on the bottom of the handset display. Return to Nomad, click Next, and the following screen displays.


Figure 4-3 – Phone number settings

4. Enter the number you are calling and the number you are calling from (handset). Click OK and the following screen displays.

Figure 4-4 – Complete phone number settings

5. Click Next and the following screen displays.


Figure 4-5 – Bluetooth Device Control


Figure 4-6 – Test Setup Tab

7. Click Start Logging Session and the following screen displays


Figure 4-7 – Current Channel Review


Figure 4-8 – Start logging Session Wizard


Figure 4-9 – File Path


10. Create the new file every time you run this option. Once the new file is accepted, clicking Next will start the Nomad test. Always start the test on the Datum handset prior to starting the Nomad test. Datum tests measure RTTM data in the background, and automatically collect this data and make it available for correlation within Nomad. When the new file is accepted, and as soon as you have started the Nomad test, click Next and the following screen displays.

Figure 4-10 – Session Duration

Warning: For Datum, always skip the current task before cancelling the Datum test. Cancelling the Datum test renders the data in the current test cycle unusable.

11. During the test phase, monitor Datum to verify Datum is running your test set. This process is the same process you would perform for any other test.

12. On the Datum handset verify all the Datum log files successfully uploaded to the server. Count and take note of the number of Datum log files sent to the server. This count can be useful for matching the RTTM data later, should you encounter any missing data.

13. When stopping the tests, always stop Nomad first followed by Datum as quickly as practical. An incomplete screen will look like the following.


Figure 4-11 – Incomplete

14. By stopping Nomad first, the system collects the same amount of RTTM and Nomad data. Failure to do so can create incomplete data and users will be required to force complete data to generate the report.

15. When the tests finish, a screen like the following displays.

Figure 4-12 – Finished Test

16. Once Nomad stops, Nomad downloads all RTTM data from Datum base and merges it with the Nomad session data. Sometimes this process can take several minutes to complete, so if you are using a slow network connection, you may need to retry auto complete multiple times or complete the data in the office when the connection is faster.

17. Once complete, you can now generate a report by placing a check mark in the file name and clicking Generate Report From Selected... When the reports are finished, the following screen displays.


Figure 4-13 – Generate Reports

18. Two reports appear in the Data>Reports>Report tab. View standard is the traditional Nomad report. View RTTM is a new report that includes RTTM data and Nomad data. This Excel output also shows the RTTM data and Nomad data per channel broken out by Nomad

19. To view these files click the link under Report and the report displays.


Documents

Real Time Test Metric (RTTM) Getting Started Guide · Web viewThese are examples of what you would see in the case of the operations current multi-Radio Access Bearer (RAB) test set