04-Keynote SIGOS New Perspectives for Customer Experience Testing

Keynote SIGOS GmbHKlingenhofstrasse 50d90411 NurembergGermanywww.keynote-sigos.com

Diversity of SITE

Date: 30th October 2008

New Perspectives forCustomer Experience Testing

©2008 Keynote SIGOS

Testing is our Competence

2

Selected SITE Functionality

Iu-PS

Iu-CS

Core Network Testing in GSM & UMTS

WLU

SITERecorder

GIS Interface

Wireless Service Quality Monitoring

Drive Service Testing

Handset SimulationMLU



3


Iu-PS

Iu-CS




Handset Simulation



4


Functionality

Emulation of entire radio access networks plus mobile terminals

Active simulation of up to 30 subscribers / LU

Support of all relevant SITE Test Applications

Perfectly geared for testing services with crucial network components in core network

Benefits

Significantly increased test performance compared to mobile probese.g. testing time f. attach/detach decreases from appr. 1 min to 3 secs

Air interface problems do not affect testing results

KPIs as well as faults can be isolated to pure core network

NEW



5

Core Network Testing in UMTS

Functionality

State-of-the-art technology: IP

Support of packet and circuit switched services

Protocol conformance checks

Enhanced Protocol decoding capabilities

New Hardware: 4-port Iu board for both ps & cs

Benefits

Early Availability of Iu ps & Iu cs over IP testing solution

Erroneous messages from the core network can be detected

Entire protocol stack can be traced



6


WLU



Handset Simulation




7

Functionality

Permanent IP connection established via GPRS/EDGE/HSPA modem

New LU: Wireless Local Unit (WLU)

WLU upholds GPRS connection with IPSec tunnel

Router gets private IP address and represents WLU

Up to 3 probes for concurrent tests

Benefits

Fast installation of test probes for fault isolation

No Dependency on wired IP connections

No fixed IP address required

Same look ‘n’ feel for LU, CLU and WLU

Synchronized tests between WLUs and LUs possible

WAN

IP connection

CU/SIMMUX

No IP connection

x

Router

IPsec

WLU

Mobile Network

Virtual SIMcard transfer

GPRS/HSPA




8

Prerequisites

General availability of mobile packet network (GPRS/EDGE/UMTS/HSPA) -> Ideally UMTS/HSPA

One probe (MC8775) solely installed for communication -> Not available for SITE test cases

Number of parallel tests determined by actual bandwidth: -> 20 kbit/sec uplink per interface required -> Main bandwidth consumers are traces and result files

Round trip delay of less than 1,5 seconds

Firewall settings for WLU recommended for security reasons

One SIM / interface to maintain permanent wireless IP connection

Geared for wireless deployment; not suitable for moving subscribers

WLU = Portable local unit for flexible deployment in the field




9


GIS Interface



Handset SimulationMLU




10

Drive Service Testing Concept

SIM Multiplexer

Clients

Central Unit

Mobile LU in car

Mobile LU in train

Geographic Information System (GIS)

LAN/WAN

Static LUPermanent IP ConnectionNon-Permanent IP Connection

Customer’s Map Server

GPS Position

TestCases / KPIs for moving subscribers performing end to end services in addition to static subscriber probes

Introduction of moving subscriber probes: Mobile Local Units (MLU)

Information displayed on a brightly coloured geographic map (GIS)

Geared for vehicles like taxis, trucks or trains



11


Functionality

MLU connected to wiring system of moving vehicle

MLU automatically started with vehicle engine ignition

After shut down of car engine MLU keeps running for 15 minutes to avoid too frequent booting

Fully automated test cases, even beyond the automation grade of previous SITE System

Every test case initiates powering up and booting of relevant wireless probes

Up to four probes can perform test cases simultaneously (3 recommended)

During entire test drive test case keeps running in cyclical mode

Test case setup done via common SITE test case editor

Non-permanent IP connection between MLU and Central Unit (via e-mail server)

Retrieval and sending of e-mails performed every 15 minutes (Proposed value)



12

Drive Service Testing GIS InterfaceFunctionality

Graphical presentation on your network quality performed by SITE Geographic Information System (GIS)

Rich set of KPIs diplayed on geographic map according to GPS data recorded for each Drive Service Test Case

WMS maps, e.g. operator’s own coverage map, display KPIs customizable by operator

MapInfo, serves as the GIS engine for SITE Drive Service Tests

Support of maps according to Web Map Service (WMS)

Display of the KPIs can be customized by operator

Periodicity of map refresh configurable

Graphic views export in JPEG format



13

non-permanent IP connection via e-mail

Central Unit

E-Mail Server

MLU

permanent IP connection

Drive Service Testing Mobile Local Unit

Functionality

MLU based upon CLU

Non-permanent IP connection

Communication between SITE Central Unit & MLU performed by transferring data via the e- mail server

E-mail server sends e-mail responses (result data, status messages) and receives test jobs

One probe used for UMTS/GPRS communication with the SITE Central Unit

Same air interfaces supported as for CLU

Duplication of some central software functions: Scheduler, Resource Manager and Test Execution Manager also implemented in MLU

Fixed SIM in MLU

Tested for off-road usage



14

MLU Random Vibration Test

Same standard random vibration test as performed for electrical and electromechanical components of MAN vehicle cabs (Random IEC 60068-2-64)

6 hours of random vibration testing in the frequency range between 10 and 2000Hz

MLU fully operational during vibrations; all probes equipped with SIM-Card & connected via Ethernet to master system

2 hours of vibrations in X-axis 2 hours of vibrations in Y-axis 2 hours of vibrations in Z-axis



15

Drive Service Testing Key Benefits

Autonomous System -> No engineer needs to be present in the vehicle

Easy Installation and Removal -> Once MLU connected to wiring system, no further human interaction required

Enhanced Service Assurance -> Independent view of end-to-end quality for static & moving customers

Network Optimization -> Troubleshooting & optimization for new technology networks

Benchmarking -> Comparison with quality of competitors’ networks

Test Cases -> Issues of most customer concern covered

Geographic View -> Challenging set of KPIs displayed on map

Interfaces -> All CLU SITE interface probes supported

Investment Protection -> Extension to existing SITE system securing current investment into Keynote SIGOS service assurance products



16


SITE Recorder



Handset Simulation

Handset Simulation



17

Handset Simulation Network / SITE Evolution

Service Tests with fixed Handset characteristics

Handset Simulation

Interactive RecorderIP, MMS, WAP Service

Handset specific servicesRingtones, Music DL,

Games

Complex Web/WAP transactions



18

Handset Simulation

SITE Testcase

Emulation of UAProfiles

Application Server

Check UAProfiles

Content

Request URL

Send MMS

Response Content

Push Content

Content Validation

against Handset capabilities

Flexible WTPHTTP

protocol stack

Handset library contains UAProfiles and capabilities of more than 1000 devices

Handset simulation does NOT test real devices or the original handset browser

Functionality

Test of specific handset characteristics against network services

Device specific monitoring of pages with fixed content/ URLs

Test cases for WAP and MMS monitoring as well as Web browsing

Compliant to ETSI STQ- Mobile



19

Handset Simulation Content Validation

Keyword Validation Search for defined words or phrases within the downloaded content

Color Support Check if device is capable of displaying colored images

Image Size Comparison of all downloaded images with maximum height and width of the device

Deck Size Comparison of downloaded size item with deck size of device (WAP1.x)

Content Support Check (based on HTTP/WTP accept header) if device can decode downloaded content.

Data Format Check if the downloaded content is valid



TC- Editor with

Template TestcaseTC- Editor with

Template TestcaseBrowser Browser

TraceTrace

Handset Simulation SITE Recorder

Functionality

Intuitive creation of browsing test cases

New frame test case and Recorder GUI



21

Handset Simulation SITE Recorder Integration in SITE

SITE Test EditorSITE Test Editor

SITE Test Editor SITE Recorder

SITE Test Case Editor

Save Browsing Session as new test

case in TC-Editor



22

Handset Simulation SITE Recorder

Benefits

Intuitive Recording of browsing actions-> No drag ‘n’ drop

Support of HTML forms

Easy creation & adaptation of browsing sessions

SITE integrated

Received page content at a glance

KPIs for each browsing action

User defined measurements / KPIs can be added with TC-Editor



Intelligent KPI MonitoringDashboard & Service Area Monitoring

Conclusion

Diversity of SITE offers new perspectives for Customer Experience Testing

Core NetworkEfficient Short-cut solutions

Handset SimulationCharacteristics of mobiles

MobilityDrive Service Testing

Revenue AssuranceCDR & billing verification

Portable UnitsWireless Service

Quality Monitoring



24

Thank [email protected]

Documents

04-Keynote SIGOS New Perspectives for Customer Experience Testing