Test automation analysis in System Testing for the Intelligent Packet Core Author: Mona Saxena Supervisor: Professor Jörg Ott Nokia Networks, Helsinki

Test automation analysis in System

Testing for the Intelligent Packet

CoreAuthor: Mona Saxena

Supervisor: Professor Jörg Ott

Nokia Networks, Helsinki

22.5.2007 Mona Saxena Test automation analysis for System Testing in the Intelligent Packet Core

2

Agenda

• Introduction• FlexiISN in Nokia Intelligent Packet Core • FlexiISN Interfaces, Protocols, Features and

procedures• Software development process and testing phases

• Research problem

• Research method

• Results

• Conclusions


3

Nokia Intelligent Packet Core


4

FlexiISN Interfaces, Protocols and Features

IPN/W

GaRadius/Diameter

GTPGn

FlexiISNSGSN

CG OSC NPS

Radius

/

LDAP

Internet

Prepaid Charging

Postpaid Charging

ContentService

Providers

ApplicationService

Providers

Gi

NetAct

SNMP

TA

Gi

WAP

CorporateIntranet

PoC


5

FlexiISN SW development process and testing phases

Network or System Requirement

Customer or Business Requirements

Product Requirement Phase

Architecture Design Phase

Functional Specification Phase

Design and Specification Phase

Source code implementationand unit testing

System Verification and Integration Phase

Acceptance Testing Phase

System Testing Phase

Functional Testing Phase

Product Integration Phase

Module Testing Phase

System Level Program

Product Level Program


6

Research problem

Software has to be tested thoroughly and requires lot of time and effort. Therefore a significant amount of time and available resources could be saved if some aspects of testing could be automated.

Which areas could be automated and how they could be automated and their benefit before the product is delivered to the end customer? How much time it could take with the necessary available tools to automate the system testing related configurations and procedures in FlexiISN and related counterpart network elements of the Intelligent Packet Core (IPC)?


7

Research method

• Participating in FlexiISN System testing

• Literature study of Intelligent Packet core network elements, protocols and procedures

• Literature study of software testing and software test automation

• Discussions with other Service Access testing teams and collecting and analyzing their automation details.

• Test Automation Analysis based on actual number of test cases for different counterpart network elements and functionality areas along side FlexiISN


8

Test automation metrics

• Maintainability

• Efficiency and its relation to cost

• Reliability of an automated testing regime

• Flexibility

• Usability

• Robustness

• Portability


9

Test Automation analysis

• Search for actual number of test cases along with involved counterpart network elements or functionality areas

• Discovered what network element or functionality area has the maximum test cases in all the product releases

• Discovered what network element or functionality area has the minimum test cases in all the product releases

• Search on benefits to specific areas of testing with involved counterpart network elements or functionality areas

• Pre-Configuration• Collection of results post execution• Analysis of results post execution


10

Test automation analysis results

• Based on actual number of test cases along with involved counterpart network elements or functionality areas the highest priority for automation is towards test cases involving OSC and CG.

• Based on benefits to specific areas of testing with involved counterpart network elements or functionality areas

• For Pre-configuration maximum benefit was for test cases using OSC, NPS, HA feature and LIG

• For collection of results maximum benefit was for test cases using CG, HA feature and LIG

• For analysis of results the maximum benefit was for test cases using CG followed by LIG


11

Test automation implementation for System Testing

• Order of implementation of automation: Based on test automation analysis

• Method of automation implementation• Use previously available scripts from other testing

teams• Use scripts from System Verification in particular as

the tasks taken by them is quite similar.

• Tool setup for Automation Implementation• LASS Test Automation setup

• Hardware Environment need for automation• Laptops for individual testers with USB ports for

mobile phone /Handset connectivity• Red Hat 8.0 Server for script storage and version

control


12

Test automation implementation for System Testing

• Effort Estimation for Test Automation Implementation

• Competence and Training needs for Automation implementation

• Maintenance of Test Automation for the System Testing team


13

Conclusions and future work

• Test automation is an effective way for testing in a fast and efficient manner.

• Everything that needs to be tested cannot be and should not be automated. Automation in testing depends upon software feature implementation and its benefit for a particular test case scenario.

• The most important future work is to have the actual test automation system in place and then analyze the overall benefits of automation for the System Testing team.


14

Thank You!

Questions?

Documents

Test automation analysis in System Testing for the Intelligent Packet Core Author: Mona Saxena Supervisor: Professor Jörg Ott Nokia Networks, Helsinki