1 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller Test Implementation using TTCN-3 Libraries TTCN-3 User Conference,

1 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller

Test Implementation using TTCN-3 Libraries

TTCN-3 User Conference, 7th of June 2005

Stephan SchulzValidation & Testing Technologies

Computing Architectures LabNokia Research Center

Helsinki

Sebastian MüllerFSCOM

Sophia AntipolisFrance


Presentation Outline

• About the ETSI IPv6 Library

• Motivation

• TTCN-3 Libraries

• Library-based Test Suite Implementation

• An Example TTCN-3 Library: ETSI Common

• Conclusions


About IPv6 Library

• Implemented at ETSI by STF 276• Conformance and interoperability test specs for IPv6

• Joint e-Europe/ETSI project• Promote deployment of IPv6 technology within Europe• Establish European player in IPv6 testing

• Developed to support IPv6 Logo program by IPv6 Forum

• Provide faster access to IPv6 market

• Test implementations are publicly (open+free) available• Insure high deployment• Make tests transparent


Motivation

• Common problem in TTCN coding is the “copy & paste” mentality of test engineers

Lots of duplicated code + large code size = hard to maintain

• Another problem: “The reinvention of the wheel“

• We desire a test suite implementation which allows as much reuse of TTCN-3 code within as well as between different test suites as possible

Much of the code developed for the IPv6 Logo certification should be reusable also in a IPv6 test suite for, e.g., 3GPP!

• A test suite should also be useful for additional test case development by other, external parties


The Idea

• Create a test suite using TTCN-3 libraries which can then be reused to compose also other related test suites

So just use TTCN-3 like a regular programming language!

• Ability to reuse depends on making important TTCN-3 code structuring decisions early on in test suite development

Strong need for a sound methodology!


Benefits of TTCN-3 libraries

• Less test suite specific TTCN-3 code

• Less room for errors in test implementations

• Use of validated TTCN-3 code segments

• “More readable” test implementations

• Less time needed for test case implementation and validation

• Avoids the “reinvention of the wheel”

Summary: Higher quality test suite implementations!


Keys to Reusable TTCN-3 Code

• Follow the TTCN-3 Framework• Methodology for Test Suite Implementation• Modularization• Rules on how to implement TTCN-3 behavior• Naming conventions

• Specify well defined interfaces • Use component type compatibility

• Document TTCN-3 code• Standard practice in any programming language• Free documentation tools are available


Libraries in TTCN-3

• A TTCN-3 library consists of modules separating • behavior(s)• types and constants• templates

• Example: Libraries in the IPv6 test suite• Common IPv6 independent code (e.g, Synchronization)• Common IPv6 code (e.g., Echo Procedure)• Common IPv6 test suite code (e.g., Ipv6 Host Preamble )But then (not reusable):• Specific IPv6 test suite code (e.g., IPv6 Core Test Cases)


Basic Structuring of IPv6 Test Cases

• Decompose test case behavior based on well established methodology

• Configuration (CF) function• for test configuration set up

• Preamble (PR) function• get IUT into testing state

• Test body (TP) function• perform test as specified by test purpose

• Postamble (PO) function • return IUT to its initial state

All of the above implemented by invoking test suite, IPv6, or common library functions!

• Implement test case or test case (TC) functions by simply invoking PR, TP, PO functions


testcase

(PR) function

(TP) function

(PO) function

(MTC)

(TC

) function(PR) function

(TP) function

(PO) function

(PR) function

(TP) function

(PO) function

(TC

) function

(PTC1) (PTCn)

testcase

(MTC)

create/start

preamble sync

TP sync

TC functions in Single and Multiple Interfaces

•Specify test case (TC) functions for each test component•Use MTC in concurrent test cases only to spawn and synchronize PTCs

No Test case (TC) functions needed


Rules for reusable TTCN-3 Behavior

• Function should not contain any stop statement since it prevents the execution of postambles!

• Test suite independent functions should avoid to use the setverdict operation

• Instead use the return value of a function!• Verdict setting harms reuse, e.g., testing of valid versus invalid behavior

• A module which defines reusable behavior should contain a test component type definition tailored its needs


About TTCN-3 Code Documentation

• Without proper documentation reuse is questionable• Documentation of library code is well established standard

practice in other programming languages (e.g., man pages)

• It should be possible to use a TTCN-3 library or modules without ever reading source files

• Any TTCN-3 code should be documented regardless of its reusability!

• Guidelines part of framework document

• A free documentation tool t3doc is available:• Follows ideas of similar tools for C and Java, e.g., doxygen• Generates HTML pages from TTCN-3 code comments• WWW tool download: www.farodyne.com\t3doc


A Library Example: ETSI Common

• Three modules with common type and value definitions

• Common basic, data string and text string constant and type definitions, e.g., c_CRLF, UInt, etc.

• Module for verdict control handling• Definition of function return code and verdict setting

functions

• Module for timing related functions, e.g., f_sleep

• Modules which implement a generic test component synchronization as well as sync examples

• Our classic “reinvention of the wheel”


The Synchronization Library Module

• Implements an easy-to-use synchronization handling for both single as well as multi component test cases

• Key concepts• Named synchronization points• TTCN-3 default used for handling shutdown

• Uses extensively TTCN-3 log statement• Results in more readable execution traces!

• Current main user API:f_selfSync(syncPointName, retCode)

f_serverSyncClients(#, syncPointName)f_serverSyncClientsTimed(#, syncPointName, time)f_serverWaitForAllClientsToStop()f_clientSync(syncPointName, retCode)[a_dummyShutDown()] // shutdown example


Example of Nice Multi-Component Sync

f_tc1_ptc2()

(PTC2)

f_tc1_ptc1()

(PTC1)

f_PR_tc1_ptc1()

f_TP_tc1_ptc1()

f_PO_tc1_ptc1()

tc1()

(MTC) create/start PTC1 and PTC2

f_serverSyncClients(2,“prDone”)

f_serverSyncClients(2,“tpDone”)

f_serverWaitForAllClientsToStop()

f_clientSync(“prDone”,e_success);

f_clientSync(“tpDone”,e_success);

f_PR_tc1_ptc2()

f_TP_tc1_ptc2()

f_PO_tc1_ptc2()


f_clientSync(“tpDone”,e_success);

Ready! Ready!

Ready. Ready.Ready!Ready!

Ready. Ready.


Example of Sync Problem Handling

f_tc1_ptc2()

(PTC2)

f_tc1_ptc1()

(PTC1)

f_PR_tc1_ptc1()

tc1()

(MTC) create/start PTC1 and PTC2

f_serverSyncClients(2,“prDone”)

f_clientSync(“prDone”,e_error);

f_PR_tc1_ptc2()


STOP! Ready!

STOP!STOP!

f_wait4AllClientsToShutDown()

a_shutDown_if1()(calls stop!)



Example of Self Synchronization

• Note that shutdown defaults are neither test case nor component but only interface specific (e.g, IPv6, SIP, etc.)

• Therefore the same default can be used to shutdown any test component (which uses that interface) in any test case.

tc2()

f_PR_tc2()

f_TP_tc2()

(MTC)

f_selfSync(“prDone”,e_success);

f_selfSync(“tpDone”,e_error);STOP!



Conclusions

• This really works in practice• The ETSI common TTCN-3 library has already been

successfully deployed within STF 276 as well as a number of different protocol test suites within Nokia

• First versions of the generic IPv6 and IPv6 Core libraries have been successfully used in the execution of the ETSI IPv6 test suite prototype against a Linux Ipv6 SUT

• At the end the real key to implementing truly reusable TTCN-3 code are

• TTCN-3 framework• Well-defined interfaces• TTCN-3 code documentation


References

• http://www.ipt.etsi.org• More info about this STF and documents• Test suite and executable download

• ETSI DTS/MTS-IPT-0001, “Methods for Testing and Specification (MTS); Internet Protocol Testing (IPT); IPv6 Testing: Methodology and Framework”, Sophia-Antipolis, 2005.

• S. Moseley et al., “IPv6 Test Specifications from ETSI”, Global Ipv6 Summit, Barcelona, (to appear), June, 2005.


About Component Type Compatibility

type component Ipv6Comp { IpPort pt_ip; SyncPort pt_sync; timer t_sync;}

type component SyncComp { SyncPort pt_sync; timer t_sync;}

function f_clientSync(…)runs on SyncComp { // some code}

function f_PR_EchoProcUp(…)runs on Ipv6Comp { var FncRetCode v_ret; v_ret := f_echoProc(); f_clientSync(“prDone”, v_ret);}

• Key to create self contained libraries is the runs on clause with TTCN-3 functions

• An IPv6 preamble may invoke, e.g., a synchronization function, if its component type definition is compatible

Documents

1 © NOKIA Test Implementation using TTCN-3 Libraries.ppt/ 07.6.2005 /S. Schulz and S. Müller Test Implementation using TTCN-3 Libraries TTCN-3 User Conference,