The Use of TTCN-3 for Software Testing Ina Schieferdecker FOKUS, Berlin [email protected]

The Use of TTCN-3for Software TestingThe Use of TTCN-3for Software Testing

Ina Schieferdecker

FOKUS, [email protected]

ITU-T Workshop, Geneva, Nov. 2002 I. Schieferdecker: TTCN-3

Content

• Overview on TTCN-3• TTCN-3 applied to software• TTCN-3 and XML• Test Execution


Content

Overview on TTCN-3• TTCN-3 applied to software• TTCN-3 and XML• Test Execution


Introduction• TTCN-3 is the Testing and Test Control Notation • The new standardised test specification and test

implementation language• Developed from 1999 – 2002 at the European

Telecommunications Standards Institute (ETSI).• Developed based on experiences from previous TTCN

editions• Removal of OSI specific concepts; Improvement of concepts;

Introduction of new concepts.• Applicable for all kinds of black-box testing for reactive

and distributed systems, e.g.,• Telecom systems (ISDN, ATM, GSM, UMTS); Internet (IP, IP

based protocols and applications); Software systems (Java, XML); Middleware platforms and component-based systems (CORBA, .Net, EJB).


Overview on TTCN-3

Presentation Format n

Graphical Format

Tabular Format

TTCN-3CoreLanguage

Other Types & Values n

Other Types & Values 2

ASN.1 Types & Values

:testcase myTestcase () runs on MTCType system TSIType{ mydefault := activate (OtherwiseFail);

verdict.set(pass);

:connect(PTC_ISAP1:CP_ISAP1,mtc:CP_ISAP1);:map(PTC_ISAP1:ISAP1, system:TSI_ISAP1);:PTC_ISAP1.start(func_PTC_ISAP1());PTC_MSAP2.start(func_PTC_MSAP2());Synchronization(); all component.done;log(”Correct Termination”);

}

:

msc mi_synch1_conc1

mtc ISAP1 MSAP2


TTCN-3 Test Case

TTCN-3 – Based Black-Box Testing

Port.send(Stimulus) Port.receive(Response)

System Under Test

Port

• Assignmentof aTest Verdict


TTCN-3 Test Case

TC TCs

TC create

create

TC

TCsMTC

create

TC

SUT

TCs

TC start

start

start

TC

Component-Based Test System


Main Elements of TTCN-3• Module covers declarations and control• Templates (test data description) and

matching mechanisms (pattern matching) • Test configurations

• Formally defined interfaces to the SUT• Dynamic creation of test component• Concurrency to describe distributed test setups

• Test cases• Small (complete) separate compilable programs • Share (type and data) information

• Test verdicts


Content

• Overview on TTCN-3 TTCN-3 applied to software • TTCN-3 and XML• Test Execution


TTCN-3 and Software Testing

• Message-Based Software• Such as Protocols,

Messaging Services, Web site and Portals, ...

• Example technology is XML

• Signature-Based Software• Such as Client-Server,

Peer-to-Peer, Component-based Systems, ....

• Example technology is IDL

Reuse of XML/IDL/... data within TTCN-3 Define a mapping from XML/IDL/... to TTCN-3



Graphical Format

Tabular Format

Other Types & Values 2

IDL

ASN.1 Types & Values

Other Types & Values n

TTCN-3CoreLanguage


TTCN-3 and Software Testing

C, C++, JAVA

XML


Content

• Overview on TTCN-3• TTCN-3 applied to software TTCN-3 and XML• Test Execution


XML in Distributed Applications

• Increasing number of distributed applications use XML for

• Description of messages in datacom protocols• Exchange format between software components• Data description in Web-applications• etc.

• XML• is a structured method for putting data into a textual

presentation by marking up data• can have attributes that describe additional information• describes both attributes and content• is intuitive and self describing


Mapping XML to TTCN-3

• Idea:

Map element tags and attributes to TTCN-3 fields

• Different grammar definitions and mappings• Schemas

• Embedded approach• Flat-Catalog approach• Named Type approach

• DTDs


Example – The Dinosaur Database

XMLPaleontologist feeding the database

Student requesting data

ExampleExampleExampleExample


The Request Interface

XML

URLhttp://www.testingtech.de/TTCN-3_Example/dinolist.xml

XML File


Web Service

Generation of test data structure

TestSystem

TestComponent

TestComponent

TestComponent

TestComponent

Compilationto Executable Tests

XML

ADAPTOR

Adaptor acc. to the

mapping rules

Generation of test behavior

Generation of test data

Principal Approach


Structured Type Defintions


XML DTD<!ELEMENT dinolist (dinosaur*)>

<!ELEMENT dinosaur (name, len, mass, time, place)>

<!ELEMENT name (#PCDATA)>

<!ELEMENT len (#PCDATA)>

<!ELEMENT mass (#PCDATA)>

<!ELEMENT time (#PCDATA)>

<!ELEMENT place (#PCDATA)>

type set of dinosaur dinolist;

type record dinosaur { charstring name, charstring len, charstring mass, charstring time, charstring place

}

Set of Type Definition

Field Definition

Record Type Definition


Test Data Definitions

template dinolist DinoList := {?, ?, Brachiosaurus, ?, ?, ?, ?};

template dinosaur Brachiosaurus := {name := "Brachiosaurus",len := ?,mass := ?,time := ?,place := ?

}


Any Value


Test Port

/** communication port type definition */ type port httpTestPortType message {

out url;in dinolist;

}

Port Definition



Test Components

/** component type definitions */ type component httpTestComponent {

port httpTestPortType httpPort;timer localTimer := 3.0;

}

type component httpTestSystemComponent {port httpTestPortType httpTestSystemPort;

}

Component Definition


Local Port

Local Timer


Test Behavior

httpPort.send(requestURL);localTimer.start;alt { [] httpPort.receive(DinoList) {

localTimer.stop;setverdict(pass);

} [] httpPort.receive {

localTimer.stop;setverdict(fail);

} [] localTimer.timeout {

setverdict(fail); }}

Alternative

reactions


Sending a

message

Starting the timer

The expected response

An unexpected

responseA timeout


altstep DinoList_Default_1() runs on httpTestComponent { [] httpPort.receive {

localTimer.stop;setverdict(fail);

} [] localTimer.timeout {

setverdict(fail); }

}

Handling of unexpected

response

Simplification: Altstep

Test Component

Type

Handling of timeouts



...activate(DinoList_Default_1());

httpPort.send(requestURL);localTimer.start;

httpPort.receive(DinoList); localTimer.stop; setverdict(pass);

…

The expected response is

given here only, All other cases are handled by

the default

Simplification: Use of the Altstep

Default activation



testcase DinoList_Test_1() runs on httpTestComponent system httpTestSystemComponent {

map(mtc:httpPort, system:httpTestSystemPort);

activate(DinoList_Default_1()); httpPort.send(requestURL);

localTimer.start; httpPort.receive(DinoList); localTimer.stop; setverdict(pass); }

A Test Case

MTC Type


Test Case Definition

TSI Type

Mapping the Ports


Graphical Formattestcase DinoList_Test_1 runs on httpTestComponent system httpTestSystemComponent

httpTestComponent

mtc

requestURL

DinoList

httpPort

activate(DinoList_Default_1)

pass

LocalTimer

map(mtc:httpPort,system:httpTestSystemPort)


Content

• Overview on TTCN-3• TTCN-3 applied to software• TTCN-3 and XML Test Execution


SUT

Real Test System Interface

Test Execution

Test System

Abstract Test System Interface

TC2TC1

IN OUT

OUT IN

Connected Ports

INOUTMapped Ports

INOUT

Real Test System Interface


TTCN-3 Execution

Test System

Test System User

TE

TM: Management

System Under Test (SUT)

CD

: C

oD

ec

CH

: C

om

po

ne

nt

Ha

nd

ling

SA: Communication PA: Timer

• Generic XML adaptor that adheres to the mapping


TTCN-3 Runtime Interface

• Adaptation to the SUT

Test System User

TE

TM: Management


CD

: C

oD

ec

CH

: C

om

po

ne

nt

Ha

nd

ling

SA: Communication PA: TimerTRI


TTCN-3 Control Interfaces

• Adaptation to the test platform/test device

Test System User

TE

TM: Management


CD

: C

oD

ec

CH

: C

om

po

ne

nt

Ha

nd

ling

SA: Communication PA: Timer

TCI


Summary of TTCN-3• New version of the only standardized test notation• Modernization: Programming-like test specification with

flexible data support and various representation formats• Wider scope of application

• applicable to many kinds of test applications not just conformance (development, system, integration, interoperability, scalability …)

• applicable in the datacom domain• Harmonization

• first choice for test specifiers, implementors and users both for standardized test suites and

• as a generic solution in industrial software development• Was successfully shown to be applicable to testing of IDL

and XML interfaces, Java and C++ classes, …• Tools are available

Thank You.

Questions?

Thank You.

Questions?

Documents

The Use of TTCN-3 for Software Testing Ina Schieferdecker FOKUS, Berlin [email protected]