Ingegneria del Software II - di.univaq.it fileLecturer: Henry Muccini and Vittorio Cortellessa Computer Science Department University of L'Aquila-Italy [email protected] – [email protected]

Lecturer:Henry Muccini and Vittorio Cortellessa

Computer Science Department University of L'Aquila - Italy

[email protected] – [email protected][www.di.univaq.it/muccini] – [www.di.univaq.it/cortelle]

Course:

Ingegneria del Software IIacademic year: 2004-2005

Course Web-site: [www.di.univaq.it/ingegneria2/]

Model-Checking plus Testing: from SoftwareArchitecture Analysis to Code Testing

mailto:[email protected]


http://www.di.univaq.it/muccini

http://www.di.univaq.it/cortelle

http://www.di.univaq.it/ingegneria2/

2SEA Group

© 2005 by H. Muccini and V. Cortellessa / Ingegneria del Software IISEA Group

Copyright Notice

» The material in these slides may be freely reproduced and distributed, partially or totally, as far as an explicit reference or acknowledge to the material author is preserved.

Henry Muccini

3SEA Group


Acknowledgment

» This work is joined with Patrizio Pelliccione (University of L’Aquila), and Pierluigi Pierini (Siemens CNX), and Antonio Bucchiarone (ISTI – CNR)

» Published in ITM 2004, Lecture Notes in Computer Science, LNCS, vol. 3236, pp. 351 - 365 (2004).

4SEA Group


Agenda

» 1. Introduction and Motivations

» 2. Proposal

» 3. Case Study and Initial Results

123

123

12

5SEA Group


Henry Muccini: main research areasSEA Group

Software Engineering and Architecture Group

» SA-based Code Testing:

- Model-Checking driven SA-based Testing

- SA-based Code Regression Testing

- Testing of Product Line Architectures

» Model-based and Component-based Testing

» Product Line:

- Testing Product Lines

- Waple: Web Applications Product Line Engineering

123

6SEA Group


Software Model Checking and Software Testing

» Model Checking:

- It checks whether a certain property is valid for a certain modelof a system [Ruys_PhDThesis]

> Model checking is a model-based, automatic technique that, given a finite-state model M of a system and a property P, checks the validity of P in M

» Testing:

- “Software testing consists of the dynamic verification of the behavior of a program on a finite set of test cases, suitablyselected from the usually infinite executions domain, against the specified expected behavior” [Bertolino_SWEBOK]

123

7SEA Group


Differences

123 skills on formal methods

generally not requiredskills on formal methods

test case identification problemstate explosion problem

code-based, model-based, specification-based

only model-based

clever selection of limited and relevant test cases

- usually left to the testerexperience

exhaustive approach tocompletely check the system

- completely automated

TestingModel Checking

8SEA Group


Goals and Motivations > 1/2

» General Goal:

-- integrationintegration of of modelmodel--checkingchecking and and testingtesting toprovide an useful tool to test modern complexsoftware systems

» In related approaches:

- By using model-checking features, counter-examples are produced, successively used to derive test cases

- Main Limitations:

> P1 : due to models complexity, the model checker techniques become inapplicable, thus not allowing to identify test cases;

> P2 : even on little examples, the number of generated test cases causes

123

9SEA Group


Goals and Motivations > 2/2

» Our Goal:

To apply Model-checking and Testing in a SoftwareArchitecture-based (SA) process, where:

> Model-checking techniques are used to validate the SA model conformance with respect to selected functionalproperties

+ avoiding state explosion problem

> while testing techniques are used to provide confidence on the implementation fulfillment to its architecturalspecification

+ Test case selection driven by model-checking

123

10SEA Group


Software Architecture: a bit of history

» In the early 90’s, SA is recognized as an independent discipline

» Initially boxes & arrows, informal diagrams; then, formal Architectural Description Languages (ADLs) are introduced. Recently, UML may be used to model SAs.

» Currently

- SA used for analysis purposes;

- SA as the basis for Product Family, Component-based paradigms

123

11SEA Group


Our Proposal

Charmy

validate the SA model conformance with respect to selected functional properties

provide confidence on the implementation fulfillment to its architectural spec

SA-based Testing

Requirements

CBSSoftware

Architecture

CBSImplementation

TestExec

[NOK]Fault removal

M.C.[NOK]

refine SA OK [test case selection]

drive

identify

Model

Testing

Checking

SA conformance to requirements through MC

SA model

Implementation conformance to SA through Testing

Test Caseselection

functionalproperties

TestCases

123

12SEA Group


CHARMY (www.di.univaq.it/charmy)

123

SA spec properties

Automaticallygenerated

Step1

Step2

Step3

http://www.di.univaq.it/charmy)

13SEA Group


CHARMY» Iterative process

» Tool support:

- offers a graphical user interface to draw state diagrams and scenarios

- a plugin which allows to input existing diagram in the XMI format

- a translation engine to automatically derive Promela code and Buchi Automaton

123

14SEA Group


Component

Component

Component

Connector

SA-based Code Testing [IEEE_TSE04]

CodeSoftware Architecture spec.

XClient

ClientA ClientB

ClientC

Class--------------------------

123

15SEA Group


SA-based Testing [IEEE TSE04, BookChapter03]

123

16SEA Group


Siemens CNX : main research areas» Siemens CNX S.p.a. is a Siemens R&D lab; its mission is the

design and development of SDH(1) TLC equipments

» relevant research areas:

- Formal design methodologies

- System and software performance analysis

- Test design methodologies

- Intelligent agent application

- Network Processors

- Ethernet first mile

- Optics and cristal properties

- Electromagnetic compatibility

1) SDH Synchronous Digital Hierarchy

123

17SEA Group


Test Design Methodology > objective

»Improve the tets design process

123

18SEA Group


Case Study > some definitions

» A SDH Network Element (NE, i.e. equipment) is modeled using the functional model standardized by ETSI and ITU-T.

» The functional model is built around two specific concepts:

- “network layering", with a client/server relationship between adjacent layers;

- “atomic functions“ (connection, termination and adaptation), to specify the behavior of each layer.

» applicative functions should reside on top of a layer providing specific processing on transmitted information

» A “virtual network connection” can be established between mate network layers (or atomic/applicative functions) belonging to different NEs by means of transport services offered by the underlying layers

123

19SEA Group


Case Study > some definitions

123

20SEA Group


Case Study > EOW architecture

123

» The EOW supports a telephone link between NEs using dedicated voice channels defined on the SDH frame (i.e. the “EOW SubNetwork” [eowSN]);

» An EOW node consists of:

- A “handset” (HS) that manage the physical phone device;

- a “conference manager” (CM) that control the handset connection to the EOW subnetwork;

eowSNHS

CM

HS

CM

HS

CM

EOWNode

21SEA Group


Case Study > EOW components

HS1

CM1

eowSN

localNumSign1

call1

callRequest

callRequest

callRequest

eowKeyDigit

eowKeyDigit

eowKeyDigit

HS2

CM2localNumSign2

call2

HS3

CM3localNumSign3

call3

123

offH

ook onH

ook?call

t imeo

ut

[cbu

sy==

true]

[cbusy==false]

timeout

onHook

!localNumSign

?call

?localNumSign/cbusy=false

[digit

==0]/

!call

22SEA Group


Case Study > Functional Requirements

» EOW Functional Requirements/Properties:

A) when an operator makes a call dialling a selective number, the target operator must receive the call.

B) it must be possible to enter a busy conference (with the special number-sign key) when a call is already in progress.

C) It must be always possible to exit to the conference (cleanly terminate a call).

123

23SEA Group


Case Study > Functional Requirements

123

24SEA Group


Case Study ResultsInteractive simulation & Test generation

» Simulation without constraint will result in an intractable number of traces;

» Simulation conditioned by the given properties;

» Up to 36 test traces was extracted;

- Most of them are eligible to become test cases;

» Test selection focus on some optimization criteria like:

- Maximization of system coverage,

- Minimization of global number of tests

- Minimization of test lenght (i.e. number of steps)

123

25SEA Group


Some Considerations

Advantages:

» Model complexity and the state explosion reduction obtained by: SA-level model chekcing, iterative approach and abstraction ;

» Charmy → easy to use, practical, approach to model-checking, hiding the modeling complexity;

» interactive simulation → we may identify traces of interest for testing the whole system or just a relevant subsystem.

» test specifications are identified from the architectural model (not from requirements)

- Easiest alignment between SA and Test specifications;

- Easiest control of the design steps and evolution

123

26SEA Group


Some Considerations

Limitations:

» The Test Generator Engine can be automated; its implementation is in progress.

» The executable tests implementation from the generated test specifications is not automated yet. We approach this point with the aim to automate also this step.

» Models dimension and complexity still remain an issue, even if the iterative approach reduces the state explosion problem.

123

27SEA Group


Ongoing and Future work

» Improve the Simulation Process and the Test Generation Engine

» automated way to produce test cases from test specifications

» Other case studies

» Approach empirical evaluation

123

28SEA Group


Test Generation Engine

123

29SEA Group


Contact Information» Henry Muccini and Patrizio Pelliccione

- Dipartimento di Informatica, Universita' dell'Aquila, L'Aquila, Italy

- [email protected], [email protected]

» Antonio Bucchiarone

- ISTI CNR

- [email protected],

» Pierluigi Pierini

- Siemens C.N.X. S.p.A., R. & D., L'Aquila, Italy

- [email protected]





Documents

Ingegneria del Software II - di.univaq.it fileLecturer: Henry Muccini and Vittorio Cortellessa Computer Science Department University of L'Aquila-Italy [email protected] – [email protected]