Software testing techniques Spring 2009 Csci565: Theory and Practice of Software Testing 1 Csci565: Graph Theory for Testers

Software testing techniquesSpring 2009

Csci565: Theory and Practice of Software Testing1

Csci565: Graph Theory for Testers

Objectives


Discuss graph theory and white-box, black-box, conformance testing, etc.GraphsDegree of a nodeIncidence matricesAdjacency matricesConnectednessCondensation graphDirected graphsPaths and semi-pathsReach-ability matrixStrong componentsModel Based testing using

Finite State Machines Petri nets Statecharts ..

Graphs theory


Graph theory?Has nothing to do with graph or graphicsAn area of math dealing with entities (nodes)

and the connections (links) between the nodes

Graph: formal definition


A graph is an abstract mathematical structure defined from two sets:V={n1, n2,…nm} of nodesE={e1,e2,…em} of edges

A graph with 7 nodes and 5 edges (undirected)


n1

n6n7

n3n5

n4

n2e1

e5

e3

e4e2

Degree of a node


The degreeRefers to the number of edges that have a

node as an endpoint , denoted by deg(n)Indicates the extent of integration testing

that is appropriate for the objectE.g. deg(n1) =2

Incidence matrices


Alternative to visual presentation of graphThe incidence matrix of G=(V,E) with m

nodes and n edges is an mn matrixWe have 1 in row i, column J

iff node i is an endpoint of edge jRow sum represents degree of nodesColumn sum represents the endpoints of an

edge

Incidence matrices

e1 e2 e3 e4 e5

n1 1 1 0 0 0

n2 1 0 0 1 0

n3 0 0 1 0 0

n4 0 1 1 0 1

n5 0 0 0 1 0

n6 0 0 0 0 1

n7 0 0 0 0 0

8Csci565: Theory and Practice of Software Testing

Adjacency matrices


A useful supplement to the incidence matrix

The Adjacency matrix of G=(V,E) with m nodes and n edges is an mm matrixWe have 1 in row i, and col. J iff

there is an edge between node i and node j, zero otherwise

Used to identify paths and hence equivalence relation to simplify a graph and hence testing

adjacency matrices

n1 n2 n3 n4 n5 n6 n7

n1 0 1 0 1 0 0 0

n2 1 0 0 0 1 0 0

n3 0 0 0 1 0 0 0

n4 1 0 1 0 0 1 0

n5 0 1 0 0 0 0 0

n6 0 0 0 1 0 0 1

n7 0 0 0 0 0 0 0


PathsA path is a

sequence of edges such that, for any adjacent pair of edges ei, ej in the sequence, the edges share a common (node) endpoint

Can be described as sequences of edges or nodes

path Nodes sequence

Edge sequences

Between n1 and n5

n1,n2,n5 e1,e4

Between n6 and n5

n6, n4, n1, n2, n5

e5,e2,e1,e4

….


Connectedness


Nodes ni and nj are connected iff they are in the same path

Connectedness is an equivalence relation can be checked easilyReflexive(every node is in path of 0 length

with itself)Symmetric n1, and 2 in same path, then n2

and n1 is also in the same pathtransitive

Component and condensation graphs


Connectedness defines a partition (or component) on the node set of a graphComponents of a graph is maximal set of

connected nodesE.g. Components

S1={n1,n2,n3,n4,n5,n6} and S2={n7}

Condensation graphUsed as a Simplification mechanismCreating a graph by replacing a set of connected

nodes (or components) by a condensing nodeThe implication for testing is that component are

stand alone elements and hence can be tested separately

Directed graph


n1

n7

n3n5

n4

n2

n6

e1

e4

e5

e3

e2

Types of nodes


source node: a node with in-degree zeroSink node: a node with out-degree=0Transfer node: node having in-degree

=out-degree of nonzero

Directed graph: The testing implication


The implication for testingInitial nodeTerminal nodeEdges represents software concepts

Sequential behaviorsControl infoTime-ordered eventsDefine/uses concepts…

Adjacency matrix of a directed graph (AMD)


The AMD of G=(V,E) with m nodes is an mm matrix where a(i,j) is a 1 iff there is an edge from node i to node j, otherwise it is 0Row sum represents outdegreesColumn sum represents indegrees

adjacency matrices for directed graph

n1 n2 n3 n4 n5 n6 n7

n1 0 1 0 1 0 0 0

n2 0 0 0 0 1 0 0

n3 0 0 0 1 0 0 0

n4 0 0 0 0 0 1 0

n5 0 0 0 0 0 0 0

n6 0 0 0 0 0 0 0

n7 0 0 0 0 0 0 0


Paths and semi-paths


Direction is important thereforeDirected path (a sequence of edges ei and ej,

the terminal node of ei is the initial node of ej )

Cycle (directed path that begins and ends at the same node)

Directed semi-path (for adjacent pair of ei, the initial (terminal) node of the first edge is the initial (terminal) node of the second edgeE.g., n1 and n3

Reachability matrices for directed graph

n1 n2 n3 n4 n5 n6 n7

n1 1 1 0 1 1 1 0

n2 0 1 0 0 1 0 0

n3 0 0 1 1 0 1 0

n4 0 0 0 1 0 1 0

n5 0 0 0 0 1 0 0

n6 0 0 0 0 0 1 0

n7 0 0 0 0 0 0 1


N-connectedness


Connectedness of directed graph0-connected (no path between ni, and nj)1-connected (semi-path between ni, and nj )2-connected(a path between ni, and nj )3-connected (a path between ni to nj, and a

path between nj, and ni)Strong components

Directed graph


n1

n7

n3n5

n4

n2

n6

e1

e4

e5

e3

e2

e6

n1 and n7 0-c

n2 and n4 1-c

n1 and n6 2-c

n3 and n6 3-c

Graphs for testing


Program graph or flowgraph ( code testing)Finite State machines (blackbox)

A computational model consisting of a finite number of states and transitions between those states, possibly with accompanying actions

Petri nets (blackbox)Sysetm behavior

Statecharts (blackbox)System behaviorBehavior diagrams specified as part of the Unified

Modeling Language (UML). statechart depicts the states that a system or

component can assume,shows the events or circumstances that cause or result

from a change from one state to another.

Graph theory and Model-Based testing


ModelsA method (way) of representing the behavior

of a systemSimpler than the system they describeHelp us to understand and predict the

system’s behavior

Life is too short for manual testing


MBST?Black-box technique that can be used

To construct the behavioral aspects of system early in the development lifecycle

Exposes ambiguities in the specification and design of the software under construction

Embodies behavioral information that can be reused in future testing, even when the specification change

Easier to update than a suite of individual tests



requirements

1. create model

model

2. Generat

e abstract

test cases

Test case

s3.

Generate

concrete test

cases

Test scrip

ts

4. Run the SUT

Test Resu

lt

5. Analyze

the result

Model Covera

ge

RTM

The model-based testing process: Build or Borrow

results

6. Revise

the model

Platform Independent Test cases

Platform Specific Test Cases

Modeling Maturity level


Level 0: No specificationLevel 1: Textual (very informal)Level 2: Text with Diagram (textual spec is

augmented by diagrams)Level 3: Models with text ( a set of well-

defined diagrams/text)Level 4: Precise models (MDA)Level 5: Models only (model-to-code)

Model-based testing experience reports


IBMUsed a model-based test generator known as

GOTCHA-TCBeans to generate test cases from FSM models

FSM model was created from English SPECMicrosoft

At least three generation of model-based testing tools developed and used within Microsoft over the past few yearsTMT: TEST MODEL TOOLKIT (1ST ) developed by IE team

and won the best testing practice awardASML/T AND SPEC EXPLORER by FSE found bugs in early

versions of web services implementations

Limitations with MBT


No insurances to find all the differences between the model and its implementation

Requires different skills from manual test designModel designers must be able to ABSTRACT

and design modelsWorks with functional testing and test

maturity

Modeling Method Guidance

Application Characteristics Suggested Modeling Method Processes formal language (e.g., web browser process HTML, compiler)

Grammar

State-rich systems (e.g., telephony systems)

Finite State Machines

Parallel system, individual components capable of being modeled by state machines

Statecharts, Petri Nets

Need to represent conditions under which inputs cause a particular response/effects,

Decision tables


Examples of models


a typical example of a model in computing isFinite state machine (or state graph)

Petri Nets

tired freshsleep


Example: A Very Simple Finite State Model of the Clock

Test cases


We could use this very simple state model as a basis for tests, where following a path in the model is equivalent to running a test:Setup: Put the Clock into its Analog display

modeAction: Click on “Settings\Digital”Outcome: Does the Clock correctly change to

the Digital display?

Actions


the following actions in the Clock:Start the Clock application Stop the Clock application Select Analog setting Select Digital setting

Operational modes


Two modes of operations:system mode

NOT_RUNNING means Clock is not runningRUNNING means Clock is running

setting mode ANALOG means Analog display is setDIGITAL means Digital display is set

State Transition Diagram for the Clock Model


Generate Sequences of Test Actions from the Model:1


Generate Sequences of Test Actions from the Model Unusual (random) combination

Start Analog Analog Analog Analog Analog Stop

Petri Nets: Informal Definition


Directed, weighted, bipartite graphplacestransitionsarcs (places to transitions or transitions to

places)weights associated with each arcInitial marking (assigns a non-negative

integer to each place)

Formal definition: Petri nets


A quadruple (P,T,F,W)P: places; T: transitions F {PT} {TP}) W: F N – {0}

Petri Nets: Properties


Properties:(1) P T = Ø(2) P T Ø(3)F (P T) (T P)(4) W: F N-{0} //relate non-zero value to f//default value of W is 1

State defined by marking: M: P N

Petri Nets (PN): Terminologies


Input placesArrows go from places to transition

Output placesArrows go from transitions to places

Enabled TransitionT is enabled if each of its input places

contains token such that M(p’) ≥ W<f>

Transition (firing) rule


A transition t is enabled if each input place ( or t) p has at least w(p,t) tokens

An enabled transition may (or may not) fireA firing on an enabled transition t removes

w(p,t) from each input place p, and adds w(t,p’) to each output place p’

Formal Semantics


Transition t is enabled iffpt‘s input places (i.e., t), M(p) W(<p,t>)

Firing of tproduces a new marking M’ in places that are either t's input

or output places or both if p =t (i.e. p is input place)

M'(p) = M(p) - W(<p,t>) if p= t (i.e., p is output place)

M'(p) = M(p) + W(<t,p>) if p= t t (i.e., p is both an input and an output place)

M'(p) = M(p) - W(<p,t>) + W(<t,p>)

Firing example


2H2 + O2 2H2O

H2

O2H2O

MakeH2o

2

2

Firing example


2H2 + O2 2H2O

H2

O2

H2O

MakeH2o

2

2

Some typical interpretations of places


Places representPre/post conditions Conditions/conclusions (in logical setting)Input signals/output signalsInput data/output dataBuffersResources needed/released

Some typical interpretations of transitions


Transitions representActionsEventsSignal processorTask or jobClause in logicprocessor

Vending machine


VendingMachine

insert_coin (x)

select_snak (X)

accept_coin

reject_coin

give_snak

No_change_returned

Vending Machine Requirements


The machine dispenses only two snack bar products 20c 15c

The machine accepts only two kinds of change 10c5c

The machine does not return any change

Vending Machine : FSM’s represntation


0 cent

5 cents

10 cents

15 cents

20 cents

Deposit

5c

Deposit 10c

Deposit 10c

Deposit 10cD

eposit 5c

Deposit 5c

Deposit 5c

Take 20c snack bar

Take 15c snack bar

Vending Machine: PN’s version


5c

Take 15c bar

Deposit 5c

0c

Deposit 10c

Deposit 5c

10c

Deposit 10c

Deposit5c

Deposit 10c20c

Deposit5c

15c

Take 20c bar

Vending Machine: Some Scenarios


Scenario 1 Deposit 5c, deposit 5c, deposit 5c, deposit

5c, take 20c snack barScenario 2

Deposit 10c, deposit 5c, take 15c snack barScenario 3

Deposit 5c, deposit 10c, deposit 5c, take 20c snack bar

Modeling concurrency


t2

t3

t1 t4marked graph:each place hasexactly oneincoming arc and one outgoingarc.

Modeling concurrency


t2

t3

t1 t4

concurrency

Modeling CC and Synchronization: In a Restaurant (A Petri Net)


Waiter-readyC1-arriving C2-arriving

Takeorder

Takeorder

cooking

Get food

C1-waiting C2-waiting

Serve food Serve food

eating eating

Example: In a Restaurant (Two Scenarios)


Scenario 1( sequential serving)Waiter takes order from customer 1

serves customer 1 takes order from customer 2serves customer 2

Scenario 2 (Interleave serving)Waiter takes order from customer 1

takes order from customer 2 serves customer 2 serves customer 1

Example: In a Restaurant (Scenario 1)


WaiterfreeCustomer 1 Customer 2

Takeorder

Takeorder

Ordertaken

Tellkitchen

wait wait


eating eating

Example: In a Restaurant (Scenario 2)


WaiterfreeCustomer 1 Customer 2

Takeorder

Takeorder

Ordertaken

Tellkitchen

wait wait


eating eating

Modeling dataflow computation


Compute x = (a+b)/(a-b)

a

a

b

b

R(a+b)

R(a-b)

+

-

/

If (a-b)0

If (a-b)=0

X=(a+b)/(a-b)

No-result

copy

copya-b

Problems with Petri nets


Petri nets (Good)The ability of easily modeling concurrencysynchronization aspects, the intuitive graphic notation, the formal semantics that supports powerful

analysis capabilities and the availability ofseveral supporting tools

Petri nets (Bad)Not widely used in software engineering,

Csci565: Theory and Practice of Software Testing62 UML

Statecharts : A Visual Formalism for Complex Systems


• Origins Of Statecharts• FSMs• STDs• Harel Statecharts

• Limitations of State Machines– Doesn’t

• Provide State HierarchyState Hierarchy• provide History Info.History Info.• support Concurrent States Concurrent States

Semantics of Statecharts


StateEventTransitionsEvent [Guard] / Action Transition


Example for Statechart Semantics

More features of Statecharts


• Generalization– Simplifies by Factorization

• Orthogonality– Simplifies by Segmentation

• History Info.– To memorize the substate substate last visited

Example for Generalization


Example for Orthogonality


Example for History state


Importance of Testing Statecharts


Statecharts play crucial role in modelingSafety-critical systemsNetworking ProtocolsGUIs (Graphical User Interfaces)OOD (Object-Oriented Design)


Methodology of Testing SCs

Coverage Metrics


State CoverageRatio of number of states covered and total number of states

in the given state model

Event CoverageRatio of number of events covered and total number

of events in the given state model

Transition CoverageRatio of number transitions exercised and total number of

transitions in the given state model

State-Event CoverageRatio of state-event pairs exercised and number of states

multiply number of events in the given state model

Statechart Spec. and Test Case’s Formats


Acknowledgement/references


Harry Robinson: Finite State Model-Based Testing on a Shoestring (1999 Software Testing Analysis and Review West Conference )

Petri nets and software engineering Practical Model-Based Testing by Mark Utting and Bruno

Legeard, Morgan Kaufmann Publisher


Any q u e s t i o n s?

Documents

Software testing techniques Spring 2009 Csci565: Theory and Practice of Software Testing 1 Csci565: Graph Theory for Testers