Testing Basics

1841f06detprob4

Testing Basics

Detection probability

L5asg – detprob for subdomain tests

What is the probability of detection with one randomly chosen test case per path?

What is the probability of detection with an equal number of randomly chosen test cases?

3841f06detprob4

Control Flow Graph

I: cout

B: type = isoscles

D: type = equilateral

F: type = not a triangle

H: type = bad input

A: type = scalene

C: if

E: if

G: if

Operational profile

3,3,3 abcdegi equi3,3,4 abcegi isos3,3,5 abcegi isos3,3,6 abcefgi not3,4,3 abcegi isos3,4,4 abcegi isos3,4,5 acegi scal3,4,6 acegi scal

All inputs are equally likely

4841f06detprob4

What are the failure probability for each color (separately)?

cin >> a >> b >> c ;type = “scalene”;if (a == b || a == c && b == c) type= “isosceles”;if (a == b || a == c) type = “equilateral”;if (a >= b+c || b >= a+c || c > a+b) type=“not a triangle”;if (a <= 0 || b <= 0 || c <= 0) type=“bad input”;cout<< type;

Blue Green Red

TTYP – smaller subdomains

What might be better smaller subdomains? Would MCC (multiple condition coverage)

be better subdomains

TTYP2 – C0 and C1 coverage

How do we deal with C0 and C1 coverage since they are not subdomain testing methodologies?

9841f06detprob4

Evaluating Testing Methods by Delivered Reliability

Frankl, Hamlet, Littlewood, Strigini

IEEE TOSE Aug98

Testing

Debug

Operational

Fault Detection Probability

Probability of a testing methodology finding a fault (if it existed)

Partition vs Random

Tests, Specifications, meets

Test or test case– single value of program input

– functional program - one input produces an output

Specification - S– set of input-output pairs

Program meets specification– iff for all x in spec, actual output matches spec output

Q: probability distribution

Q - probability distribution over input domain– Q:D -> [0,1] and Q(t) = 1

: Failure Probability

- failure probability for a randomly drawn point is Q*– Where(t) = 1 if and 0 if – and-phi(failure) and -sigma(success)

How does this relate to our notation?

Reliability

R(N) = (1- )N

Assumptions of initial model

Terms

q

d

3.2 SFR, w/o subdomains

d = tinF V(t)

P() = 1-(1-d)T

P(q) = (1-d)T

E() = 0* P() +q* P(q) = q(1-d)T

Thurs, Sep 6

Read next section of article