ECE355 Fall 2004Software Reliability1 ECE-355 Tutorial Jie Lian

ECE355 Fall 2004 Software Reliability 1

Software Reliability

ECE-355 Tutorial

Jie Lian

ECE355 Fall 2004 Software Reliability 2

Outline

• Part I: Software Reliability Model

– Musa’s Basic Model

– Musa/Okumoto Logarithmic Model

• Part II: Control Flow Graph

ECE355 Fall 2004 Software Reliability 3

Definition of Software Reliability

• Reliability is usually defined in terms of a statistical measure for the operation of a software system without a failure occurring

• Software reliability is a measure for the probability

of a software failure occurring

• Two terms related to software reliability– Fault: a defect in the software, e.g. a bug in the code

which may cause a failure

– Failure: a derivation of the programs observed behavior

from the required behavior

ECE355 Fall 2004 Software Reliability 4

Parameters of Software Reliability

• Average total number of failures (t)

Average refers to n independent instantiations of an identical software.

• Failure intensity (t)

Number of failures per time unit, derivative of (t).

• Mean Time To Failure (MTTF):

• t may denote elapsed execution calendar or machine clock time

)(

1

tMTTF

ECE355 Fall 2004 Software Reliability 5

Importance of Software Reliability

• In safety-critical systems, certain failures are fatal. This requires pushing reliability to very high levels at very high costs (code redundancy, hardware redundancy, recovery blocks, n version programming…).

• In non-safety-critical systems a certain failure rate is usually tolerable.– This is a question of quality of service.

– Which failure rate is tolerable is mainly a question of customer acceptance. (customer lifts receiver and receives neither fast busy nor dial tone one every 10/10000 calls?)

• We will only talk about non-safety-critical systems

ECE355 Fall 2004 Software Reliability 6

Software Reliability Growth Model (SRG)

• Purpose of SRG models

SRGs rely on observation of failure occurrence and try to predict future failure behavior

• Two different SRG models (appr 40 models totally):– Musa linear model

– Musa/Okomoto logarithmic model

ECE355 Fall 2004 Software Reliability 7

Basic Assumptions of Musa’s Model

• Faults are independent and distributed with constant rate of encounter.

• Well mixed types of instructions, execution time between failures is large compared to instruction execution time.

• Test space covers use space. (Tests selected from a complete set of use input sets).

• Set of inputs for each run selected randomly.

• All failures are observed, implied by definition.

• Fault causing failure is corrected immediately, otherwise reoccurrence of that failure is not counted.

ECE355 Fall 2004 Software Reliability 8

Musa’s Basic Model• Assumption: decrement in failure intensity function

is constant.• Consequence: failure intensity is function of average

number of failures experienced at any given point in time (= failure probability).

(): failure intensity. 0: initial failure intensity at start of execution. : average total number of failures at a given point in time.– v0: total number of failures over infinite time.

00 1)(

v

ECE355 Fall 2004 Software Reliability 9

Example 1• Assume that we are at some point of time t time units in the

life cycle of a software system after it has been deployed.

• Assume the program will experience 100 failures over infinite

execution time. During the last t time unit interval 50 failures

have been observed (and counted). The initially failure

intensity was 10 failures per CPU hour.

• Compute the current (at t) failure intensity:

HourCPU

failures

v

5100

50110)50(

1)(0

0

ECE355 Fall 2004 Software Reliability 10

Musa/Okumoto Logarithmic Model

• Decrement per encountered failure decreases:

: failure intensity decay parameter.

• Example 2 0 = 10 failures per CPU hour.

=0.02/failure.

– 50 failures have been experienced ( = 50).

– Current failure intensity:

e0)(

68.31010)50( 1)5002.0( ee

ECE355 Fall 2004 Software Reliability 11

Model Extension (1)

• Average total number of counted experienced failures () as a function of the elapsed execution time ().

• For basic model

• For logarithmic model

0

0

1)( 0vev

1ln1

)( 0

ECE355 Fall 2004 Software Reliability 12

Example 3 (Basic Model)

0 = 10 [failures/CPU hour].

• v0 = 100 (number of failures over infinite execution time).

= 10 CPU hours:

= 100 CPU hours:

0

0

1)( 0vev

failuresee 6311001100)10( 110

100

10

failuresee 10011001100)100( 10100

100

10

ECE355 Fall 2004 Software Reliability 13

Example 4 (Logarithmic Model)

0 = 10 [failures/CPU hour].

= 0.02 / failure.

= 10 CPU hours:

= 100 CPU hours:

5511002.010ln02.0

1)10(

1ln1

)( 0

152110002.010ln02.0

1)100(

(63 in basic model)

(100 in basic model)

ECE355 Fall 2004 Software Reliability 14

Model Extension (2)

• Failure intensity as a function of execution time.• For basic model:

• For logarithmic Poisson model

0

0

0)( ve

1)(

0

0

ECE355 Fall 2004 Software Reliability 15

Example 5 (Basic Model)

0 = 10 [failures/CPU hour].

• v0 = 100 (number of failures over infinite execution time).

= 10 CPU hours:

= 100 CPU hours:

hourCPU

failuresee 68.31010)10( 1

10100

10

0

0

0)( ve

hourCPU

failuresee 000454.01010)100( 10

100100

10

ECE355 Fall 2004 Software Reliability 16

Example 6 (Logarithmic Model)

0 = 10 [failures/CPU hour]. = 0.02 / failure.

= 10 CPU hours:

= 100 CPU hours:

hourCPU

failures33.3

11002.010

10)10( (3.68 in basic model)

(0.000454 in basic model)

1)(

0

0

hourCPU

failures467.0

110002.010

10)100(

ECE355 Fall 2004 Software Reliability 17

Model Discussion

• Comparison of basic and logarithmic model:

– Basic model assumes that there is a 0 failure intensity,

logarithmic model assumes convergence to 0 failure intensity.

– Basic model assumes a finite number of failures in the

system, logarithmic model assumes infinite number.

• Parameter estimation is major problem: 0, , and v0.

Usually obtained from:

– system test,

– observation of operational system,

– by comparison with values from similar projects.

ECE355 Fall 2004 Software Reliability 18

Part II: Control Flow Graph (CFG)

• A graph representation of a set of statements is called

a flow graph or control flow graph.

• Nodes in the flow graph represent computations and

the edges represent the flow of control.

• A basic block is a sequence of consecutive three-

address statements in which flow of control enters at

the beginning and leaves at the end without halt or

possibility of branching except at the end.

• A CFG consists of a set of basic blocks.

ECE355 Fall 2004 Software Reliability 19

Three-Address Statements• Assignment statements of the form x: = y op z or x: = op z, where op is a

binary or unary arithmetic or logical operation.

• Copy statements x: = y where the value of y is assigned to x.

• Unconditional jump goto L. Execution jumps to the statement labeled by

L.

• Conditional jump if x relop y goto L.

• Indexed assignments of the form x: = y[i] and x[i] := y.

• Address and pointer assignments of the form x := &y, x := *y, and *x := y.

• Param x and call p, n, and return y, where

return value of y is optional. For a procedure

call p(x1, x2, … , xn), the transformed

three-address statements are:

param x1

param x2

…

param xn,

call p, n

ECE355 Fall 2004 Software Reliability 20

Partition into Basic Blocks

• Input: A sequence of three-address statements.

• Output: A list of basic blocks with each three-address

statements in exactly one block.

• Method

1. Determining leaders (the first statement of basic blocks) by three rules:

i. The first statement is a leader.

ii. Any statement that is the target of a conditional or unconditional goto is a

leader.

iii. Any statement that immediately follows a goto or conditional goto

statement is a leader.

2. For each leader, its basic block consists of the leader and all

statements up to but not including the next leader or the end of the

program.

ECE355 Fall 2004 Software Reliability 21

Example

I = 1;

TI = TV = 0;

sum = 0;

DO WHILE (v[I] <> –999 and TI < 1) {

      TI++;

      IF (v[I] >= min and v[I] <= max) {

TV++; sum = sum + v[I];

}

I++;

}

IF TV >0 )

    av = sum/TV;

ELSE

    av = –999 ;

…

1 I = 1;

TI = TV = 0;

sum = 0;

2 IF (v[I] == –999) GOTO 10

3 IF (TI >= 1) GOTO 10

4 TI++;

5 IF (v[I] < min) GOTO 8

6 IF (v[I] > max) GOTO 8

7 TV++;

sum = sum + v[I];

8 I++;

9 GOTO 2

10 IF (TV <= 0) GOTO 12

11 av = sum/TV;

goto 13

12 av = –999;

13 …

While loop

IF ELSE

Basic Block

We do not strictly follow the transformationfrom source code to three-address statements.Note that each statement with a label is a leader.

ECE355 Fall 2004 Software Reliability 22

Transformation from Basic Blocks to CFG1

2

3

4

5

6

8 7

9

10

12 11

13

R4

R1

R3

R5

R2

R6

Outer region

predicate node

…

1 I = 1;

TI = TV = 0;

sum = 0;

2 IF (v[I] == –999) GOTO 10

3 IF (TI >= 1) GOTO 10

4 TI++;

5 IF (v[I] < min) GOTO 8

6 IF (v[I] > max) GOTO 8

7 TV++;

sum = sum + v[I];

8 I++;

9 GOTO 2

10 IF (TV <= 0) GOTO 12

11 av = sum/TV;

goto 13

12 av = –999;

13 …

ECE355 Fall 2004 Software Reliability 23

Cyclomatic Complexity

• McCabe’s cyclomatic complexity

– V(G) = E – N + 2, E: number of edges, N: number of nodes.

– V(G) = p + 1, p is a number of predicate (decision) nodes.

– V(G) = number of regions (area surrounded by nodes/edges).

• V(G): upper bound on the number of independent paths

– Independent path: A path with at least one new node/edge.

• Example (pp. 22) :

– V(G) = E – N + 2 = 17 – 13 + 2 = 6

– V(G) = p + 1 = 5 + 1 = 6

– V(G) = 6

• Advantage: # of test cases is proportional to the program size.

ECE355 Fall 2004 Software Reliability 24

References

[1] Musa, JD, Iannino, A. and Okumoto, K., “Software Reliability:

Measurement, Prediction, Application”, McGraw-Hill Book

Company, NY, 1987.

[2] A. V. Aho, R. Sethi, and J. Ullman, "Compilers: Principles,

Techniques, and Tools", Addison-Wesley, Reading, MA, 1986.