163
SOFTWARE QUALITY AND TESTING

Software Quality and Testing_Se lect18 btech

  • Upload
    iiita

  • View
    670

  • Download
    1

Embed Size (px)

DESCRIPTION

18/18

Citation preview

Page 1: Software Quality and Testing_Se lect18 btech

SOFTWARE QUALITY AND TESTING

Page 2: Software Quality and Testing_Se lect18 btech

What is quality ?Quality popular view:

– Something “good” but not quantifiable

– Something luxurious and classy

Quality professional view:– Conformance to requirement

(Crosby, 1979) The requirements are clearly stated and the product must conform to it Any deviation from the requirements is regarded as a defect A good quality product contains fewer defects

– Fitness for use (Juran, 1970): Fit to user expectations: meet user’s needs A good quality product provides better user satisfaction

2

Page 3: Software Quality and Testing_Se lect18 btech

DEFINITION OF QUALITY

Page 4: Software Quality and Testing_Se lect18 btech

Definition of Quality

ISO 8402 definition

of QUALITY:

The totality of features and

characteristics of a product

or a service that bear on

its ability to satisfy stated

or implied needs

4

ISO 9216 Model:

Quality characteristics

1.Functionality2.Reliability3.Usability4.Efficiency5.Maintainability6.Portability

Page 5: Software Quality and Testing_Se lect18 btech

QUALITY ASPECTS

Page 6: Software Quality and Testing_Se lect18 btech

Quality Concepts

• Quality of design– refers to characteristics designers specify for the end product to

be constructed• Quality of conformance

– degree to which design specifications are followed in manufacturing the product

• Quality control– series of inspections, reviews, and tests used to ensure

conformance of a work product (artifact) to its specifications• Quality assurance

– auditing and reporting procedures used to provide management with data needed to make proactive decisions

6

Page 7: Software Quality and Testing_Se lect18 btech

Quality Control versus Quality Assurance

Page 8: Software Quality and Testing_Se lect18 btech

Quality Control versus Quality Assurance

• Quality Control (QC) is a set of activities carried out with the main objective of withholding products from shipment if they do not qualify.

• Quality Assurance (QA) is meant to minimize the costs of quality by introducing a variety of activities throughout the development process and maintenance process in order to prevent the causes of errors, detect them, and correct them in the early stages of the development. As a result, quality assurance substantially reduces the rate of non-qualifying products.

8

Page 9: Software Quality and Testing_Se lect18 btech

Quality as Dealing with defects

• When people associate Quality or High Quality with software system, it is an indication that few, if any, defects are expected to occur during its operations or when problems do occur, the negative impact is expected to be minimized.

• Key of the correctness aspect of software quality is the concept of defect, failure, fault and error.

• The term “defect” refers to some problem with the software either with its external or with internal characteristics.

9

Page 10: Software Quality and Testing_Se lect18 btech

Causes of software defects

Page 11: Software Quality and Testing_Se lect18 btech

Causes of software defects

1. Faulty requirements definition

2. Client-developer communication failures

3. Deliberate deviations from software requirements

4. Logical design errors

5. Coding errors

6. Non-compliance with documentation and coding instructions

7. Shortcomings of the testing process

8. User interface and procedure errors

9. Documentation errors

11

Page 12: Software Quality and Testing_Se lect18 btech

Software errors, software faults and software failures

Software development process

software fault

software failure

software error

12

Page 13: Software Quality and Testing_Se lect18 btech

Software errors, software faults and software failures

More precisely:

• An error can be a grammatical error in one or more of the code lines, or a logical error in carrying out one or more of the client’s requirements.

• Not all software errors become software faults. in some cases, the software error can cause improper functioning of the software. In many other cases, erroneous code lines will not affect the functionality of the software as a whole.

• A failure is said to occur whenever the external behavior of a system does not conform to that prescribed in the system specification. A software fault becomes a software failure only when it is “activated”

13

Page 14: Software Quality and Testing_Se lect18 btech

Error, faults, failures

14

Page 15: Software Quality and Testing_Se lect18 btech

Tools for Improving Quality

• Formal specification

• Self-checking code

• Program verification and validation

• Testing

• Deploy with capabilities to repair

Page 16: Software Quality and Testing_Se lect18 btech

Software Testing

• Software testing is a popular risk management strategy. It is used to verify that functional requirements were met.

• The limitation of this approach, however, is that by the time testing occurs, it is too late to build quality into the product

16

Page 17: Software Quality and Testing_Se lect18 btech

“Testing is the process of establishing confidence that a program or system does what it is supposed to.”by Hetzel 1973

“Testing is the process of executing a program or system with the intent of finding errors.” by Myers 1979

“Testing is any activity aimed at evaluating an attribute or capability of a program or system and determining that it meets its required results.” by Hetzel 1983

What is Software Testing?

Page 18: Software Quality and Testing_Se lect18 btech

Testing is NOT

• Code inspections• Design reviews• Configuration management• Bug tracking

These are, along with testing, are part of Software Quality Assurance (SQA). Together these

improve the quality of the product

Page 19: Software Quality and Testing_Se lect18 btech

TYPES OF TESTING

Page 20: Software Quality and Testing_Se lect18 btech

Types of Testing:

Scope Unit Module Integration Interface Coverage

Purpose Functional Stress

Execution Manual Automated

Phases/Triggers Nightly Builds Defect Removal Change Order Completion Alpha Beta

Page 21: Software Quality and Testing_Se lect18 btech

WHO ARE THE TESTERS?

Page 22: Software Quality and Testing_Se lect18 btech

Who Tests?

DevelopersUnit testing

QA TeamModule testing, automated testing

Software Automated testing

DesignersEngineers, Analysts

UsersUnpleasant but true

Page 23: Software Quality and Testing_Se lect18 btech

Testing, debugging, Verification

Page 24: Software Quality and Testing_Se lect18 btech

TESTING AND DEBUGGING

Page 25: Software Quality and Testing_Se lect18 btech

Testing and debugging

Testing is the process of determining if a program has anyerrors.

When testing reveals an error, the process used to determine the cause of this error and to remove it, is known as debugging.

Page 26: Software Quality and Testing_Se lect18 btech

A test/debug cycle

Page 27: Software Quality and Testing_Se lect18 btech

WHAT I TEST PLAN

Page 28: Software Quality and Testing_Se lect18 btech

Test plan

A test cycle is often guided by a test plan.

Example: The sort program is to be tested to meet the requirements given earlier. Specifically, the following needs to be done.

• Execute sort on at least two input sequences, one with ``A'' and the other with ``D'' as request characters.

Page 29: Software Quality and Testing_Se lect18 btech

Test plan (contd.)

• Execute the program on an empty input sequence.

• Test the program for robustness against erroneous inputs such as ``R'' typed in as the request character.

• All failures of the test program should be recorded in a suitable file using the Company Failure Report Form.

Page 30: Software Quality and Testing_Se lect18 btech

Test case/data

A test case is a pair consisting of test data to be input to the program and the expected output. The test data is a set of values, one for each input variable.

A test set is a collection of zero or more test cases.

Sample test case for sort:

Test data: <''A'’ 12 -29 32 >Expected output: -29 12 32

Page 31: Software Quality and Testing_Se lect18 btech

TESTING AND VERIFICATION

Page 32: Software Quality and Testing_Se lect18 btech

Testing and verification

Program verification aims at proving the correctness of programs by showing that it contains no errors. This is very different from testing that aims at uncovering errors in a program.

Program verification and testing are best considered as complementary techniques. In practice, one can discard program verification, but not testing.

Page 33: Software Quality and Testing_Se lect18 btech

Testing and verification (contd.)

Testing is not a perfect technique in that a program might contain errors despite the success of a set of tests.

Verification might appear to be perfect technique as it promises to verify that a program is free from errors. However, the person who verified a program might have made mistake in the verification process; there might be an incorrect assumption on the input conditions; incorrect assumptions might be made regarding the components that interface with the program, and so on.

Page 34: Software Quality and Testing_Se lect18 btech

Software quality assurance and testing in

Structured programming

Page 35: Software Quality and Testing_Se lect18 btech

Structure Programming and Unstructured Programming

Page 36: Software Quality and Testing_Se lect18 btech

Structured Programming

• structured programming: A technique for organizing and coding computer programs in which a hierarchy of modules is used.

• Each having a single entry and a single exit point, and in which control is passed downward through the structure.

Page 37: Software Quality and Testing_Se lect18 btech

STRUCTURED PROGRAMMING

• In structured Programming each construct has a predictable logical structure.

• This could enabling a reader to follow procedural flow more easily.

• The constructs are sequence, condition, and repetition.

Page 38: Software Quality and Testing_Se lect18 btech

STRUCTURED PROGRAMMING

• Sequence implements processing steps that are essential in the specification of any algorithm.

• Condition provides the facility for selected processing based on some logical occurrence.

• Repetition allows for looping.

• These three constructs are fundamental to structured programming—an important component-level design technique.

Page 39: Software Quality and Testing_Se lect18 btech

STRUCTURED PROGRAMMING

• The structured constructs were proposed to limit the procedural design of software to a small number of predictable operations.

• Complexity metrics indicate that the use of the structured constructs reduces program complexity and thereby enhances readability, testability, and maintainability.

Page 40: Software Quality and Testing_Se lect18 btech

STRUCTURED PROGRAMMING

• The use of a limited number of logical constructs also contributes to a human understanding process that is known as chunking.

• To understand this process, consider the way in which you are reading this page.

• You do not read individual letters but rather recognize patterns or chunks of letters that form words or phrases.

Page 41: Software Quality and Testing_Se lect18 btech

STRUCTURED PROGRAMMING

• The structured constructs are logical chunks that allow a reader to recognize procedural elements of a module, rather than reading the design or code line by line.

• Understanding is enhanced when readily recognizable logical patterns are encountered.

Page 42: Software Quality and Testing_Se lect18 btech

STRUCTURED PROGRAMMING

• Any program, regardless of application area or technical complexity, can be designed and implemented using only the three structured constructs.

• It should be noted, however, that strict use of only these constructs can sometimes cause practical difficulties.

Page 43: Software Quality and Testing_Se lect18 btech

Unstructured programming

All the program code written in a single continuous main program.All the program code written in a single continuous main program.

Many disadvantages for large programs

• difficult to follow logic

• if something needs to be done more than once must be re-typed

• hard to incorporate other code

• not easily modified

• difficult to test particular portions of the code

• ...

Many disadvantages for large programs

• difficult to follow logic

• if something needs to be done more than once must be re-typed

• hard to incorporate other code

• not easily modified

• difficult to test particular portions of the code

• ...

Page 44: Software Quality and Testing_Se lect18 btech

Advantages of Structured Programming

Page 45: Software Quality and Testing_Se lect18 btech

Advantages of Structured Programming• Easy to write:

Modular design increases the programmer's productivity by allowing them to look at the big picture first and focus on details later.

Several Programmers can work on a single, large program, each working on a different module

Studies show structured programs take less time to write than standard programs.

Procedures written for one program can be reused in other programs requiring the same task. A procedure that can be used in many programs is said to be reusable

Page 46: Software Quality and Testing_Se lect18 btech

Advantages of Structured Programming

• Easy to debugSince each procedure is specialized to perform just one task, a procedure can be checked individually. Older unstructured programs consist of a sequence of instructions that are not grouped for specific tasks. The logic of such programs is cluttered with details and therefore difficult to follow.

Easy to UnderstandThe relationship between the procedures shows the modular design of the program.

Easy to ChangeSince a correctly written structured program is self-documenting, it can be easily understood by another programmer.

Page 47: Software Quality and Testing_Se lect18 btech

Structured programming using graphical design notation

Page 48: Software Quality and Testing_Se lect18 btech

Graphical Design Notation

• "A picture is worth a thousand words," but it's rather important to know which picture and which 1000 words.

• There is no question that graphical tools, such as the flowchart or box diagram, provide useful pictorial patterns that readily depict procedural detail.

• However, if graphical tools are misused, the wrong picture may lead to the wrong software.

Page 49: Software Quality and Testing_Se lect18 btech

Graphical Design Notation

• A flow chart is quite simple pictorially.

• A box is used to indicate a processing step.

• A diamond represents a logical condition, and arrows show the flow of control.

Page 50: Software Quality and Testing_Se lect18 btech

Graphical Design Notation

• The sequence is represented as two processing boxes connected by an line (arrow) of control.

Page 51: Software Quality and Testing_Se lect18 btech

Graphical Design Notation

• Condition, also called if-then-else, is shown as a decision diamond that if true, causes then-part processing to occur, and if false, invokes else-part processing.

Page 52: Software Quality and Testing_Se lect18 btech

Graphical Design Notation

• The selection (or select-case) construct shown in the figure is actually an extension of theif-then-else. A parameter is tested by successive decisions until a true condition occurs and a case part processing path is executed.

Page 53: Software Quality and Testing_Se lect18 btech

Graphical Design Notation

• Repetition is represented using two slightly different forms. The do while tests a condition and executes a loop task repetitively as long as the condition holds true.

• A repeat until executes the loop task first, then tests a condition and repeats the task until the condition fails.

Page 54: Software Quality and Testing_Se lect18 btech

Graphical Design Notation

• The structured constructs may be nested within one another as shown in Figure

Page 55: Software Quality and Testing_Se lect18 btech

Structured programming using Box diagram

Page 56: Software Quality and Testing_Se lect18 btech

The box diagram

Page 57: Software Quality and Testing_Se lect18 btech

The box diagram

Page 58: Software Quality and Testing_Se lect18 btech

The box diagram

Page 59: Software Quality and Testing_Se lect18 btech

The box diagram

Page 60: Software Quality and Testing_Se lect18 btech

Structure Programming and testing

Page 61: Software Quality and Testing_Se lect18 btech

Structure Programming and testing

(1) guarantee that all independent paths within a module have been exercised at least once,

(2) exercise all logical decisions on their true and false sides,

(3) execute all loops at their boundaries and within their operational bounds, and

(4) exercise internal data structures to ensure their validity.

Page 62: Software Quality and Testing_Se lect18 btech

BASIS PATH TESTING

Page 63: Software Quality and Testing_Se lect18 btech

Structure Programming and testing

• BASIS PATH TESTING : Test cases derived to check the execution of every statement in the program at least one time during testing.– Flow Graph Notation– Cyclomatic Complexity– Deriving Test Cases– Graph Matrices

Page 64: Software Quality and Testing_Se lect18 btech

What is Flow Graph Notation

Page 65: Software Quality and Testing_Se lect18 btech

Flow Graph Notation

• Draw a flow graph when the logical control structure of a module is complex. The flow graph enables you to trace program paths more readily.

Page 66: Software Quality and Testing_Se lect18 btech

Types of Logic Coverage

• Statement: each statement executed at least once

• Branch: each branch traversed (and every entry point taken) at least once

• Condition: each condition True at least once and False at least once

• Branch/Condition: both Branch and Condition coverage achieved

Page 67: Software Quality and Testing_Se lect18 btech

Types of Logic Coverage (cont’d)

• Compound Condition: all combinations of condition values at every branch statement covered (and every entry point taken)

• Path: all program paths traversed at least once

Page 68: Software Quality and Testing_Se lect18 btech

Control Flow Graphs

“nodes”

“edges”

Page 69: Software Quality and Testing_Se lect18 btech

Statement Coverage

• Statement Coverage requires that each statement will have been executed at least once.

• Simplest form of logic coverage.• Also known as Node Coverage.

Page 70: Software Quality and Testing_Se lect18 btech

Statement Coverage

• Statement should be executed completely or not at all.

• For example:

IF a THEN b ENDIF

is considered as more than 1 statement since b may or may not be executed depending on the

condition a.

Page 71: Software Quality and Testing_Se lect18 btech

Branch Coverage

• Branch Coverage requires that each branch will have been traversed, and that every program entry point will have been taken, at least once.

• Also known as Edge Coverage.

Page 72: Software Quality and Testing_Se lect18 btech

Branch Coverage

• Attempting to cover all the paths in the software is called path coverage

• The simplest form of path testing is called branch testing

________________________________Print “Hello World”IF Date == “01-01-2005” THEN

Print “Happy New Year”ELSE

Print “Have a good day!”END IFPrint “The date is: ”+DatePrint “The time is: ”+TimeEND

________________________________• To test each branch you would need to execute 2 test

cases, one with the date set to 1st Jan 2005, and another with any other date.

Page 73: Software Quality and Testing_Se lect18 btech

Condition Coverage

• A branch predicate may have more than one condition.

input(X,Y)if (Y<=0) or (X=0) then

Y := -Yend_ifwhile (Y>0) and (not EOF) do

input(X)Y := Y-1

end_while

Page 74: Software Quality and Testing_Se lect18 btech

Condition Coverage (cont’d)

• Condition Coverage requires that each condition will have been True at least once and False at least once.

Page 75: Software Quality and Testing_Se lect18 btech

Condition Coverage

________________________________Print “Hello World”IF Date == “01-01-2005” AND Time ==“00:00:00” THEN

Print “Happy New Year”ELSE

Print “Have a good day!”END IFPrint “The date is: ”+DatePrint “The time is: ”+TimeEND

________________________________• Condition Coverage takes extra conditions into

consideration• To achieve full condition coverage you would need 4 test

cases

Page 76: Software Quality and Testing_Se lect18 btech

Branch/Condition Coverage

• Branch/Condition Coverage requires that both Branch AND Condition Coverage will have been achieved.

• Therefore, Branch/Condition Coverage subsumes both Branch Coverage and Condition Coverage.

Page 77: Software Quality and Testing_Se lect18 btech

Compound Condition Coverage (cont’d)

• Compound Condition Coverage requires that all combinations of condition values at every branch statement will have been covered, and that every entry point will have been taken, at least once.

• Also know as Multiple Condition Coverage.

• Subsumes Branch/Condition Coverage, regardless of the order in which conditions are evaluated.

Page 78: Software Quality and Testing_Se lect18 btech

Path Coverage

• Path Coverage requires that all program paths will have been traversed at least once.

• Often described as the “strongest” form of logic coverage?

• Path Coverage is usually impossible when loops are present. (How many test cases would be required to cover all paths in the example below?)

Page 79: Software Quality and Testing_Se lect18 btech

Loop Coverage

• Loop Coverage requires that the body of loops be executed 0, 1, 2, t, max, and max+1 times, where possible.

Page 80: Software Quality and Testing_Se lect18 btech

Loop Coverage (cont’d)

• Rationale: (cont’d)– Check loop re-initialization.– Check typical number of iterations.– Check upper (valid) bound on number of times

body may be executed.– If the maximum can be exceeded, what behavior

results?

Page 81: Software Quality and Testing_Se lect18 btech

Flow Graph Notation

• Each circle, called a flow graph node, represents one or more procedural statements.

• A sequence of process boxes and a decision diamond can map into a single node.

• The arrows on the flow graph, called edges or links, represent flow of control and are analogous to flowchart arrows.

Page 82: Software Quality and Testing_Se lect18 btech

Flow Graph Notation

• An edge must terminate at a node, even if the node does not represent any procedural statements (e.g., see the symbol for the if-then-else construct).

• Areas bounded by edges and nodes are called regions.

• When counting regions, we include the area outside the graph as a region.

Page 83: Software Quality and Testing_Se lect18 btech

Flow Graph Notation

• When compound conditions are encountered in a procedural design, the generation of a flow graph becomes slightly more complicated.

• A compound condition occurs when one or more Boolean operators (logical OR, AND, NAND, NOR) is present in a conditional statement.

Page 84: Software Quality and Testing_Se lect18 btech

Flow Graph Notation

• Note that a separate node is created for each of the conditions a and b in the statement IF a OR b.

• Each node that contains a condition is called a predicate node and is characterized by two or more edges emanating from it.

Page 85: Software Quality and Testing_Se lect18 btech

How to draw Control flow graph?

• Sequence:– 1 a=5;– 2 b=a*b-1;

85

1

2

Page 86: Software Quality and Testing_Se lect18 btech

How to draw Control flow graph?

• Selection:– 1 if(a>b) then– 2 c=3;– 3 else c=5;– 4 c=c*c;

86

1

2 3

4

Page 87: Software Quality and Testing_Se lect18 btech

How to draw Control flow graph?

• Iteration:– 1 while(a>b){– 2 b=b*a;– 3 b=b-1;}– 4 c=b+d;

87

1

2

3

4

Page 88: Software Quality and Testing_Se lect18 btech

Example

• int f1(int x,int y){ • 1 while (x != y){• 2 if (x>y) then • 3 x=x-y;• 4 else y=y-x;• 5 }• 6 return x; }

88

1

2

3 4

5

6

Page 89: Software Quality and Testing_Se lect18 btech

How to draw Control flow graph?

• Iteration:– 1 while(a>b){– 2 b=b*a;– 3 b=b-1;}– 4 c=b+d;

89

1

2

3

4

Page 90: Software Quality and Testing_Se lect18 btech

Cyclomatic Complexity

• Cyclomatic complexity is a software metric that provides a quantitative measure of the logical complexity of a program.

• It defines the number of independent paths in the basis set of a program and provides us with an upper bound for the number of tests that must be conducted to ensure that all statements have been executed at least once.

Page 91: Software Quality and Testing_Se lect18 btech

Cyclomatic Complexity

Page 92: Software Quality and Testing_Se lect18 btech

Complexity is computed in one of three ways

• The number of regions of the flow graph correspond to the cyclomatic complexity.

• Cyclomatic complexity, V(G), for a flow graph, G, is defined as

• where E is the number of flow graph edges, N is the number of flow graph nodes.

Page 93: Software Quality and Testing_Se lect18 btech

Complexity is computed in one of three ways

• Cyclomatic complexity, V(G), for a flow graph, G, is also defined as

• where P is the number of predicate nodes contained in the flow graph G.

Page 94: Software Quality and Testing_Se lect18 btech

Complexity is computed in one of three ways

Page 95: Software Quality and Testing_Se lect18 btech

How to decide no. of test cases

• More important, the value for V(G) provides us with an upper bound for the number of independent paths that form the basis set and, by implication, an upper bound on the number of tests that must be designed and executed to guarantee coverage of all program statements.

Page 96: Software Quality and Testing_Se lect18 btech

How to derive test cases

• Using the design or code as a foundation, draw a corresponding flow graph.

• Determine the cyclomatic complexity of the resultant flow graph.

• Determine a basis set of linearly independent paths.

• Prepare test cases that will force execution of each path in the basis set.

Page 97: Software Quality and Testing_Se lect18 btech

Example Control Flow Graph

97

1

2

3 4

5

6

Cyclomatic complexity = E-N+2 7-6+2 = 3.

Page 98: Software Quality and Testing_Se lect18 btech

Cyclomatic complexity

• Another way of computing cyclomatic complexity: – inspect control flow graph

– determine number of bounded areas in the graph

• V(G) = Total number of bounded areas + 1

98

Page 99: Software Quality and Testing_Se lect18 btech

Example Control Flow Graph

99

1

2

3 4

5

6

From a visual examination of the CFG:the number of bounded areas is 2. cyclomatic complexity = 2+1=3.

Page 100: Software Quality and Testing_Se lect18 btech

Example• int f1(int x,int y){ • 1 while (x != y){• 2 if (x>y) then • 3 x=x-y;• 4 else y=y-x;• 5 }• 6 return x; }

100

1

2

3 4

5

6

• Number of independent paths: 3– 1,6 test case (x=1, y=1)– 1,2,3,5,1,6 test case(x=1, y=2)– 1,2,4,5,1,6 test case(x=2, y=1)

Page 101: Software Quality and Testing_Se lect18 btech

101

2

3

4 6

7

8

9 11

12

y == 0 y != 0

a > 0 a ≤ 0

1. void myflow(int x, int y) {2. int a;3. if (y == 0) {4. a = 1;5. } else {6. a = y; }7. y = y – 1;8. if (a > 0) {9. a = x;10. } else {11. a = x + 1; }12. printf(“%d\n”, a); }

Page 102: Software Quality and Testing_Se lect18 btech

White-box Test Methods

• Statement Coverage

• Decision/Branch Coverage

• Condition Coverage

• Decision/Condition Coverage

• Path Coverage

Page 103: Software Quality and Testing_Se lect18 btech

Example Code Fragment

• If ((A>1) & (B=0)) then Do;

• X=X/A;• END;• If ((A==2) | (X>1)) then

Do;• X=X+1;• END;• END;

Page 104: Software Quality and Testing_Se lect18 btech

Statement Coverage

• Exercise all statements at least once

• How many test cases?A=2 and B=0 (ace)

Page 105: Software Quality and Testing_Se lect18 btech

Decision/Branch Coverage

• Each decision has a true and a false outcome at least once

• How many test cases?A=2 and B=0 (ace)A=1 and X=1 (abd)

Page 106: Software Quality and Testing_Se lect18 btech

Condition Coverage

• Each condition in a decision takes on all possible outcomes at least once

• Conditions: A>1, B=0, A=2, X>1

• How many test cases?A=2, B=0, and X=4 (ace)A=1, B=1, and X=1 (abd)

Page 107: Software Quality and Testing_Se lect18 btech

Decision/Condition Coverage

• Each condition in a decision takes on all possible outcomes at least once, and each decision takes on all possible outcomes at least once

• How many test cases? A=2, B=0, and X=4 (ace) A=1, B=1, and X=1 (abd)

• What about these? A=1, B=0, and X=3 A=2, B=1, and X=1

(abe)

(abe)

Page 108: Software Quality and Testing_Se lect18 btech

Multiple Condition Coverage

• Exercise all possible combinations of condition outcomes in each decision

• Conditions:

A>1, B=0A>1, B<>0A<=1, B=0A<=1, B<>0

A=2, X>1A=2, X<=1A<>2, X>1A<>2, X<=1

Page 109: Software Quality and Testing_Se lect18 btech

Multiple Condition Coverage

• How many test cases?A=2, B=0, X=4A=2, B=1, X=1A=1, B=0, X=2A=1, B=1, X=1

(ace)

(abe)

(abe)(abd)

Page 110: Software Quality and Testing_Se lect18 btech

Path Coverage

• Every unique path through the program is executed at least once

• How many test cases?A=2, B=0, X=4 (ace)A=2, B=1, X=1 (abe)A=3, B=0, X=1 (acd)A=1, B=1, X=1 (abd)

Page 111: Software Quality and Testing_Se lect18 btech

Graph Matrices

• A graph matrix is a square matrix whose size (i.e., number of rows and columns) is equal to the number of nodes on the flow graph.

• Each row and column corresponds to an identified node, and matrix entries correspond to connections (an edge) between nodes.

Page 112: Software Quality and Testing_Se lect18 btech

Graph Matrices

Page 113: Software Quality and Testing_Se lect18 btech

Graph Matrices

• However, by adding a link weight to each matrix entry, the graph matrix can become a powerful tool for evaluating program control structure during testing.

• The link weight provides additional information about control flow.

• In its simplest form, the link weight is 1 (a connection exists) or 0 (a connection does not exist).

Page 114: Software Quality and Testing_Se lect18 btech

Graph Matrices

Page 115: Software Quality and Testing_Se lect18 btech

STRUCTURED PROGRAMMING

Page 116: Software Quality and Testing_Se lect18 btech

Graph Matrices for testing

• A graph matrix is a square matrix whose size (i.e., number of rows and columns) is equal to the number of nodes on the flow graph.

• Each row and column corresponds to an identified node, and matrix entries correspond to connections (an edge) between nodes.

Page 117: Software Quality and Testing_Se lect18 btech

Graph Matrices for testing

Page 118: Software Quality and Testing_Se lect18 btech

Graph Matrices for testing

• However, by adding a link weight to each matrix entry, the graph matrix can become a powerful tool for evaluating program control structure during testing.

• The link weight provides additional information about control flow.

• In its simplest form, the link weight is 1 (a connection exists) or 0 (a connection does not exist).

Page 119: Software Quality and Testing_Se lect18 btech

Graph Matrices for testing

Page 120: Software Quality and Testing_Se lect18 btech

1. Loop Testing

2. Data Flow Testing

Structured Programming and TestingStructured Programming and Testing

Page 121: Software Quality and Testing_Se lect18 btech

.

Loop testing

Four loops can be defined:

•Simple loops

•Concatenate loops

•Nested loops and

•Unstructured loops

Page 122: Software Quality and Testing_Se lect18 btech

Simple loop

• 7 different test cases made:– Skip loop entirely– Only one pass through loop– Two passes through loop– m passes through loop (m < n, where n

is the maximum number)– n – 1– n– n + 1

Page 123: Software Quality and Testing_Se lect18 btech

Simple loop example

int i;

while (i >= 1 && i <= 10){

i++;

}i Number of

iterations

0 or 11 0 (Skip loop entirely)

10 1 (one pass)

9 2 (two passes)

5 6 (pass M times)

2 9 (pass n – 1 times)

1 10 (pass n times)

0 0 (pass n + 1 times)

Page 124: Software Quality and Testing_Se lect18 btech

Nested loops

• Start with inner loop. Set all other loops to minimum values.

• Conduct simple loop testing on inner loop

• Work outwards

• Continue until all loops tested

Page 125: Software Quality and Testing_Se lect18 btech

Nested loops example

int n = 10;

for (int i = 0; i < n; i++){

for (int j = 0; j < n; j++){

}

}

Page 126: Software Quality and Testing_Se lect18 btech

Concatenated loops

• Concatenated loops can be tested using the approach defined for simple loops, if each of the loops is independent of the other.

• However, if two loops are concatenated and the loop counter for loop 1 is used as the initial value for loop 2, then the loops are not independent.

• When the loops are not independent, the approach applied to nested loops is recommended.

Page 127: Software Quality and Testing_Se lect18 btech

Concatenated Loops

• If independent loops, use simple loop

testing

• If dependent, treat as nested loops

Page 128: Software Quality and Testing_Se lect18 btech

Concatenated loops example

int n = 10;int m = 20;while (i < n){…

}int j = i;while (j < m){…

}

Page 129: Software Quality and Testing_Se lect18 btech

Unstructured loops

• This kind of loop must be redesigned instead of testing.

Page 130: Software Quality and Testing_Se lect18 btech

Data Flow Testing

• The data flow testing method selects test paths of a program according to the locations of definitions and uses of variables in the program

• Data flow testing is a powerful tool to detect improper use of data values due to coding errors– Incorrect assignment or input statement– Definition is missing (use of null definition)– Predicate is faulty (incorrect path is taken which leads

to incorrect definition)

130

Page 131: Software Quality and Testing_Se lect18 btech

Data Flow Testing

Variables that contain data values have a defined life cycle: created, used, killed (destroyed).

The "scope" of the variable { // begin outer block int x; // x is defined as an integer within this outer block ...; // x can be accessed here { // begin inner block int y; // y is defined within this inner block ...; // both x and y can be accessed here } // y is automatically destroyed at the end of // this block ...; // x can still be accessed, but y is gone } // x is automatically destroyed

131

Page 132: Software Quality and Testing_Se lect18 btech

The General Idea

• Data flow testing can be performed at two conceptual levels.– Static data flow testing

– Dynamic data flow testing

• Static data flow testing– Identify potential defects, commonly known as data flow anomaly.

– Analyze source code.

– Do not execute code.

• Dynamic data flow testing– Involves actual program execution.

– Bears similarity with control flow testing.

• Identify paths to execute them.

• Paths are identified based on data flow testing criteria.

132

Page 133: Software Quality and Testing_Se lect18 btech

Data Flow Anomaly

• Anomaly: It is an abnormal way of doing something.

– Example 1: The second definition of x overrides the first.x = f1(y);x = f2(z);

• Three types of abnormal situations with using variable.

– Type 1: Defined and then defined again

– Type 2: Undefined but referenced

– Type 3: Defined but not referenced

133

Page 134: Software Quality and Testing_Se lect18 btech

Data flow analysis (DFA)’s anomalies

Detects Defines/reference anomalies, such as A variable that is defined but never used

(unused variable) A variable that is used but never defined

(undefined variable) A variable that is defined twice before it is used

(redundant operations) Data reads from location not previously written to

(uninitialized variables) The anomalies can be discovered by “static

analysis”

Page 135: Software Quality and Testing_Se lect18 btech

Data Flow Testing

• Three possibilities exist for the first occurrence of a variable through a program path:– ~d - the variable does not exist (indicated by the ~),

then it is defined (d)– ~u - the variable does not exist, then it is used (u)– ~k - the variable does not exist, then it is killed or

destroyed (k)

135

Page 136: Software Quality and Testing_Se lect18 btech

Data Flow Testing

Time-sequenced pairs of defined (d), used (u), and killed (k):dd - Defined and defined again—not invalid but suspicious.

Probably a programming error.du - Defined and used—perfectly correct. The normal case.dk - Defined and then killed—not invalid but probably a

programming error.uu - Used and used again—acceptable.uk - Used and killed—acceptable.kd - Killed and defined—acceptable. A variable is killed and then

redefined.ku - Killed and used—a serious defect. Using a variable that does

not exist or is undefined is always an error.kk - Killed and killed—probably a programming error

136

Page 137: Software Quality and Testing_Se lect18 btech

Data Flow Testing

• A data flow graph is similar to a control flow graph in that it shows the processing flow through a module.

In addition, it details the definition, use, and destruction of each of the module's variables.

• Technique– Construct diagrams– Perform a static test of the diagram– Perform dynamic tests on the module

137

Page 138: Software Quality and Testing_Se lect18 btech

Data Flow Testing

Perform a static test of the diagramFor each variable within the

module we will examine define-use-kill patterns along the control flow paths

The define-use-kill patterns for x (taken in pairs as we follow the paths) are:~define - correct, the normal

casedefine-define - suspicious,

perhaps a programming errordefine-use - correct, the

normal case

138

Page 139: Software Quality and Testing_Se lect18 btech

Limitation

• The tester must have sufficient programming skill to understand the code and its control flow.

• Can be very time consuming because of all the modules and basis paths that comprise a system

139

Page 140: Software Quality and Testing_Se lect18 btech

Tabular Design Notation

• In many software applications, a module may be required to evaluate a complex combination of conditions and select appropriate actions based on these conditions.

• Decision tables provide a notation that translates actions and conditions into a tabular form.

Page 141: Software Quality and Testing_Se lect18 btech

Tabular Design Notation

• The table is difficult to misinterpret and may even be used as a machine readable input to a table driven algorithm.

Page 142: Software Quality and Testing_Se lect18 btech

Tabular Design Notation

• The table is divided into four sections.

Page 143: Software Quality and Testing_Se lect18 btech

Tabular Design Notation

• The upper left-hand quadrant contains a list of all conditions . The lower left-hand quadrant contains a list of all actions that are possible based on combinations of conditions.

Page 144: Software Quality and Testing_Se lect18 btech

Tabular Design Notation

• The right-hand quadrants form a matrix that indicates condition combinations and the corresponding actions that will occur for a specific combination. Therefore, each column of the matrix may be interpreted as a processing rule.

Page 145: Software Quality and Testing_Se lect18 btech

Tabular Design Notation

The following steps are applied to develop a decision table:

1. List all actions that can be associated with a specific procedure (or module).

2. List all conditions (or decisions made) during execution of the procedure.

Page 146: Software Quality and Testing_Se lect18 btech

Tabular Design Notation

• Associate specific sets of conditions with specific actions, eliminating imposible combinations of conditions; alternatively, develop every possible permutation of conditions.

• Define rules by indicating what action(s) occurs for a set of conditions.

Page 147: Software Quality and Testing_Se lect18 btech

Example

• If the customer account is billed using a fixed rate method, a minimum monthly charge is assessed for consumption of less than 100 KWH (kilowatt-hours).

• Otherwise, computer billing applies a Schedule A rate structure.

• However, if the account is billed using a variable rate method, a Schedule A rate structure will apply to consumption below 100 KWH, with additional consumption billed according to Schedule B

Page 148: Software Quality and Testing_Se lect18 btech
Page 149: Software Quality and Testing_Se lect18 btech

Program Design Language

• Program design language (PDL), also called structured English or pseudocode, is "a language in that it uses the vocabulary of one language (i.e., English) and the overall syntax of a structured programming language.

• A program design language may be a simple transposition of a language such C.

Page 150: Software Quality and Testing_Se lect18 btech

Program Design Language

• At first glance PDL looks like a modern programming language.

• The difference between PDL and a real programming language lies in the use of narrative text (e.g., English) embedded directly within PDL statements.

• Given the use of narrative text embedded directly into a syntactical structure, PDL cannot be compiled (at least not yet).

• However, PDL tools currently exist to translate PDL into a programming language “skeleton” and/or a graphical representation (e.g., a flowchart) of design.

Page 151: Software Quality and Testing_Se lect18 btech

PDL characteristics

• A fixed syntax of keywords that provide for all structured constructs, data declaration, and modularity characteristics.

• A free syntax of natural language that describes processing features.

• Data declaration facilities that should include both simple (scalar, array) and complex (linked list or tree) data structures.

• Subprogram description and calling techniques that support various modes of interface description.

Page 152: Software Quality and Testing_Se lect18 btech

PDL characteristics

• A basic PDL syntax should include – Interface description, – Data declaration, – Techniques for block structuring, – Condition constructs,– Repetition constructs, – I/O constructs. The format and semantics for some of

these

Page 153: Software Quality and Testing_Se lect18 btech

A PDL Example

• The system monitors alarms for fire, smoke, burglar, water, and temperature (e.g., furnace breaks while homeowner is away ) and produces an alarm bell and calls a monitoring service, generating a voice-synthesized message.

Page 154: Software Quality and Testing_Se lect18 btech

A PDL Example

• Recall that PDL is not a programming language. The designer can adapt as required without worry of syntax errors.

Page 155: Software Quality and Testing_Se lect18 btech

A PDL Example

Page 156: Software Quality and Testing_Se lect18 btech

A PDL Example

Page 157: Software Quality and Testing_Se lect18 btech

A PDL Example

Page 158: Software Quality and Testing_Se lect18 btech

A PDL Example

• Note that the designer for the security.monitor component has used a new construct PARBEGIN . . . ENDPAR that specifies a parallel block.

• All tasks specified within the PARBEGIN block are executed in parallel.

Page 159: Software Quality and Testing_Se lect18 btech

COMPARISON OF DESIGN NOTATION

• Modularity. Design notation should support the development of modular software and provide a means for interface specification.

• Overall simplicity. Design notation should be relatively simple to learn, relatively easy to use, and generally easy to read.

• Ease of editing. The procedural design may require modification as the software process proceeds. The ease with which a design representation can be edited can help facilitate each software engineering task.

Page 160: Software Quality and Testing_Se lect18 btech

COMPARISON OF DESIGN NOTATION

• Machine readability. Notation that can be input directly into a computer-based development system offers significant benefits.

• Maintainability. Software maintenance is the most costly phase of the software life cycle. Maintenance of the software configuration nearly always means maintenance of the procedural design representation.

Page 161: Software Quality and Testing_Se lect18 btech

COMPARISON OF DESIGN NOTATION

• Structure enforcement. The benefits of a design approach that uses structured programming concepts have already been discussed. Design notation that enforces the use of only the structured constructs promotes good design practice.

• Automatic processing. A procedural design contains information that can be processed to give the designer new or better insights into the correctness and quality of a design. Such insight can be enhanced with reports provided via software design tools.

Page 162: Software Quality and Testing_Se lect18 btech

COMPARISON OF DESIGN NOTATION

• Data representation. The ability to represent local and global data is an essential element of component-level design. Ideally, design notation should represent such data directly.

• Logic verification. Automatic verification of design logic is a goal that is paramount during software testing. Notation that enhances the ability to verify logic greatly improves testing adequacy.

Page 163: Software Quality and Testing_Se lect18 btech

COMPARISON OF DESIGN NOTATION

• "Code-to" ability. The software engineering task that follows component-level design is code generation. Notation that may be converted easily to source code reduces effort and error.