1

Chapter 13 & 14Software Testing

Strategiesand Techniques

Chapter 13 & 14Software Testing

Strategiesand Techniques Software Engineering: A Practitioner’s Approach, 6th edition

by Roger S. Pressman

2

Software Testing

Testing is the process of exercising aTesting is the process of exercising aprogram with the specific intent of findingprogram with the specific intent of findingerrors prior to delivery to the end user.errors prior to delivery to the end user.

3

What Testing Shows

errorserrors

requirements conformancerequirements conformance

performanceperformance

an indicationan indicationof qualityof quality

4

Who Tests the Software?

developerdeveloper independent testerindependent tester

Understands the system Understands the system

but, will test "gently"but, will test "gently"

and, is driven by "delivery"and, is driven by "delivery"

Must learn about the system,Must learn about the system,but, will attempt to break itbut, will attempt to break itand, is driven by qualityand, is driven by quality

5

Validation vs VerificationValidation vs Verification

Verification – Are we building the product right? Is the code correct with respect to its

specification?

Validation – Are we building the right product? Does the specification reflect what it should?

Verification – Are we building the product right? Is the code correct with respect to its

specification?

Validation – Are we building the right product? Does the specification reflect what it should?

6

Testing Strategy

unit testunit test integrationintegrationtesttest

validationvalidationtesttest

systemsystemtesttest

7

Testing StrategyTesting Strategy Begin with unit testing and work your way up to

system testing. Unit testing – test individual components (modules in

procedural languages; classes in OO languages) Integration testing – test collections of components

that must work together Validation testing – test the application as a whole

against user requirements System testing – test the application in the context of

an entire system

Begin with unit testing and work your way up to system testing.

Unit testing – test individual components (modules in procedural languages; classes in OO languages)

Integration testing – test collections of components that must work together

Validation testing – test the application as a whole against user requirements

System testing – test the application in the context of an entire system

8

Unit Testing

modulemoduleto beto betestedtested

test casestest cases

resultsresults

softwaresoftwareengineerengineer

9

Unit Testing

interface interface local data structureslocal data structures

boundary conditionsboundary conditionsindependent pathsindependent pathserror handling pathserror handling paths

modulemoduleto beto betestedtested

test casestest cases

10

Unit Test Environment

ModuleModule

stubstub stubstub

driverdriver

RESULTSRESULTS

interface interface

local data structureslocal data structures

boundary conditionsboundary conditions

independent pathsindependent paths

error handling pathserror handling paths

test casestest cases

11

Integration Testing Strategies

Options:Options:•• the “big bang” approachthe “big bang” approach•• an incremental construction strategyan incremental construction strategy

12

Top Down Integration

top module is tested with top module is tested with stubsstubs

stubs are replaced one at stubs are replaced one at a time, "depth first"a time, "depth first"

as new modules are integrated, as new modules are integrated, some subset of tests is re-runsome subset of tests is re-run

AA

BB

CC

DD EE

FF GG

13

Bottom-Up Integration

drivers are replaced one at a drivers are replaced one at a time, "depth first"time, "depth first"

worker modules are grouped into worker modules are grouped into builds and integratedbuilds and integrated

AA

BB

CC

DD EE

FF GG

clustercluster

14

Regression TestingRegression Testing

The selective retesting of a modified system to help ensure that no bugs have been introduced during modification.

Fixing one part of the code can break another

The selective retesting of a modified system to help ensure that no bugs have been introduced during modification.

Fixing one part of the code can break another

15

High Order TestingHigh Order Testing Validation testing

Focus is on software requirements System testing

Focus is on system integration Alpha/Beta testing

Focus is on customer usage Recovery testing

forces the software to fail in a variety of ways and verifies that recovery is properly performed Security testing

verifies that protection mechanisms built into a system will, in fact, protect it from improper penetration

Stress testing executes a system in a manner that demands resources in abnormal quantity, frequency, or volume

Performance Testing test the run-time performance of software within the context of an integrated system

Validation testing Focus is on software requirements

System testing Focus is on system integration

Alpha/Beta testing Focus is on customer usage

Recovery testing forces the software to fail in a variety of ways and verifies that recovery is properly performed

Security testing verifies that protection mechanisms built into a system will, in fact, protect it from improper

penetration Stress testing

executes a system in a manner that demands resources in abnormal quantity, frequency, or volume Performance Testing

test the run-time performance of software within the context of an integrated system

16

What is a “Good” Test?What is a “Good” Test?

A good test is one that has a high probability of finding an error.

A good test is one that has a high probability of finding an error.

17

Test Case Design

"Bugs lurk in corners "Bugs lurk in corners and congregate at and congregate at boundaries ..."boundaries ..."

Boris BeizerBoris Beizer

OBJECTIVEOBJECTIVE

CRITERIACRITERIA

CONSTRAINTCONSTRAINT

to uncover errorsto uncover errors

in a complete mannerin a complete manner

with a minimum of effort and timewith a minimum of effort and time

18

Exhaustive Testing

loop < 20 Xloop < 20 X

There are 10 possible paths! If we execute oneThere are 10 possible paths! If we execute onetest per millisecond, it would take 3,170 years totest per millisecond, it would take 3,170 years totest this program!!test this program!!

1414

19

Selective Testing

loop < 20 Xloop < 20 X

Selected pathSelected path

20

Software Testing

Methods

Strategies

white-boxmethods

black-box methods

21

White-Box Testing

... our goal is to ensure that all ... our goal is to ensure that all statements and conditions have statements and conditions have been executed at least once ...been executed at least once ...

22

Why Cover?

logic errors and incorrect assumptions logic errors and incorrect assumptions are inversely proportional to a path's are inversely proportional to a path's execution probabilityexecution probability

we often we often believebelieve that a path is not that a path is not likely to be executed; in fact, reality is likely to be executed; in fact, reality is often counter intuitiveoften counter intuitive

typographical errors are random; it's typographical errors are random; it's likely that untested paths will contain likely that untested paths will contain some some

23

Basis Path TestingFirst, we compute the cyclomatic

complexity:

number of simple decisions + 1

or

number of enclosed areas + 1

In this case, V(G) = 4

24

Cyclomatic Complexity

A number of industry studies have indicated A number of industry studies have indicated that the higher V(G), the higher the probability that the higher V(G), the higher the probability or errors.or errors.

V(G)V(G)

modulesmodules

modules in this range are modules in this range are more error pronemore error prone

25

Basis Path TestingNext, we derive the Next, we derive the

independent paths:independent paths:

Since V(G) = 4,Since V(G) = 4,there are four pathsthere are four paths

Path 1: 1,2,3,6,7,8Path 1: 1,2,3,6,7,8Path 2: 1,2,3,5,7,8Path 2: 1,2,3,5,7,8Path 3: 1,2,4,7,8Path 3: 1,2,4,7,8Path 4: 1,2,4,7,2,4,...7,8Path 4: 1,2,4,7,2,4,...7,8

Finally, we derive testFinally, we derive testcases to exercise these cases to exercise these paths.paths.

11

22

3344

55 66

77

88

26

Basis Path Testing Notes

you don't need a flow chart, you don't need a flow chart, but the picture will help when but the picture will help when you trace program pathsyou trace program paths

count each simple logical test, count each simple logical test, compound tests count as 2 or compound tests count as 2 or moremore

basis path testing should be basis path testing should be applied to critical modulesapplied to critical modules

27

Black-Box Testing

requirementsrequirements

eventseventsinputinput

outputoutput

28

Equivalence Partitioning

useruserqueriesqueries

mousemousepickspicks

outputoutputformatsformats

promptsprompts

FKFKinputinput

datadata

29

Sample Equivalence Classes

user supplied commandsuser supplied commandsresponses to system promptsresponses to system promptsfile namesfile namescomputational datacomputational data physical parameters physical parameters bounding valuesbounding values initiation valuesinitiation valuesoutput data formattingoutput data formattingresponses to error messagesresponses to error messagesgraphical data (e.g., mouse picks)graphical data (e.g., mouse picks)

data outside bounds of the program data outside bounds of the program physically impossible dataphysically impossible dataproper value supplied in wrong placeproper value supplied in wrong place

Valid dataValid data

Invalid dataInvalid data

30

Boundary Value Analysis

useruserqueriesqueries

mousemousepickspicks

outputoutputformatsformats

promptsprompts

FKFKinputinput

datadata

outputoutputdomaindomaininput domaininput domain

31

OOT Methods: Behavior TestingOOT Methods: Behavior Testing

emptyacctopen setup Accnt

set upacct

deposit(initial)

workingacct

withdrawal(final)

deadacct close

nonworkingacct

deposit

withdrawbalance

creditaccntInfo

Figure 14.3 State diagram for Account class (adapted from [ KIR94])

The tests to be The tests to be designed designed should achieve should achieve all state all state coverage coverage [KIR94]. That [KIR94]. That is, the is, the operation operation sequences sequences should cause should cause the Account the Account class to make class to make transition transition through all through all allowable allowable statesstates