ISTQB / ISEB Foundation Exam Practice - 4

Dynamic Testing Techniques

1 Principles 2 Lifecycle

4 Dynamic testtechniques

3 Static testing

5 Management 6 Tools

Software Testing

ISTQB / ISEB Foundation Exam Practice

Chapter 4

Contents

What is a testing technique?

Black and White box testing

Black box test techniques

White box test techniques

Error Guessing


1 2 3

4 5 6


Why dynamic test techniques?

Exhaustive testing (use of all possible inputs and conditions) is impractical- must use a subset of all possible test casesmust use a subset of all possible test casesmust have high probability of detecting faultsmust have high probability of detecting faults

Need thought processes that help us select test cases more intelligently- test case design techniques are such thought test case design techniques are such thought

processesprocesses


a procedure for selecting or designing tests based on a structural or functional model of

the software successful at finding faults 'best' practice a way of deriving good test cases a way of objectively measuring a test effort

Testing should be rigorous, thorough and systematicTesting should be rigorous, thorough and systematic

Using techniques makes testing much more effectiveUsing techniques makes testing much more effective

Advantages of techniques

Different people: similar probability find faults- gain some independence of thoughtgain some independence of thought

Effective testing: find more faults- focus attention on specific types of faultfocus attention on specific types of fault- know you're testing the right thing know you're testing the right thing

Efficient testing: find faults with less effort- avoid duplicationavoid duplication- systematic techniques are measurablesystematic techniques are measurable

Measurement

Objective assessment of thoroughness of testing (with respect to use of each technique)- useful for comparison of one test effort to anotheruseful for comparison of one test effort to another

E.g.

Project A60% Equivalence

partitions50% Boundaries75% Branches

Project B40% Equivalence

partitions45% Boundaries60% Branches

Contents





Error Guessing


1 2 3

4 5 6


Three types of systematic technique

Static (non-execution)

• examination of documentation,examination of documentation,source code listings, etc.source code listings, etc.

Functional (Black Box)

• based on behaviour /based on behaviour /functionality of softwarefunctionality of software

Structural (White Box)

• based on structurebased on structureof softwareof software

Some test techniquesStaticStatic DynamicDynamic

StructuralStructural

BehaviouralBehavioural

FunctionalFunctionalNon-functionalNon-functional

ReviewsReviews

WalkthroughsWalkthroughs

Desk-checkingDesk-checking

DataFlowDataFlow

SymbolicExecutionSymbolicExecution

Definition-UseDefinition-Use

StatementStatement

Branch/DecisionBranch/Decision

Branch ConditionBranch Condition

Branch ConditionCombinationBranch ConditionCombination

LCSAJLCSAJ

ArcsArcs

EquivalencePartitioningEquivalencePartitioning

BoundaryValue AnalysisBoundaryValue Analysis

Cause-Effect GraphingCause-Effect Graphing

RandomRandom

UsabilityUsability

PerformancePerformance

Static AnalysisStatic AnalysisInspectionInspection

ControlFlowControlFlow

etc.etc.

etc.etc.

etc.etc.

etc.etc.

etc.etc.

State TransitionState Transition

Black box versus white box?

IntegrationIntegration

ComponentComponent

AcceptanceAcceptance

SystemSystem

Black box appropriateat all levels butdominates higherlevels of testing

White box usedpredominatelyat lower levelsto complimentblack box

Contents





Error Guessing


1 2 3

4 5 6


Black Box test design and measurement techniques

Techniques defined in BS 7925-2- Equivalence partitioningEquivalence partitioning- Boundary value analysisBoundary value analysis- State transition testingState transition testing- Cause-effect graphingCause-effect graphing- Syntax testingSyntax testing- Random testingRandom testing

Also defines how to specify other techniques

= Yes= No

Also a measurementtechnique?

Equivalence partitioning (EP)

- divide (partition) the inputs, outputs, etc. into areas divide (partition) the inputs, outputs, etc. into areas which are the same (equivalent)which are the same (equivalent)

- assumption: if one value works, all will workassumption: if one value works, all will work- one from each partition better than all from oneone from each partition better than all from one

1 100 1010

valid invalidinvalid

Boundary value analysis (BVA)

- faults tend to lurk near boundariesfaults tend to lurk near boundaries- good place to look for faultsgood place to look for faults- test values on both sides of boundariestest values on both sides of boundaries

1 100 1010

valid invalidinvalid

Example: Loan application

Customer Name

Account number

Loan amount requested

Term of loan

Monthly repayment

Term:

Repayment:

Interest rate:

Total paid back:

6 digits, 1stnon-zero

£500 to £9000

1 to 30 years

Minimum £10

2-64 chars.

Customer name

Number of characters:

2 64 65invalid valid invalid

1

Conditions ValidPartitions

InvalidPartitions

ValidBoundaries

InvalidBoundaries

Customername

2 to 64 charsvalid chars

< 2 chars> 64 charsinvalid chars

2 chars64 chars

1 chars65 chars0 chars

Valid characters:Anyother

A-Za-z-’

space

Account number

5 6 7invalid

validinvalid

number of digits:

first character:invalid: zero

valid: non-zero


InvalidPartitions

ValidBoundaries

InvalidBoundaries

Accountnumber

6 digits1st non-zero

< 6 digits> 6 digits1st digit = 0non-digit

100000999999

5 digits7 digits0 digits

Loan amount

500 9000 9001

invalid valid invalid

499


InvalidPartitions

ValidBoundaries

InvalidBoundaries

Loanamount

500 - 9000 < 500>90000non-numericnull

5009000

4999001

Condition template


Tag InvalidPartitions

Tag ValidBoundaries

Tag InvalidBoundaries

Tag

Customername

2 - 64 charsvalid chars

V1V2

< 2 chars> 64 charsinvalid char

X1X2X3

2 chars64 chars

B1B2

1 char65 chars0 chars

D1D2D3

Accountnumber

6 digits1st non-zero

V3V4

< 6 digits> 6 digits1st digit = 0non-digit

X4X5X6X7

100000999999

B3B4

5 digits7 digits0 digits

D4D5D6

Loanamount

500 - 9000 V5 < 500>90000non-integernull

X8X9X10X11X12

5009000

B5B6

4999001

D7D8

Design test cases

TestCase

Description Expected Outcome New TagsCovered

1

2

Name: John SmithAcc no: 123456Loan: 2500Term: 3 years

Name: ABAcc no: 100000Loan: 500Term: 1 year

Term: 3 yearsRepayment: 79.86Interest rate: 10%Total paid: 2874.96

Term: 1 yearRepayment: 44.80Interest rate: 7.5%Total paid: 537.60

V1, V2,V3, V4,V5 .....

B1, B3,B5, .....

Why do both EP and BVA?

If you do boundaries only, you have covered all the partitions as well- technically correct and may be OK if everything works technically correct and may be OK if everything works

correctly!correctly!- if the test fails, is the whole partition wrong, or is a if the test fails, is the whole partition wrong, or is a

boundary in the wrong place - have to test mid-partition boundary in the wrong place - have to test mid-partition anywayanyway

- testing only extremes may not give confidence for testing only extremes may not give confidence for typical use scenarios (especially for users)typical use scenarios (especially for users)

- boundaries may be harder (more costly) to set upboundaries may be harder (more costly) to set up

Test objectives?

For a thorough approach: VP, IP, VB, IB Under time pressure, depends on your test objective

- minimal user-confidence: VP only?minimal user-confidence: VP only?- maximum fault finding: VB first (plus IB?)maximum fault finding: VB first (plus IB?)

Condition ValidPartition

Tag InvalidPartition

Tag ValidBoundary

Tag InvalidBoundary

Tag

Decision tables

explore combinations of inputs, situations or events,

it is very easy to overlook specific combinations of input

start by expressing the input conditions of interest so that they are either TRUE or FALSE

– record found– file exists– code valid

– policy expired– account in credit– due date > current date

Example: student access

A university computer system allows students an allocation of disc space depending on their projects.

If they have used all their allotted space, they are only allowed restricted access, i.e. to delete files, not to create them. This is assuming they have logged on with a valid username and password.

What are the input and output conditions?What are the input and output conditions?

List the input and output conditions

• list the ‘output conditions’ under the input conditions

Input ConditionsValid usernameValid passwordAccount in creditOutput ConditionsLogin acceptedRestricted access

• list the ‘input conditions’ in the first column of the table

Determine input combinations

add columns to the table for each unique combination of input conditions.

each entry in the table may be either ‘T’ for true, ‘F’ for false.

Input ConditionsValid username T T T T F F F FValid password T T F F T T F FAccount in credit T F T F T F T F

Rationalise input combinations

some combinations may be impossible or not of interest

some combinations may be ‘equivalent’ use a hyphen to denote “don’t care”

Input ConditionsValid username F T T TValid password - F T TAccount in credit - - F T

Complete the table

determine the expected output conditions for each combination of input conditions

Input ConditionsValid username F T T TValid password - F T TAccount in credit - - F TOutput ConditionsLogin accepted F F T TRestricted access - - T F

Determine test case groups each column is at least one test case

Input ConditionsValid username F T T TValid password - F T TAccount in credit - - F TOutput ConditionsLogin accepted F F T TRestricted access - - T F

Tags A B C D

Design test cases

usually one test case for each column but can be none or several

Test Description Expected Outcome Tag1 Username BrbU Invalid username A2 Username

usernametoolongInvalid username A

3 Username BobUPassword abcd

Invalid password B

4 Valid user, no discspace

Restricted access C

5 Valid user with discspace

Unrestricted access D

Rationalising outputs

X T F F

Y F T FZ F F T Output X Y Z

if outputs or effects are mutually exclusive, I.e. T occurs in only one place in each column, we can combine them

for example:

is equivalent to:

Rationalising dangers

rationalising is based on assumptions assumptions may be wrong! assumptions should be stated assumptions may change over time be aware of the dangers

- filling in the full table may find errors which will be filling in the full table may find errors which will be missed if you rationalisemissed if you rationalise

- it is possible to rationalise too farit is possible to rationalise too far

Extending decision tables

Entries can be more than just ‘true’ or ‘false’- completing table needs to be done carefullycompleting table needs to be done carefully- rationalising becomes more importantrationalising becomes more important

E.g.

Code = 1, 2, or 3 1 1 1 1 2 2 2 2 3 3 3 3Exp.date < now T T F F T T F F T T F FClass A product T F T F T F T F T F T F

Decision Tables in relation to EP and BVA

Input value Output value

FALSE

TRUE

Contents





Error Guessing


1 2 3

4 5 6


White Box test design and measurement techniques

Techniques defined in BS 7925-2- Statement testingStatement testing- Branch / Decision testingBranch / Decision testing- Data flow testingData flow testing- Branch condition testingBranch condition testing- Branch condition combination testingBranch condition combination testing- Modified condition decision testingModified condition decision testing- LCSAJ testingLCSAJ testing

Also defines how to specify other techniques

= Yes= No

Also a measurementtechnique?

Stronger structural techniques (different structural elements)

More tests

Using structural coverage

Increasing coverage

Coverage OK?

What'scovered?

Results OK?

Enoughtests?

Spec

Spec SoftwareSoftware

Tests

More testsMore testsMore testsMore testsMore testsMore testsMore tests

The test coverage trap

Structure exercised,insufficient function

Structure exercised,insufficient function

Function exercised,insufficient structure

Function exercised,insufficient structure

better testingbetter testing

% Statement % Decision % Condition Combination

Structural testedness

Functionaltestedness

100% coverage doesnot mean 100% tested!

100% coverage doesnot mean 100% tested!

Coverage is notThoroughness

Coverage is notThoroughness

Statement coverage

percentage of executable statements exercised by a test suite

number of statements exercisednumber of statements exercised

total number of statementstotal number of statements example:

- program has 100 statementsprogram has 100 statementstests exercise 87 statementstests exercise 87 statements- statement coverage = 87%statement coverage = 87%

=

Typical ad hoc testing achieves 60 - 75%Typical ad hoc testing achieves 60 - 75%

Statement coverageis normally measured

by a software tool.

Statement coverageis normally measured

by a software tool.

?

Example of statement coverage

Testcase

Input Expectedoutput

1 7 7

As all 5 statements are ‘covered’ bythis test case, we have achieved

100% statement coverage

read(a)

IF a > 6 THEN

b = a

ENDIF

print b

1

2

3

4

5

Statementnumbers

Decision coverage(Branch coverage)

percentage of decision outcomesexercised by a test suite

number of decisions outcomes exercisednumber of decisions outcomes exercised

total number of decision outcomestotal number of decision outcomes example:

- program has 120 decision outcomesprogram has 120 decision outcomestests exercise 60 decision outcomestests exercise 60 decision outcomes- decision coverage = 50%decision coverage = 50%

Typical ad hoc testing achieves 40 - 60%Typical ad hoc testing achieves 40 - 60%

=

Decision coverageis normally measured

by a software tool.

Decision coverageis normally measured

by a software tool.

True

False?

Paths through code

? ?

1 2

? ?

1231 2 ?

?

1234

Paths through code with loops

?

1 2 3 4 5 6 7 8 ….

for as many times as itis possible to go roundthe loop (this can beunlimited, i.e. infinite)

End

Selecttrans...

Yes

Example 1

Wait for card to be inserted

IF card is a valid card THEN

display “Enter PIN number”

IF PIN is valid THEN

select transaction

ELSE (otherwise)

display “PIN invalid”

ELSE (otherwise)

reject card

End

Display“Enter..

Yes

ValidPIN?

No

Rejectcard

Display“PIN in..

No

Wait for card to be insertedWait for card to be inserted













select transaction





select transaction

ELSE (otherwise)






select transaction

ELSE (otherwise)


ELSE (otherwise)

reject card

Validcard?

Wait

Read AIF A > 0 THEN IF A = 21 THEN

Print “Key” ENDIFENDIF

IF A > 0 THEN

ENDIF

Print “Key”

IF A = 21 THEN

ENDIF

Example 2

- Cyclomatic complexity: _____Cyclomatic complexity: _____- Minimum tests to achieve:Minimum tests to achieve:

• Statement coverage: ______Statement coverage: ______

• Branch coverage: _____Branch coverage: _____

3

13

Read A

Print

YesA=21

Yes

No

End

A>0

No

Read

Example 3




Read ARead BIF A > 0 THEN IF B = 0 THEN

Print “No values” ELSE

Print BIF A > 21 THEN Print AENDIF

ENDIFENDIF

4

24

PrintYes

Print

A>21

No

No

B=0

Print

Yes

Yes

Read

A>0

End

No




Read ARead BIF A < 0 THEN Print “A negative”ELSE Print “A positive”ENDIFIF B < 0 THEN Print “B negative”ELSE Print “B positive”ENDIF

3

22

Example 4 Read

End

Print

No

PrintYes

B<0

Print

No

PrintYes

A<0

Note: thereare 4 paths

End

No




Read ARead BIF A < 0 THEN Print “A negative”ENDIFIF B < 0 THEN Print “B negative”ENDIF

3

12

Example 5 Read A<0 PrintYes

No

B<0 PrintYes




Read AIF A < 0 THEN Print “A negative”ENDIFIF A > 0 THEN Print “A positive”ENDIF

3

22

Example 6 Read PrintYes

No

End

No

A>0 PrintYes

A<0

Contents





Error Guessing


1 2 3

4 5 6


Non-systematic test techniques

Trial and error / Ad hoc Error guessing / Experience-driven User Testing Unscripted Testing

A testing approach that is onlyrigorous, thorough and systematic

is incomplete

A testing approach that is onlyrigorous, thorough and systematic

is incomplete

Error-Guessing

always worth including after systematic techniques have been used can find some faults that systematic

techniques can miss a ‘mopping up’ approach supplements systematic techniques

Not a good approach to start testing withNot a good approach to start testing with

Error Guessing: deriving test cases

Consider:- past failurespast failures- intuitionintuition- experienceexperience- brain stormingbrain storming- ““What is the craziest thing we can do?”What is the craziest thing we can do?”

Summary: Key Points

Test techniques are ‘best practice’: help to find faults

Black Box techniques are based on behaviour

White Box techniques are based on structure

Error Guessing supplements systematic techniques


1 2 3

4 5 6


Technology

ISTQB / ISEB Foundation Exam Practice - 4