software testing design_1

8/7/2019 software testing design_1

1/41

Test Design Techniques

Asking the right questions


2/41

Information objectives 1

! Find defects

! Block premature product releases

! Help managers make ship / no-ship decisions

! Minimize technical support costs

! Assess conformance to specification

Copyright (c) Cem Kaner and James Bach, [email protected]


3/41

Information objectives 2

! Conform to regulations

! Minimize safety-related lawsuit risk

! Find safe scenarios for use of the product

! Assess quality

! Verify correctness of the product

! Assure qualityCopyright (c) Cem Kaner and James Bach, [email protected]


4/41

To different degrees, good tests have

these attributes:

! Power.When a problem exists, the test will reveal it.

! Valid. When the test reveals a problem, it is a genuine problem.

! Value. It reveals things your clients want to know about the product or

project.

! Credible. Your client will believe that people will do the things that aredone in this test.

! Performable. It can be performed as designed.

! Maintainable. Easy to revise in the face of product changes.

! Repeatable. It is easy and inexpensive to reuse the test.

Modified Slide: Original by Cem Kaner and James Bach, [email protected]


5/41

Test attributes! Coverage. It exercises the product in a way that isn't

already taken care of by other tests.

! Easy to evaluate.

! Supports troubleshooting. Provides useful information forthe debugging programmer.

! Appropriately complex. As the program gets more stable,you can hit it with more complex tests and more closelysimulate use by experienced users.

! Accountable. You can explain, justify, and prove you ran it.

! Cost. This includes time and effort, as well as direct costs." See Kaner, What IS a Good Test Case?Copyright (c) Cem Kaner and James Bach, [email protected]


6/41

Requirements and How Testers can Help

making them better

! Official requirements specification (in theory)# Whatever information we can find in the project!

# Plus functionality that is NOT written but needed

# Plus all possible/likely problems that we need to test

! Create a Test Design Model# Quality assurance of the requirements

! Test analysis and design# Is about asking the right questions

# Good Test Design often finds more defectsthan test execution!


7/41

Unified Process (in Theory)

! System Requirements# Use Cases

# Supplementary Specification

# System Vision

# Additional Document describing Business Rules?

! Business Requirements# Information Model

# Processes

! Technical# System Architecture Document

# Navigation Map


8/41

Test Design The Four-Step Workflow

i. Analyse test basis and create a model

ii. Cover the model with high level test cases

iii.Add test data for low level test cases

iv.Finally do advanced testing using some creative

thinking


9/41

Examples of Models

-

Bokbestllning Testfall

Beskrivning

Prio 1 2 2 2 3 3 3

Parameter Vrde Prio Id 1 2 3 4 5 6 7 Antal Giltiga grupper

Heltal 1-99 1 X X X

Heltal 99 2 XDecimaltal 1.01-99.99 3Bokstver 3+,-,*rkneoperatorer

3

Ogiltiga grupper

vriga ickenumeriska tecken 31 2 XAntal

giltiga grnsvrden 99 X0 2Antal Ogiltigagrnsvrden

100 3

England, Nordirland, Skottland,Wales

1 X

vriga lnder i Europa 1 X X

Land grupper

vriga vrlden 1 XFrsta landet p listan 3 XLand

grnsvrden Sista landet i listan 3 XFaktura 1 X XVISA, MasterCard 1 X X

Betalningsstt grupper

Postfrskott 1 X X

H

Stoppa in kort

Verifiera giltighet

Be omkod

Mata in kod

Kontrollera kod

Mata utkort

Be om transaktion tkortet

Mata in transaktion

Kontrollera transaktion

Mata utkort ,pengar& kvitto

Felaktigt kort

Felaktig kod

A1

Felaktigtransaktion

Felaktig kod -tredje

gngen

A2

A3

A4

Sug in pengar &skrivtill fellogg

Kunden tarej pengarna

A5

Annan bankej on- line

A6


10/41

All Science is Based on Models

All models are wrong but some are useful.- Gerald Weinberg

! A good model is easy to understand

! A good model is a true simplification of reality

! Too little information makes the model incomplete

! Too much information makes the model hard to understand

! Quality Assurance of the model is very important


11/41

Test Design Models

! All techniques use some kind of model

! Problem Areas# Data tables, trees, diagrams

# Flows flow graphs

# Logic tables and trees

# Combinatorics free tools, formulas

# Event-based systems graphs, tables

# Overall tests graphs, tables, scenarios, userprofiles, syntax


12/41

Test Design Techniques 1

! Data# Equivalence Partitioning# Boundary Value Analysis# Domain Testing# Classification Trees

! Logic# Decision Tables# Decision Trees# Modified Condition Decision Coverage

! Event-based# State graph# State table


13/41

Test Design Techniques 2

! Combinatorics# All-Pairs or more using tools like PICT

! Overall Testing

# Scenarios# Charters# Data Cycles# Time Cycles# Syntax

! Flows# General thoughts on coverage# Control and data flows for unit testing# Business Process Testing# Use Case Testing


14/41

Test Data

A Test Design Technique

But Also Step Three in the Test

Design Process


15/41

Test Data

! Equivalence Partitions

# How do you group values so that it is enough(in theory) to test only one value in eachgroup?

! Boundary Values

# Some, but not all, groups have boundaries

# Defects are often found here, e.g.,

! Confusion


16/41

Example: Numerical Partition

! Order books on the Internet. Number ofbooks per ordered title is 1-99. Whichare the partitions?

1 99

Invalid Valid Invalid

Parameter Partition

Number of books

Valid

Integer 1 -99

Integer 99

-3 17 121


17/41

Is the Partitioning Correct?

! Possible factors that play a role:

# Ordering of more than one book with the same title maygive a discount

# Ordering for a certain sum may give you free postage

#

Odering over a certain amount might require pre-paiddelivery

# Ordering many books may require delivery in severalpackages

# Default values may exist

! Solution: test several values for each partition

# Min, max, default, typical, plus another one in themiddle


18/41

What else is invalid?

Parameter Partition

Number of books

Valid

Integer 1 -99

Integer 99

Decimals 1.01 -99.99Letters

+,-,*

operators

Other non -numericals like @,$

Number of books

Invalid

Empty field

1Ask for a robust design!

That will make it easier to test AND build.


19/41

and the boundary values

! For number of books 1-99

Parameter Boundary Value

1Number of books

Valid boundaries 99

0Number of books

Unvalid boundaries 100


0 1 99 100


20/41

The Complete List

! For number of books 1-99


-3 17 1210 1 99 100

! Additional test data# Empty field

# Any non-integers


21/41

PostageAdress in Great Britain: free postage

For the rest of Europe there is a special postage list.

For the rest of the world outside Europe there is another, dearer, list.

Parameter Partition

England, North ern Ireland, Scotland,

Wales

Rest of Europe

Country

Valid

Outside Europe

Country

Invalid

Empty field

all other values. May not bepossible but try anyhow!

SWE

NOREtc.

F:Postage

GBRCount

ries


22/41

Variations on partitioning I

! Interval: continuum of numerical values

# Has valid and invalid boundaries

! Sets: countries, working-days etc

# Are usually seen not to have boundaries

! Free format input field: both values and number ofcharacters

# Has valid and invalid partitions

# Has boundaries

! Logical Values: Yes/No/Not selected

# No boundaries


23/41


24/41

All partitions

Parameter PartitionInteger 1 -99

Integer 99

Decimals 1.01 -99.99

Letters

+,-,*

operators

Number of books

Valid

Number of books

Invalid

Other non -numericals like @,$1Valid boundaries

99

0Invalid boundaries

100

England, Nor thern Irelan d,ScotlandWales

Rest of Europe

Country valid

Rest of worldEmpty fieldInvalid

Other values

First Country on list

Last country on list

Country


25/41

Data 1: EP&BV: Use a Table

Employee test data Employees

Description

Parameter Partition Id 1 2 3 4 5 6 7

Name Regular name X X X

Double-name X

Short Name X X

Long Name XChanged when married X

Foreign Name X

Below 18 X XAge

18-65 X

Above 65

Regular employee X X

Temporary X

Classification

Monthly pay X XSalary

Hourly pay X

Amount

Morepartitions=better coverage


26/41

Data 2: Classification Trees

Card

Unreadable Readable

Valid

OtherATM

Invalid

BlockedExpired

Petrol Library

Our bank Other

Deeper tree=better coverage

DemoCTreeEditor


27/41

Data 3: Domain testing! Partitions created by several parameters working together

! In principle this is a set of rules

More boundary tests= better coverage


28/41

Exercise

! Define equivalence partitions andboundary values for the following:

$%Last Name(20 char)

&%Age (of a person)

'%Fulltime worker Y/N

! Any questions regarding the parameters# Believe nothing, question everything


29/41

Examples of Myers' categories

1. {5,6,7}

2. {15,15,15}

3. {3,3,4; 5,6,6; 7,8,7}

4. {0,1,1; 2,0,2; 3,2,0; 0,0,9; 0,8,0; 11,0,0; 0,0,0}

5. {3,4,-6}

6. {1,2,3; 2,5,3; 7,4,3}

7. {1,2,4; 2,6,2; 8,4,2}

8. {Q,2,3}

9. {2,4; 4,5,5,6}

(examples courtesy of Doug Hoffman)


30/41

Testing Logic

Decision Tables

Decision Trees


31/41

Logic

! Variations

# Set of Rules

# Conditions

# Formulas

! Design techniques

# Decision Table

# Decision Tree

Explicit

variations

Age Sex Car Type Fee

1


32/41

Format used

! Either put the rules in rows or in columns

! One rule per column one test case per row -makes is easy to create test data

! Enter Y/N, 1/0 or X or the value itself

NoX1NResult 2

3000YResult 1

TrueX0NCondition 2

< 5X1YCondition 1

Rule 4Rule 3Rule 2Rule 1


33/41

Business Rules: Car Insurance! Insurance price depends on a number of factors

$% Business Rule 1: As a starting point,the excess for all policy holdersis 500 kronor.

&% Business Rule 2: Men under 25 years of age are subject to an excessincreased by 1500 kronor.

'% Business Rule 3: Everyone over 25 with a familytype are subject toan excess reduced by 200 kronor.

(% Business Rule 4: All persons between 45 and 65 are subject to anexcess reduced

)% Clients above 25 with car type family have a reduced fee of 100 SEK

*% Every one between 45 and 65 have an extra reduced fee of 100 SEK

+% Car type sport gives an extra fee of 1000 SEK

,% Car type regular does not effect the fee

-% Everyone above 65 has no reductions whatsoever

! How do you test this?


34/41

Step 1: Identify Parameters?

Parameters:

age,

car type,

gender

Partitions:

Age: 65

Car Type: family, regular, sport

Gender: male, female


35/41

List Rules in a Table Are They Correct?

Explicit

variations

Age Gender Car Type Fee

1


36/41

Decision trees

! Start with the table you created. Draw a tree byselecting one rule and one parameter at a time

# Complete?

# Not overlapping?

! Create

# One high level test case for each leaf

# Low level test cases depending on boundary values

# Low level level test cases depending on logic

More combinations (rules)=better coverage


37/41

Create a Decision Tree

Missing


38/41

Add missing rule

Explicit

variations

Age Gender Car Type Fee

1


39/41

Exercise: Decision Table and Tree

! On Saturday Peter has a lot to do. What he will do dependson the circumstances. If he wakes early and it is sunnyoutside he will cut the grass. But if he sleeps late he willinstead hang the laundry.

! If he awakes early and the weather is bad and he still hasany money left then he will go shopping.

! Part A:# Create a decision table for the rules above.

# Is anything unclear?

# How many test cases do you suggest using? Why?

! Part B: Create a decision tree from the information above.# Which model is the best for testing purposes?

#Exercise: adapted from Clive Bates; Grove Consultants


40/41


41/41

Exercise: Advanced Decision Tree

Income/AgeIncomeLossNonPmtAgePropertyRule

YY8

120>426N50, 120>100, 42Y5

>75>120>100N100,

Documents

software testing design_1