Test Fundamentals For Expert Testerslets-test.com/wp-content/uploads/2014/06/2014_05_25_Test_Funde… · Conform to requirements Value to stakeholders Suit purpose. AmiBug.Com, Inc

AmiBug.Com, Inc.

Test FundamentalsFor Expert Testers

Robert Sabourin

President

AmiBug.Com, Inc.

Montreal, Canada

[email protected] 5, 2014 © Robert Sabourin, 2014 Slide 1

AmiBug.Com, Inc.

June 5, 2014 © Robert Sabourin, 2014 Slide 2

• Robert Sabourin ,

Software Evangelist

• President

• AmiBug.Com Inc.

• Montreal, Quebec,

Canada

• [email protected]

• www.amibug.com

Test FundamentalsFor Expert Testers

AmiBug.Com, Inc.

Quiz

• Application screens

are selected with

three controls:

(a) has 5 options

(b) has 6 options

(c) has 2 options

• How many screens can a user choose?


AmiBug.Com, Inc.

Quiz

• Total Combinations

= 6 x 5 x 2 = 60

• To exercise each

combination once a

total of 60 tests

would be required.


AmiBug.Com, Inc.

• How many tests

would be required to

exercise all possible

screens in every

possible order?.

Quiz


AmiBug.Com, Inc.

• To exercise all screens in every

possible order would require

60! Test cases

60! = 60 x 59 x 58 x ... 3 x 2 x 1

60! ≈ 8.32 x 10**81

Pop Quiz


AmiBug.Com, Inc.

Pop Quiz

From 7.0 × 10**79

To 1.5 × 10**82

How many atoms are in

the observable

universe?


AmiBug.Com, Inc.


Fundamental Question

• How do you know when you are finished?

AmiBug.Com, Inc.

What is Quality?


AmiBug.Com, Inc.


Crosby on Quality

• “Quality is defined as conformance to

requirements”

• “Quality is not a measure of

GOODNESS”

– Phil B. Crosby, Quality is Free

AmiBug.Com, Inc.


Gerald M. Weinberg

“Quality is value to some person”

Exploring Requirements Quality Before Design

Dorset House

AmiBug.Com, Inc.


“Quality is fitness for use”

Joseph Juran

Quality Control Handbook

AmiBug.Com, Inc.

What is Quality?


Conform to requirements

Value to stakeholders

Suit purpose

AmiBug.Com, Inc.


Edsger W. Dijkstra

• “Program testing can be used to show

the presence of bugs, but never to show

their absence”

AmiBug.Com, Inc.

Marcus Vitruvius PollioDe Architectura (25 BCE)

An architect should be ingenious …

… and apt in the

acquisition of

knowledge.


AmiBug.Com, Inc.

Marcus Vitruvius Pollio

• Skill set of an architect:

– Good writer

– Skilled draftsman

– Versed in geometry and optics

– Expert in figures

– Acquainted with history

– Informed on the principles of

natural and moral philosophy

– Musician

– Not ignorant of the sciencesJune 5, 2014 © Robert Sabourin, 2014 Slide 16

AmiBug.Com, Inc.

Marcus Vitruvius Pollio

• Quality Principles

– Firmatis (Durability) - It should stand

up robustly and remain in good

condition.

– Utilitas (Utility) - It should be useful

and function well for the people using

it.

– Venustatis (Beauty) - It should delight

people and raise their spirits.


AmiBug.Com, Inc.

Circa 25 BC


AmiBug.Com, Inc.

Circa 1979


AmiBug.Com, Inc.


"No! Try not, Do. Or do not.

There is no try."

Yoda

AmiBug.Com, Inc.


C. Northcote Parkinson

Parkinson’s Law:

“…work expands so as to fill the time available for its completion…”

AmiBug.Com, Inc.

Pareto Principle

• Vilfredo Pareto, 1848 - 1923, Economist

– 80% of the wealth was in the hands of 20%

of the population


AmiBug.Com, Inc.

Pareto Principle

Joseph Juran, 1904 - 2008

• 1950 Quality Control Handbook

• 20% of the study population accounts for 80% of the measure under consideration

• The critical few and the trivial many


AmiBug.Com, Inc.

Bugs on Bugs

Some Boolean algebra


AmiBug.Com, Inc.

Duck Season

• Who is the duck?

• Who is the rabbit?

• Isn’t it obvious?

• Peer rivalry

• Common nemesis


AmiBug.Com, Inc.

Rabbit Season

• Just because it

was written down

doesn't make it

so!


AmiBug.Com, Inc.

Duck Season

• Ambiguous

pronouns

• Fallacious

negation

• Role playing


AmiBug.Com, Inc.

Which Season?


ME HOME RESULT COMMENT

F F Shoot Rabbit now Probably the opposite the Rabbit Wanted

F T Shoot Rabbit at home Another interesting opposite

T F Shoot me now Funny opposite

T T Shoot me at home Suggested target

SHOOT

AmiBug.Com, Inc.

Boolean Logic

• Conditions– TRUE

– FALSE

• Primary Operators– AND

– OR

– XOR

– NOT

• Composite Operators– NAND

– NOR


AmiBug.Com, Inc.

Boolean Logic

• Demorgan’s Rule

– NOT ( (A) AND (B) ) = (NOT (A)) OR (NOT (B))


AmiBug.Com, Inc.

DeMorgan’s Law

A B Not A Not B A and B NOT (A and B) (Not A) or (Not B)

0 0 1 1 0 1 1

0 1 1 0 0 1 1

1 0 0 1 0 1 1

1 1 0 0 1 0 0


AmiBug.Com, Inc.

Boolean Logic

• Composed Logic

– Minterms

• For “n” logical variables there are 2n Minterms

• Any logical function of “n” variables can be

constructed from the logical OR of one or more

Minterms


AmiBug.Com, Inc.

Boolean Logic

• Composed Logic

– 2 Variable Minterms

MINTERM A B

M0 F F

M1 F T

M2 T F

M3 T T


AmiBug.Com, Inc.

Boolean Logic

• Composed Logic

– 3 Variable Minterms

MINTERM A B C

M0 F F F

M1 F F T

M2 F T F

M3 F T T

M4 T F F

M5 T F T

M6 T T F

M7 T T T


AmiBug.Com, Inc.

Boolean Logic

MINTERM A B C Y

M0 F F F F

M1 F F T T

M2 F T F T

M3 F T T F

M4 T F F T

M5 T F T F

M6 T T F F

M7 T T T T

Y = M1 OR M2 OR M4 OR M7

M1 = (NOT A) AND (NOT B) AND (C)

M2 = (NOT A) AND (B) AND (NOT C)

M4 = (A) AND (NOT B) AND (NOT C)

M7 = (A) AND (B) AND (C)June 5, 2014 © Robert Sabourin, 2014 Slide 35

AmiBug.Com, Inc.

Logic Reduction

• Express a logical relationship with fewer

terms

– Karnaugh (K) mapping

• Visual

– Quine-McCluskey

• Tabular


AmiBug.Com, Inc.

Ambiguous Requirements

• Is it really perfectly

clear?

• Do not tick off the

stakeholders

• Decision tables?

• Real communications?


AmiBug.Com, Inc.

Test Design

Decision Tables


AmiBug.Com, Inc.

• The following instructions are taken from FAFSASM,

the Free Application for Federal Student Aid form:

Step Four: Who is considered a parent in this step? Read these notes to

determine who is considered a parent for purposes of this form. Answer all

questions in Step Four about them, even if you do not live with them. Are you an

orphan, or are you or were you (until age 18) a ward/dependent of the court? If

Yes, skip Step Four. If your parents are both living and married to each other,

answer the questions about them. If your parent is widowed or single, answer the

questions about that parent. If your widowed parent is remarried as of today,

answer the questions about that parent and the person whom your parent

married (your stepparent). If your parents are divorced or separated, answer the

questions about the parent you lived with more during the past 12 months. (If you

did not live with one parent more than the other, give answers about the parent

who provided more financial support during the last 12 months, or during the

most recent year that you actually received support from a parent.) If this parent

is remarried as of today, answer the questions on the rest of this form about that

parent and the person whom your parent married (your stepparent).

System Requirements | Visual Models | Decision Tables


AmiBug.Com, Inc.

Decision Tables

Construction

1. Identify Conditions

2. Identify Actions

3. Relate Conditions to Actions with Rules

4. Logic Reduction

5. Tests each Rule


AmiBug.Com, Inc.

Decision Tables

Logic Modeling

Application Logic

Business Rules

Regulations

Multiple Conditions

Multiple Actions


AmiBug.Com, Inc.

Decision Tables

Conditions

Rules

Actions


AmiBug.Com, Inc.

Decision Tablesw

ww

.wik

ipedia

.co

m


AmiBug.Com, Inc.

Fizzbin

• Complex rules

• Conditions?

• Actions?


AmiBug.Com, Inc.

Fizzbin

• Each player gets six cards, except for the player on the dealer's right,

who gets seven

• The second card is turned up, except on Tuesdays

• Two jacks are a half-fizzbin

• If you have a half-fizzbin

• a third jack is a shralk and results in disqualification

• one wants a king and a deuce, except at night, when one wants a

queen and a four

• if a king had been dealt, the player would get another card, except

when it is dark, in which case he'd have to give it back

• The top hand is a "royal fizzbin", but the odds against getting one are

said to be "astronomical". June 5, 2014 © Robert Sabourin, 2014 Slide 45

AmiBug.Com, Inc.

University Acceptance

Decision Table


AmiBug.Com, Inc.

Decision TablesExample

Conditions R01 R02 R03 R04 R05 R06 R07 R08 R09 R10

SAT SCORE High High High High High High High High High High

Prerequisites All All All Core Only Core Only Core Only Partial Partial Partial None

Residency In State Out of State Foreign In State Out of State Foreign In State Out of State Foreign *

Actions

Accept x x x

Redirect x x x x

Conditional x x x

Reject

Grant x x x x x x


AmiBug.Com, Inc.

Decision TablesExample

Conditions R11 R12 R13 R14 R15 R16 R17 R18 R19 R20

SAT SCORE Medium Medium Medium Medium Medium Medium Medium Medium Medium Medium

Prerequisites All All All Core Only Core Only Core Only Partial Partial Partial None

Residency In State Out of State Foreign In State Out of State Foreign In State Out of State Foreign *

Actions

Accept x x x

Redirect x x x

Conditional x x x

Reject x

Grant x x x


AmiBug.Com, Inc.

Decision TableExample

Conditions R21 R22 R23 R24

SAT SCORE Low Low Low Low

Prerequisites All Core Only Partial None

Residency * * * *

Actions

Accept

Redirect

Conditional x

Reject x x x

Grant


AmiBug.Com, Inc.

DVD Rental

Decision Table


AmiBug.Com, Inc.

• The following rules describe how we process

DVDs that are not returned. Create a decision table

that represents these rules:

“When a DVD is “lost” (it has not been returned in 90 days) the following

rules apply: If the customer has not lost any DVDs in the preceding 12

months we do not suspend the account nor do we bill for the lost DVDs. If

the customer has lost 1-2 DVDs in the preceding 12 months and if they

have not filed a complaint with the USPS we do not suspend or bill, If the

customer has lost 1-2 in the preceding 12 months and if they have filed a

complaint with the USPS we do not suspend or bill. If the customer has

lost 3-4 DVDs in the preceding 12 months and if they have not filed a

complaint then we suspend their account but do not bill for the lost DVDs.

If the customer has lost 3-4 DVDs in the preceding 12 months and if they

have filed a complaint then we do not suspend their account nor do we bill

them. If the customer has lost 5 or more DVDs in the preceding 12 months

we suspend their account and bill them for the lost DVDs.”

System Requirements | Visual Models | Decision Tables | Exercise


AmiBug.Com, Inc.


Binary Ranges

Conditions Rule 1 Rule 2 Rule 3 Rule 4 Rule 5 Rule 6

Lost DVDs in past 12 Months (YES or NO) NO YES YES YES YES YES

Lost 1-2 DVDS in past 12 Months (YES or NO) NO YES YES YES YES YES

Complaint to USPS Dcare NO YES NO YES Dcare

Lost 3-4 DVDs in past 12 Months (YES or NO) NO NO NO YES YES YES

Lost 5 or more DVDs in past 12 Months (YES or NO) NO NO NO NO NO YES

Actions

Suspend account NO NO NO YES NO YES

Bill for lost DVDs NO NO NO NO NO YES


AmiBug.Com, Inc.


Conditions Rule 1 Rule 2 Rule 3 Rule 4 Rule 5 Rule 6

Number of DVDs lost in past 12 Months 0 1-2 1-2 3-4 3-4 5 +

Filled Complaint with USPS Dcare YES NO YES NO Dcare

Actions

Suspend account NO NO NO NO YES YES

Bill for lost DVDs NO NO NO NO NO YES

Value Ranges


AmiBug.Com, Inc.


Conditions Rule 1 Rule 2 Rule 3 Rule 4

Number of DVDs lost in past 12 Months 0-2 3-4 3-4 5 +

Filled Complaint with USPS * YES NO Dcare

Actions

Suspend account NO NO YES YES

Bill for lost DVDs NO NO NO YES

Reduced


AmiBug.Com, Inc.

Customer Business Rules

Decision Tables


AmiBug.Com, Inc.

Decision Tables

Elicited from customer


AmiBug.Com, Inc.

Decision Tables

Reduced


AmiBug.Com, Inc.

Monopoly®

House Purchase Rules

Decision Tables


AmiBug.Com, Inc.

Decision Tables

• House Purchase

Rules

• Property must be in

a color group

• A player must own

all properties of a

color group

• No properties can

be mortgaged


AmiBug.Com, Inc.

Decision Tables

• House Purchase

Rules

• Houses must be

available for sale

• Houses must be

built progressively

• Each property can

have a maximum of

4 houses


AmiBug.Com, Inc.

Decision Tables

• House Purchase

Rules

• If multiple players

attempt to purchase

the same house the

highest auction

bidder gets it


AmiBug.Com, Inc.


AmiBug.Com, Inc.

Wrap-O-Matic

Decision Tables


AmiBug.Com, Inc.

Decision Tables

• Wrapping Rules:• Disallows ribbons applied to unwrapped

chocolates.

• Disallows hollow chocolates tied with metallic

ribbon.

• Uses the gentle wrapping algorithm with tissues

wrappers.


AmiBug.Com, Inc.

Decision Tables

• Wrapping Rules:• Uses the rapid wrapping algorithm whenever

chocolates do not have ribbons and do not

have tissue wrappers.

• Uses the gentle algorithm whenever hollow

chocolates are tied with ribbons.

• Uses the normal algorithm for all other cases.


AmiBug.Com, Inc.

Decision Tables

Rules

R01 R02 R03 R04 R05 R06 R07 R08 R09 R10 R11 R12 R13 R14 R15

Co

nd

itio

ns

Viscosity Hollow Not Hollow

Ribbon Metallic Other None Metallic or Other None

WrapperMetallic

or PaperTissue None

Metallic

or PaperTissue None

Metallic

or PaperTissue None

Metallic

or PaperTissue None

Metallic

or PaperTissue None

Act

ions

Disallow x x x . . x . . . . . x . . .

Rapid

Algorithm. . . - . - X . x . . . x . x

Normal

Algorithm. . . . . . . . . x . . . . .

Gentle

Algorithm. - . x x x . x . . x . . x .

Wrapping Rules


AmiBug.Com, Inc.

Problem Solving

How to solve it?


AmiBug.Com, Inc.

Amazon Problem

• So what would you do next?

– Give 3 alternatives


AmiBug.Com, Inc.

Problem Solving


George Pólya

1945

AmiBug.Com, Inc.

Pólya onProblem Solving

How to solve problems

• Understand it

• Plan it

• Do it

• Review it


AmiBug.Com, Inc.


Understand it

• What are you asked to find or show?

• Can you restate the problem in your own words?

• Can you make a diagram to help understand the problem?

• Is there enough information to find a solution?

• Do you understand concepts used in stating the problem?

• Do you need to ask a question to get the answer?


AmiBug.Com, Inc.


Plan it

• Guess and check

• Make an orderly list

• Eliminate possibilities

• Use symmetry

• Consider special cases

• Use direct reasoning

• Solve an equation


AmiBug.Com, Inc.


Plan it

• Look for a pattern

• Draw a picture

• Solve a simpler problem

• Use a model

• Work backward

• Use a formula

• Be creative

• Use your head


AmiBug.Com, Inc.


Do it

• Prepare

• Follow plan

• Learn

• Adapt


AmiBug.Com, Inc.


Review it

• Did we solve it?

• Any partial solution?

• Next step?

• New strategy?


AmiBug.Com, Inc.

Pólya Heuristics

Analogy

• Can you find a problem analogous to your problem and solve that?

Generalization

• Can you find a problem more general than your problem

Induction

• Can you solve your problem by deriving a generalization from some examples?

Variation of the Problem

• Can you vary or change your problem to create a new problem (or set of problems) whose solution(s) will help you solve your original problem?

Auxiliary Problem

• Can you find a subproblem or side problem whose solution will help you solve your problem?


AmiBug.Com, Inc.

Pólya Heuristics

Pattern recognition

• Can you find a problem related to yours that has already been solved and use that to solve your problem?

Specialization

• Can you find a problem more specialized?

Decomposing and Recombining

• Can you decompose the problem and "recombine its elements in some new manner"?

Divide and conquer

• Working backward Can you start with the goal and work backwards to something you already know?

Backward chaining

• Draw a Figure Can you draw a picture of the problem


AmiBug.Com, Inc.


Test Design

Variable identification

AmiBug.Com, Inc.


Identify Variables

• What is a variable?

– To VARY is to CHANGE

• A variable is something which can

change

– Software behavior depends on the

VALUES of many VARIABLES

• Anything which influences the

behavior of software could be a

variable

AmiBug.Com, Inc.


Identify Variables

• Review test basis

• Identify variables– Influencers

– Outcomes

• Potential Sources– Conditions

– Environment

– Rules

– Constraints

– Actions

– States

AmiBug.Com, Inc.


Wrap-O-Matic


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


Gmail Image Attachment


AmiBug.Com, Inc.


AmiBug.Com, Inc.


Examples

AmiBug.Com, Inc.


Examples

AmiBug.Com, Inc.


Insurance Policy Cancelation


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.

Identify Variables


Act on variables

Ignore

Default values

Specific values

Observe

AmiBug.Com, Inc.


Test Design

Equivalence Classes

AmiBug.Com, Inc.


• Equivalence class– A subset of all possible test values to a variable

– Each member assumed provide the same info

– Each variable may have many classes

– Equivalence class are not mutually exclusive

– Focus testing

– Reduce the number of test cases

Taking AIM

AmiBug.Com, Inc.


Sets – Venn DiagramEquivalence Classes

b

a

c

a

b c

d

a b

c d

e

f

AmiBug.Com, Inc.


Equivalence Classes

– Sources• Requirements

– Business logic

– Capabilities

– Ranges

– Constraints

• Code

– Decisions

– Intermediate computations

• Data

– Input fields

– Database

– Internal structures

Equivalence

AmiBug.Com, Inc.


AIM - Equivalence Classes

AmiBug.Com, Inc.


Square Root Function


AmiBug.Com, Inc.


Example: SQRT XEquivalence Classes

Mind Map

AmiBug.Com, Inc.


Date Field


AmiBug.Com, Inc.


Example: Date FieldEquivalence Classes

Mind Map

AmiBug.Com, Inc.


Insurance Claim Amount


AmiBug.Com, Inc.


Example: Claim AmountEquivalence Classes

Mind Map

AmiBug.Com, Inc.


Gmail “to composed”


AmiBug.Com, Inc.


Example: Gmail To ComposedEquivalence Classes

Mind Map

AmiBug.Com, Inc.



Mind Map

AmiBug.Com, Inc.



Mind Map

AmiBug.Com, Inc.



Mind Map

AmiBug.Com, Inc.



Mind Map

AmiBug.Com, Inc.


Scenario Based Testing

AmiBug.Com, Inc.

Whiteboarding


Use

r M

ind

Ma

p

AmiBug.Com, Inc.


Buying a Book

Usage Scenarios

AmiBug.Com, Inc.

Whiteboarding


Sto

ry B

oa

rd

AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


AmiBug.Com, Inc.


Social Networking

Usage Scenarios

AmiBug.Com, Inc.


AmiBug.Com, Inc.



AmiBug.Com, Inc.



AmiBug.Com, Inc.



AmiBug.Com, Inc.



AmiBug.Com, Inc.



AmiBug.Com, Inc.



AmiBug.Com, Inc.



AmiBug.Com, Inc.



AmiBug.Com, Inc.


Test Design

Control Flow Testing

AmiBug.Com, Inc.



• Exercise paths through a

system

– Data Flow

– Transaction Flow

– Code Flow

– Process Flow

AmiBug.Com, Inc.



• Model flow

– Create control flow diagram

– Find basis paths• Minimal set of transactions

• Exercise at least once

– Each step

– Each decision

AmiBug.Com, Inc.



– Process Steps

• A flows to B

• B flow to C

B

A

C

AmiBug.Com, Inc.



– Decisions

• A flow to B or C

CB

A

AmiBug.Com, Inc.



– Junctions

• A or B flow to C

A

C

B

AmiBug.Com, Inc.



– Minimal basis paths

• N – number of nodes

• E – number of edges

• P – number of basis paths

• P = E – N + 2

• McCabe Cyclomatic Complexity

B

A

C D

F G

E

H

I

N=9

E=10

P=10-9+2

P=3

AmiBug.Com, Inc.



– A set of basis paths:

• A B D E G H I

• A B C E G H I

• A B D E F H I

B

A

C D

F G

E

H

I

N=9

E=10

P=10-9+2

P=3

AmiBug.Com, Inc.



– Finding basis paths:

1. Start with a typical baseline

2. Flip first decision keep rest as similar as possible

3. Continue flipping decisions on baseline

4. After all decisions on baseline have been flipped continue on next path

5. Stop when all paths have been exhausted

B

A

C D

F G

E

H

I

N=9

E=10

P=10-9+2

P=3

AmiBug.Com, Inc.


Keyword Optimization

Control Flow Diagram

AmiBug.Com, Inc.


AmiBug.Com, Inc.


ST A

B

E

C

END

IHG

D

F

AmiBug.Com, Inc.


AmiBug.Com, Inc.



AmiBug.Com, Inc.


AmiBug.Com, Inc.


Insurance Workflow

Control Flow Diagram

AmiBug.Com, Inc.



• Application screens

are selected with

three controls:

(a) has 5 options

(b) has 6 options

(c) has 2 options

• How many screens can a user choose?

AmiBug.Com, Inc.



• Total Combinations

= 6 x 5 x 2 = 60

• To exercise each

combination once a

total of 60 tests

would be required.

AmiBug.Com, Inc.



• How many tests

would be required to

exercise all possible

screens in every

possible order?.

AmiBug.Com, Inc.



• To exercise all screens in every

possible order would require

60! Test cases

60! = 60 x 59 x 58 x ... 3 x 2 x 1

60! ≈ 8.32 x 10**81

AmiBug.Com, Inc.



From 7.0 × 10**79

To 1.5 × 10**82

How many atoms are in

the observable

universe?

AmiBug.Com, Inc.



AmiBug.Com, Inc.


AmiBug.Com, Inc.


Control Flow Diagram to Identify Basis Paths

Basis Paths

p01 Start Dispatch L M N P Q R

p02 Start Dispatch X Y AA AB R

p03 Start Dispatch AC AF AJ AI AK R

p04 Start Dispatch AD AE AF AJ AI AK R

p05 Start Dispatch O P Q R

p06 Start Dispatch K P Q R

p07 Start Dispatch I J C D H P Q R

p08 Start Dispatch A B C D H P Q R

p09 Start Dispatch L M N P S T Q R

p10 Start Dispatch L M N P S T U R

p11 Start Dispatch L M N P S T U V W R

p12 Start Dispatch X Z AA AB R

p13 Start Dispatch X Z AA AM R

p14 Start Dispatch X Z AA AM AN AP AQ R

p15 Start Dispatch X Z AA AM AN AO AQ R

p16 Start Dispatch AC AF AG R

p17 Start Dispatch AC AF AH R

p18 Start Dispatch AC AF AJ AL AM R

p19 Start Dispatch AC AF AJ AL R

p20 Start Dispatch I J C D E F R

p21 Start Dispatch I J C D E G R

p22 Start Dispatch A B R

Nodes 45

Edges 65

Complexity 22 E-N+2

AmiBug.Com, Inc.

Control Flow

• Some interesting

control flow test ideas

– All nodes

– All edges

– All paths


AmiBug.Com, Inc.


AmiBug.Com, Inc.


Test Design

State Models

AmiBug.Com, Inc.


Gerald M. Weinberg

“A state is a situation which can be

recognized if it occurs again”

An Introduction to General Systems

ThinkingDorset House

AmiBug.Com, Inc.


State Models

“Stateful” Systems

Transactions

Embedded Systems

Process

Workflow

User Interface

AmiBug.Com, Inc.


State Models

State

A

State

B

Transition i

Transition j

AmiBug.Com, Inc.


State Models

Construction

1. Identify: States

2. Identify: Transitions

3. Identify: Triggers and Outcomes

4. Test: Get to Each State

5. Test: Exercise Each Transition

6. Test: Cover Each Path

AmiBug.Com, Inc.


San Diego Singles Pattern

State Models

AmiBug.Com, Inc.


State ModelsSan Diego Singles Pattern

State

C

Transition k

Transition l

State

B

Transition i

Transition j

State

A

AmiBug.Com, Inc.


State ModelsSan Diego Singles Pattern

Non

Existing

Departed

Come to be

Together

Get Together

Separate

Single

AmiBug.Com, Inc.


Stack Pattern

State Models

AmiBug.Com, Inc.


Loaded

Push

Pop beyond Min

State ModelsStack Pattern

Full

Push beyond Max

Pop

Empty

Attempted

Pop

Attempted

PushPush

Pop

AmiBug.Com, Inc.


Embedded Systems

State Models

AmiBug.Com, Inc.


State ModelsCoffee Machine

power up

idleinserting

coins

user

choose

make

coffee

service

needed

coin insertedreset button

coin return

right amount

enteredcoin return

button pushed

no cups

OR no coffee

OR sensor jam

cup removed

AmiBug.Com, Inc.


Insurance Transactions

State Models

AmiBug.Com, Inc.


State ModelsTransaction

Cancel

Null

Warning

Approved ActiveIssued PaidRequest

Ord

er

su

bm

itte

d

Cre

dit a

nd R

isks O

K

Cre

ate

Po

licy

Re

ce

ive

Pa

ym

en

t

Po

licy A

ctiva

ted

Late Pay

No Pay

AmiBug.Com, Inc.


Wrap-O-Matic

State Models

AmiBug.Com, Inc.

State Models


IdleError State

Wrapping

Sta

rt Wra

p-o

-Matic

Pre

ssed S

top o

r

Wra

ppin

g C

om

ple

te

Power Up

Attemp to Start without

Manifest or Insufficient

Materials Needed

Pro

cessin

g E

rror

Recovery

System Reset

Recovery Complete

Inputting ManifestBacking up Reports

Inputting Manifest

Backing up

Reports

AmiBug.Com, Inc.


State Models

Tools of the Trade

Baskets

Index Cards

Excel

Visio

Power Point

Case Tools

AmiBug.Com, Inc.

State Models

Model Based Testing

Finite State Machines

Robots

ProbesJune 5, 2014 © Robert Sabourin, 2014 Slide 181

AmiBug.Com, Inc.


Test Design

Failure Mode Analysis

AmiBug.Com, Inc.


Failure Mode and Effects Analysis

Create block diagram

List classes of users

Find elements which can fail

Create usage/failure table

Rank potential failure impacts


AmiBug.Com, Inc.


Create block diagram

Blocks

• System Elements

• Processes

• Subsystems

Connector

• Logical relationships

• Physical relationships

• Process invocations


AmiBug.Com, Inc.


List classes of usersWho are the users?

What are they doing?

Classes

Personas

Context


AmiBug.Com, Inc.


Find elements which can fail

• Break

• Constrained

• Slow

• ABEND

• Crash


AmiBug.Com, Inc.


Create usage/failure table

•Scenarios

• Interesting blocks


AmiBug.Com, Inc.



Rank potential failure impacts

S4 –

Cosmetic

• User can accomplish the task with some inconvenience

S3 –

Work around

• User can accomplish task by working around the problem

S2 –

No Work around

• User cannot accomplish some tasks

S1 –

Show stopper

• User cannot accomplish any tasks

S0 –

No Impact

• User can accomplish the task

AmiBug.Com, Inc.


Wrap-O-Matic


AmiBug.Com, Inc.

Whiteboarding


Fa

ilure

Mo

de

s

AmiBug.Com, Inc.



AmiBug.Com, Inc.



AmiBug.Com, Inc.


Prize Server


AmiBug.Com, Inc.



Prize

Manageme

nt Interface

Prize TableGame

ServerPrize Server

Prize

Spinner

Prize Status

ReporterGame Server

System Elements

AmiBug.Com, Inc.


Users

Prize Administrator

Database Administrator

Auditor

Player


AmiBug.Com, Inc.



Prize System Failure Modes

Pri

ze

Man

age

me

nt

Inte

rfac

e

Pri

ze S

tatu

s

Re

po

rter

Pri

zeT

ab

le

Pri

zeSe

rve

r

Ga

me

Ser

ver

Pri

zeS

pin

ner

User Scenario What if the process fails?

Prize Administrator Add prize S1 S0 S1 S0 S0 S0

Prize Administrator Change prize dates S3 S0 S1 S0 S0 S0

Prize Administrator Change prize amounts S3 S0 S1 S0 S0 S0

Prize Administrator Stop prizes S3 S0 S1 S0 S0 S0

Prize Administrator Start prizes S3 S0 S1 S0 S0 S0

Database Administrator Report prize statusS0 S0 S1 S0 S0 S0

Database Administrator Modify prize statusS0 S0 S1 S0 S0 S0

Auditor View prize schedule S0 S3 S1 S0 S0 S0

Auditor View prizes awarded S0 S3 S1 S0 S0 S0

Auditor View prizes remaining S0 S3 S1 S0 S0 S0

Player Spin for prize S0 S0 S1 S1 S1 S1

AmiBug.Com, Inc.

Thank You

• Questions?


Documents

Test Fundamentals For Expert Testerslets-test.com/wp-content/uploads/2014/06/2014_05_25_Test_Funde… · Conform to requirements Value to stakeholders Suit purpose. AmiBug.Com, Inc