Testing Insepction

Increasing Responsiveness and Economy of Software Inspection

Marko Komssi

[email protected]

2004-12-03

2

Contents

Problems with inspection processes and rules

Generating a minimal number of inspection rules based on risks and goals

Progressively increasing the precision of inspection practices

3

Traditional Inspection Process1

Entryand

Planning

KickoffMeeting

Indi-vidual Check-

ing

Specifi-cation

andProcessMeet-ings

Editing EditAuditing ExitLogging

Meeting

4

Inspection Process Variables

• Amount of inspection rules and phases

• Number of inspectors and their roles

• Duration of inspection sub-phase and total time spent

• Checking rate

• Proportion of product under inspection

• Experience of inspectors

• Rigidity of inspection entry and exit criteria

5

Inspection Rules

A common way to seek errors is to verify the product against inspection rules

Defects in a code or a document are violations of a rule

CUSTOMER — customer requirements must be real• Does the statement in the specification describe the customer?

• Is the need of the customer correctly presented?

6

Problems with Inspection Rules and Processes

A rigorous and stable inspection process• Different company maturity levels and development processes

• Alternative quality goals and development phases of projects

• A high threshold for using a heavy inspection model

Ineffective inspection rules• Inspection rules may reveal only minor defects

• Defects may have high severity but low priority

7

Effectiveness of Inspection Rules (Requirements)

8

Effectiveness of Inspection Rules (User Interface)

9

Not Too Much Precision, Please!

Write long and precise specifications only if you know what you are doing

• “Odd though it may sound, you will typically do less damage if you write too little rather than too much4.“

• “Even the smallest features pack more calories than you might think5.“

Take compactness as part of a inspection strategy• “Then you will have a short, readable document, which means that

people will bother to read it and then ask questions4.“

Include formality in review practices just as much as is needed6

10

Goal- and Risk-Based Inspection Rules

The three most effective rules cover 60–80 % of problems

It is economical to inspect issues that:• support the goal of the product

• are likely to harm the project in the current development phase

Rule generation can be done by imitating the concepts of Goal-Question-Metric2 (GQM) and Risk-based testing3

11

Progressive Inspection

Progressive inspection (PI) is a concept for improving the traditional inspection process:

• Increases the precision of review practices as the project development progresses

• Uses the cheapest possible review techniques associated with thequality goals and the risks of the product6

• Weights different stages of inspection

12

Emphasis in the Beginning of Development

Entryand

Plan-ning

KickoffMeeting

Indi-vidual Check-

ing

Specifi-cation

andProcess

Meet-ings


Meeting

13

Slim Quality Goals in the Beginning of Development

Entryand

Plan-ning

KickoffMeeting

Indi-vidual Check-

ing

Specifi-cation

andProcess

Meet-ings


Meeting

Few rules Soft exit criteria

14

Emphasis in the End of Development

Entryand

Planning

KickoffMeeting

Indi-vidual Check-

ing

Specifi-cation

andProcessMeet-ings


Meeting

15

Increasing Precision in the End of Development

Entryand

Planning

KickoffMeeting

Indi-vidual Check-

ing

Specifi-cation

andProcessMeet-ings


Meeting

A larger set of rules Rigid exit criteria

16

PI inside XP and EVO

In XP7, high-level goals of user stories can be questioned with targeted inspections

In EVO8, inspection and its techniques can be tailored based on the needs of the project

• Inspections are started by concentrating in planning and brainstorming

• In each inspection cycle, practices are developed based on comments from the inspection data and the team

17

Conclusions

Traditional inspection practices may be cumbersome

Three inspection rules suffice in the early stages of development

In PI, inspection practices increase in precision and progress during project development

18

References

1. Gilb, T., Graham D., Software Inspection. Addison-Wesley (1993).2. Basili, V.R., Rombach, H.D., The TAME Project: Toward Improvement-

Oriented Software Environments. IEEE Transactions on Software Engineering, 14, 6 (1988), 758–773.

3. Bach, J., James Bach on Risk-Based Testing. How to Conduct Heuristic Risk Analysis? Software Testing & Quality Engineering, 1, 6 (1999), 23–28.

4. Cockburn, A., Writing Effective Use Cases. Addison-Wesley (2001).5. McConnell, S., Achieving Leaner Software. IEEE Software, 14, 6 (1997),

127–128.6. Wiegers, K.E., Peer Reviews in Software: A Practical Guide. Addison-

Wesley (2001).7. Beck, K., Embracing Change with Extreme Programming. Computer, 32, 10

(1999), 70–77.8. Gilb, T., Principles of Software Engineering Management. Addison-Wesley

(1988).

g

Agile Development

Dietmar Strasser,Borland, Austria

Europe’s Premier Software Testing Event

World Forum Convention Centre, The Hague, Netherlands

WWW.QUALTECHCONFERENCES.COM

“The Future of Software Testing”

Traditional QA Meets

Traditional Testing meets

Agile Development

Dietmar Strasser

Director QA, Lifecycle Quality Management

[email protected]

mailto:[email protected]

Agenda

Journey towards an Agile Team

Our Environment we live in

How do we provide Visibility?

Q & A

Journey towards an Agile Team

ConfidentialCopyright © 2008 Borland Software Corporation. 5

Starting Point

SilkPerformer

Developers

Test Manager

Developers

SilkTest

DevelopersQA Doc

PM

PM

PM

PM ... Product Manager

PO ... Product Owner


SilkPerformer

Developers

Test Manager

Developers

SilkTest

DevelopersQA Doc

PO

PO

PO

PM

PM

PM



Adding Product Owner


User Story Workflow – Testers not integrated

Unassigned

In Progress

Dev

In Progress

QA

Drafted QA Ready

Approved

Drop Ready

RTM Ready

PO

PO

QA

Not Started In Progress

PO

Drop Ready

RTM Ready

QA

Dev

QA


User Story Workflow - „Small Waterfall“

Unassigned

In Progress

Dev

In Progress

QA

Drafted QA Ready

Approved

Drop Ready

RTM Ready

PO

PO

QA


PO

Drop Ready

RTM Ready

QA

Dev

QA

Coding•User story•Iteration x•4 weeks

Release Testing•All user stories•Last Iteration•4 weeks

Testing•User story•Iteration x+1•4 weeks


Lesson Learned

“Ask the Team”


Adding Tester Skills

SilkPerformer

Developers

Test Manager

Developers

SilkTest

DevelopersDoc

PO

PO

PO

PM

PM

PM



Tester

Tester

Tester


Adding Documentation Skills

PO

PO

PO

PM

PM

PM



SilkPerformer

Developers

Test Manager

Developers

SilkTest

Developers

Tester

Tester

Tester

Doc

Doc

Doc


SilkPerformer Engineering Team

Transition to Engineering Team

PO

PO

PO

PM

PM

PM



SilkTest Engineering Team

Test Manager Engineering Team


Adding QM Coach

PO

PO

PO

PM

PM

PM



QM Coach






Splitting & Re-Locating Teams

PO

PO

PO

PM

PM

PM



QM Coach




In Progress

User Story Workflow - Agile

Unassigned

Drafted

Approved

PO

PO


PO

DONEScrum

Team

DONE

AgileFinish user story in

one4-weeks Iteration


Lesson Learned

“Agile is a journey,

not a destination”


Product Scrum Team(s)

Distributed Team Environment

EQCContact

Daily

Quarterly

EQC

Resource Pool

QM Coach


Lesson Learned

“Communicate, communicate,

communicate, …”



Scrum Team(s)

Iteration Management


RBT Environment

Scrum Team(s)

Test Management

Product Owner

Requirements Management

Management

Project Management, Reporting

Scrum

Team(s)

xUnit

Scrum Team(s)

Functional/

Performance Testing

Scrum Team(s)

Source

Management


Lesson Learned

“People are more important

than processes & tools”

How do we provide Visibility?


Types of Visibility

• Internal Visibility

• Daily Stand-Ups

• Iteration Review Meetings

• Regular Updates on Production Systems

• Project Dashboard

• External Visibility

• Regular „Drops“ for customers and field people


Project Dashboard

Goal Story Report(Executive Summary)

Scrum Team Reports

Iteration-Related Quality-Related

User Story Reports

Gettin

g in

to D

eta

ils

Pro

vid

e V

isib

ility


Goal Story Report


User Story Report

Q & A

Email to:[email protected]

mailto:[email protected]

Documents

Testing Insepction