August 18, 2005 Jim Nindel-Edwards How Early, Proactive Test Planning Contributes to Project Success Based on a paper to be presented at the International

August 18, 2005 Jim Nindel-Edwards

How Early, Proactive Test Planning Contributes to Project Success

Based on a paper to be presented at the International Information Management

Association in Dublin, Ireland - September, 2005


Presentation Outline

Introduction Brief primer on requirements Where does the test plan fit in? Addressing requirements issues with the test

plan Conclusion Q & A, Comments, Discussion


Survey of the Requirements Process

The importance of requirements Just cannot succeed without them

Requirements can be missed And too often are missed until late the process

Missed functional requirements vs. other types of requirements

Development methodologies can help, or hinder the requirements process

How can the test plan help?


Requirements – Definitions

Dorfman and Thayer (1990)

A software capability needed by the user to solve a problem to achieve an objective

A software capability that must be met or possessed by a system or system component to satisfy a contract, standard, specification, or other formally imposed documentation


Unknown Requirements

Watts Humphrey (1990)“Each error, when found, is a surprise

whose correction is both expensive and disruptive.”

Unknown“The users think they know what is wanted, but

during initial use they discover that their real needs are not what they had thought.”

Misunderstood Requirements“Even when the requirements are known and

stable, the implementers often do not understand them in the detail required to produce a satisfactory product.”


What types of requirements are there anyway? Functional – gets most of the attention Application Architecture Environmental Standards Functional needs - secondary user

groups Security


Where does the Test Plan fit?

Functional requirements: Test what is supposed to work (of course)

Test what is not supposed to work (Negative testing)

Testing in priority sequence (Pri1, Pri2, etc.)

Regression Testing Testing conflicting requirements (Rich feature

set, ease of use)


Application Architecture

Target hardware and OS environments Performance Test Stress Test Limits Testing Target installation environment vs. actual

implementation environment How does the application respond to

installation in unanticipated environments?


Environmental Requirements

Glenford Myers’s Project Objective List Efficiency Compatibility Security (more later) Reliability Maintainability Upgradeability

Shared environments


Standards Requirements

Standards – Abstractions of hardware and software Target Operating System(s) Target Hardware Supported vs. non-supported environments Industry standards (XML, SOAP, UDDI, etc.)

Validation that standards are met Edge, boundary, and “corner” tests


Secondary User Groups

Secondary users are users also! Installers, super users, DBAs Testing the documentation for all types of

users groups Testing the installation, upgrade, rollback,

and trouble shooting processes Conversion and upgrades from other software

packages and/or old versions Reliability, efficiency, storage requirements


Product and System Security

Security requirements – Application or Environment? Security holes typically lie in-between these

two areas Making certain the security requirements are

clearly documented, and testable Continuous testing of security features, and

searching for security holes Tests must include all user groups


Back to the test plan

Verify functionality, find defects Use a checklist (handout) Build the test plan early in the project, and

keep it up to date You will affect the requirements document in

this process! Test all aspects of the product at every

milestone Early test failure(s) need not indicate a poor

product, just an “immature” product


Conclusion

Not having a test plan is not a good idea!

Having a test plan, even late in the project, is better than not having one at all At least you can document what was tested, and not

tested, in the development process

Early test planning leads to more comprehensive tests earlier in the development process

Test plans at the very start of a project can uncover missing and misunderstood requirements before they add delays to the project


Q & A, Comments, Suggestions

Question: What is a “good” bug? And what might you change in your project to catch

this one early?


Thanks!

Jim Nindel-Edwards, SQA Lead, Costco Wholesale, Costco.com

Email: [email protected]

My thanks to SASQAG And my Graduate Advisor, Dr. Gerhard Steinke,

Seattle Pacific University

Documents

August 18, 2005 Jim Nindel-Edwards How Early, Proactive Test Planning Contributes to Project Success Based on a paper to be presented at the International