View
530
Download
2
Category
Tags:
Preview:
DESCRIPTION
To assist in risk analysis, prioritization of testing, and test reporting (telling your testing story), you need a thorough Test Coverage Outline (TCO)—a road map of your proposed testing activities. By creating a TCO, you can prepare for testing without having to create a giant pile of detailed test cases. Paul Holland says that a comprehensive TCO helps the test team to get buy-in for the overall test strategy very early in the project and is valuable for identifying risk areas, testability issues, and resource constraints. Paul describes how to create a TCO including the use of heuristic-based checklists to help ensure you don’t overlook important elements in your testing. Learn multiple approaches for critical information gathering, the artifacts used as input for creating a TCO, and how you can use a TCO to maintain testing focus. Take back a new, lightweight tool to help you tell the testing story throughout your project.
Citation preview
W2 Test Techniques
5/1/2013 11:30:00 AM
The Test Coverage Outline: Your
Testing Road Map
Presented by:
Paul Holland
Testing Thoughts
Brought to you by:
340 Corporate Way, Suite 300, Orange Park, FL 32073
888-268-8770 ∙ 904-278-0524 ∙ sqeinfo@sqe.com ∙ www.sqe.com
Paul Holland
An independent software test consultant and teacher, Paul Holland has more than sixteen years of hands-on testing and test management experience, primarily at Alcatel-Lucent where he led a transformation of the testing approach for two product divisions, making them more efficient and effective. As a test manager and tester, Paul focused on exploratory testing, test automation, and improving testing techniques. For the past five years, he has been consulting and delivering training within Alcatel-Lucent and externally to companies such as Intel, Intuit, Progressive Insurance, HP, RIM, and General Dynamics. Paul teaches the Rapid Software Testing course for Satisfice. For more information visit www.testingthoughts.com.
1
Product Coverage Outlines
and How to Use Them
Paul HollandTest Consultant and Teacherat Testing Thoughts
My Background
• Independent S/W Testing consultant since Apr 2012
• 16+ years testing telecommunications equipment and reworking test methodologies at Alcatel-Lucent
• 10+ years as a test manager
• Presenter at, STAREast, STARWest, Let’s Test, CAST, STARCanada, and EuroStar
• Keynote at KWSQA conference in 2012
• Facilitator at 25+ peer conferences and workshops
• Teacher of S/W testing for the past 5 years
• Teacher of Rapid Software Testing– through Satisfice (James Bach): www.satisfice.com
• Military Helicopter pilot – Canadian Sea Kings
April, 2013 ©2013 Testing Thoughts 2
2
Attributions
• Many of the concepts that I am presenting
come from the Rapid Software Testing
class developed by James Bach and
Michael Bolton
April, 2013 ©2013 Testing Thoughts 3
Test Strategy Development
• A Test Strategy is often developed by:
– Following a template
– Envisioning the entire product as a whole
– Using elements from previous features/products
– Using estimates from the past
– Some use development estimates of complexity
and effort to determine test effort
April, 2013 ©2013 Testing Thoughts 4
3
An Alternative
• Each feature/product is different from other
features/products
• Start with “raw” project specific information:
– Detailed description of the elements of the
feature/product
– Identify specific risks related to the project
– Develop a list of “test ideas”
April, 2013 ©2013 Testing Thoughts 5
Product Coverage Outline
• A Product Coverage Outline can be
described as:
– A parts list
– An inventory of elements
– Components of a product
• Essentially a list of items of a project that can
be tested in some way
April, 2013 ©2013 Testing Thoughts 6
4
Benefits of a PCO
• Helps eliminate tunnel vision
• Will help display your thoroughness to others
– Help you gain respect
• Will allow you to get “buy-in” at a high level of
what you are going to consider when testing
• Helps prioritize your testing
• Helps in reporting
April, 2013 ©2013 Testing Thoughts 7
Product Coverage Outline
How to create a PCO
April, 2013 ©2013 Testing Thoughts 8
5
Product Coverage Outline
• Take a tour of the feature/product
– Investigate every menu item
– List sub-menus
– Context specific items
– Text boxes
– Radio buttons / Check boxes
– Visual elements (pictures, background, logos)
– Pop-up boxes
April, 2013 ©2013 Testing Thoughts 9
Product Coverage Outline
• Talk with designers, business people,
customers, customer service, etc.
• Look at previous versions
• Look at marketing information
• Read the functional specification document
(*although, I would do this later – seriously)
April, 2013 ©2013 Testing Thoughts 10
6
Product Coverage Outline
• Consider the following (SFDIPOT):
– Structure of the program (smallest components)
– Functionality (individual features)
– Data (I/O, create, store, manipulate, backup, F)
– Interface (User interfaces, APIs, F)
– Platform (Computer, CPU, OS, browser, F)
– Operations (How is it used by customers)
– Timings (Race conds, time of day/wk/mth/yr, F)
(Taken from the Rapid Software Testing Course, Bach and Bolton)
April, 2013 ©2013 Testing Thoughts 11
Product Coverage Outline
• As you are investigating the feature/product
– Create a list of ALL the elements that MAY need
to be tested in some way
• Make a mind map (I use XMind)
• Enter the elements into Excel (or a word processor)
• Create a list on paper, flip chart, or whiteboard
• Make a bunch of sticky notes
(I would encourage electronic versions for ease of
sharing and re-use – but it is not mandatory)
April, 2013 ©2013 Testing Thoughts 12
7
Risk List
• Create a list of project specific risks
• These are in addition to “generic” risks
• Talk with designers, business managers,
customers, customer support, etc.
• Try to identify risks during the creation of the
Product Coverage Outline
April, 2013 ©2013 Testing Thoughts 13
Test Ideas
• This is NOT a list of specific test charters
• List ways to test that you came across during investigation. For example:
– Overflow input buffers
– Check interop between two components
– Verify information in text boxes/help screens
– Modify values, change all values from default
(These are only a very small set of examples)
April, 2013 ©2013 Testing Thoughts 14
8
3 Raw Lists: Now What?
• Use these three lists as the starting point
for your test strategy
• Get agreement from stakeholders of the
components and their importance
• They are important in “traditional” testing
environments as well as for agile teams
April, 2013 ©2013 Testing Thoughts 15
3 Raw Lists: agile testing
Use these 3 lists to:
• Create your list of Charters
– Combine the test ideas, risks and components to
create specific charters
– Add “test ideas” under each Charter
• Help with the initial prioritization
• Assist in creation of new Charters during testing
• Help generate a better definition of done
April, 2013 ©2013 Testing Thoughts 16
9
3 Raw Lists: Traditional
• Create your list of Test Cases (shudder)
• If you must create scripted test cases,
using these three lists will help:
– Give you better coverage (not just using the
requirements)
– Reduce rework (because you toured SUT)
– Prioritize the test cases for execution and
reuse
April, 2013 ©2013 Testing Thoughts 17
Hands on
Creation of a PCO for a simple program
• Lego MindstormsTM robot which changes
behavior depending on “sensed” color
• A Flash program that simulates the robot
• Generation of a PCO for the robot
• How would PCO differ for the Flash
simulation?
April, 2013 ©2013 Testing Thoughts 18
10
April, 2013 ©2013 Testing Thoughts 19
Recommended