Embed Size (px)
Citation preview
Practical Test Strategy Using HeuristicsEuroStar Webinar, September 2016
Huib Schoots
Thanks to:• Ruud Cox for the many discussions on this topic
• Jean-Paul Varwijk for helping me making it awesome
• Fiona Charles and Rikard Edgren for inspiration
• Obviously James Bach and Michael Bolton for sharing
their knowledge about Rapid Software Testing and the
Heuristic Test Strategy Model
Many slides are taken from Rapid Software Testing and are used with
permission. Rapid Software Testing is developed by James Bach and
Michael Bolton. Also see: http://www.satisfice.com/info_rst.shtml
Acknowledgements
2Practical Test Strategy Using Heuristics
What is test strategy?
Think about these questions:
1. What is test strategy to you?
2. Why do you make your test strategy?
3. What does your test strategy look like?
Test Strategy (according to ISTQB Glossary definition)
A high-level description of the test levels to be performed
and the testing within those levels for an organization or
programme (one or more projects).
3Practical Test Strategy Using Heuristics
Test strategy to me
4Practical Test Strategy Using Heuristics
Source: EuroStar Webinar Thinking Strategically About Testing by Fiona Charles
Creating a test strategy: problem solving!
1. Define the testing problems (or test missions)
2. Define solutions to these problems
3. Communicate / capture / execute
A test strategy is a solution to a complex problem: How do we meet the information needs of the stakeholders in the most efficient way possible?
5Practical Test Strategy Using Heuristics
Test strategy
Strategy:Thesetofideasthatguideyourtestdesign
Logistics:Thesetofideasthatguideyourapplicationofresources tofulfillingtheteststrategy
Plan: Thesetofideasthatguideyourtestproject
plan=strategy+logistics
6Practical Test Strategy Using Heuristics
Strategy – execution – reporting
Test Strategy Test ReportingTest Execution
Practical Test Strategy Using Heuristics 7
Testing
Strategy – execution – reporting
Practical Test Strategy Using Heuristics 8
A growing test strategy
Over time we learn & discover more
Practical Test Strategy Using Heuristics 9
Things to consider…
10Practical Test Strategy Using Heuristics
Source: Nordic Testing Days Workshop Test Strategy the next level by Rikard Edgren
Test strategy in agile…
US
Iteration 0 HardeningSprint
ReleaseIteration Test Strategy
Story Test Strategy Release Test Strategy
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
US
Sprint 1 Sprint 2 Sprint 3 HardeningSprint
Sprint 4 Sprint 5 Sprint 6
Overall Test Strategy
Release
Practical Test Strategy Using Heuristics 11
Tests
ProjectEnvironment
ProductElements
QualityCriteria
PerceivedQuality
Heuristic Test Strategy Model
Practical Test Strategy Using Heuristics 12
Tests
ProjectEnvironment
ProductElements
QualityCriteria
PerceivedQuality
Heuristic Test Strategy Model
Practical Test Strategy Using Heuristics 13
Heuristic Test Strategy Model
14Practical Test Strategy Using Heuristics
ProjectEnvironmentWaystounderstandourcontext
15Practical Test Strategy Using Heuristics
MIDTESTD§ Mission § Equipment & tools
§ Information § Schedule
§ Developer relations § Test items
§ Test team § Deliverables
QualityCriteriaIdentifyingvalueandthreatstoit…
16Practical Test Strategy Using Heuristics
§ Capabililty § Scalability
§ Reliability § Compatibility
§ Usability § Performance
§ Charisma § Installability
§ Security § Development
CRUCSS CPID
Coverage
Interestingcoverage:• Product coverage
• Risk coverage
• Requirements coverage
17
____ coverage is how thoroughly you have examined
the product with respect to some model of ____
Practical Test Strategy Using Heuristics
Product ElementsWant to cover the product?
18Practical Test Strategy Using Heuristics
SFDIPOT§ Structure § Platform
§ Function § Operations
§ Data § Time
§ Interfaces
TheGeneralTestTechniques“Ways to test…”?
19Practical Test Strategy Using Heuristics
FDSFSCURA§ Function testing § Claims testing
§ Domain testing § User testing
§ Stress testing § Risk testing
§ Flow testing § Automatic checking
§ Scenario testing
Diversify!
20Practical Test Strategy Using Heuristics
http://www.satisfice.com/presentations/etta.pdf
Thirty-FourTestStrategyHeuristics
21Practical Test Strategy Using Heuristics
MissionInformationDeveloperrelationsTeamEquipment&toolsScheduleTestItemsDeliverables
StructuresFunctionsDataInterfacesPlatformsOperationsTime
CapabilityReliabilityUsabilityCharismaSecurityScalabilityCompatibilityPerformanceInstallabilityDevelopment
FunctiontestingDomaintestingStresstestingFlowtestingScenariotestingClaimstestingUsertestingRisktestingAutomatictesting
ProjectEnvironment
ProductElements
QualityCriteria
TestTechniques
Balanced test strategy
Project Environment
Practical Test Strategy Using Heuristics 22
Ask for testability!
23Practical Test Strategy Using Heuristics
• Controllability• Observability• Configurability• Availability• Simplicity• Stability• Information
Testing is far more rapidwhen the product is more testable
Log files!
ScriptableInterface!
Test strategy is …
24Practical Test Strategy Using Heuristics
Source: Nordic Testing Days Workshop Test Strategy the next level by Rikard Edgren
Make your test strategy visual!
25Practical Test Strategy Using Heuristics
Make your test strategy visual!
Practical Test Strategy Using Heuristics 26
Make your test strategy visual!
27Practical Test Strategy Using Heuristics
Questions?
!!!???
Practical Test Strategy Using Heuristics 28
References
30Practical Test Strategy Using Heuristics
• RapidSoftwareTesting– JamesBach&MichaelBoltonhttp://www.satisfice.com/info_rst.shtml
• HeuristicTestStrategyModel– DesignedbyJamesBachhttp://www.satisfice.com/tools/htsm.pdf
• HeuristicTestPlanning– JamesBachhttp://www.satisfice.com/tools/satisfice-cm.pdf
• HeuristicRisk-BasedTesting– JamesBachhttp://www.satisfice.com/articles/hrbt.pdf
• BasicsRevisited:TestStrategy- FionaCharleshttp://www.quality-intelligence.com/articles/BasicsRevisited-TestStrategy.pdf
• Webinar:ThinkingStrategicallyAboutTesting- FionaCharleshttp://testhuddle.com/resource/thinking-strategically-about-testing-with-fiona-charles/
• Whatisagoodteststrategy– Rikard Edgrenhttp://thetesteye.com/blog/2013/09/what-is-a-good-test-strategy
• SoftwareQualityCharacteristics– Thetesteye.comhttp://thetesteye.com/blog/2011/11/software-quality-characteristics-1-1/
• WorkshopTestStrategythenextlevel– Rikard Edgrenhttp://nordictestingdays.eu/sites/default/files/NTD2014%20Presentations/TestStrategyNextLevel_FullDayTutorial.pdf
• Testabilityheuristics- JamesBachhttp://www.satisfice.com/tools/testable.pdf
