Where is test strategy with an

agile team?In search of long-term ideas that guide test design

Maaret Pyhäjärvi

Twitter: @maaretp

Atypical Agile Team

• Product Development

• Team of 9 (1 testing specialist) + PO

• Release in production when done (includes

tested)

– Continuous Deployment with little test automation

– Kanban with conversational WIP limits

– NoEstimates, focus on identifying slices of value

– NoProjects

– Testing = Checking + Exploring

Ideas that Guide All Testing

• Knowing the product (by asking around)– Purpose of existence

– Functionality, Performance

– Browsers, .NET MVC

– Choosing the right features into development pipeline (lean startup)

• Delivering professionally– Done means done – value in use delivered

– Production monitoring is an option for getting information

– Reporting on product (Lead time; Net Promoter Score) not on testing

• Actionable information first– Awareness of reporting time

Strategy should be more

specific to the product at

hand?

Supporting Documents

• Quality Target

– “Awareness thing”

– Outlining rough types of testing with split to

roles

• System Testing Support List

– “Feature Breakdown”

– Connections in the system between the

features

• Elisabeth Hendrickson’s cheat sheet

What Really Happens

• Split Jira item into smallest possible testable chunk; talk about value and design (dev-test-po)

• Implement & Pair-test to introduce to testing specialist

• Explore sympathetically and extending as long as needed– Feature in isolation, split to browsers starting from

most likely to break

– Feature in combination with other features

– …

– Monitoring in production

• Get better (scope of test automation; refactoring; pairing and group work; individuals’ skills)

Key Observations from the

Experience• CONTINUOUS DEPLOYMENT IS TESTING GAME-CHANGER

– Continuous deployment allows for applying indefinite time on selected tactics, so the prioritizing of the next tactic to use is on selecting the next move that reveals information that is immediately useful

• TACTICS OVER STRATEGY FOR TESTING– There’s so much commonality on things to do in testing of different projects that

strategic ideas seem almost invisible, and the focus on is applying the right tactics to reveal the right information more efficiently and timely.

• TEMPORARY AND TACTICAL AIDS FOR AGREEING AND REMEMBERING

– Two documents have seemed relevant in the lifecycle of this product: a quality target agreeing who does what kinds of testing activities and a feature breakdown to remind of connections in the system between the features

• STRATEGY IS ENABLING TESTING IN PRODUCT DEVELOPMENT– Most of strategic effort goes into software development in general: shortening the

feedback cycle to enable flexibility and relevance of testing and building in testing from the inception of a feature idea, driving through feature splitting to the ideas of tactics we’d apply on a particular feature.