Title
Acceptance and Story Testing PatternsCharles BradleyProfessional
Scrum TrainerExperienced Scrum Coachhttp://www.ScrumCrazy.com
To download presentation, see Presentations section
at:http://www.ScrumCrazy.com 2012, Charles Bradley, All Rights
Reserved
1
About MeProfessional Scrum Trainer,
Scrum.orgCourses:Professional Scrum Foundations(2 days)Professional
Scrum Master (2 days)Started Scrum in 2008, Scrum Team Member
(Java), Scrum CoachB.S., Computer Science2
www.synerzip.comWhy should I care about Story Testing
Patterns?Increase Productivity and Efficiency Identify Tests up
front (ATDD), before development beginsFlushes out non-obvious test
scenariosEnable more Test Automation!Tests can be automated while
code is being writtenAllows us to move faster without being sloppy,
and without comprehensive documentation3www.synerzip.comYou may
notice that my slides are pretty low-tech.3OverviewQuick Review of
User Stories (15% of our time)Basic Story Testing Patterns
(50%)Exercise (15%)Advanced Story Testing Patterns (10%)Special
Story Testing Patterns (10%)4www.synerzip.comUser
StoriesCard/Titletoken used for planning and reminder to have
conversationsConversationsConfirmationsaka Story Testsaka
Acceptance Testsaka Test Confirmationsaka Acceptance CriteriaUser
Stories are not part of Scrum, but they are one good way to
represent Product Backlog Items in Scrum
5www.synerzip.comA User Story Worst PracticeAs a , I want , so
that The Technique is not a worst practice, but thinking this is a
User Story IS a worst practice!^^ This is not a User Story!^^ This
is 1/3 of a User Story^^ This is the least important 1/3 of the
User Story!I argue that Story tests are the most important part,
and they are achieved via
Conversations6www.synerzip.comWarning!!Story Tests and these
patterns are NOT: Documentation techniques.Living documents that
are maintained.User Stories only convey *new* behavior of a
system.Strict contracts.
7www.synerzip.comWhat Story Tests AreStory Tests are: A
Communication and testing technique.The results of collaboration
and conversation.Defined during backlog grooming, before
development begins, at least at a high/conceptual level.Ideally
automated to create an Agile Specification.Not all automated Story
Tests are at the UI level
8www.synerzip.comRelevant only until the story is tested and
delivered.A means to an end. The end is software that delights
customers.
8Agile Manifesto RelevanceSummation of previous two slides:We
valueWorking software over comprehensive documentationIndividuals
and interactions over processes and tools9www.synerzip.comTwo Story
Tests that Always ApplyOld behavior still works correctlyPO has
accepted the storyBest Practice: Always get immediate signoff from
the PO Dont wait until Sprint End.10www.synerzip.comTest that the
old behavior of the system, that wasnt changed, still works the way
it used to. (Regression Test)Test that the Product Owner has
approved that the User Story has been implemented to her
satisfaction. (Usually a small demo and then a verbal signoff)Best
Practice: Always get immediate signoff from the PO Dont wait until
Sprint End.
10How were going to proceedFor each Pattern CategoryAdvice on
Communication MediumsFor each PatternDescriptionContext (Good
For/Bad For)An Example or two11www.synerzip.comBasic Story Testing
PatternsTest thatGiven/When/Then (aka Gherkin style)Specification
By Example ConceptualSpecification By Example
Concrete12www.synerzip.comCommunication Mediums Basic Story Testing
PatternsBestA wiki or whiteboard is (Best)Hand written
(paraphrased) on 5 X 8 cards (2nd Best)Word Processing Document
(last resort)WorstAlmost all ALM Tools
13www.synerzip.comAlmost all ALM ToolsPoor support for inline
tables and imagesTip: Just put User Story title in ALM tool
13Optimize Your DocumentationQ: How much do I document?A: The
minimum amount that could possibly work.
Q: How will I know if its working?A: If your Product Owner often
has to point out things not done, that were already mentioned in
previous User Story conversations. This is a sign that you might
need to document more, and/or add more Story Tests. If this rarely
happens, then maybe you should try documenting less! Iterate to the
optimum!The optimum amount will vary widely by
team.14www.synerzip.comPattern: Test thatThe techniqueTest that
Test that (less frequent)ConceptualAvoid using specific test
data
15www.synerzip.comPattern Context: Test thatGood ForBeginner
Story TestersSimple TestsTests that are hard to describe or
understand using the other patterns16www.synerzip.comPattern
Context : Test thatBad ForExperienced Story Testers who know of a
more appropriate pattern to useTests with a lot of setup logic or
behavior logic.Tests where behavior depends on numerous test
inputs
17www.synerzip.comUsed in some of the examples
18www.synerzip.comExamples: Test thatAT-1. Test that, when a
user enters an incorrect old password, they get an error message
indicating incorrect credentials.AT-2. Test that three incorrect
submissions of the old password within 1 hour results in the user
being logged out from the system.AT-3. Test that, when a user
clicks on the "Continue Shopping" link, it takes them to the home
page.AT-4. Test that the order confirmation contains:An order
numberEstimated arrival dateCustomer service email
address19www.synerzip.comPattern: Given/When/ThenThe techniqueGiven
When Then (or less frequently, Then )Might use specific test
dataUse AND, OR to join statements as
necessary20www.synerzip.comPattern Context: Given/When/ThenGood
ForTests that require a lot of preconditions, setup.Tests that
require setup that is important or easily forgottenTests that have
a specific, non obvious, trigger.Tests where there are few expected
outputs21www.synerzip.comPattern Context: Given/When/ThenBad
ForSimple testsTests that have unimportant/simple/obvious
preconditionsTests where there are multiple different inputs and
multiple different outputsTests where a single Given/When/Then only
describes one of numerous very similar test scenarios
22www.synerzip.comExamples: Given/When/ThenAT-1. GivenA user who
has submitted an incorrect old password 2 times in the last
hourWhenThe user submits an incorrect password (for the 3rd
time)ThenThe system logs the user out ANDThe system displays:the
customer service phone number.a message telling them to call
customer service.23www.synerzip.comExamples:
Given/When/ThenAT-2.GivenA user that is logged in ANDthe user is an
admin user OR the user's account has been flagged by FraudWhenThe
user submits an incorrect password (for the 3rd time) ThenThe
system logs the user out ANDThe system generates an email to the
production support team with the following info:user id of the user
ANDthe user's phone number on file24www.synerzip.comPattern:
Specification By Example (SBE)The techniqueCreate a table of the
most important Examples (testing scenarios) that specify test
inputs and expected test outputs.Similar to the "decision table"
concept in tools like FitnesseBe sure to create enough examples to
exercise all important logic/test paths25www.synerzip.comPattern
Context: SBEGood ForTests that have numerousInputs that affect
output behaviorOutputs/expected behaviorsTests where its important
to test a lot of different data scenariosTests where the trigger
event is somewhat obviousAny test where it seems like a table would
be useful to:describe the test better, orhelp explore all of the
possible inputs and outputs for a test.26www.synerzip.comPattern
Context: SBEBad ForSimple testsTests that are more about verifying
simple UI behaviorFor instance Test that an error message is
displayed when the user enters an incorrect password.Test where
there is really only one input or
precondition.27www.synerzip.comSBE-Conceptual vs.
SBE-ConcreteTechnique differencesFor the conceptual form, avoid
using specific data, but instead describe the data conceptually.For
the concrete form, use actual test data.Choosing Conceptual vs.
ConcreteTry to at least have the Conceptual form done before
development begins on a story.Concrete form is usually better,
though it is harder for teams to get done before development
begins.28www.synerzip.comExample: SBE-Conceptual29
www.synerzip.com29Example: SBE-Concrete30
www.synerzip.comExercises31www.synerzip.comExample 1Pat: "For
this story, we need to calculate the shipping for orders. Our
shipping prices are done by weight. For under 10 pounds, the cost
is $25. For under 50 pounds, the cost is $35, and for under 100
pounds, the cost is $75. As you know, we dont accept orders over
100 pounds."Payton: Sounds good, that seems pretty easy.Pat: I
thought so too."
32www.synerzip.comCharacteristics of this storyFairly simple
(favors Test That pattern)Numerous expected outputs, inputs affect
outputs (favors Specification By Example patterns)The test setup
and trigger events are fairly obvious or unimportant.
(Given/When/Then is not favored)33www.synerzip.comExample 1:
Possible Story TestsAT-1. Test that orders under 10 lbs are charged
$25 for shipping.AT-2. Test that orders that are between 10lbs and
under 50lbs are charged $35 for shipping.AT-3. Test that orders
50lbs and over are charged $75 for shipping.
34www.synerzip.comExample 1: Possible Story TestsOrder
WeightShipping Cost0-10lbs$2510.01lbs-50lbs$35> 50lbs$7535Order
WeightShipping Cost0lbs$2510.00 lbs$2510.01 lbs$3550.00 lbs$3550.01
lbs$7599.99lbs$75Specification By Example - ConcreteSpecification
By Example - Conceptualwww.synerzip.comExample 2User Story Title:
RR154 Logout goes to home page
User Story Conversations:Pat: "For this next story, here is what
I want. Right now, when a user clicks on the 'logout link in the
upper right hand menu, it takes them to a basically blank page that
tells them that we've logged them out. In the future, rather than
going to basically a blank page, I want the system to take them to
the home page with all of our products, and some sort of message up
near the top that uses that same message."Payton: "I think it says
'You have successfully logged out.' "Pat: "Yes, that's right, that
one. Just make sure that the logout message is highly visible near
the top of the page -- I don't want it to get lost in the
page."Charles: "On other web sites, I've seen it presented like in
a small rectangle with a green background, to make it stand out
from a mostly white page."Pat: "That works fine, but I don't want
to specify 'The How', right? You folks have taught me that I only
get to specify 'The What'"Bailey: "Aahhhhh,
right!"36www.synerzip.comCharacteristics of this storyFairly UI
related (favors Test That pattern)Fairly simple story tests (favors
Test That pattern)The test setup and trigger events are important,
but probably not hard to remember, either. (moderately favors
Given/When/Then)Input/Output scenarios are not numerous only a
couple (Specification By Example patterns not favored)
37www.synerzip.comExample 2: Possible Story TestsUsing 'Test
That' patternAT-1Test that, when a user clicks the logout link, the
system navigates the user to the home page.
AT-2Test that, after navigating the user to the home page, the
system displays a message like "You have successfully logged out.
(logout message)
AT-3Test that the logout message is highly visible and near the
top of the page. (Ask Pat to approve
aesthetics)38www.synerzip.comExample 2: Possible Story TestsUsing
Given/When/Then patternAT-1.Given a user that is logged in,When the
user clicks on the "logout link"Then test that the system navigates
the user to the home page.
AT-2.Given a user that has just logged outWhen the system
navigates the user to the home pageThen test that the system
displays the existing logout message in a highly visible way, near
the top of the page.39www.synerzip.comExample 2: Possible Story
TestsMixing Given/When/Then and Test That patterns
AT-1.Given a user that is logged in,When the user clicks on the
"logout link"Then test that the system navigates the user to the
home page.
AT-2Test that, after a logout navigates the user to the home
page, the system displays a message like "You have successfully
logged out. (logout message)40www.synerzip.comExample 3User Story
Title: MKG-44 Limit Product Quantities
User Story Conversations:Pat: "For Marketing story 44, here is
what I'm looking for. I'd like to limit users to only being able to
purchase a maximum quantity of 5 of any one item in any one
order."Payton: "Hmmm... That seems weird. I would assume we want to
make as many sales as possible?"Pat: "Yes, I know, but what we've
found is that we often don't have the inventory to be able to
supply that many units. Also, many of the orders for more than 5
items turn out to be fraudulent anyway. So, for now, we just want
to limit the quantity to be 5 items."Charles: "Pat, do you have any
vision for how we do this?"Pat: "I don't really care -- just
something that looks nice. Maybe show me what you come up with and
we can collaborate from there."Charles:
"Ok."41www.synerzip.comExample 3: Possible Story TestsAT-1. Test
that a user is unable to purchase more than 5 of the same item in
any one order.
AT-2. Test that, when a user attempts to purchase more than 5,
the system displays a message indicating no more than 5 of any item
can be purchased.42www.synerzip.comTest Automation
Pyramid/Tools
JMeterFitnesseCucumberGreenPepperxUnit,
TestNGSoapUISelleniumQTPWatirTelerikJMeter
xUnit,
TestNGhttp://www.mountaingoatsoftware.com/blog/the-forgotten-layer-of-the-test-automation-pyramid
www.synerzip.comAdvanced Story Testing PatternsBullet PointsTest
with
Work best with:Teams that are highly co-located with POStories
that are very small (2-3 days)Tests that are very simple.Tests with
fairly obvious expected behavior
44www.synerzip.com44Communication Mediums Advanced Story Testing
PatternsBestHand written (paraphrased) on 5 X 8 cards (Best)A wiki
or whiteboard (2nd Best)ALM tool (Last resort)
45www.synerzip.comExample Bullet Points
46www.synerzip.comExample Test with
47www.synerzip.comSpecial Story Testing PatternsFlowchartsState
Diagrams
48www.synerzip.comCommunication Mediums Special Story Testing
PatternsBestPicture of handwritten diagram from a whiteboard
(Best)Diagram in drawing tool, but only if the logic is complex
enough to warrant something more fancy than a hand written diagram
(2nd Best)WorstFancy diagram in some fancy drawing tool that takes
way more time than it should.
49www.synerzip.comExample 1 Flow Charts50
www.synerzip.comExample 2 Flow Charts51
www.synerzip.comExample State Diagrams
52www.synerzip.comSummaryDont be afraid to mix and match, even
within the same story!Story Tests are the most important part of
any User Story, and they are the result of conversations.Tip for
Beginners: Start with Test that and work your way up from there.Tip
for Intermediates: Start working in new patterns, where they fit
nicely.Try to get as many Story Tests automated as you can. Test
automation is essential to superior Agility.Closing Tip: Story
tests are often a great way to slice User Stories!
53www.synerzip.comResourcesOne Page PDF Chart with all Story
Testing Patterns http://ScrumCrazy.com/STPChartArticle: User Story
Basicshttp://www.scrumcrazy.com/User+Story+Basics+-+What+is+a+User+Story%3FArticle:
The Bradley User Story Maturity
Modelhttp://www.scrumcrazy.com/The+Bradley+User+Story+Maturity+ModelTo
contact me:[email protected] download
presentation:http://www.ScrumCrazy.comClick on Presentations
54www.synerzip.com555555Questions?
www.synerzip.comwww.synerzip.comHemant
[email protected] in a
Nut-shellSoftware product development partner for small/mid-sized
technology companiesExclusive focus on small/mid-sized technology
companies, typically venture-backed companies in growth phaseBy
definition, all Synerzip work is the IP of its respective
clientsDeep experience in full SDLC design, dev, QA/testing,
deploymentDedicated team of high caliber software professionals for
each clientSeamlessly extends clients local team, offering full
transparencyStable teams with very low turn-overNOT just staff
augmentation, but provide full mgmt supportActually reduces risk of
development/deliveryExperienced team - uses appropriate level of
engineering disciplinePractices Agile development responsive, yet
disciplinedReduces cost dual-shore team, 50% cost advantageOffers
long term flexibility allows (facilitates) taking offshore team
captive aka BOT optionwww.synerzip.com5656
Our Clients
www.synerzip.comOur Clients57585858Call Us for a Free
Consultation!
Thanks!www.synerzip.comwww.synerzip.com
Hemant [email protected]
