Lightweight Use of an Established Best Practice · Agile quality risk analysis process (iteration planning) Gather the agile team List iteration backlog items Identify functional

Agile Risk-based TestingLightweight Use of an Established Best Practice

Time Tested. Testing Improved

I t d tiIntroduction Risk-based testing is a proven best practice Risk-based testing is a proven best practice for test analysis, planning, estimation, design, execution, and results reporting, p gCan you apply this best practice during agile projects?p jSure, with a few minor modifications (and, of course, iteration) Let’s see how this works…

Agile Risk-based Testingwww.rbcs-us.com

Copyright (c) RBCS 2014 Page 2


A il Lif l d Ri k b d T tiAgile Lifecycle and Risk-based Testing

Order of feature creation, depth of feature testing is

Quality risk analysis occursduring sprint planning at

p ginfluenced by level of risk

Depth of tests in automatedCI system is influenced by



during sprint planning atthe start of each sprint

CI system is influenced by level of risk, as are acc. tests


A i Q lit Ri k A il P j tAssessing Quality Risks on Agile ProjectsSelect, allocate, and prioritize test conditions to maximize , , peffectiveness and efficiency Quality risk analysis supports this process

Risk: a possible negative outcome Risk: a possible negative outcome Level of risk: based on likelihood and impact Quality risks: potential problems with product qualityProject risks: potential problems for project successj p p p j

Agile quality risk analysis occurs:At a high level during release planning by business stakeholdersAt a detailed level during iteration planning by the whole team

In each iteration, the tester designs, implements, and executes tests for the risks




Q lit Ri kQuality RisksQuality risks include all features and attributes that Quality risks include all features and attributes that can affect customer, user, stakeholder satisfaction

Incorrect calculations (functional) Slow response time (non-functional performance risk)Slow response time (non functional performance risk)Confusing interface (non-functional usability risk)

Risk analysis prioritizes tasks and guides the sizing of the tasksof the tasks

High risks require extensive testing, come earlier, and involve more story pointsLow risks receive cursory testing come later and involve Low risks receive cursory testing, come later, and involve fewer story points

Risk-based prioritization also includes release and iteration backlog items iteration backlog items




P f Q lit Ri k A l iProcess of Quality Risk Analysis Agile quality risk analysis process (iteration planning)Agile quality risk analysis process (iteration planning)

Gather the agile teamList iteration backlog itemsIdentify functional non functional quality risks for each item Identify functional, non-functional quality risks for each item Assess identified risks: categorize each risk, determine risk levelBuild consensus and ensure a good distribution of risk ratingsU l l f i k h f iUse level of risk to choose extent of testingSelect appropriate test techniques for each risk item

Adjustments may occur during an iterationRisk analysis may detect opportunities for early defect removal (e.g., problems in user stories)




Quality risks are potential system problems which could reduce user satisfactionQuality risks are potential system problems which could reduce user satisfaction

Impact: business customer or operational risk of problem

Risk priority number: Aggregate measure of problem riskTracing

information to

Likelihood: technical risk of problem

Impact: business, customer, or operational risk of problemrequirements, user story, or other risk

bases

Quality Risk Likeli-hood Impact

RiskPri. #

Extent ofTesting Tracing

Risk Category 1

Risk 1

Risk 2

Risk nA hierarchy ofA hierarchy of risk categories can help organize the list and jog

1 = Very high2 = High3 = Medium4 = Low

1-5 = Extensive6-10 = Broad11 15 C

The product of technical and business risk,



list and jog your memory.

5 = Very low 11-15 = Cursory16-20 = Opportunity21-25 = Report bugs

from 1-25.


E ti ti T ti Eff tEstimating Testing EffortDuring iteration planning user stories are During iteration planning, user stories are estimatedStory size give implementation effort y g pRisk level should influence story size Techniques such as planning poker can be used to reach consensus, involve whole team, and avoid missing anything Reliable estimation including testing is Reliable estimation, including testing, is necessary for smooth work pace and meaningful velocityg y




E pl All ti T t Eff tExample: Allocating Test EffortExtensive: run large number of tests, both broad and deep, g , p,combine and vary interesting conditions, use all relevant techniques with strong coverage criteriaBroad: run medium number of tests, exercise many different interesting conditions use most relevant different interesting conditions, use most relevant techniques with medium coverage criteria Cursory: run small number of tests, sample most interesting conditions, use efficient techniques with weak interesting conditions, use efficient techniques with weak coverage criteria Opportunity: leverage other tests or activities to test 1-2 interesting conditions, investing very little time and effort,

i i h i i ll using reactive techniques especially Report bugs only: allocate only a small amount of extra time to report and manage these accidental bugs




C ll b ti U St C tiCollaborative User Story CreationDevelopers, testers, and business stakeholders collaborate p , ,to capture requirements in user storiesUser stories include:

Functional and non-functional elementsFunctional and non functional elementsAcceptance criteria for each element

Testers bring a unique perspective to this process Identify missing elements Identify missing elements Ask open-ended questionsIdentify quality and project risks Suggest tests for the user storySuggest tests for the user storyConfirm the acceptance criteria

Acceptance criteria clarify the feature and establish clear completion measures



completion measures


C ti U St iCreating User StoriesINVEST technique 3C elements:INVEST technique

IndependentNegotiable Valuable

3C elements:Card: physical description of story and its benefitsValuable

EstimatableSized appropriatelyTestable

Conversation: how the software will be usedConfirmation: h ki f th Testable

Collaborators can also brainstorm and mind map

checking of the acceptance criteria (positive and negative) by various participantsmap y p p

User story docs: concise, sufficient, necessary necessary




E pl A pt C it iExample: Acceptance CriteriaAssume you are testing a browser-based Assume you are testing a browser based applicationWorking with the product owner, you Working with the product owner, you might define the following acceptance criteria

All screens display correctly on current and previous versions of Internet Explorer, Firefox, Safari, and ChromeSafari, and ChromeAt 100% magnification, no horizontal scrolling is required to view any part of any screen




E pl I d p d t T ti d A ilExample: Independent Testing and Agile




St t f T tiStatus of TestingIn agile projects, change happens, rapidly and ofteng p j g pp p yWhen the features change, so does product quality and quality risk!Change can make a mess of your status reporting processes if you’re not processes if you re not carefulChange also means that accurate test status is critical f t t d i ifor smart team decisionsChange can have a retroactive impact on features from previous piterationsSo, change often meansexisting tests must changeand risks must be re-evaluated




C i ti T t R ltCommunicating Test ResultsTest progress can be recorded using automated test Test progress can be recorded using automated test results, agile task boards, and burndown charts Test status can be communicated via wikis, standard test management tools, and during stand-ups Project, product, and process metrics can be gathered (e g customer satisfaction test pass/fail defects (e.g., customer satisfaction, test pass/fail, defects found/fixed, test basis coverage, risks mitigated, etc.)

Metrics should be relevant and helpfulMetrics should never be misused

Automating the gathering and reporting of status and metrics allows testers to focus on testingand metrics allows testers to focus on testing




D il St d p M tiDaily Stand-up MeetingsIn agile task boards, tasks move into columns (to In agile task boards, tasks move into columns (to do, work in progress, verify, and done)Done means all tests for the task pass and all risks associated with the task are sufficiently mitigatedassociated with the task are sufficiently mitigatedTask board status reviewed during stand-ups, which include testers and developers (whole team)team)Each attendee should address:

What have you completed since the last meeting?What do you plan to complete by the next meeting?What is getting in your way?

Team discusses any blockages or delays for any y g y ytask, and works collaboratively to resolve them




E pl T k B dExample: Task Board




C l iConclusionsRisk-based testing is a long-proven testing best Risk-based testing is a long-proven testing best practicesIn any lifecycle, proper risk-based testing y y , p p grequires complete integration into the lifecycleIn Agile lifecycles, risk-based analysis, planning, estimation, design, execution, and results reporting permeate each iterationRisk awareness complements and heightens Risk awareness complements and heightens other Agile techniques (e.g., planning poker)Use risk-based testing for great agilityUse risk based testing for great agility



Documents

Lightweight Use of an Established Best Practice · Agile quality risk analysis process (iteration planning) Gather the agile team List iteration backlog items Identify functional