Bram Bronneborg - Agile Testing Isn’t Risking It! - EuroSTAR 2012

Agile Testing isn’t Risking IT! Wednesday 7 november 2012

Eurostar 2012

© All rights reserved

B.W.F.P.M. BRONNEBERG

TEST MANAGER / CONSULTANT

REQUIREMENT & QUALITY MANAGEMENT

[email protected]

WWW.BRAMBRONNEBERG.NL

@BRAMBRONNEBERG

+31 6 55 11 35 10

4 © All rights reserved

Agile Testing

isn’t Risking IT!

Introduction

What is it all about?

Which problems did I face?

How to do it right?

Q & A


Agile Testing

isn’t Risking IT!

Introduction


Which problems did

I face?

How to do it right?

Q & A


Agile in a nutshell

6

Product Backlog

Sprint Backlog

Sprint 2-4 wk

Scrum 24 u

Product Owner Team Scrum Master

Demo

Planning Meeting

Retrospective


Risk & Requirement Based Testing in a nutshell

Risk without requirement:

• Add requirement (a fault found early)

• Remove Risk (no worries about nothing)

Requirement without Risk

• Add product risk (improves coverage of test)

• Remove Requirement (only make what is needed)

Requirements

Product Risks


“A process or method to analyse the risks to the product, with the intention to utilize the testing effort in the most valuable way.”

Product Risk Analysis in a nutshell

Risk class: High

Risk class: Medium

Risk class: Low

Product Risk Requirement

“The likelihood that a product fails in relation to the conceivable damage when a product fails in production.”

8


Agile Testing

isn’t Risking IT!

Introduction


Which problems did I face?

How to do it right?

Q & A


Risk analysis were not translated to the Backlog which makes it difficult to utilise a risk based approach.

Merging or splitting features invalidate the PRA.

High

Medium

Low

10


Risk analysis were not updated every iteration, which could mean that they will be out dated within a sprint.

PRA isn’t updated

11


Risk analysis were only done from the requirements which meant that some product risks were missed.

PRA are done with tunnel vision

Specify Test Cases

Determine Risk

Order Requirements

Analyse Requirements

Analyse Risk

Analyse Risk

Order Requirements

Analyse Requirements

Specify Test Cases

Agile Testing isn’t Risking IT! 12


Risk analysis were done, but the outcome wasn’t taken into account in the planning meetings.

Risk class isn’t used in planning meeting

Team Velocity =

7 Story Points

Hoog = n * 4

Middel = n * 2

Laag = n * 1


Risk analysis were done, but the risks where prioritised using MoSCoW, which doesn’t automatically fit in a Agile environment.

Testing is prioritized based on product risks

Sprint 1 Sprint 2 Sprint 3

14


Agile Testing

isn’t Risking IT!

Introduction


Which problems did

I face?

How to do it right?

Q & A


Gather Risk Items

•Gather & Order Requirements

•Gather Product Risks

Plot Risk Items

•Map Requirements & Product Risks

Calculate Risk Items

•Determine Impact & Likelihood Factors

•Score & Calculate Risk Items/Factors

Strategy to tactic

•Risk class tactic

•Sprint tactic

Gather Risk Items

16


Gather & Order Requirements

REQ 04

REQ 02

REQ 07

REQ 03

REQ 01

REQ 05

REQ 06

REQ 08 REQ

09 REQ 10

Rank #

1

2

3

4

5

6

7

8

9

10

17


Gather Product Risks

PR 02

PR 05

PR 01

PR 04

PR 03

PR 08

PR 07

PR 06

PR 09

18


Gather Risk Items



Plot Risk Items





Strategy to tactic


•Sprint tactic

Plot Risk Items

19


Requirements

Map Requirements & Product Risks 1/3

Product Risks

REQ 04

PR 02

PR 05

PR 01

PR 04

PR 03

PR 08

PR 07

PR 06

REQ 02

REQ 07

REQ 03

REQ 01

REQ 05

REQ 06

REQ 08 PR

09

REQ 09 REQ

10


Requirements


Product Risks

REQ 04

PR 02

PR 05

PR 01

PR 04

PR 03

PR 08

PR 07

PR 06

REQ 02

REQ 07

REQ 03

REQ 01

REQ 05

REQ 06

REQ 08 PR

09

REQ 09 REQ

10



22


PR 01

PR 02

PR 03

PR 04

PR 05

PR 06

PR 07

PR 08

PR 09

REQ 01 ● ● ● ●

REQ 02 ● ● ●

REQ 03 ●

REQ 04 ● ●

REQ 05 ● ●

REQ 06 ● ●

REQ 07 ● ●

REQ 08 ● ●

REQ 09 ● ●

REQ 10 ● ● ●


Gather Risk Items



Plot Risk Items





Strategy to tactic


•Sprint tactic


23


Impact factors e.g.

• Frequency of Use

• Visibility

• Cost of failure

• …

Determine Impact & Likelihood Factors

Likelihood Factors e.g.

• Complexity

• Re-use

• Interfaces

• Dependencies

• Size

• …

𝐼𝑚𝑝𝑎𝑐𝑡 = 𝐹𝑟𝑒𝑞𝑢𝑒𝑛𝑐𝑦 𝑜𝑓 𝑈𝑠𝑒 ∗ 𝑉𝑖𝑠𝑖𝑏𝑖𝑙𝑖𝑡𝑦 ∗ 𝐶𝑜𝑠𝑡 𝑜𝑓 𝑓𝑎𝑖𝑙𝑢𝑟𝑒 ∗ …

𝐿𝑖𝑘𝑒𝑙𝑖ℎ𝑜𝑜𝑑 = 𝐶𝑜𝑚𝑝𝑙𝑒𝑥𝑖𝑡𝑦 ∗ 𝑅𝑒 − 𝑢𝑠𝑒 ∗ 𝐼𝑛𝑡𝑒𝑟𝑓𝑎𝑐𝑒𝑠 ∗ …


Score & Calculate Risk Item/Factors

25

LIKELIHOOD REQUIREMENT

RISK CLASS

IMPACT

Factor A

Factor B

Factor C

TOTAL TOTAL Factor A

Factor B

Factor C

High High High High REQ 01 Medium Low Low Low Low

Low Medium Medium Low REQ 02 Medium Medium High High Low

Medium Medium Low Medium REQ 03 Medium Medium Medium Low Low

High High Medium High REQ 04 High Medium High Medium High

High Low Low Medium REQ 05 Medium Medium Low Medium Low

Medium Low Low Low REQ 06 Low Low Low Low Medium

Medium Medium Medium Medium REQ 07 High High Medium High High

Low Medium High Low REQ 08 Low Low Medium Low Low

Low Low Low Low REQ 09 Low Low Medium Low Low

Medium Medium Medium Medium REQ 10 Medium Medium Low Medium Medium


Gather Risk Items



Plot Risk Items





Strategy to tactic


•Sprint tactic

Strategy to tactic

26


High Medium Low

Check documentation Inspection Walkthrough Peer review

UT test case review Test lead Tester -

ST test case review Test lead Tester -

Include test case in regression set

Sprint + release

Release -

Automate test case CodedUI Record & Playback

-

Test execution on browsers

Top 3 Top 2 Top 1

Risk class tactic

High = test effort * 4 Medium = test effort * 2 Low = test effort * 1

27


Sprint Tactic

Rank #

1

2

3

4

5

6

7

8

9

10

Spri

nt

1

28


PR 01

PR 02

PR 06

REQ 03 ●

REQ 07 ● ●


Sprint Tactic

Spri

nt

2

Rank #

1

2

3

4

5

6

7

8

9

10

29


PR 01

PR 04

PR 07

PR 08

PR 09

REQ 01 ● ● ● ●

REQ 05 ● ●

REQ 08 ● ●

© All rights reserved 30

Scrum bord

To Do Doing Done

REQ 8

REQ 1

REQ 5

S P R I N T 2


Insight into Product Risk & Requirement coverage

31


PR 01

PR 02

PR 03

PR 04

PR 05

PR 06

PR 07

PR 08

PR 09

REQ 01 2 2 2 2

REQ 02 ● ● ● ?

REQ 03 1

REQ 04 ● ● ?

REQ 05 2 2

REQ 06 ● ● ?

REQ 07 1 1

REQ 08 2 2

REQ 09 3 3

REQ 10 ● ● ● ?

? ? ? ? ? ?


Agile Testing

isn’t Risking IT!

Introduction


Which problems did

I face?

How to do it right?

Q & A


B.W.F.P.M. BRONNEBERG TEST MANAGER / CONSULTANT

REQUIREMENT & QUALITY MANAGEMENT

[email protected]

WWW.BRAMBRONNEBERG.NL

@BRAMBRONNEBERG

+31 6 55 11 35 10

33

Technology

Bram Bronneborg - Agile Testing Isn’t Risking It! - EuroSTAR 2012