Assess Prioritize Improve Tutorial at 6th World Congress For Software Quality by Rik Marselis

Assess. Prioritize. Improve.

London, 1 July 2014

|

Assess, prioritize and do parallel & step-by-step improvement Rik Marselis, Sogeti, the Netherlands

London, 1 July 2014 © Sogeti

2 A half-day tutorial

|

Agenda

13:30 Welcome and introductions Why improve? Assess Exercise A

15:00 Coffee break Prioritize Exercise B Improve Exercise C

17:00 End

3

|

Introductions: who are you?

  Name

  Organization

  Experience with process improvement

  Your main learning objective of today

4

|

Introducing: Rik Marselis

5

Management Consultant Quality & Testing at Over 30 years IT experience, of which > 15 years quality & testing

Advisor, process-improver & coach at many organizations Prince2 Practitioner, CMMI and CISA

Trainer for various courses, e.g. Agile testing TMap, TPI en ISTQB accredited

Research Author various books & articles Fellow of SogetiLabs, Speaker at many conferences

Chairman (association of testers, 1600 members)

@rikmarselis

|

How we work together

Questions are welcome

Interaction is good

There are no “silly questions”

Let’s learn from each other

Let’s also have fun !!

6

Why Improve?

|

London Stock Exchange

LA Airport flights grounded

AT&T take out ⅓ of US ‘phones

London Underground free travel with Oystercard

British Passport failure

Intel Pentium chip maths division wrong

Software failure at Stanstead closed check in

Therac 25 radiation overdosing

Airbus A380 incompatible software

Patriot Missile System – Dhahran

Mars Climate Orbiter & Polar Lander

Why do we need to improve?

Because we don’t reach

the expected success

and want to get better!

|

Why improve?

  The project took too much time   The costs were too high   The desired result was not completely delivered

  Summarized: We didn’t get business success

  So we have a need to start the journey towards increased success

9

|

The basic assumption for process improvement

10

+ = People Process Product

+ + +

= = =

| 11

Adequate quality, not too little, not too much

Faults must be prevented; Frontload the process with quality measures

People are fallible perform early reviews and collaborate to improve

PointZERO principles

And remember: Quality can’t be ‘tested in’ at the end

|

PointZERO roadmap

12

Root Cause Analysis

Prioritize improve-

ments

Benefits Deming cycle

Plan

Do

Check

Act

Assess Application

Lifecycle

Continuous Improve-

ment

Improvement Backlog

Assess. -Assess application lifecycle -Root Cause Analysis

|

PointZERO roadmap

14

Root Cause Analysis Assess

Application Lifecycle

Improvement Backlog

|

Assess your application lifecycle

15

|

The traditional IT-lifecycle model

16

|

The holistic view of creating and implementing business solutions

17

|

Detailed Application Lifecycle model

This lifecycle shows

activities. These activities can be done

sequentially (e.g. waterfall)

or

in parallel (e.g. agile).

Each activity is important, you can’t skip any.

18

|

Assess your application lifecycle: use 3 perspectives

Product What is the result that we are working towards? What intermediate products are created and why?

People Who are involved in creating intermediate and end products? Who hands over to who?

Process What are the important activities? Where are the important handover moments?

19

|

Root Cause Analysis

20

|

What is the origin of a problem?

During the assessment of the application lifecycle you will find problems / difficulties / inefficiencies / ineffectiveness

To solve this you need to know the origin of the problem

That can be found using a Root Cause Analysis

21

|

5-why approach

Act like a curious kid Why, … Why, … Why, … Why, … Why, …

22

|

5-why approach, example The vehicle will not start. (the problem) • Why? - The battery is dead. (first why) • Why? - The alternator is not functioning. (second why) • Why? - The alternator belt has broken. (third why) • Why? - The alternator belt was well beyond its useful service life and

not replaced. (fourth why) • Why? - The vehicle was not maintained according to the

recommended service schedule. (fifth why, a root cause)

Five iterations of asking why is generally sufficient to get to a root cause. The key is to encourage the trouble-shooter to avoid assumptions and logic traps and instead trace the chain of causality in direct increments from the effect through any layers of abstraction to a root cause that still has some connection to the original problem.

Note that, in this example, the fifth why suggests a broken process or an alterable behaviour, which is indicative of reaching the root-cause level.

23 Source: Wikipedia

|

RCA is often based on incidents as discovered in tests or live operation

What is the cost of an incident in live operation?

What caused the incident?

What would it have cost to detect and fix the cause?

24

|

RCA based on test incidents: a start…

Often, in an immature lifecycle process, the test incidents are the best available source to determine the root cause of issues with the quality of the product.

Important to know: Defect Detection Phase where was the defect found? Escape Phase where could the defect be found? Defect Injection Phase where was the defect created?

And remember: the Root Cause may even be in a phase before the Defect Injection Phase.

25

|

The result of your assessment:

Improvement backlog

26

|

What does an improvement backlog look like?

27

|

Improvement backlog item - info

28

ID Status Title Business Success factor Risk current situation Risk improvement Relative effort Priority Improvement measures Metric

How does the improvement relate to success?

Quick win / medium / long term

What is the actual improvement?

|

The improvement backlog evolves

Improving is a continuous activity So your improvement backlog must be maintained Priorities may change (will change!!) New improvements will need to be added

Use your improvement backlog as a live document

Or even better, use a tool, e.g. in excel or a Kanban tool (like Trello)

29

|

- Take an Application Lifecycle you know from practice - Assess the collaboration and handovers in the lifecycle - Use the 5-why approach - Derive improvement items and put them at the backlog (in no particular order)

Exercise A “Assess”

30 Groups of 3 or 4 persons. 20 minutes.

Prioritize. - What to improve first?

|

PointZERO roadmap

32

Root Cause Analysis

Prioritize improve-

ments

Assess Application

Lifecycle

Improvement Backlog

|

A mountain can’t be moved in one day

33

Maturing is a long process of small steps forward

Thus: Parallel & step-by-step improvement

Kaizen: continuous improvement

Using the “improvement backlog”

Start improving at the “weak spots”

Don’t elevate the peaks; start improving by filling the valleys

|

But this success is not just process….

34

|

It takes time before the actual improvements are realized

35

When ambitions for change are set high, It takes too much time before the actual

successes are realized

|

Improve gradually

36

parallel and step-by-step improvement with small but measurable effects

|

Parallel and step-by-step improvement

Long term

Medium term

Short Short Short Short Short Short Short Short

Good feeling Fast progress

Useful improvement with high outcome

If you don’t start now you will never have the benefits

Medium term Medium term

37

Consider the ability to change

in your organization

|

How to prioritize?

Unfortunately a mountain can’t be moved in one day. So we will need parallel and step-by-step improvement. We need to prioritize, based on risk. The risk assessment has two components: •  the risk for the organization if the problem is

not solved •  but the risk the organization runs when we

start changing

38

|

Improvement backlog items related to risk

39

ID Status Title Business Success factor Risk current situation Risk improvement Relative effort Priority Improvement measures Metric

What if we don’t change?

What if we do change?

Quick win / medium / long term

How does the improvement relate to success?

|

Risk level influences quality, time and cost

Risk poker is done prior to planning poker 40

|

Risk poker: based on Agile ways of working

41

2 2

1

Discussion

Improvement

3

Note: this is

an example of

risk assessment,

other possibilities

exist.

|

- Take the improvement backlog of the previous exercise, if necessary extend the list you already have

- Prioritize the improvement items - Determine which improvement items you will pick up first (and prepare to explain WHY)

Exercise B

42 Groups of 3 or 4 persons. 15 minutes.

Improve. -How to improve? -How to keep improving? (show benefits!!)

|

PointZERO roadmap

44 The steps in the roadmap are selected for

and tuned to the specific situation

Root Cause Analysis

Prioritize improve-

ments

Benefits

Assess Application

Lifecycle

Continuous Improve-

ment

Improvement Backlog

Deming cycle

Plan

Do

Check

Act

|

Collaboration across the lifecycle of people and artifacts

Quality Gate: collaboration towards handover based on previously agreed criteria

Application Lifecycle management tooling

45

|

Supervision keeps the holistic view of the application lifecycle

46

The points of collaboration for hand-over are important in the overall

lifecycle process

|

Shift focus from time & cost

to Quality & Risk

47 Fit for purpose quality supports business success!

|

Improvement backlog item - info

48

ID Status Title Business Success factor Risk current situation Risk improvement Relative effort Priority Improvement measures Metric

What is the actual improvement?

How will you know if it worked?

|

Measuring

“It is better to use imprecise measures of what is wanted, rather than precise measures of what is not.” - R. Ackoff

Start by defining what it is that we want to achieve And how we will measure the result Try not only to measure the simple indicators (e.g. time & cost) but focus the indicators that really matter (e.g. customer satisfaction)

Start by measuring the current situation you need to compare to know if the improvement works

49

|

Measure, think of 3 dimensions: People, e.g. •  Knowledge

Process, e.g. •  Activities (e.g. nr. of activities, nr. of involved people) •  Time & cost

Product •  Customer satisfaction •  Mean Time Between Failure •  Risks covered

Indicators of Business Success

50

|

Choosing the right measurement & metric

What indicator is suited to support the improvement goal?

Use the metrics to create your dashboard of: people, process and product

It is: your business success monitor

51

|

Shift left

52

To enable “right first time” & “no faults forward”:

Shift the quality focus to early lifecycle activities

Frontload the lifecycle

with quality measures

enable a sustainable

increase of your

business success

|

How will you know your improvements actually contribute to your business success? (what can you measure & how?)

Exercise C

53 Groups of 3 or 4 persons. 20 minutes.

Recap. Assess Prioritize Improve

|

Recap: the Improvement backlog

55

ID Status Title Business Success factor Risk current situation Risk improvement Relative effort Priority Improvement measures Metric

How does the improvement relate to success?

What if we don’t change?

What if we do change?

Quick win / medium / long term

What is the actual improvement?

How will you know if it worked?

|

PointZERO® roadmap

56 The steps in the roadmap are selected for

and tuned to the specific situation

Root Cause Analysis

Prioritize improve-

ments

Benefits Kaizen

Plan

Do

Check

Act

Assess Application

Lifecycle

Continuous Improve-

ment

Improvement Backlog

| 57

So why would you improve? Measure increased success!!

|

PointZERO: the result

58

PointZERO achieves reduced effort, While the focus shifts to early lifecycle activities

|

the PointZERO® vision

59 Stop wasting time and money