How can metrics help in improving our effectiveness?

agile42 | The Agile Coaching Company www.agile42.com | All rights reserved. Copyright © 2007 - 2016

Metrics Workshop

Gaetano Mazzanti

@mgaewsj

agile42


Expectations


Q:

Which metrics are you using?

What is the goal?

Problems, anyone?


Exercise

Penny Game

How large (and valuable) are your batches?


Velocity, oh my


Velocity, oh my


Velocity, oh my


Correlation between Story Points and Time?


Correlation between Story Points and Time?

Ian Carroll collected data from 25 teams across 4 different organizations, 12500 work items in total – here are data from one of those teams

https://iancarroll.com/2016/04/18/no-correlation-between-estimated-size-and-actual-time-taken/


Story Points vs # of Stories

time

SP

# of Stories


Exercise

Throughput Forecast


Throughput Forecast

Trial1

Trial2 Trial3 Trial4 Trial5 Trial6 Trial7 Trial8 Trial9 Trial10

Trial11

7 11 7 5 17 5 10 16 19 7 10 5 13 13 5 7 6 5 5 3 5 16 6 5 6 19 5 3 5 3 6 3 5 7 10 5 6 8 8 6 5 7 19 10 16 8 10 16

48 56 56 31 62 53 45 53 40+ 50+ 50+ 30+ 60+ 50+ 40+ 50+

Sum of all samples for each trial by column (upper) / Nearest “tens” grouping rounded down (lower)

by Troy Magennis https://creativecommons.org/licenses/by-nc-sa/2.0/


Throughput Forecast

by Troy Magennis https://creativecommons.org/licenses/by-nc-sa/2.0/

1. MinimumSum(a): 31 MaximumSum(b): 70

2. MinimumSumwithlowestremoved(c): 39 MaximumSumwithhighest

removed(d): 68

3. Sumalltrials(e): 581 Averagealltrials(e/11):

53

4. Sumalltrialswithextremesremoved(f): 480 Averagewithextremes

removed(f/9): 53

Actualdataaverage6weekthroughput=57.75.Howclosewasyouraverage?

5. Probabilitiesofachievingatleastnstoriesforasix-weektimespan

SixWeek

Throughput

Counttrialsumgroupsat

least30,40,50,etc.stories

(Count/11)

Likelihood

Atleast30stories 11 1 Atleast40stories 9 0.82 Atleast50stories 7 0.63 Atleast60stories 3 0.28 Atleast70stories 1 0.09 Atleast80stories 0 0 Atleast90stories 0 0

Thisvalueis0to1

Multiplyitby100togetapercentage.0%=nochance,

100%meanseverytrialachievedatleastthislevel.


but… beware


Exercise

Simpson’s Paradox


Simpson’s paradox Global Natural Treat

Live 108 153

Die 123 120

Natural 47% live Treat 56% live

Women Natural Treat

Live 57 32

Die 100 57

Natural 36,3% live Treat 36,0% live

Men Natural Treat

Live

Die

Natural live Treat live


Simpson’s paradox

Global Natural Treat

Live 108 153

Die 123 120


Women Natural Treat

Live 57 32

Die 100 57

Natural 36,3% live Treat 36,0% live

Men Natural Treat

Live 51 121

Die 23 63



Q: why do we need metrics?


really, why do we need metrics?

key reason: to improve


really, why do we need metrics?

decisions


why do we need metrics?

predictability



why?

key questions about your metrics


what do you want to improve?



what do you want to learn?



which metrics?

matter to customers no yes

end-to-end (global)

functional (local)

typical

ideal


pizza delivery

fast delivery

accuracy and quality

predictability

what matters to customers


just three metrics?

Lead Time

Throughput

WIP


Little’s Law

Items In Queue = Arrival Rate * Waiting Time

Lead Time = Work In Progress / Throughput


ideas elaborate do delivered validate

B C

D F

E A

H

G

I

Lead Time

3 2 2

Lead Time

commitment point


0

1

2

3

4

5

6

7

8

1-Feb 3-Feb 5-Feb 7-Feb 9-Feb 11-Feb 13-Feb 15-Feb 17-Feb 19-Feb 21-Feb

scatterplot

54%

71%

88%

96%

days average


CFD

0

5

10

15

20

25

30

35

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20

Backlog To Do In Progress Done


throughput


more about CFDs

0

5

10

15

20

25

30

35

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20


arrivals

departures


CFD

arrivals

departures


CFD

arrivals

departures

average lead time

WIP


CFD

0

5

10

15

20

25

30

35

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20


average lead time

WIP

leading

lagging


some CFD issues

flat line

growing area

staircase


scatterplot

source ActionableMetrics


scatterplot (only bugs)

source ActionableMetrics


frequency distribution the shape of uncertainty*

source ActionableMetrics *Dominica DeGrandis


Weibull distribution

Weibull(1, 0.5)

Weibull(2, 0.5)

Weibull(3, 1)


___

efficiency

process efficiency = total time active time ________

20 5 = 25%

!1!!2!!3!!4!!5!!6!!7!!8!!9!10!11!12!13!14!15!16!17!18!19!20!

elaborate do validate deliver

waitingactive


correlations…

Productivity (throughput)

Quality (# of bugs)

Predictability (variance in throughput)

Responsiveness (Time in Process)

WIP

Focus

Stability

&


correlation with WIP

source Larry Maccherone


correlation with Team Focus



correlation with Team Stability



Exercise

FeatureBan

by Mike Burrows of Positive Incline http://creativecommons.org/licenses/by-sa/4.0/ https://blog.agendashift.com/2016/06/02/featureban-2-0/


tempus fugit


focus on trends

(not on absolute values)


"If you torture the data long enough, it will confess to anything you'd like"

Ronald Coase, economist


Gaetano Mazzanti

@mgaewsj

[email protected]

agile42

Education

How can metrics help in improving our effectiveness?