Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
Construx®
Tools for Dealing with Uncertainty
Succeeding with Software in
a Complex, Chaotic World
Construx®
These presentation materials are © 1996-2018
Construx Software Builders, Inc.
All Rights Reserved. No part of the contents of this
presentation may be reproduced or transmitted in any
form or by any means without the written permission
of Construx Software Builders, Inc.
Copyright Notice
Construx®
Construx
10900 NE 8th Street, #1350
Bellevue, WA 98004
+1 (866) 296-6300
www.construx.com
Steve
McConnell
Introduction to
Uncertainty
5 What Do I Mean by “Uncertainty”?
► Unknowns (more on this later)
► Variability (in individual productivity, team productivity, defect
rates, interpretation of functionality, etc.,etc.)
► Risk (financial definition of “Risk”)
► Literal “uncertainty”—“The state of not able to be relied on, not
definite”
6 In other words,
Uncertainty ≈
Unknowns ≈ Variability ≈ Risk ≈ Unreliability
7 A Case Study in Uncertainty
An Early Extreme
Programming
Project
8 A major software products
company adopted XP very early
Team grew to 100+ people
9 Project Characteristics
Onsite coaches
Onsite customer
Focus on always doing the “next
most important thing”
Short iterations
Delivery of working software
every iteration
10 This was a Well-Run Project!
This was a highly disciplined,
rigorous, well-supported
Extreme Programming project
This project should have been
successful (in some sense)
11 Project Result
After 18 months the project was
cancelled
Company wrote off $90 million
Company laid off 200 staff
The company said,
“We are never doing another
Extreme Programming project
again”
12 Cause of Failure?
► The discipline of “always do the next most useful thing” was not
converging to a viable product
► After 18 months the team had failed to converge to anything the
company thought was coherent enough to sell
► The company ultimately lost confidence that the team would ever
produce a coherent product
13 An Ineffective Approach to Uncertainty
► This is not a criticism of Extreme Programming,
per se
► It is a criticism of the way that uncertainty was addressed on this
particular project (especially in product concept & requirements)
14 Variations on a Theme
► This basic pattern of projects failing because of mishandled
uncertainty has been repeated thousands of times on projects of
all kinds (Scrum, Kanban, Lean, Waterfall, Scrummerfall, etc.)
Software Projects are Beset by
Uncertainty From All Sides
16 Example Sources of Uncertainty
Acquisitions
Additional customers
Changes in implementation
Architecture changes
Art requirements
Budget changes
Budget cuts
Bug fixes
Bugs being found during late
testing
Business crime (e.g., embezzlement)
Business logic / regulatary changes
Change in market requirements
Change in priorities
Change in quality criteria
Change in technology
Change in your company's credit rating
(i.e., staff size changes, resource
changes)
Changes due to scale (10 users =>
10,000 users)
Changes in clients or third parties
Changes in in-house or third-party tools
Changes in requirements
Changes in staffing
17 Example Sources of Uncertainty
Changes to delivery time (i.e., the
deadline changes)
Changing customers
Changing graphics
Code drop from the vendor
Code reuse
Coding errors
Communication protocol changes
between client and server
Company culture changes (e.g., if
you get acquired)
Company direction
Competition features change
Compiler changes (toolset changes in
general)
Compliance changes
Cost reduction and scope reduction
Target customer changes
Dependencies change
Dependencies not fulfilling promises
Depracation of old libraries (client
software)
End of life for certain parts
18 Example Sources of Uncertainty
End of life of system you're
working on
End or support contracts for
supported technologies
Environment (client
environment)
External APIs change
Facebook APIs change
Design approach fails
Financial benefit (need to
release a new version sooner
than expected for cash flow)
Funding changes
Infrastructure/framework changes
Integration changes (system integration)
Integration of acquired company's code
base to fit your standards and what not
Integration with another code base
Internationalization issues (language,
currency)
IP infringement
Lead developer resigns
Management changes
New research that can affect your existing
stuff
Where does Uncertainty come from?
It comes from developing software in our
complex, chaotic world.
Role of Uncertainty
21 Role of Uncertainty
Uncertainty is a common blind spot—
Most projects don’t realize that Uncertainty
is a factor that needs to be addressed
22 Uncertainty is Usually Not Addressed Well
Common responses to Uncertainty tend to be ineffective
► Plan everything! (Waterfall)
► Plan nothing! (Early Agile)
► Don’t have a plan! (Since the beginning of time)
These are caricatures, but too close to the truth for comfort
Talk Roadmap
24 Understanding Software Projects Lectures
25
Size Human
Variation
Defects Uncertainty
The Four Factors Lifecycle Model
26 Talk Roadmap
► Introduction
► Categories of Uncertainty
► Tools for Dealing with Uncertainty
Uncertainty and Size
Uncertainty and Defects
Uncertainty and Human Variation
Uncertainty and the SLM
Uncertainty and the Four
Factors Lifecycle Model
► Conclusions
Categories of Uncertainty
Tool:
Categories of Uncertainty
29 Categories of Uncertainty
Known Unknown
Knowable Unknowable
Static Situation Dynamic/Emergent Situation
Current State Future State
Common Cause Special Cause
30 Categories of Uncertainty
Some categories of uncertainty are more problematic than others
31 Categories of Uncertainty
Known Unknown
Knowable Unknowable
Static Situation Dynamic/Emergent Situation
Current State Future State
Common Cause Special Cause
32 Categories of Uncertainty
Known Unknown
Knowable Unknowable
Static Situation Dynamic/Emergent Situation
Current State Future State
Common Cause Special Cause
33 Categories of Uncertainty
Known Unknown
Knowable Unknowable
Static Situation Dynamic/Emergent Situation
Current State Future State
?
Common Cause Special Cause
34 Categories of Uncertainty
Known Unknown
Knowable Unknowable
Static Situation Dynamic/Emergent Situation
Current State Future State
?
Common Cause Special Cause
35 Categories of Uncertainty
Known Unknown
Knowable Unknowable
Static Situation Dynamic/Emergent Situation
Current State Future State
?
Common Cause Special Cause
36 Summary of Categories of Uncertainty
Again, some categories of uncertainty (and combinations) are more
problematic than others
Tool:
Essential vs. Accidental Uncertainty
38 Essential vs. Accidental Uncertainty
Adapting Fred Brooks’ distinction between Essential and Accidental,
“Essential” Uncertainty inherent in the problem we’re trying to
solve
“Accidental” Uncertainty created by our solution
39 Examples of Accidental Uncertainty
We decide to hire new staff with unknown capabilities
We decide to develop functionality in an area where our users don’t
quite know what they want
We decide to try a new design approach
We decide to use a new programming language
We decide to use a new methodology
All the above, at the same time!
40 Accidental Uncertainty
This is Uncertainty that we take on, through our decisions, and which
in many cases is discretionary
Tools for Dealing with
Uncertainty
The Interaction Between
Uncertainty and Size
Human
Variation
Defects
Size
Uncertainty
43 Uncertainty and Size
The larger the project is, the disproportionately larger impact
uncertainty will have
44 Cocomo II and Uncertainty – Small Project
Cumulative Effect of
Uncertainty Factors:
5x
45 Cocomo II and Uncertainty – Large Project
Cumulative Effect of
Uncertainty Factors:
8x
Tool:
Keep Projects Small
47 Project Outcomes by Project Size
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
10 FP
500-1000 LOC
100 FP
5K-10K LOC
1,000 FP
50K-100K LOC
10,000 FP
500K-1M LOC
100,000 FP
5M-10M LOC
Perc
enta
ge
Failed
Late
On Time or Early
Data from Economics of Software Quality, Capers Jones, 2012
48 Essential vs. Accidental Uncertainty and Size
Size is often an Essential
source of uncertainty, but
… sometimes …
it’s Accidental
49 Final Word on Uncertainty and Size
High Uncertainty and Large Size is a Bad Combination
50 Remember That Early XP Project?
Known Unknown
Knowable Unknowable
Static Situation Dynamic/Emergent Situation
Current State Future State
Now Add Large Size to This (200 Staff)
Common Cause Special Cause
The Interaction Between
Uncertainty and Defects
Size Human
Variation
Defects Uncertainty
Tool:
Intellectual Phase Profile
53 Intellectual Phase Profile for Projects
and Activities
Schedule
Focus
Discovery
(Requirements)
Invention
(Design) Construction
(Implementation)
54 Intellectual Phase Profile
Overlap = Wicked Problem
Schedule
Focus
Discovery
(Requirements)
Invention
(Design) Construction
(Implementation)
“Wickedness”
(not just iteration)
55 Intellectual Phase Profile
Schedule
Focus
Discovery
(Requirements)
Invention
(Design) Construction
(Implementation)
Overlap =
►Dependencies
► Potential for work
that will need to
be redone
► Potential for
defects
► Risk
►Uncertainty
56 Intellectual Phase Profile and Defects
Overlap = Uncertainty = Defect Potential
57 Uncertainty and Defects
Uncertainty handled badly Rework (aka Defects)
Plan wrong
Experiment wrong
Design wrong
Build wrong
Build wrong thing
58 Uncertainty and Defects
Uncertainty Defects Extra Cost, Longer Schedules
This becomes a vicious circle, with
Extra unplanned work More uncertainty
59 The Intellectual Phase Profile Provides
Key Questions to Ask
► How much uncertainty do I expect in requirements?
► How much uncertainty do I expect in design?
► How much interaction should I expect between requirements and
design, wrt uncertainty?
► How much uncertainty should I expect in construction?
► How much interaction should I expect between construction,
requirements and design, wrt uncertainty?
60 Variations in Sources of Uncertainty-
Concentrate on High Focus Areas
61 Expect More Defects!
Expect more defects in the
areas of high uncertainty
Focus
High Focus
Area
High Activity
Overlap
(wickedness) in a
High Focus Area
= High Defect
Potential
62 Ask About Essential vs. Accidental Uncertainty
How much Uncertainty is inherent in Discovery, Invention, and
Construction (“Essential”) vs. how much is being created by your
team’s solution? (“Accidental”)?
Avoid Accidental Uncertainty
63 An Example of Accidental Uncertainty
Related to Defects
“Defect Detection Gap”
Defect creation
Defect removal
Well-Run Project
Time Time
Cumulative
Defects
Typical Project
This area is
accidental
uncertainty This is too,
but better
controlled
The Interaction Between
Uncertainty and Human Variation
Size
Defects
Human
Variation
Uncertainty
65 Uncertainty and Human Variation
How much uncertainty can you safely take on?
Tool:
MSF Risk Offset
67 MSF =
Motorcycle Safety
Foundation
68 MSF Risk
Offset Tool
69
Resu
lt
What is Your Risk Offset?
Skill Scale:
High
Skill Scale:
Low
Risk Scale:
High
Risk Scale:
Low
70
Resu
lt
What is Your Risk Offset?
Skill Scale:
High
Skill Scale:
Low
Average Risk with Higher Skill:
Good Result
Average Risk with Average
Skill: Average Result
Average Risk with Lower Skill:
Bad Result
Average Risk with Lower Skill:
Worse Result
Low Risk with Higher Skill:
Best Result Risk Scale:
High
Risk Scale:
Low
71 What Are The
Software
Implications
of the MSF
Risk Offset
Tool?
72
Resu
lt
Software Implications: All These Apply!
Average Uncertainty with
Higher Skill: Good Result
Average Uncertainty with
Average Skill: Average Result
Average Uncertainty with
Lower Skill: Bad Result
Average Uncertainty with
Lower Skill: Worse Result
Low Uncertainty with Higher
Skill: Best Result Skill:
High
Skill:
Low
Uncertainty:
High
Uncertainty:
Low
73
Skill:
High
Skill:
Low
Uncertainty:
High
Uncertainty:
Low
Resu
lt
Software Implications
Typical Result With an Average Skill Level?
Average Result ?
74
Skill:
High
Skill:
Low
Uncertainty:
High
Uncertainty:
Low
Resu
lt
Software Implications
Best Result With an Average Skill Level?
Good Result
?
Best Result is Not Attainable at
this Skill Level
75
Skill:
High
Skill:
Low
Uncertainty:
High
Uncertainty:
Low
Resu
lt
Software Implications
Result With a High Skill Level?
Best Result is Readily
Attainable at this Skill Level
Good Result is Attainable
Even With High Risk ?
76
Skill:
High
Skill:
Low
Uncertainty:
High
Uncertainty:
Low
Resu
lt
Software Implications
Best Result with a High Risk/Uncertainty Level?
Good Result ?
Bad Result
Best Result is Not Attainable at
this Risk/Uncertainty Level
77
Skill:
High
Skill:
Low
Uncertainty:
High
Uncertainty:
Low
Resu
lt
Software Implications
Best Result with a Low Skill Level?
Bad Result
Bad Result (Best Possible
Result with Low Skill Level) ?
78
Skill:
High
Skill:
Low
Uncertainty:
High
Uncertainty:
Low
Resu
lt
XP Case Study
Bad Result
?
?
79
Skill:
High
Skill:
Low
Uncertainty:
High
Uncertainty:
Low
Resu
lt
XP Case Study
?
?
Even Worse Result
Extending the MSF Tool
81
Resu
lt
Uncertainty and Size
Size:
Small
Size:
Large
Average Uncertainty with
Small Size: Good Result
Average Uncertainty with
Average Size: Average Result
Average Uncertainty with
Med-Large Size: Bad Result
High Uncertainty with
Med-Large Size: Worse Result
Low Uncertainty with
Small-Medium Size: Best Result Uncertainty:
High
Uncertainty:
Low
82 Extending the MSF Tool
We can create analogous diagrams with any pair of factors from the
Four Factors Lifecycle Model
Size Human
Variation
Defects Uncertainty
The Interaction Between
Uncertainty and the SLM
Size Human
Variation
Defects Uncertainty
84 Interaction Between Uncertainty and the SLM
► More uncertainty calls for tighter feedback loops, i.e., more
iteration
► Briefly, …
85 The Lifecycle Building Block
Require-
ments
Archi-
tecture
Construction System
Test
86 Variation: Length of Iteration
87 Variation: Iterating Within an Activity
88 Variation: Iterating Across Activities
89 Interaction Between Uncertainty and the SLM
► Overall, iteration is a powerful tool for addressing Uncertainty
► This is covered extensively in the USP Lecture series
Uncertainty and the Whole Four
Factors Software Lifecycle Model
Size Human
Variation
Defects Uncertainty
Tool: Four Factors Lifecycle Model Radar Chart
92 Four Factors Lifecycle Model Radar Chart
The kind of Uncertainty we’re talking about here is the Uncertainty
of, “Should we do this project?”
93 Four Factors
Lifecycle Model
Radar Chart
Size
Defects
(Defect Potential)
Uncertainty Human
Variation
4
3
2
1
94 Proposed Radar
Chart Scale (Draft)
15+ Green
13-14 Yellow
11-12 Orange
≤10 Red
Defects
(Defect Potential)
Uncertainty Human
Variation
Size
4
3
2
1
95
4
3
2
1
Ideal Project
Score: 16, Green
Size
Defects
(Defect Potential)
Uncertainty Human
Variation
96
4
3
2
1
Worst Possible
Project
Score: 4, Red
Defects
(Defect Potential)
Uncertainty Human
Variation
Size
97
4
3
2
1
Best Possible
Project With High
Uncertainty
Score: 13,
Yellow
Defects
(Defect Potential)
Uncertainty Human
Variation
Size
98
4
3
2
1
Best Possible
Project With
Large Size
Score: 13,
Yellow
Defects
(Defect Potential)
Uncertainty Human
Variation
Size
99
4
3
2
1
Best Possible Project
With Both High
Uncertainty and
Large Size
Score: 10,
Red
Defects
(Defect Potential)
Uncertainty Human
Variation
Size
100
4
3
2
1
XP Case Study
Project
Score: ~10, Red
Defects
(Defect Potential)
Uncertainty Human
Variation
Size
New methodology
New product
Lots of requirements
volatility
200 staff
>18 months
Large team
(regression to
mean)
Team new to XP
Extensive
coaching
Disciplined use of XP’s
quality practices
101
4
3 2
1
What Do We Do With Our Score?
Suggested actions
Green Proceed!
Yellow Take steps to reduce the
score, if possible
Orange Proceed only if reducing
the score to yellow or green
is possible
Red Proceed only if reducing
the score to yellow or green
is possible
Defects
(Defect Potential)
Uncertainty Human
Variation
Size
3
2
1
Conclusions
103 Summary
► Software projects with high uncertainty are common
► Software projects with high uncertainty are difficult
► Numerous tools exist to help understand and deal with
uncertainty
104 These Tools Help Understand Uncertainty
► Categories of Uncertainty
► Essential vs. Accidental Uncertainty
► Project Size and Cocomo II’s Uncertainty-Related Factors
► Defects and Intellectual Phase Profiles
► Human Variation and MSF Uncertainty Offset Tool
► Software Lifecycle Model and focused iteration
► Four Factors Lifecycle Model Radar Chart
105 These Additional Tools Can Help Too
► Hardest Part First
► Cone of Uncertainty
► Cynefin
► Feature Build Up
► Prototyping
► Spikes
► Defect Detection Gap
Size Human
Variation
Defects Uncertainty
And Much More!
106 See Much More Detail in the Understanding
Software Projects Lectures!
Construx®
Construx
10900 NE 8th Street, Suite 1350
Bellevue, WA 98004
+1 (866) 296-6300
www.construx.com
We believe that each and every software
project can be successful.
We believe that developing the professional
skills of companies, teams, and individuals is
the best way to make software projects
more successful.
For information about our training and
consulting services, contact