Upload
janelle-klein
View
208
Download
0
Embed Size (px)
Citation preview
Janelle Kleinopenmastery.org @janellekz
Let’s Make the PAIN Visible!
Tour Speaker
We Start with the Best of Intentions
High Quality Code
Low Technical Debt
Easy to Maintain
Good Code Coverage
Stages of Escalating Project Risk
Product Owner: “We’ve got more important things to do.”
Deferring(Problems(
Deferring(Problems(
Painful(Releases(
Manager: “Good job everyone! Keep up that great work ethic!”
Stages of Escalating Project Risk
Deferring(Problems(
Painful(Releases(
Thrashing)
Manager: “We need to go faster! Let’s hire more developers.”
Stages of Escalating Project Risk
Deferring(Problems(
Painful(Releases(
Thrashing) Project(Meltdown(
Developer: “I give up. I don’t care anymore if the project fails.”
Stages of Escalating Project Risk
RESET
“A description of the goal is not a strategy.”
-- Richard P. Rumelt
What’s wrong with our current strategy?
RESET“A good strategy is a specific and coherent response to—and approach for overcoming—the obstacles to progress.”
-- Richard P. Rumelt
The problem is we don’t have a strategy...
What are the biggest obstacles that prevent us from breaking the cycle?
Start%Over%
Unmaintainable%So0ware%
Why Can’t We Break the Cycle?
Consulting with Failing Projects
Engineers: “We’re going to CRASH!”
Manager: “What do I do? We can’t miss these deadlines.”
Obstacle #2: We spend tons of time working on improvements that
don’t actually make much difference.
Great Team Disciplined with Best Practices Constantly Working on Improvements+
Project FAILURE
About 8 years ago…
We made significantly more mistakes in code that we didn’t write ourselves.
Lower Familiarity
More Mistakes=
There had to be more to the story...
Unexpected Behavior
Problem Resolved
Measure Painful Interaction with the Code (Friction)
Troubleshooting
Progress
5 hours and 18 minutes of troubleshooting...
PAINFUL
The amount of PAIN was caused by…
Likeliness(of((Unexpected(Behavior(
Cost(to(Troubleshoot(and(Repair(
High(Frequency(Low(Impact(
Low(Frequency(Low(Impact(
Low(Frequency(High(Impact(
PAIN(
What Causes Unexpected Behavior (likeliness)?
What Makes Troubleshooting Time-Consuming (impact)?
Semantic Mistakes
Stale Memory Mistakes
Association Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
Most of the pain was caused by human factors.
What causes PAIN?
What Causes Unexpected Behavior (likeliness)?
What Makes Troubleshooting Time-Consuming (impact)?
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
What causes PAIN?
Most of the pain was caused by human factors.
Semantic Mistakes
Stale Memory Mistakes
Association Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
What Causes Unexpected Behavior (likeliness)?
What Makes Troubleshooting Time-Consuming (impact)?
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
What causes PAIN?
Semantic Mistakes
Stale Memory Mistakes
Association Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
Most of the pain was caused by human factors.
What Causes Unexpected Behavior (likeliness)?
What Makes Troubleshooting Time-Consuming (impact)?
Non-Deterministic Behavior
Ambiguous Clues
Lots of Code Changes
Noisy Output
Cryptic Output
Long Execution Time
Environment Cleanup
Test Data Creation
Using Debugger
What causes PAIN?
PAIN is a consequence of how we interact with the code.
Semantic Mistakes
Stale Memory Mistakes
Association Mistakes
Bad Input Assumption
Tedious Change Mistakes
Copy-Edit Mistakes
Transposition Mistakes
Failed Refactor Mistakes
False Alarm
The Consequences of Problem-Breakdown
7:01
Iterative Validation with Unit Tests7:010:00
14:230:00
Skipping Tests and Validating at the End
7:01
Iterative Validation with Unit Tests7:010:00
14:230:00
Skipping Tests and Validating at the End
Urgency Leads to High-Risk Decisions
If I make no mistakes I save ~2 hours.
If I make several mistakes I lose ~8 hours.
The Consequences of Problem-Breakdown
Product(Requirements( So1ware(Task(
Possible(Decision(Paths(
So1ware(
Software is a Reflection of our Decisions
Product(Requirements( So1ware(Task(
High%Risk%Decision%Habits%
So1ware((and(all(our(problems)(
(
RESET
Software is a Reflection of our Decisions
Product(Requirements( So1ware(Task(
High%Risk%Decision%Habits%
So1ware((and(all(our(problems)(
(
Technical Debt is just a Symptom of the Problem
Software is a Reflection of our Decisions
PAIN occurs during the process of understanding and extending the software
Complex(So*ware(
PAIN
Not the Code.
Optimize “Idea Flow”
My team spent tons of time working on improvements that didn’t make much difference.
We had tons of automation, but the automation didn’t catch our bugs.
My team spent tons of time working on improvements that didn’t make much difference.
We had well-modularized code,
but it was still extremely time-consuming to troubleshoot defects.
The hard part isn’t solving the problems it’s identifying the right problems to solve.
“What are the specific problems that are causing the team’s pain?”
measures the time spent on:
Idea Flow
x
Troubleshooting
x
Learning
x
Rework
Quality Risk Familiarity Risk Assumption Risk
Why Measure These Things?
x
Troubleshooting
x
Learning
x
Rework
measures the time spent on:
Idea Flow
The Rhythm of Software Development
Write a little code.
Work out the kinks.
Write a little code.
Work out the kinks.
Write a little code.
Work out the kinks.
The Rhythm of Software Development
Conflict Confirm
Rework'
Learn'
Validate(
Modify'
Progress Loop Conflict Loop
Troubleshoot'
The Rhythm of Software Development
Conflict Confirm
Rework'
Learn'
Validate(
Modify'
Progress Loop Conflict Loop
Troubleshoot'
Conflict Confirm
Rework'
Learn'
Validate(
Modify'
Progress Loop Conflict Loop
Troubleshoot'
The Rhythm of Idea Flow
Write a little code. Work out the kinks.
Conflict Confirm
Rework'
Learn'
Validate(
Modify'
Progress Loop Conflict Loop
Troubleshoot'
The Rhythm of Idea Flow
“Friction” in Idea Flow
Conflict Confirm
Rework'
Learn'
Validate(
Modify'
Progress Loop Conflict Loop
Troubleshoot'
Idea Flow Mapping Tools
(Open Source, Supported GA ~June 2016)github.com/ideaflow/tools
Reading Visual Indicators in Idea Flow Maps
Le#$Atrium$
Le#$Ventricle$
Right$Ventricle$
Right$Atrium$
What’s$causing$this$pa7ern?$
Similar to how an EKG helps doctors diagnose heart problems...
...Idea Flow Maps help developers diagnose software problems.
Problem-Solving Machine
Reading Visual Indicators in Idea Flow Maps
= Solution Strategy
The “Heart” of Software Development(the problem-solving machine)
Choose a general strategy
= Solution Strategy
The “Heart” of Software Development(the problem-solving machine)
Understand the system
= Solution Strategy
The “Heart” of Software Development(the problem-solving machine)
Code & work out the kinks
= Solution Strategy
The “Heart” of Software Development(the problem-solving machine)
Back to the drawing board
The type of friction we experience depends on where the disruptions are in the problem-solving process
Struggling with Understanding the Code
The Friction is Here
Troubleshooting
Progress
Learning
Rework
The Ten Pains of Software Development
Design-Fit Pain - When the new feature doesn’t fit well into the existing design.
The Ten Pains of Software Development
Requirements Pain - Bad assumptions about what functionality to build
The Ten Pains of Software Development
Collaboration Pain - Challenges collaborating with other developers on the team.
The Ten Pains of Software Development
Modeling Pain - When it’s difficult to build a conceptual model of the existing software.
The Ten Pains of Software Development
Cognitive Pain - Challenges with complexity and intense thinking
The Ten Pains of Software Development
Alarm Pain - Challenges with false alarms and test maintenance
The Ten Pains of Software Development
Experiment Pain - Challenges with getting feedback by running experiments
The Ten Pains of Software Development
Execution Pain - When changing the code is highly mistake-prone
100 hours
50 hours
Troubleshooting
Learning
Rework
Why categorize problems this way?
Focused Effort = Visible Improvement
The hard part isn’t solving the problems it’s identifying the right problems to solve.
Visibility Changes Everything!!
Troubleshooting
Progress
Learning
7:070:00
0:00 19:52
12 year old project after all original developers left.
Case Study: Huge Mess with Great Team
70-90% of dev capacity on “friction”
The Team’s Improvement Focus: Increasing unit test coverage by 5%
Case Study: Huge Mess with Great Team
“What seems to be our biggest cause of pain?”
Add up the Friction by Tag
#ReportingEngine
#Hibernate
#MergeHell
Case Study: Huge Mess with Great Team
1. Test Data Generation
2. Merging Problems
3. Repairing False Alarms
1000 hours/month
The Biggest Problem: ~700 hours/month generating test data
Experiment Time
Our Perception of Time was WAY OFF
Setup Experiment Analyze Results & Decide Next Experiment
Execute
waiting - time goes slow
doing - time zooms by
Experiment*Pain*
!Numerous!Feedback!Loops!
Itera&ve(Narrowing(
Ambiguous(Clue(
Non6determinis&c(
Lots(of(Test(Cases(
Lots(of(Bugs(
Bad(Input(Assump&on(
Uncertainty(
Difficult(to(Interpret(
Experiment*Pain*
!Slow!Feedback!Loops!
Long%Execu+on%
Noisy%Output%
Limited%Output%
Complex%Behavior%
Non<determinis+c%Behavior%
Lots%of%Code%Changes%
Difficult%to%Interpret%
Test%Input%Genera+on%
Environment%Setup%
Environment%Cleanup%
Have%to%significantly%modify%a%test%
Debugger%
Breakdown the Causes of Pain into Patterns
Distill Lessons Learned into “Decision Principles”
Answers two questions
How do I evaluate my situation?
What should I optimize for?
Trade-off Decisions
The Code Sandwich Principle
The thickness of the sandwich increases
troubleshooting difficulty
Behavior Complexity
Observability
Ease of Manipulation
Optimize for low diagnostic difficulty.
The Waxy Coating Principle
Optimize the signal to noise ratio.
Software (before)
Software (after)
Tests are like a waxy coating poured over the code.
This is Painful!
Application
Usability Problems
3rd Party Dependencies
Upstream Components
Usability problems cascade through dependencies
90% 10%This is Painful!
Application
Usability Problems
3rd Party Dependencies
Upstream Components
The Complexity is in the Supply Chain
The Challenge of our Industry: Growing Complexity
Cost &
Risk
Complexity
Cumulative Complexity in Software Development
Human Limitations
Cost &
Risk
Complexity
Human Limitations
Cumulative Complexity in Shared SW Supply Chain (i.e. shared dumping ground of crappy software)
Bad software has become the new environment pollution.
The Challenge of our Industry: Growing Complexity
“The Idea Flow Factory” (supply chain model)
Optimize the Rate of Idea Flow
Across the Organization (or the Industry)
18 months after a Micro-Services/Continuous Delivery rewrite.
Troubleshooting
Progress
Learning40-60% of dev capacity on “friction”
0:00 28:15
12:230:00
Case Study: From Monolith to Microservices
18 months after a Micro-Services/Continuous Delivery rewrite.
Troubleshooting
Progress
Learning40-60% of dev capacity on “friction”
0:00 28:15
12:230:00
Case Study: From Monolith to Microservices
Long-Term Pain
0%
100%
Release 1 Release 2 Release 3
Troubleshooting
Progress
Learning
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
(extrapolated from samples)
0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
Figure out what to do Learning is front-loaded
Troubleshooting
Progress
Learning
Long-Term Pain
0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
Rush Before the Deadline Validation is Deferred
Troubleshooting
Progress
Learning
Long-Term Pain
0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
Pain Builds Baseline friction keeps rising
Troubleshooting
Progress
Learning
Long-Term Pain
0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
Chaos Reigns Unpredictable work stops
fitting in the timebox
Troubleshooting
Progress
Learning
Long-Term Pain
Developers tried to explain to management: “Technical debt is building up in the code!”
Managers: “We’re already behind schedule!!”
PAIN
Eventually the problems got so bad, they couldn’t be ignored.
Long-Term Pain
Builds were breaking
Releases were painful
Productivity slowing to a crawl
Begging for time
The Team’s Focus: Reducing technical debt in the micro-services code + test automation
The Biggest Problem: Integration problems across teams leading to environment down time.
1000 hours/month!!Cost:
Case Study: From Monolith to Microservices
The cost of bad architecture in the microservices world
is EXTREMELY HIGH.
Visibility gives us a way to manage long-term technical risk.
Risk is the Bridge Language Between Managers and Developers
Quality Risk Familiarity Risk
0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
x
Troubleshooting
x
Learning
x
Rework
Quality Risk Familiarity Risk Assumption Risk
Visibility gives us a way to manage long-term technical risk.
Quality Risk (Troubleshooting)
Likelihood)of))Unexpected)Behavior)
Cost)to)Troubleshoot)and)Repair)
High)Frequency)Low)Impact)
Low)Frequency)Low)Impact)
Low)Frequency)High)Impact)
PAIN)
Familiarity Risk (Learning)
Likelihood)of))working)with)Unfamiliar)
Code)
Cost)to)Learn)
High)Frequency)Easy)to)Learn)
Low)Frequency)Easy)to)Learn)
Low)Frequency)Hard)to)Learn)
PAIN)
Assumption Risk (Rework)
Likelihood)of))making)a))
Bad)Assump4on)
Cost)to)Correct)Decisions)
High)Uncertainty)Low)Delay)
Low)Uncertainty)Low)Delay)
Low)Uncertainty)High)Delay)
PAIN)
How much work does it take to complete a software task?
= Side-Effects from Ignoring the Risk
Trade-off Decisions
Writing Unit Tests Troubleshooting MistakesDirect Cost Indirect Costs
or
We make high-risk decisions because the indirect costs are hard to quantify.
Likeliness of Event
Potential Impactx=
Risk
Ignore Tango(risk not obvious)
Escala&ng)Risk)
Time)Pressure)
Defer)Problems)
Increase)Risk)&)Impact)
of)Extra)Work)
More)Frequent)Bugs)and)Higher)
Task)Effort)
IgnoreRisk
Cycle of Chaos
TangosMultiply
Escala&ng)Risk)
Time)Pressure)
Defer)Problems)
Increase)Risk)&)Impact)
of)Extra)Work)
More)Frequent)Bugs)and)Higher)
Task)Effort)
IgnoreRisk
Cycle of Chaos
More MoppingWork
Escala&ng)Risk)
Time)Pressure)
Defer)Problems)
Increase)Risk)&)Impact)
of)Extra)Work)
More)Frequent)Bugs)and)Higher)
Task)Effort)
IgnoreRisk
Cycle of Chaos
Further Behind on Deliverables
Escala&ng)Risk)
Time)Pressure)
Defer)Problems)
Increase)Risk)&)Impact)
of)Extra)Work)
More)Frequent)Bugs)and)Higher)
Task)Effort)
Productivity plummets because of the overwhelming side-effects
IgnoreRisk
I can’t go any faster!
Cycle of Chaos
Fewer%Problems%to%
Fix%
Stop%%and%Think%
Mi8gate%the%Risk%
Increased%Produc8vity%
Safety'Identify Risk
Cycle of Safety
Fewer%Problems%to%
Fix%
Stop%%and%Think%
Mi8gate%the%Risk%
Increased%Produc8vity%
Safety'Put Tango on a Leash
Cycle of Safety
Fewer%Problems%to%
Fix%
Stop%%and%Think%
Mi8gate%the%Risk%
Increased%Produc8vity%
Safety'
Predictable Completion
Cycle of Safety
Fewer%Problems%to%
Fix%
Stop%%and%Think%
Mi8gate%the%Risk%
Increased%Produc8vity%
Safety'
We improve productivity by improving control not by trying to go faster
Slack Time
Cycle of Safety
Troubleshooting Learning Rework
Optimize Idea Flow by systematically mitigating risk.
Control = Risk Mitigation
Quality Risk Familiarity Risk
0%
100%
Release 1 Release 2 Release 3
Percentage Capacity spent on Troubleshooting (red) and Learning (blue)
“How do we control the risk?”
Risk is the Bridge Language Between Managers and Developers
“Pain Control” in Software Development
Average Pain per Incident
This is Control
Target
Control Limit
Idea Flow Learning Framework
Input: Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term loop long-term loop
1. Visibility
2. Clarity
3. Awareness
F ocus!
Output: “Friction” in Idea Flow
Improve Quality of Decisions
Input: Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term loop long-term loop
1. Visibility
2. Clarity
3. Awareness
F ocus!
Output: “Friction” in Idea Flow
Target - The direction of “better”
Target: Optimize the Rate of Idea Flow
Idea Flow Learning Framework
Input: Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term loop long-term loop
1. Visibility
2. Clarity
3. Awareness
F ocus!
Output: “Friction” in Idea Flow
Input - The constraints that limit our short-term choices…
Idea Flow Learning Framework
Input: Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term loop long-term loop
1. Visibility
2. Clarity
3. Awareness
F ocus!
Output: “Friction” in Idea Flow
Output - The pain signal we’re trying to improve
Idea Flow Learning Framework
Input: Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term loop long-term loop
1. Visibility
2. Clarity
3. Awareness
F ocus!
Output: “Friction” in Idea Flow
Focus on the biggest pain…
F ocus!
Idea Flow Learning Framework
Input: Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term loop long-term loop
1. Visibility
2. Clarity
3. Awareness
F ocus!
Output: “Friction” in Idea Flow
1. Visibility - Identify the specific patterns.
1. Visibility
Idea Flow Learning Framework
Input: Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term loop long-term loop
1. Visibility
2. Clarity
3. Awareness
F ocus!
Output: “Friction” in Idea Flow
2. Clarity - Understand cause and effect.
2. Clarity
Idea Flow Learning Framework
Input: Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term loop long-term loop
1. Visibility
2. Clarity
3. Awareness
F ocus!
Output: “Friction” in Idea Flow
3. Awareness
3. Awareness - Learn strategies to avoid the pain.
Idea Flow Learning Framework
Input: Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term loop long-term loop
1. Visibility
2. Clarity
3. Awareness
F ocus!
Output: “Friction” in Idea Flow
Improve Quality of Decisions
Idea Flow Learning Framework
“The Idea Flow Factory” (supply chain model)
Optimize the Rate of Idea Flow
Across the Software Supply Chain
Brevity?
What should we be optimizing for?
Modularity?
Code Coverage?
What does “better” really mean?
Product(Requirements( So1ware(Task(
High%Risk%Decision%Habits%
So1ware((and(all(our(problems)(
(
Technical Debt is just a Symptom of the Problem
Software is a Reflection of our Decisions
Data-Driven Software Mastery
Input: Decision Constraints
Target: Optimize the Rate of Idea Flow
short-term loop long-term loop
1. Visibility
2. Clarity
3. Awareness
F ocus!
Output: “Friction” in Idea Flow
Improve Quality of Decisions
measures the time spent on:
Idea Flow
x
Troubleshooting
x
Learning
x
Rework
Quality Risk Familiarity Risk Assumption Risk
x
Troubleshooting
x
Learning
x
Rework
Quality Risk Familiarity Risk Assumption Risk
Idea Flow gives us a universal definition of effective practice.
Quality Risk Familiarity Risk Assumption Risk
What are the consequences of our decisions?
Idea Flow gives us a universal definition of effective practice.
Quality Risk Familiarity Risk Assumption Risk
What strategies tend to minimize the friction in Idea Flow?
Idea Flow gives us a universal language for sharing our experiences.
And in what contexts?
Quality Risk Familiarity Risk Assumption Risk
We have an opportunity to learn together like we’ve never had before.
Idea Flow gives us the capability to learn together as an industry.
Why? Because this REALLY Sucks.
Seeing your creation get stomped on by organizational dysfunction is an emotionally damaging experience.
Collaboration Platform
IFM Tools Team Mastery Platform
TeamJoe Sally Mark Eric
Industry Collaboration Platform
Anonymized Data
Integrated #HashTag Glossary
ProjectTiger
ProjectBear
(REST)
Change Starts with Making the PAIN Visible!
Janelle Kleinopenmastery.org @janellekz
Janelle Kleinopenmastery.org @janellekz
Check out openmastery.org for details.
Read my Book.
Think About It.
FREE with Reading GroupBuy It
How to Measure the PAIN in Software Development
Janelle Klein
Why read Idea Flow? Because Rene and Matt said so!
“If you just want to read ONE book about software engineering, this year, read this one: leanpub.com/ideaflow #ideaflow”
Rene Gröschke, Gradle Inc.
“If you don't know about @janellekz's work on Idea Flow Learning, you're missing out. leanpub.com/ideaflow”
Matt Stine, NFJS Tour Speaker
@breskeby
@mstine
Check out openmastery.org for details.