Embed Size (px)
DESCRIPTION
There is a trap hidden inside Lean Startup's CustDev cycle. If entrepreneurs want to be successful, they have to be passionate about their ideas. They need to understand more about their customers than anyone else. I’ve spent the last year lecturing to and teaching entrepreneurs and I've observed that their passion for their idea and their belief that they already DO know everything about their customers can prevent them from actually LEARNING what they need to know in order to create as successful business. The same passion and positive psychology required to succeed in the face of uncertainty is hindering them from learning fast enough to survive. How can we take our passion, our vision, a couple “wild ass guesses”, and produce meaningful, validated learning? The question of how to learn as an organization and how to DEMONSTRATE learning has been explored by philosophers of science and by business theorists for years. What can the Lean Startup Community learn about creating scientifically valid experiments that create actionable knowledge? Learn how to fail well and fail faster by keeping your passion focused on the vision and our dispassionate logic focused on the assumptions.
Citation preview
blog: jabe.co
FAILINGprinciples and practices of
Vanity Validation a Paradox of Passionate Commitment
WELL
@cyetain
What was the last thing you failed at?
What did you learn?
Did you share your failure with anyone?
Pre-Talk Questions
blog: jabe.co
FAILINGprinciples and practices of
Vanity Validation a Paradox of Passionate Commitment
WELL
HELLLOOOO McFLYTLC LABS
blog http://jabe.co
Send Anonymous Feedback
http://sayat.me/jabebloom
Joshua (Jabe) BloomCTO : The Library Corporation
& Consulting Practioner TLC Labs
#agile2013
@cyetain
Fail Fast
@cyetain
@cyetain
Learning occurs when we detect and correct error. Error is
any mismatch between what we intend an action to produce
and what actually happens when we implement that action.
-Chris Argyris
@cyetain
How Do We Make Better Choices?
Why is it so hard to Fail?
Could we design a system to help?
@cyetain
One must treat his theory-in-use as both a psychological certainty and an intellectual
hypothesis.-Chris Argyris
@cyetain
How wonderful that we have met with a paradox. Now we have some hope of making
progress.-Niels Bohr
@cyetain
@cyetain
fake dictionary page from colbertnation.com
@cyetain
Although theory without experiment is empty,
experiment without theory is blind.
-Paul Thagard
@cyetain
3 Things To Leave With
• Failing Well Produces more Information than Failing Poorly
• “Passionate Beliefs, Loosely Held”
• Reducing Variability too, Soon risks suboptimal result, too Late increases Failure blindness
@cyetain
We simply cannot rely on randomness to correct
the problems that randomness creates.
-Don Reinertsen
@cyetain
undifferentiated streams of data
@cyetain
@cyetain
“Research is what I’m doing when I don’t know what
I’m doing.”-Wernher von Braun
@cyetain
@cyetain
The Principle of Optimum Failure Rate
0% 100%
Probability of Failure
50%
Pote
ntia
l Inf
orm
atio
n
@cyetain
0% 100%
Probability of Failure
50%
Pote
ntia
l Inf
orm
atio
n
Greater Asserted Information
Greater Asserted Information
@cyetain
0% 100%
Probability of Failure
50%
Pote
ntia
l Inf
orm
atio
n
Greater Asserted Information
Greater Asserted Information
Pretty Sure
theory is wrong
Pretty Sure
theory is right
Interesting Ideas
@cyetain
0% 100%
Probability of Failure
50%
Pote
ntia
l Inf
orm
atio
n
Greater Asserted Information
Greater Asserted Information
Pretty Sure
theory is wrong
Pretty Sure
theory is right
Interesting Ideas
Uncomfortable
Confident
@cyetain
Experience of Failure
Num
ber
of S
ampl
esThe Competency Trap
@cyetain
0% 100%
Probability of Failure
50%
Pote
ntia
l Inf
orm
atio
n
@cyetain
0% 100%
Probability of Failure
50%
Pote
ntia
l Inf
orm
atio
n
Pretty Sure
theory is wrong
Pretty Sure
theory is right
Interesting IdeasHidden
RiskHidden Value
The Line of SURPRISE!
@cyetain
0% 100%
Probability of Failure
50%
Pote
ntia
l Inf
orm
atio
n
Pretty Sure
theory is wrong
Pretty Sure
theory is right
Interesting IdeasHidden
RiskHidden Value
1
2
3
During Customer Development Focus on Interesting Ideas
Before Scaling Validate Your "We Know This Assumptions" to reduce risk of Failure Demand
After Customer Validation Run experiments to Validate Assumptions of Failure
2 3
1
@cyetain
“The typical sequence of coin tosses has high information content but little value; an
ephemeris, giving the positions of the moon and
planets every day for a hundred years, has no more
information than the equations of motion and
initial conditions from which it was calculated, but saves
it’s owner the effort of recalculating these positions.”
-Charles H. Bennett
@cyetain
Based on what we know right now, what problems
do we have the least amount of information
about that we can reasonably expect to
understand?
@cyetain
@cyetain
Risk vs Uncertainty
@cyetain
Alteaory vs Epistemic
Uncertainties
@cyetain
Gamble Invest
@cyetain
Justified MVP
Value of Information
Cost of Acquisition
Cost of MVP
UnjustifiedMVP
Over JustifiedMVP
JustifiedMVP
@cyetain
The first principle is that you must not fool
yourself--and you are the easiest person to fool.
-Richard Feynman
@cyetain
“Most people don’t know how to learn. What’s more, those
members of the organization that many assume to be the best at learning are, in fact, not very
good at it. I am talking about the well-educated, high-powered, high
commitment professionals” -Chris Argyris
@cyetain
“Expertise … breeds an inability to accept new
views.” -Laski
@cyetain
@cyetain
Vanity Validation
@cyetain
I need to be right even if I'm wrong.
@cyetain
Defensive Reasoning
@cyetain
Remain in unilateral control
@cyetain
Maximize "winning"
Minimize "losing"
@cyetain
Suppress negative feelings
@cyetain
Be as "rational" as possible -- by which people mean
defining clear objectives and evaluating their behavior in
terms of whether or not they have achieved them
@cyetain
Mindset Actions
Results Match
Results Mismatch
Single-loop
Double-loop
@cyetain
Valid Public Information
@cyetain
@cyetain
whenever we propose a solution to a problem, we ought to try as hard as we
can to overthrow our solution, rather than defend it.
-Karl Popper
@cyetain
• Identify Your Assumptions and Conclusions CLEARLY AS POSSIBLE PUBLICLY
• Question Your Assumptions and Conclusions
• Seek Contrary Data
• Learn when to correct your Actions and when to correct your Mindset
@cyetain
@cyetain
Abductionnot just for
Aliens
@cyetain
@cyetain
[Abduction] goes upon the hope that there is sufficient affinity between the reasoner's mind and nature's to render
guessing not altogether hopeless, provided each guess is checked by comparison with observation... The effort should therefore be to make
each hypothesis... as near an even bet as possible.
-Charles Peirce
@cyetain
ABDUCE
DEDUCE
INDUCE
Predictive
Probable
Plausible
The Way
Computers "Think"
The Way
Humans Think
Binary
Probability
Analogue
Justifiable
@cyetain
ABDUCE
DEDUCE
INDUCE
Experiences Hypothesises
ExpectedOutcomes
If Coherent
If Expected Outcomes Match Reality
EffectiveMatch
@cyetain
ABDUCE
DEDUCEINDUCE
@cyetain
ABDUCE
DEDUCEINDUCE
SURPRISE!!!
@cyetain
Multi-Hypothesis Research
!=
@cyetain
BRAINSTORM
@cyetain
TheoriesOpinions
Hypothesizes
The Facts and Just the Facts
@cyetain
TheoriesOpinions
Hypothesizes
ConstraintsCriteria
@cyetain
TheoriesOpinions
Hypothesizes
QuestionFacts
@cyetain
TheoriesOpinions
Hypothesizes Request More Information
@cyetain
NO TALKING!
@cyetain
How Would I Validate my
understanding of this
problem?How Would I
solve this
Problem?
•Based on your experiences, what would you do to solve this problem? This is your Hypothesis.•Identify What Needs to Be True if your Hypothesis is true.•Assert, Presume, Assume Truth•Imagine Experiments that would justify the Assumptions
@cyetain
I Assert that this I know this
0% 100%
Probability of Failure
50%
Pote
ntia
l Inf
orm
atio
n
Pretty Sure
theory is wrong
Pretty Sure
theory is right
Interesting Ideas
I Presume Somebody knows this
I am going to Assume
this is true for my
Hypothesis to be true
0% 100%
Probability of Failure
50%
Pote
ntia
l Inf
orm
atio
n
Pretty Sure
theory is wrong
Pretty Sure
theory is right
Interesting Ideas
0% 100%
Probability of Failure
50%
Pote
ntia
l Inf
orm
atio
n
Pretty Sure
theory is wrong
Pretty Sure
theory is right
Interesting Ideas
@cyetain
This is my Hypothesis,
Assumptions and
Experiments
ChallengeAssumptions & Experiments
Rotate Pairs 2-3 TimesAllow Time for Revision Between Rounds
@cyetain
This is my Hypothesis,
Assumptions and
Experiments
@cyetain
Multiple Smaller Experimentsagainst
Multiple Abductive Hypotheses
instead of
Single Large Experimentagainst
Single Hypotheses
@cyetain
Failing Well Produces more Information than Failing
Poorly
@cyetain
What are You Doing w All that Information?
Incremental: Confirm. Disconfirm.
Iteratively: Select Next Step. Generate More Options
@cyetain
Having “Passionate Beliefs, Loosely Held”
FAILURE
MUST
BE AN OPTION
@cyetain
Reducing Variability too soon risks suboptimal
result, too late increases failure demand
@cyetain
@cyetain
Influences &
Sources of More Information
@cyetain
Joshua (Jabe) BloomCTO : The Library Corporation
& TLC Labs
blog http://jabe.coSend Anonymous Feedback
http://sayat.me/jabebloom
