Upload
devops-israel
View
107
Download
0
Tags:
Embed Size (px)
DESCRIPTION
As devops practitioners we focus on improving the culture of collaboration so that others play nicely with us & we play nicely with others - but what if the biggest thing holding us back from change is our own brains? Cognitive biases can deeply affect our behaviours towards others by herding us towards mental shortcuts that are optimised for timeliness over accuracy, at the expense of rationalising irrational behaviour. You are probably pushing these biases onto other people every day but don't even know it. Does that idea make you feel unconfortable? You are probably experiencing the Semmelweis reflex kicking your confirmation bias right now. Knowing is half the battle. This talk will delve into some of the well-known and less well-known biases that may be affecting your ability to work with your peers, and your team's ability to work constructively with other teams. Attendees will leave the talk with an overview of biases they run into every day, how to hack their brains to use these biases to their advantage, and some tips on how to mitigate the effects of the limitations baked into their wetware. We have met the enemy and he is us. Speaker: Lindsay Holmwood Lindsay Holmwood is a engineering manager living in the Australian Blue Mountains. He is the creator of Visage & cucumber-nagios, and organises the Sydney DevOps Meetup. He runs a distributed infracoders team at Bulletproof Networks, that builds hassle free tools, and was responsible for ensuring 100% uptime for the 2010 + 2011 + 2012 Movember campaigns. In his spare time, Lindsay organises the monthly Sydney DevOps Meetups. He also won third place at the 1996 Sydney Royal Easter Show LEGO building competition.
Citation preview
Devops field guide to
understanding
cognitive biases
Hi!I'm Lindsay Holmwood
@auxesis
Engineering manager
@
Bulletproof Networks
Co-organise Sydney
Devops meetups
Started
Devops Down Under
New iPhoneNew iPhone
http://www.flickr.com/photos/williamhook/4742869256/
Your current
iPhone crashes
http://www.flickr.com/photos/10268891@N07/856565056/
Your current
iPhone is slower
http://www.flickr.com/photos/brenderous/3671106784/
Time to upgrade!
Why?
Apple has made
my device slower!
Clustering
illusion
See patterns in
randomness
See correlations
where they are none
Your device has
always been that slow
Clustering illusion is a
Cognitive bias
What is
cognitive bias?
"Mental shortcut"
Timeliness
Accuracy
http://www.flickr.com/photos/frostnova/2268471558/
• Problem solving
• Heuristic
• Correct result
• Rational choice
• Problem solving
• Heuristic
• Incorrect result
• Cognitive bias!
Heuristic?
Pattern matching
Heuristics are simple, efficient rules often used by people to form judgements and make decisions.Involve focusing on specific information, and ignoring others.
http://www.flickr.com/photos/buttim/1297081125/sizes/o
What helped
your ancestors
survive!
http://www.flickr.com/photos/dhdesign/1096464615/
http://www.flickr.com/photos/princess_l_88/4948893120/
http://www.flickr.com/photos/cphotos/4780625450/
http://www.flickr.com/photos/amesis/3777130839/
http://www.flickr.com/photos/artotemsco/5465529987/
http://www.flickr.com/photos/fish2000/4648423198/
see it go over here
http://www.flickr.com/photos/cphotos/4780625450/
http://www.flickr.com/photos/cphotos/4780625450/
was actually here
http://www.flickr.com/photos/cphotos/4780625450/
Confirmation bias
Seek information that
reinforces position
Ignore alternative
explanations
Interpret ambiguous
information in your
favour!
http://www.comedycentral.com/press/images/colbertreport/stephencolbert2.jpg
•Who is clueless enough to think Colbert is a conservative?
•Why do conservatives like Stephen Colbert?
•Ohio study finds conservatives believe Colbert dislikes liberalism
http://althouse.blogspot.com.au/2006/05/who-is-clueless-enough-to-think.htmlhttp://www.alternet.org/story/137918/why_do_conservatives_like_stephen_colberthttp://crooksandliars.com/john-amato/ohio-study-finds-conservatives-believe
http://www.youtube.com/watch?v=Ahg6qcgoay4
http://www.youtube.com/watch?v=Ahg6qcgoay4
Debugging code
My code is perfect!
I know the
problem is in
this other library
I just can't find it!
*30 minutes pass*
The bug was in
my code
Counter this
behaviour
Take opposing
viewpoints
In studies of stock markets, investors made more profit when the resisted bias.
To combat the effects, investors can adopt a contrary viewpoint "for the sake of argument".
Krueger, David; Mann, John David (2009), The Secret Language of Money: How to Make Smarter Financial Decisions and Live a Richer Life, McGraw Hill Professional, pp. 112–113,ISBN 978-0-07-162339-1, OCLC 277205993
Is your conclusion
negative?
It could be biased.
Cognitive therapy is a type of psychotherapy developed in the 1960's that has become a popular approach to combating cognitive biases.
As new data tends to reinforce existing opinions, cognitive therapy teaches people to interpret new data impartially.
Harness this
behaviour
Good collaboration?
Free pass on infractions
When people are given feedback that conflicts with their self image, they regard it as unreliable and are less likely to act on it.
Confirmation reinforces,
You ignore other sources
Provisioning
automation
https://twitter.com/aussielunix
•Provisioning: •We're being hurt badly by VMware provs!
•Software: •We'll automate that now!
2 weeks later
•Provisioning: •We're being hurt badly by load balancer provs!
•Software: •We'll automate that now!
1 week later
•Provisioning: •We're being hurt badly by VMware provs!
•Software: •We'll automate that now!
4 weeks later
•Provisioning: •We're being hurt badly by EC2 provs!
•Software: •We'll automate that now!
18 months
What is
happening?
Working on the thing
closest to us that
hurts us most.
But it's not the biggest
problem overall!
Would you work
like this?
Hindsight bias
"knew it all
along" effect
memory distortion
you alter your memories to fit a narrative in your head of how you perceived things
•Asiana pilot "couldn't see runway before crash"
•Pilots in Asiana crash relied on automatic equipment for airspeed
•Asiana says crash pilot was in training
•Inexperienced pilots may be cause of Asiana jet crash
•Asiana pilots "stayed silent until two seconds before crash"
•Fears pilots' manual flying skills eroding a"er Asiana crash
•
http://news.ninemsn.com.au/world/2013/07/10/12/43/asiana-pilot-couldn-t-see-runway-before-crashhttp://au.news.yahoo.com/thewest/a/-/world/17940112/asiana-flight-crew-saw-trouble-at-500-feet-u-s-safety-board-says/http://www.theaustralian.com.au/news/world/crash-pilots-wanted-to-abort-landing-as-boeing-777-approached-airport-too-low-investigators/story-e6frg6so-1226675762698http://www.theguardian.com.au/story/1628513/inexperienced-pilots-may-be-cause-of-asiana-jet-crash/?cs=5http://www.independent.co.uk/news/world/americas/asiana-airline-pilots-stayed-silent-until-two-seconds-before-crash-at-san-francisco-airport-8698477.htmlhttp://www.news.com.au/world-news/fears-pilots-manual-flying-skills-eroding-after-asiana-airlines-crash-in-san-francisco/story-fndir2ev-1226676769126
Judgement
"You were capable
of preventing
a bad outcome"
Monitoring thresholds
"I don't want that thing
that woke me last night
to wake me again"
"I'm going to make that
problem go away."
But...
How often does
that alert fire?
What percentage
of all alerts is that?
Are there other alerts
that are more frequent?
What makes this alert the
most important to fix?
Counter this
behaviour
Collect data!
http://en.wikipedia.org/wiki/Scientific_method
Analyse that data
http://en.wikipedia.org/wiki/Scientific_method
Make decisions
based on that data
http://en.wikipedia.org/wiki/Scientific_method
Act on data,
not feelings
Scientific method works great for experimentation, but things fall down once you publish your evidence:
http://en.wikipedia.org/wiki/Publication_bias
Explain in terms
of foresight
http://en.wikipedia.org/wiki/Hindsight_bias#Judicial_system
Hindsight blinds,
Foresight finds
http://www.flickr.com/photos/studiogabe/5535916719/
http://www.flickr.com/photos/sjliew/1311490235/
Impossible deadline
http://www.flickr.com/photos/pcb85/4258475681/
???
"I don't know what
you're complaining
about"
"It's just a
bunch of typing"
http://www.flickr.com/photos/jjackowski/3812946635/
Dunning-Kruger effect
https://www.math.ucdavis.edu/~suh/metacognition.pdf
The less you know
about a topic, the
better you think
you are at it
The less you know
about a topic, the
poorer your ability to
recognise skill in others
The more you know
about a topic, the
worse you think
you are at it
Why?
• More skill
• More practice
• More experience
• See complexity + nuance
"Curse of knowledge"
• Less skill
• Less practice
• Less experience
• No peer feedback
"Poor performers
don't learn from
feedback suggesting
a need to improve"
Examples
"I can write code
better than any of our
developers"
"I can scale our
systems way better
than our sysadmins"
"I can lead
people way better
than my boss"
NO. YOU
CAN'T.
You are not a
unique snowflake
Counter this
behaviour
Minimal training
Improved self-assessment
DevOps in a nutshell
grossly incompetent subjects improved their ability to self-assess own skills level after minimal trainingregardless of an actual improvement in that skill
Make engineers
managers
this is a large hammer to wield, but it can be effective
Non-technical
manager?
Lean on your engineers!
Ethnic diversity
most of the studies of dunning-kruger have been in western culturespreliminary research suggests there may be an opposite effect in east-asian cultures
The incompetent gloat,
The capable choke
Confirmation bias
Hindsight bias
Dunning-Kruger effect
How vulnerable to
these biases are you?
Quick show of hands - given everything you've just heard about these biases, how vulnerable to them do you think you are?
• "I'm aware of my biases"
• "These don't affect me"
• "I am better than average"
Better-than-average effect
&
Illusory superiority
Bias blind spot
• Positive attributes
• Above average
• Negative attributes
• Below average
Why?
Self-esteem
Control
Confirmation reinforces,
You ignore other sources
Hindsight blinds,
Foresight finds
The incompetent gloat,
The capable choke
Rhyme-as-reasoneffect
Aesthetics affect truth
• Fluent, Fast, Smooth
• Higher value
This is a test
1.Read the statement once
2.Count the letter F
FINAL FOLIOS SEEM TO RESULT FROM YEARS OF DUTIFUL STUDY OF TEXTS ALONG WITH YEARS OF SCIENTIFIC EXPERIENCE.
How many
did you see?
The answer is 8
Fluency heuristic
http://library.mpib-berlin.mpg.de/ft/rh/RH_Fluency_2008.pdf
FINAL FOLIOS SEEM TO RESULT FROM YEARS OF DUTIFUL STUDY OF TEXTS ALONG WITH YEARS OF SCIENTIFIC EXPERIENCE.
Brain expects pattern
to continue
Brain skips
other information
Counter this
behaviour
http://www.wrayherbert.com/blog/memory-learning-and-the-fluency-heuristic
Trigger more
deliberate
thinking
Use checklists
Induce
randomness
Harness this
behaviour
Rhyme your
mantras
Make text
styling simple
Have you learnt
anything?
http://www.youtube.com/watch?v=IGQmdoK_ZfY
http://www.youtube.com/watch?v=IGQmdoK_ZfY
How many passes?
What's on the
gorilla's chest?
Did you see the
curtains change
colour?
Or one of the
players leave?
http://www.youtube.com/watch?v=IGQmdoK_ZfY
http://www.youtube.com/watch?v=IGQmdoK_ZfY
•It ain't what you don't know that gets you into trouble.•It's what you know for sure that just ain't so. • Mark Twain
Creating "good culture"
is a distraction.
Culture is made
from people
Understand how
you think.
Become more
rational.
Become less
rationalising.
Improve ourselves.
Culture will flow.
Thank you!♥ the talk?
Let @auxesis know!
•Further reading
•You Are Not So Smart - David McRaney
•Sway: The irresistible pull of irrational behavior - Ori Brafman
•Thinking, Fast and Slow - Daniel Kahneman
•Checklist Manifesto - Atul Gawande
•The field guide to understanding human error - Sidney Dekker
http://www.amazon.com/You-Are-Not-So-Smart/dp/1592407366/http://www.amazon.com/Sway-Irresistible-Pull-Irrational-Behavior/dp/0385530609/http://www.amazon.com/Thinking-Fast-Slow-Daniel-Kahneman/dp/0374533555/http://www.amazon.com/The-Checklist-Manifesto-Things-Right/dp/0312430000/http://www.amazon.com/Field-Guide-Understanding-Human-Error/dp/0754648265/