Embed Size (px)
Citation preview
1
Strengthening Human Performance in the Control Room
Mike Legatt, Ph.D., CPT
WECC Human Performance Conference
“Lighting the Path”
October 24, 2017
2
Core Philosophy:
“All organizations are perfectly alignedto get the results they get.”
Arthur W. Jones
Photo: J. Merlo, NERC
3
Core Philosophy:
In many ways, human beings are like componentson the grid. They each have their tolerances andoptimal ranges of function.
If you interconnect them well, you can build astrong, reliable and self-healing system. Conversely,you can create sub-synchronous resonances byincorrectly interconnecting them: in this you createinefficient circuits that waste energy and time, andcreate heat, risk and cost.
Managing people is no more and no less than beingan organizational reliability coordinator.
4
Please Think About…• A great deal of information in this and past Human Performance conferences
• Just culture
• High Reliability Organizational Culture
• Situation awareness
• Circadian rhythms, fatigue & error
• Visual attention, sustained vigilance
• Physiology and Psychology
• User Experience
• Cause analysis
• How can we combine all these pieces to improve human performance in the control room?
5
Agenda• Reframing the Conversation• Introduction• The changing control room• A look into humans within the organization• Shift work, sleep, and food• Human visual and attentional systems• Looking for meaning• Situational awareness• What are humans, and what is human error?• Cognitive biases• High reliabilty organizational culture• Just culture• Collaboration in emergencies• Near miss reporting• Wrap-up
6
Examples• Please think of an example of where you saw these occur / not go well:
• Operator error in the control room
• Management error in the control room
• A change in a process or procedure
• Adding a new user interface to an operator’s screen
• Promoting a person to management and training them to be successful.
• Emergency response to a dynamic, unanticipated situation
• Human resources events in the control room
• Operator injury as a result of a vehicle accident, fatigue, etc.
7
Reframing the Conversation
8
Reframing
Society
Regulators
Organizations
People
IT Systems
In-the-field devices,
regulated assets
Habits Expectations
9
• Like much of power systems engineering involves applying knowledge from physics, so does human factors engineering apply knowledge from psychology, sociology & anthropology.
• New view of a power system: humans and technology intertwined into a larger system.
• Humans enhance (not threaten) reliability.• Thus, co-optimizing both each part and their interaction is a key goal --
engineering for success.
Human Factors Engineering
10
• A great many human factors influence the overall reliability of the system, for example:
Exercise Visual system function
Sleep Cognitive biases
Corporate culture Mood
Genetic factors Training
Self-monitoring Abstract reasoning, empathy
Diet Self-actualization
Stress & fear Positive thinking
Perceived safety Key performance indicators / HR metrics
Human Factors Engineering
11
Reliability, Resiliency, and Robustness
– Reliability: “Is it working now?”
• Evaluating the consistency of service in real-time, often measured by interruptions (Clark-Ginsberg, 2016)
– Resiliency: “How well/quickly can it bounce back/transform from an adverse event?”
• Evaluating the ability of the system to reduce the magnitude and/or duration of disruptive events…depends upon its ability to anticipate, absorb, adapt to, and/or rapidly recover from a potentially disruptive event” (NIAC, 2009)
– Robustness: “How much can it absorb/withstand disturbances and crises before they become adverse events”?
12
Reliability, Resiliency, and Robustness– Critical infrastructure resiliency: both physical infrastructure, and the
human components
– Resiliency Engineering: Humans are the primary source of resiliency for an infrastructure
• This is especially true as growing infrastructure nexus challenges occur (e.g., electric-natural gas)
– Therefore, to strengthen an organization’s resiliency, you need:
• Strong ways of measuring resiliency
• Creating the optimal environment for human resiliency– High Reliability Organizational Culture
– Just Culture
– “Team of Teams” approaches
• Building habits of resiliency, rather than unprepared emergency response
13
Introduction
14
Who Am I?
15
What do I do?
Nonlinear Contrast Gain Control
16
What do I do?
TechnicalSocio-
Emergency OpsNormal Ops
17
Does It Work?
18
What do I do?
19
What do I do?
20
The Changing Control Room
21
Interconnected Critical SystemsOften, focus on:
– Electric power
– Natural gas
– Water
– Telecommunications
– CybersecurityBut, they all have in common:
– Humans
• Individuals
• Within-organization collaboration
• Between-organizations collaboration
• Human-computer interactions
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
22
What is energy consumption?Can view as the aggregation of:
– Human behavior based on their needs
– Human behavior resulting from policy decisions and financial incentives
– Devices facilitating human behavior
– Human behavior in control rooms, policy decision making, and in the residential and consumer spaces.
– Technical changes to the system as understood by humans
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
23
Interconnected Critical Systems• Modes of operation:
– Compliance
– Reliability
– Resilience
– Robustness• Stress?
– Distress
– Eustress• Cascades in human error
A Dynamic Model of Stress and Attention, from Hancock & Warm (1989)
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
24
HumansIndividuals
– Humans responsible for building, testing, installing, and maintaining the physical and software components necessary for its function.
– Oftentimes, the interface between the humans and these systems are the fastest growing weakest link.
– Increased number and speed of telemetry devices in the field
• -> New systems or subsystems to handle additional data
• -> Additional UIs
• -> Additional monitors and wallboards
• -> Lowered situational awareness, increased misplaced salience
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
25
What are our new patterns?
• Standards &
Procedures
• Metcalf Substation
• Increased media
attention
• Faster news cycles
• Increased breadth and depth of data
• Increasingly dynamic systems
26
What are our new patterns?
• Our management,
culture, and
colleagues
• Praise from our
management
• Our management’s support of us
• Our mental models – what we believe is
what we see, and who we are is what
we’ve done (Fundamental Attribution Error)
27
A Look into Humans Within the Organization
28
Activating Better Habits
Use a positive psychology principal:• Ratio of positive to negative > 2.901 to 1• > Linked to higher performance, lower anxiety• > 7:1 for “superstar” teams• < associated with punitive / fear-based cultures.
29
Activating Human Performance Improvement
1) What are we creating inside our organizations? Learned helplessness or learned optimism?
2) What are we fighting for inside our organizations? Control or excellence?
3) Are we making it easier for people to do the right things, and harder to do the wrong things? Have we set
them up to be successful?
30
Unintended Consequences
The Cobra Effect
31
Unintended Consequences
In the Workplace
32
Habits of Human Performance
Cue -> Routine -> Reward -> Cue…
– Patterns unfold automatically – brain spends resources on other tasks
– Much (> 40%) of what we do each day not a decision, rather a habit.
33
Effectiveness of Habits
Brushing vs. Flossing
Images: Colgate & Dentist4Kidz
34
Creating / Changing HabitsTakes a signifiant amount of willpower.
(Put another way, willingness and ability for self-control).
Self-control as a muscle?
Ego Depletion
35
In Chemistry, activation energy is what’s needed to make a reaction happen; without it, the reaction won’t occur.
In Psychology, it’s the motivation needed to start a task.
Activating Better Habits
From http://www.nature.com/scitable/content/enzymes-and-activation-energy-14711476
36
Activating Better Habits
37
Shift Work, Sleep, and Food
38
Sleep
• Slow-Wave Sleep (SWS)
• Rapid Eye Movement (REM) sleep
• The average person needs 7-8 hours
uninterrupted sleep
• Not just “falling asleep”, but “going through
the sleep process”
• Some areas especially susceptible to
interruption
40
Circadian Rhythms
Source: Neuroscience Education Institute
41
Circadian Rhythms
• Slightly longer than 24 hour schedule
• Body temperature, blood pressure, heart
rate, blood chemistry, attention, error
rates, SWS, and others predictable.
• “Synchronized” vs. desynchronized
Source: FAA
42
Circadian Rhythms
• Time keepers (“zeitgebers”) – cues that
help set our internal clock
–Some of them are social (e.g., SRM-II-5):
Source: Schwartz, H.A. (2013)
43
Circadian Rhythms
Source: Neuroscience Education Institute
44
Fatigue
• “After more than 100 years of research on fatigue, we do
not really know very much about it” - Hockley
• “A lack of sufficient steady-state energy to power physical
and/or cognitive work” – Hancock, Desmond & Matthews
• “Fatigue makes cowards of us all” – Vince Lombardi
45
Fatigue
• Stark & Ash (1917): “When an error occurs it is followed
immediately by other errors and more and more frequently
as the period of work continues”
• Fatigue is not boredom!
• Every hour awake after 10 hours, the equivalent of .004%
BAL decrease in reaction time (Dawson & Reid, 1997).
• Increased error risk, lowered self-monitoring.
46
Fatigue Research
• Thorndike (1900): “An emotional repugnance to the idea of mental work”
• Bills (1931): Lapses in attention• WWII: Polygraphs of pilots in
flight• Drew, Bartlett & Davis (1948):
Cambridge Cockpit• Eddy (2000): Fatigue
Avoidance Scheduling Tool
47
• Serotonin is a neurotransmitter (traveling between nerve cells)
• Low serotonin levels lead to clinical depression
• Excessive serotonin leads to extreme feelings of well-being and happiness
– Serotonin syndrome
– Serotonergic neurotoxicity
Serotonin
Image Sources: Wikipedia
48
• Digesting carbohydrates increases blood sugar, and the body in turn produces more insulin.
• Type 2 diabetes begins when muscle and other cells stop responding to insulin, and eventually insufficient insulin is produced and blood sugar levels stay too high.
Carbohydrates
Image Source: Harvard School of Public Health
49
• Vigilance is: The ability to maintain attention over long periods of time• Vigilance can decrease due to:
– Memory load
– Event rate
– Adverse environmental factors
– Changes in motivation• Maintaining vigilance is stressful, and exhausting (ego depleting), especially
under high stress
Vigilance over time
50
• Ford Motor Company (1917) found two 10-minute breaks every day produced greater worker output over the whole day.
• Fatigue is not “one size fits all”:
• Fatigued extroverts perform better in stimulating environments around people, worse in quiet steady-state environments
• The opposite effect is observed for introverts
Studies on fatigue
51
• Chances of getting acquitted in a criminal trial is higher right after lunch, lower right before (Danziger, Levav & Avnaim-Pesso, 2011).
• Some cultures incorporate early afternoon / post-lunch rest / nap (siesta) to insulate against heat & fatigue.
• In the medical field, a higher probability of human error in the early afternoon (often on long shifts) as opposed to the beginning of a shift.
Time of day
52
Many drugs used to treat depression are SSRIs – they lead to more serotonin in the brain95% of serotonin is secreted in the gut in response to foodUnder stress, most people tend to increase “snack food” intake, reduce “meal food” intake (Oliver & Wardle, 1998).
Diet and Stress
53
Experimentally, stress tends to shift people towards higher saturated fat, sugar and calorie foods (Zellner et al, 2006).
– After eating these foods, participants reported feeling better for a short while.
– People already dieting reported much higher levels of stress during these times.
– Suppressing the impulse to eat fatty foods, especially under stress, may increase the risk of ego depletion!
Diet and Stress
54
• There is a well-known relationship between what you eat and how tired you feel.
• Eating high-carbohydrate foods can lead to hyperinsulemia (excessive insulin)• For people engaged in strenuous exercise, high-carbohydrate foods 3-4 hours
prior lead to reduced fatigue.• For people not engaged in strenuous exercise, high carbohydrate foods are
associated with lightheadedness, fatigue, and occasional fainting.• Fatigue is not just physical; people can also become fatigued mentally.• The brain uses sugar as a fuel source.• Complex mental problems (including vigilance and self-control) require
increased use of sugar in the brain.• People given complex mental puzzles tend to also search out high fat / sugar
/ calorie snacks, compared to people given simple mental puzzles.
Diet and Fatigue
55
• Large food intake requires more blood flow to the gut, in order to support digestion & absorption.
• Carbohydrate intake also leads to decreases in blood pressure.
Diet and Blood Pressure
56
Central fatigue hypothesis: Increased serotonin in the brain can lead to decreased sport/exercise performance (Davis, 1996).
Diet and Exercise
57
Assessment and Monitoring
• Relieve work, or boost performance
• Manage risks of chronic fatigue
• Adaptive automation
• On-task efficiency, vs. driving home
58
Circadian Rhythm Disorders
• Loss of synchronization between internal
clock and external cues
• Sleep/wake schedules out of phase with
circadian rhythms
• Advanced sleep phase rhythms (early to
bed / early to rise)
• Delayed sleep phase rhythms (late to bed
/ late to rise)
Source: Neuroscience Education Institute
59
Shift Work Disorder
• Early morning shifts may be associated
with difficulty falling asleep and waking up
• Frequent evening shifts may be
associated with difficulty falling asleep
• Reduced alertness may be associated
with reduced performance and increased
risk, even outside shift hours
• Major portions of free time may be spent
compensating for sleep loss.
Source: Neuroscience Education Institute
60
Shift Work Disorder
• Most common symptoms:
–Not enough sleep (avg. 5.5 hours)
–Tendency to doze at work
–Reporting low-quality sleep
–Low feelings of well-being
–Decreased physical and mental well-being
– Insomnia (53%), excessive sleepiness (36%)
or both (23%)
Source: Neuroscience Education Institute
61
Shift Work Disorder
• Social Rhythm Disruption is also
something to consider:
–Loss of social cues predicts a bipolar event
more than general psychological stress
–Social support may be lacking
–Spending time with loved ones and friends
may be harder on night shift rotations, etc.
–Even outside shiftwork, social media doesn’t
make people feel as connected as in-person
interactions
Source: Schartz, H. (2013)
62
Diet and Fatigue
• Maintaining a fairly constant blood sugar
level helps maintain energy levels
–Eat within an hour of waking, snack every 2-4
hours
–Make sure strong fiber (soluble and insoluble)
in diet, especially at breakfast
• Fruits and vegetables great, but some
better than others
–Broccoli, kale, brussel sprouts, cauliflower
–Less processed generally better
Source: Schartz, H. (2013)
63
Three-Mile Island (1979)
• Significant increase in reported fatigue
after event at Three-Mile Island.
• 0400: High variability in performance,
operator misinterpreted value.
Source: FAA
• The ones who identified and prevented the tragedy were better rested, newly on-shift.
64
Three-Mile Island (1979)
• Additional human
factors failures:
hidden indicator
light, ambiguous
indicators
Source: FAA
65
Tracking sleep
• Take a sleep history! Track:
–Your work schedule
–Your sleep/wake schedule (what times, and
what quality)
–Any snoring, sleep apneas, etc.?
–Use of drugs and medications
–Quality of your surroundings – supportive of
sleep?
Source: Neuroscience Education Institute
66
Tracking health
• Keep a close eye on:
–Body-mass index (BMI)
–Cholesterol and triglycerides levels
–Blood pressure
–Mood
–Perceived quality of sleep, physical and
mental health
Source: Neuroscience Education Institute
67
Making shiftwork work
• When sleeping:
–Avoid bright light: sunglasses, dark room, etc.
–Keep your bedroom cool
• Strategically use caffeine in the first half of
a shift, no stimulants in the second half if
possible
• Plan naps
• Avoid working late after a shift end, if
possibleSource: Neuroscience Education Institute
68
Making Shiftwork Work
• Making social rhythms – within your team,
other friends who have shift work
• Make sure to be “finely tuned”
–Diet
–Exercise
–Social contact
–Quality of sleep
–Low-stress
–Regular physical examsSource: Neuroscience Education Institute
69
Making shiftwork work
• Bright Light: 10k lux for 30 minutes a day
• Dawn simulation: at end of sleep cycle,
light slowly brightens
Source: Neuroscience Education Institute
70
Melatonin
• Melatonin in the evening advances the
circadian rhythm – helps make falling
asleep earlier.
• If given early in the morning, it doesn’t
help the person feel sleepy until later.
• Recommended use at 03:00 (circadian
rhythm low)
Source: Neuroscience Education Institute
71
Sleep aid medications
• Benzodiazepine Hypnotics
–Greater risk of tolerance and withdrawal
–Estazolam, Flurazepam, Quazepam,
Temezepam, Triazolam
• Non-benzodiazepine hypnotics
–Eszopiclone, Zaleplon, Zolpidem
Source: Neuroscience Education Institute
72
Awake aid medications
• Modafinil and Armodafinil
–Activates areas in the brain that control
wakefulness (the hypothalamus, prefrontal
cortex and anterior cingulate cortex)
–Side effects may include headache, nausea,
dizziness and insomnia
–Modafinil both short- and long-acting,
Armodafinil only long-acting.
Source: Neuroscience Education Institute
73
Awake aid medications
• Stimulants
–Caffeine (300 mg)
–Energy drinks (no evidence to support use,
plus noted associations with heart rhythm
issues with long-term use)
–Simulant use may lead to increased fatigue
when it wears off.
Source: Neuroscience Education Institute
74
Positive Psychology
• Positive attitudes/perspectives linked with:
–Better immune system functioning
–Better creativity & building solutions
–Working memory (how many things you can
process at once)
–Positive attitudes lining up for success
–Positive feedback among team members
highly predictive of team success (Losada
Line)
Source: Achor, S. (2011)
75
Sports Psychology
• Visualizing success (Imagery)
• Goal Setting
–Measurable, written, tracked
• Self-Talk
–Phrases or cues that prompt you in to the
mood you want to be in
• Arousal Regulation
–Relaxation and energizing exercises
Source: AASP
76
Summary
• Shift-work is stressful (but is it
eustress/good vs distress/bad?)
• Any existing other conditions (e.g.,
depression, bipolar disorder, CVD) likely
exacerbated by shiftwork
• However – with tracking and self-
monitoring, you can perform far more
optimally
76
77
Summary
• At the end of the day, we appreciate you.
• You do something our society completely
relies on.
• So, we need to respect and support you in
your work – respect your need to sleep,
the hours of time you spend working, and
the sacrifices you make to do what you do.
77
78
Human Visual and Attentional Systems
79
Visual Attention
Simons & Chabris: Selective Attention Test (1999)
80
Starting Small…
• Eye to LGN to several areas of brain:
– “What” system
– “Where / how” system
–Both involved in visual attention
81
Starting Small…
• Magnocellular (large cells):
–Edges, motion, low contrast
– “Low resolution”
–10% of LGN area
–Fast processing
–Strong linkages to areas
involved in attention
–Nonlinear contrast gain control
system – speeds up visual
processing to high contrast
Source: Legatt, 2005
82
Starting Small…
• Parvocellular (small cells):
–Red/green color, form, high
contrast, slower processing
– “High resolution”
–80% of LGN area
• Koniocellular (“sand” cells)
–Yellow/blue color
–Recently discovered
–Hidden between LGN layers
82Source: Legatt, 2005
83
Signal Detection Theory
• Green & Swets (1966)– Ability to correctly identify
stimuli on a radar screen
• How accurately can you
detect stimuli?– Hit / False Alarm Proportion
– Accuracy (% correct)
– Sensitivity (accuracy)
– Response criterion (bias in
ambiguous situations)
83
Target Noise
Response Hit False Alarm
No Response Miss Correct Rejection
Source: Legatt, 2005
84
Measuring Visual Attention:
VC-CPT
84
Source: Legatt, 2005
85
• Take out a paper and pen
• When you see the next slide, write down exactly what you see on the slide.
• As soon as you’re done, raise your hand.
• The first person to complete this gets $20!
Starting with a Challenge
86
Jewelry is shiny
1 2 3 4 5 6 7 8 9 10 11 12 13 14
Source: JMA Human Error Solutions
87
• How fast was that?
• How hard was it?
– Did any of you make any errors?
– Did any of you have any near misses?
Starting with a Challenge
88
• Now, let’s try that again, but this time do it differently.
• We’ll write out the same lines, • Except we’ll take turns, going between letters and numbers. For
example:
• The first person to complete this gets $20!
Starting with a Challenge
89
• How fast was that?
• How hard was it?
– Did any of you make any errors?
– Did any of you have any near misses?
• So, wait a minute… what just happened here?
Starting with a Challenge
90
How we make mistakes
91
Humans
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
Individuals
– “Multitasking” is a myth: switch-tasking / mental set-shifting
Sources: The Simpsons, Frontiers in Human Neuroscience
92
Change Blindness – Sometimes we don’t see changes in a scene or some critical detail
Problems with Visual Attention
93
“Multitasking” is the attempt to carry on two or more tasks or activities at the same time.But, really what is happening is both
– Habit / automation – learned behaviors being repeated with little thought
– Frequent “mental set shifting” / “context switching” which is computationally intensive and risky.
– Should something (e.g., driving a car) need a jump in attention, there may be insufficient resources available to help.
“Multitasking” or switchtasking
93
94
• Generally, we can do more than one thing, as long as:
– Only one thing requires our conscious attention – and the other tasks can be processed automatically
• e.g., Sending an email while chewing gum
– The task that is requiring our conscious attention can be immediately stopped without consequence and there is time to re-orient
• e.g., we can pause writing out the email when we start to choke on our gum, knowing we can pick up the email later.
“Multitasking” or switchtasking
94
95
• A circuit from Motivation -> Attention -> Memory.
– Therefore, weaknesses in attention mean we have weaknesses storing, holding on to, and recalling memories
– Ever find you had a conversation while driving and a critical detail you knew you needed to know, but couldn't recall?
• Abilities to multitask not significantly different by sex or generation.• It has been estimated $650 billion a year is wasted in the US business
sector due to multitasking: errors and loss of productivity
“Multitasking” or switchtasking
95
96
• When we’re mental set shifting,
– We need extra time every time we move from one task to another, to readjust.
– We lower our performance speed (tasks take between 1½ and 3 times as long
– Our chances of making a mistake grow significantly
– Our IQ temporarily drops by about 10-15 points (worse than a night of no sleep, alcohol, marijuana, or a stroke)
– Our bodies show signs of stress: increased blood pressure, salivary cortisol, heart rate
– We are more likely to cut corners on a task
“Multitasking” or switchtasking
96
http://business.tutsplus.com/tutorials/how-to-do-one-thing-at-a-time-and-stop-multitasking--cms-25159
97
However, this also may feel exciting, and even have addictive components
– More dopamine is released in the brain when we’re trying to multitask
High prolonged multitaskers tend to have smaller anterior cingulate cortex
– A region of the brain responsible for empathy, and mental/emotional control.
Multitasking in meetings and other social settings tends to relate to lower self- and social-awareness / lower EQ.
“Multitasking” or switchtasking
97
http://www.forbes.com/sites/travisbradberry/2014/10/08/multitasking-damages-your-brain-and-career-new-studies-suggest/2/
98
Drivers trying to talk and drive at the same time are both:
– Worse drivers
• 4x more likely to be in a vehicle accident
• Reacted more slowly to brake lights and stop signs
• Involved in more rear-end collisions
• Sped up and braked more slowly than intoxicated drivers (BAL = .08)
– Worse conversationalists
• More forgetful about what discussed
• Struggled to maintain the conversation – especially when it was complexWhether a handsfree device was used or no effect.Listening to audio books did not have a negative effect on driving performanceNew infotainment systems, GPS devices, etc., seem to have similar distracting properties
“Multitasking” or switchtasking: Cell Phones & Driving
98
http://www.apa.org/research/action/drive.aspx
99
“Super-taskers”
– About 2 – 3% of the population can perform better when multi-tasking than when uni-tasking. There is likely a genetic influence – they’re born with it, and you can only learn to improve slightly.
– However, people’s ability to gauge their own multitasking abilities is usually inaccurate.
“Multitasking” or switchtasking
99
100
“Multitasking” or switchtasking
10
101
Field of Vision
• Resolution on retina highest at center, lower towards periphery.
– Farther out, you can only detect motion & vague shapes.
– Motion in the periphery can be distracting
Source: Lean Valley
102
Area of Reach
• Lay out your workspace, and user interfaces so the most commonly-used interfaces and materials are in front of you, and less frequently used materials/interfaces are on the periphery.
Source: UN Enable
103
Blind Spot
• Close one eye and stare at the cross (if
right eye open) or dot (if left eye open).
Move your head back or forward
104
Ambient Lighting
Source: Lean Valley
• High contrast differentials between environment and your
screen/UI can cause stress
–Full brightness in the middle of the night
–Low brightness screen in direct sunlight
–Transitioning between the screen and other factors of the
environment can be frustrating, especially if you have to do it often
or in a high-stress situation.
105
Colors
• Color is a fascinating subject
–Cultural values associated with colors
–Evolutionary issues
• Favorite color blue (60% of men, 35% of women)
Source: Color Blindness Simulator
• Processing speed
increases when
large red objects
seen
– In the eye, three
types of cones
process color
signals
106
Issues with color
• Color discriminability decreases both with
age and nicotine use
• Color deficiency / blindness
Source: Chroma: A wearable augmented reality…
107
Targeting our visual system
• Based on knowledge on the visual system:
– If colors are being used, critical information
should have strong red/green components
(but check users for color blindness)
–Motion captures attention – only use when
you want to capture attention
–Critical information should not have low
contrast differentials
Source: UN Enable
108
Information Organization
• Try to group UIs based on task, frequency
of use, and/or how critical its information
• Try to minimize redundant data across
multiple screens
• If possible, integrate information from
multiple sources to support operations
• If possible, stay involved in
development/vendor efforts; make sure
the screens you receive work for you.
109
Information Organization
• When possible, show as little “all good”
information – especially in complex
systems
• Provide straightforward paths to navigating
to detail and searching
• Whenever possible, test screens in
simulations under a variety of conditions
before deploying.
110
Information Processing
Source: Hancock & Warm (1989)
• Most difficult to
process information
under extreme
stress (reptilian
complex)Working memory
decreases under
stress (7±2 => 3-5
or lower w/ long
term damage)
111
User-Centered Design
• Organize around:
–User procedures and abilities
–The way we process information and make decisions
Source: Bernardo Malfitano
–Users understanding the
state of the system (e.g.,
avoid early auto-pilot
failures)
112
Focus on the message
• “This system is powerful”
Source: StackExchange
113
Focus on the message
• “This system is powerful”
Source: Rudy Vessup
114
Focus on the message
• “This system is powerful” + a lot going on
Source: Gizmodo
115
Focus on the message
• “This system is powerful”
Source: YouTube
116
Focus on the message
• “This system is powerful”
Source: Design Directions
117
Focus on the message
“This system is powerful”
vs.
“This system is optimized so you
can quickly understand what is
going on”
Source: UN Enable
118
Focus on the message
• “This system is trying to optimize SA”
Sources: SA Tech, DHS,
119
Focus on the message
• Why “this system is powerful?”
–Dignitaries and others visiting control rooms
– “cool” / “wow” factor
• Why “Optimizing SA?”
– Important, mission critical systems
–Data that may be changing rapidly, and is of
importance
–Trust in core systems – show primarily the
exceptions, not all information
Source: UN Enable
120
Avoid Pitfalls
• Attentional narrowing – attention grabbed,
drawing focus away from critical
information
• Data overload – More/faster information
presented than a human can process
• Misplaced salience – Avoid the wrong
things standing and catching your
attention
Source: Endsley, 2011
121
Avoid Pitfalls
• Complexity Creep – Additions to a UI or
procedure may affect far more than one
screen
• Low operator confidence in systems
• Uncertainty in data not represented to
users
Source: Endsley, 2011
122
Avoid Pitfalls
• Alarm screens
– “Alarm storms” – critical information scrolling
off the screen due to other important alarms
– “Dust storms” – critical information scrolling off
the screen due to unimportant alarms
Source: Endsley, 2011
123
Watch for Data Context &
Misinterpretation Risk
Source: J. Merlo
124
Watch for Data Context &
Misinterpretation Risk
Source: J. Merlo
125
Focus On What’s Important
Source: J. Merlo
126
Focus On What’s Important
Source: J. Merlo
127
Looking for Meaning
128
Why does corporate culture matter?
Esteem Needs
Belongingness and Love Needs
Safety Needs
Physiological Needs
SelfActualization
129
Why do we need good corporate culture?
Job market transparency employee expectations
Employees are becoming less engaged and satisfied (less than30%, despite education gains, >$500B losses –Gallup, 2012)
The trend toward education over hands-on experience
“What happens on the internet stays there… forever.”
Social media is changing communications and introducingboth benefits and risks
Existing sources of pressure and risk are growing
130
What makes for good management?
Google Oxygen ProjectManagers are successful who:
1) Are good coaches2) Empower their teams and don’t micromanage3) Express interest in team members’ success and personal well-being4) Are proactive and results-oriented5) Are good communicators and listen to their team6) Help their employees with career development7) Have a clear vision and strategy for the team8) Have key technical skills so they can help advise the team
Managers are NOT successful when they:1) Fail to recognize that managing is different than individually contributing2) Are inconsistent in performance management and career development3) Spend too little time managing and communicating
131
Mike Legatt
The How and Why of Why
The Emerging field of Positive Psychology
Corporate culture and employee engagement are key
Fear suppresses
– Creativity
– Motivation
– Positive ownership
Employee management is a skill, not just the next titlechange or compensation strategy
Need for cost reductions: health care, PTO, other risks
132
Fredrick Winslow Taylor– Mechanical engineer who sought to improve industrial efficiency
– One of the first management consultants
– Wrote The Principles of Scientific Management:
• Replace “rule-of-thumb” methods with those based on scientific study
• Actively select, train, and develop each employee
• Provide detailed instruction and supervision of each worker in the performance of that worker’s discrete task
• Divide work between managers and workers: managers plan/think, workers do.
– Moved from skilled individuals doing many things to less skilled individuals, each doing one thing over and over
– Motivating them simple: financial awards can make them work fasterSource: McChrystal, S. (2015). Team of Teams.
133
Fredrick Winslow Taylor• Taylor efficiencies not working in today’s world
– Much of what we do is now on thought not repetitive, automatic actions.
– Financial bonuses now decrease performance and increase error rates.
– Many employees now searching for more meaning in their work; cultural and short term KPI misalignments abound.
Source: McChrystal, S. (2015). Team of Teams.
134
Managing Human Beings and Organizations
W. Edwards Deming(1900-1993)
Source: deming.org
• “What you can measure, you can manage”
• “If you can’t measure it, you can’t manage it.”
135
Managing Human Beings and Organizations
W. Edwards Deming(1900-1993)
Source: deming.org
“It is wrong to suppose that if you can’t measure it, you can’t manage it – a costly myth…
It is the relationship with people, the development of mutual confidence, the identification of people, the creation of a community.
This is something only you can do. It cannot be measured or easily defined.
But it is not only a key function. It is one only you can perform.”
136
Situational Awareness
137
Situational Awareness
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
138
Situational Awareness
• “The perception of elements in the environment within a volume of time and space, the comprehension of their meaning, and the projection of their status into the near future” (Endsley, 1988).
• “Continuous extraction of environmental information, integration of this information with previous knowledge to form a coherent mental picture, and the use of that picture in directing further perception and anticipating future events” (Dominguez, 1994).
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
139
Situation Awareness
140
Situational Awareness
Three levels of situational awareness:
– Level 1: Perception
• What is going on?
– Level 2: Comprehension
• What does it mean?
– Level 3: Projection
• Where is it going? What am I going to do about it?
Situational awareness is necessary both in individuals and within teams.
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
141
Three components of situation awareness:
– Spatial awareness - where you and other objects are in space, orientation to location.
– System awareness – understanding about the state of the system(s)
– Task awareness – understanding about the tasks being engaged in
Situation Awareness
142
Performance
Situational Awareness
Based On: Steve Kass, Military Psychology
Situational
Awareness
WorkloadAttention
Pattern
Recognition Working
Memory
Mental
Models
Stress/
Perceived
Safety
143
Situational Awareness
“Loss Of Situational Awareness” is NOT one of many phrases to represent individual human error, leading to punishment of that individual.
Instead, it is one of several lenses into human performance that helps to understand why a mismatch between environment and human capabilities occurred.
144
Mental Models
How people internally represent the task they are performing / situation they’re in.
• Schema – Representing and organizing information into interconnected chunks
– Experts organize into larger, more effective, more accessible chunks
– Novices struggle to organize schemata
145
Working MemoryShort-term / processing memory (7 ± 2; Miller, 1956)These 7±2 points can be “chunks”, allowing for better storage and processing.Working memory decreases under stress (7±2 => 3-5, or even lower with long term damage)
747 cockpit
146
Attention and Human Performance
148
Situation Awareness: Is this a good example?
149
Situation Awareness: Is this a good example?
150
Situation Awareness: Are these good examples?
151
Situation Awareness – Common Pitfalls• Attentional narrowing
– Attention grabbed, drawing focus away from critical information• Data overload
– More/faster information presented than a human can process• Misplaced salience
– The wrong things standing and catching your attention• “Out of the loop”
– Uncertainty in data and system status not represented to users• Short-term memory over-reliance
– Holding too many chunks, or for too long (> 30 sec)
152
Situation Awareness – Common Pitfalls• Complexity Creep
– Additions to a UI or procedure may affect far more than one screen• Low operator confidence in systems• Workload, anxiety, fatigue, loss of perceived safety• Incorrect mental models• Inattention
153
Situation Awareness – Common Pitfalls• Alarm screens
– “Alarm storms” – critical information scrolling off the screen due to other important alarms
– “Dust storms” – critical information scrolling off the screen due to unimportant alarms
– Alarm accuracy – False Alarms, percent correct
Target Noise
Response Hit False Alarm
No Response Miss Correct Rejection
154
What are Humans, and What Is Human Error?
155
Human Error
156
Human Error
“Human error is a consequence, not a cause. Errors are shaped by upstream workplace and organizational factors… Only by understanding the context of the error can we hope to limit its reoccurrence”
James Reason
157
Human Performance Under Stress
A Dynamic Model of Stress and Attention, from Hancock & Warm (1989)
158
How we make mistakes
From: NERC Cause Analysis Methods
159
Cognitive Biases
160
A baseball and bat together cost $11. The bat costs $10 more than the ball. How much does the ball cost?
Did you say $1 for the ball?
– Most people do, but:
– The correct answer is $0.50:
– $10.50 - $0.50 = $10.00
Why do many of us make these kinds of mistakes?
– Mental shortcuts tend to shorten the amount of time it takes us to come up with an answer, but doesn’t necessarily work in all situations.
A quick thought exercise…
16
161
Humans
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
Individuals
– Cognitive biases
IDENTITY THEFT“Apparently, when you publish your social security number prominently on your
website and billboards, people take it as an invitation to steal your identity.”
LifeLock CEO’s Identity Stolen 13 Times–K. Zetter, Wired.com, April 2010
162
• Recognizing some of the factors that contribute to human error doesn’t necessarily protect you from consequences of it
• Overconfidence can creep in, especially from feeling more competent than you actually are
• Danger exits from illusion of explanatory depth(IOED)
Protecting the Self From Consequence of Error
163
• People often overestimate their ability to explain mechanical, natural, and social processes (Alter, Oppenheimer, & Zelma, 2010; Keil, 2003)
– You likely know that a zipper closes because teeth somehow interlock. But do you know how the teeth actually interlock to enable the bridging mechanism?
– You probably have a vague notion that an earthquake occurs because two geological plates collide and move relative to one another. But do you know more about the mechanism that initially produces these collisions?
– You may think you know your preferred political candidate’s stance on a policy, but do you really know the intricacies of what they believe on an issue?
Illusion of Explanatory Depth
164
• You look at a checklist or procedure, and think you understand the intent of the writers and that they anticipated all possible risks.
• You understand how a piece of equipment works in normal operations (> 95% of the time). Can you accurately anticipate how it will work in an abnormal condition?
• You think may think you know your coworkers’ or employees’ knowledge level and skill set, but do you actually know how they would respond to an emergency or crisis?
• Setting up a projector or conference call is simple… or not.
IOEDs At Work
165
• People are better calibrated when assessing their knowledge for declarative facts and trivia (Rozenblit & Keil, 2002)
– E.g., country capitals• Hints at why IOED occurs – awareness of our lack of specific knowledge
in these domains• People know generally why manmade objects, natural events, and
social events occur, but lack insight on specifically how they work or happen
But, the Domain Matters
166
• Confusion occurs between why and how, and people end up believing they understand complicated manmade, natural, or social concepts quite deeply
• Further, people are often surprised by the shallowness of their own explanations when prompted to describe the concepts thoroughly
IOED
167
• People understand certain concepts at an abstract level quite well
• But, they only superficially understand their more concrete characteristics
• So, when people mistake their mastery of abstract for concrete characteristics, IOED will emerge.
Why Does IOED Happen?
168
• Find a way to focus on the information that illuminates the depth of your understanding
• Steer away from focusing on uninformative cues• Focus on concrete rather than abstract knowledge,
and you should realize your ignorance and hopefully take steps to remedy it
How Can You Avoid IOED?
169
Focus on differences rather than similarities (Namkoong& Henderson, 2014; 2016)Games played from a very young age encourage us to develop this skill
How Can You Shift Your Focus?
Spot the difference
170
How Can You Shift Your Focus?
• Focus on instances rather than categories (Fujita, Trope, Liberman, & Levin-Sagi, 2006)
–Particularly necessary for people in a position of power (Smith & Trope, 2006)
171
How Can You Shift Your Focus?
Minimize psychological distance (time, space) during decision-making (Trope & Liberman, 2010)
Revisiting issues that were
settled in the distant past
Revisiting decisions that were
made in distant locations
(e.g., retreats, conferences,
working from home)
172
Cognitive Biases (a sampling)
• Anchoring – something you’ve seen before seems like the benchmark (e.g., first time you paid for gas)
• Attentional bias – You’re more likely to see something if you’re thinking about it
• Cognitive dissonance – uncomfortable to have two conflicting thoughts
• Confirmation bias – pay attention to things that support your belief
173
Cognitive Biases (a sampling)
• Diffusion of responsibility – “someone else will take care of it”
• Fundamental attribution error – you do things because of the situation, others because of their personality
• Google effect – easy to forget things that are easily available electronically
• Groupthink – people less likely to contradict ideas in a large group
174
Cognitive Biases (a sampling)
• Hindsight bias – the past seems perfectly obvious• IKEA effect – things you’ve built seem more valuable to
you than things others have built• Illusion of transparency-expect others to understand
your thoughts/feelings more than they can• Loss aversion – you’re more likely to try avoid losing
than gaining
175
Cognitive Biases (a sampling)
• Organizational bias – you’re likely to think ideas within your organization are better
• Outgroup homogeneity – you’re likely to think that people in another group all think the same
• Semmelweis reflex – rejecting new ideas that conflict with older, established ones
• Zero-risk bias – likely to choose worse overall solutions that seem less risky
176
High Reliability Organizational Culture
177
• Rhode Island Hospital
– A top medical institution
– Deep tensions between doctors and nurses
• “This place can be awful. The doctors can make you feel like you’re worthless, like you’re disposable. Like you should be thankful to pick up after them.” (A nurse interviewed by a reported in 2000)
– “Time-out” procedure called by a nurse
• Surgeon stormed out, saying “If I want your damn opinion, I’ll ask for it. Don’t ever question my authority again. If you can’t do your job, get the hell out of my OR”. (Duhigg, 156)
An example…
178
• Compensatory techniques informally developed
– Doctors names written on whiteboards – different colors depending on their abrasiveness
– Nurses would communicate ad-hoc when a doctor made a mistake; nurses tended to check up on more error prone doctors
An example…
179
• July 30, 2007• Neurosurgeon J. Frederick Harrington Jr., asked to perform an
emergency surgery on an 86 year-old patient with a subdural hematoma on the left side
• At the time, Dr. Harrington was in the midst of a lumbar laminectomy –quickly reviewed the slides but didn’t write down the side of the hematoma; his nurse failed to note it as well
Setting the stage…
180
• In the surgical suite..• A nurse noted that the consent form lacked a side, and asked for a time
out• Harrington snapped “we have no time for this!” and encouraged the
nurse she could log into a computer and check the CT scans, but he would have her terminated if she did.
• He misremembered the side of the hematoma, and operated on the wrong side of the patient’s skull.
• When Harrington realized the mistake, sealed up the skull and skin, turned the patient over and operated on the correct side, a great deal of time was lost. The patient died two weeks later.
• State board fined the hospital $50k
Setting the stage…
181
• A few months earlier, a doctor and nurse both admitted they weren’t trained to use a checklist, having failed to mark the spot of another neurosurgery.
• A few months later, a doctor and nursing staff discussed a surgery (identifying the correct side), then proceeded to operate on the wrong side of an elderly woman.
– “I don’t know if [the nurse] even knows why she didn’t speak up” • A year and a half later, a surgeon operated on the wrong part of a child’s
mouth during cleft palate surgery• Ten months after that, a drill bit was left inside a patient ($450k fine)
Setting the stage…
182
High Reliability Organizations• “Fishbowl” industries – exceptionally strong social and/or political
pressure crashing down in a failure event.• Complicated systems/fields with a high probability of error.• Major mistakes are so severe, you can’t afford to learn by trial and error.
– Human Error rates typically (3-7 errors per hour, 11-15 under high stress, fatigue, etc.)
• An organization with relatively error free operations over a long period of time.
• An organization that again and again accomplishes a high hazard mission while avoiding catastrophic events.
• An organization in a field in which failures are so severe, they should be avoided at almost any cost.
183
High Reliability Organizations• Five Principles:
1. Preoccupation with failure
2. Reluctance to simplify
3. Sensitivity to operations
4. Commitment to resilience
5. Deference to Expertise• Not “error free”, but “not disabled by an error”• Physical, social and external facets combine in complex operations• Formal and informal expectations for peak performance• Ability to decentralize in an agile fashion in emergencies. TRUST.• Redundancy in people, roles, training
184
High Reliability Organizations• “As they strive for high reliability, organizations shift away from having
outside bodies solely determine their quality agenda to developing an agenda that incorporates the organization’s most important goals.”
Mark R. Chassin, M.D., F.A.C.P.The Joint Commission Center for Transforming Healthcare
• Higher job satisfaction, lower turnover, greater willingness to share learning (Egan & Bartlett, 2004)
• Better at interviewing and selecting candidates who fit in well with HRO mission (Ericksen & Dyer, 2007)
• Less losses due to lost wages, training, severance, sick days, etc.
185
Just Culture
186
Responding To Human Error
Outcome Bias
“The single greatest impediment to error prevention…
is that we punish people for making mistakes.”– Dr. Lucian Leape, Professor, Harvard School of Public Health,
1999 Testimony before Congress on Health Care Quality Improvement
Over-Reaction Under-Reaction
• Discipline of discrete error
• Discipline person who didn’t see risk
• Over-reaction to singular events
• Turn a blind eye to risky choices
• Allow reckless people to go unchecked
• Pass over severe system design flaws
187
The case for just culture
187
Traditional management styles lagging research & employee expectations
Vast amount of work frustration due to inequity and ambiguous signals
188
RecoveryBarriers Redundancy
Ways of addressing events
First, we must first identify At-Risk Behavior Types
189
Human Error (3-7 to 11-15 errors per hour)
190
(The better we are the harder it is to perceive drift)
At-Risk Behavior
191
(< 2% but extremely dangerous)
Reckless Behavior
192
Just Culture Approach
Managing Risk – Fairly and Consistently
Repetitive Human Error
Repetitive At-Risk
Reckless Behavior
At-Risk Behavior
Human Error
Console Remediate or Discipline
Remediate or Discipline
DisciplineCoach
▪ One approach that works across all values
▪ One approach that works both pre and post event
▪ One approach that applies to everyone at all levels
193
Collaboration in Emergencies
194
February 2, 2011, in ERCOT region
–Unexpectedly cold weather
–Several power plants (weatherized to handle 110°+F weather) were unable to come or stay online
–ERCOT initiated EEA level 3 (rotating outages) due to insufficient physical responsive capability
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
195
February 2, 2011, in ERCOT region
– A transmission operator, receiving the order, opened a breaker to a circuit, without realizing that circuit fed a natural gas compressor
– By doing so, the compressor went offline and the pipeline pressure began to drop, offlining another natural gas power plant and an NG feeder to a neighborhood, further worsening conditions
– Coordination and collaboration between parties?
– How long did it take for the parties to collaborate?
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
196
Near Miss Reporting
197
Near-Miss Reporting
Near-misses are times when errors ALMOST occur but don’t –due to luck, an error catch, or barriers just barely working (or successfully working but in the wrong context)In some organizations with after action reviews they may come up but often aren’t documented in a taxonomy
Peer Check
Safety Check
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
198
Near-Miss Reporting
– Every time a near miss occurs, it is an OPPORTUNITY.
– But, it needs:• Corporate culture that
supports near miss reporting (e.g., just culture)
• Treatment of the individual as an SME on the error, not a liability
Peer Check
Safety Check
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
199
Near-Miss Reporting
– But it needs:
• An organization focused on being proactive, not reactive
• An organization with the ability to not only capture, but also track and trend near misses (organizational situational awareness)
Peer Check
Safety Check
200
Near-Miss ReportingTake the analogy of user experience (UX) in software development:
A bug fixed during the software development (alpha) stage: 1x
A bug fixed during the testing (beta) stage: 10x
A bug fixed in the production phase: 1000x!
This study was focused on financials (dollars) in terms of development costs.
But, it also can apply to stock price, public reputation, strength of relationships between entities, etc. So – a near miss reporting system can be a significant pathway to cost and reputational risk savings.
Erhlich & Rohn, 1994
201
Near-Miss ReportingChallenges and solutions to near miss reporting:
Natural disinclination to admit to an error.
Anonymous or at least confidential reporting
Suspicion that a report can be counted against the reporter
An organizational policy of indemnity.
Separation of group functions
Source: DOE Human Performance Improvement Handbook, Volume 2
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
202
Near-Miss ReportingChallenges and solutions to near miss reporting:
Skepticism – management won’t take action
Feedback to reporting community
Reporting will take too much time or effort
Reporting infrastructure needs to be easy and available to all
Source: DOE Human Performance Improvement Handbook, Volume 2
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
203
Near-Miss ReportingA pathway to implementing a near miss reporting structure:
Inform, organization-wide
Designate an event report coordinator
Develop a report format
Make the report form readily available
Provide examples of reports
Encourage reporting of errors
Track and trend errors
Use data in ongoing training and lessons learned
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
204
Near-Miss ReportingSome additional suggestions for near miss reporting:
Positive reinforcement: A positive recognition for a reporting event (aligning with an organization’s well stated mission of continuous improvement). Strongly suggest social-domain, not financial domain.
Tracking reports: Allow reporters to see, at some level, that their report is being taken seriously, and tracking and growth is occurring.
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
205
“All organizations are perfectly aligned to get theresults they get.”
Arthur W. Jones
The same is true for critical infrastructures.Having the right cultures and collaborations canlead to our getting the results we want.
Photo: J. Merlo, NERC
Core Philosophy?
206
Wrap-up
207
Closing Thoughts
©2016, Resilient Grid. All rights reserved. | [email protected] | 512-766-4743 | resilientgrid.com
208
Closing Thoughts
In many ways, human beings are like components on the grid. Theyeach have their tolerances and optimal ranges of function.
If you interconnect them well, you can build a strong, reliable and self-healing system. Conversely, you can create sub-synchronousresonances by incorrectly interconnecting them: in this you createinefficient circuits that waste energy and time, and create heat, riskand cost.
Managing people is no more and no less than being an organizationalreliability coordinator.
