Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
2008 © Copyright Incremental Systems Corporation
Using Simulators to Determine How an Expert Operator
Makes Decisions
EPCC Dublin, Ireland
June 14-17, 2009
by Robin Podmore
Incremental Systems, Issaquah, WA
By Understanding how an Expert Operator Makes Decisions we can:
• More accurately assess operator performance.• Develop better training courses• Develop and test better applications• Develop and test better human interfaces
Outline
• What is an Expert?• Simulator Configurations• Operator Decision Making• Summary
ExpertsWhat is an “Expert”?
“Someone widely recognized as a reliable source of technique or skill whose faculty for judging or deciding rightly, justly, or wisely is accorded authority and status by the public or their peers. An expert, more generally is a person with extensive knowledge or ability in a particular area of study.”
Source: Wikipedia, 2008
• Stage 1 Novice – Explicit rules susceptible to context and nuance• Stage 2 Advanced Beginner – Nuance and context begin to be
recognized and incorporated• Stage 3 Competence - Transition from calculated effort to intuitive
solutions• Stage 4 Proficiency – Scenarios are now being recognized as whole
parts, some analysis and conscious choice remains• Stage 5 Expertise – Complete contexts are recognized and
performance is fluid and unselfconscious.
Levels of Expertise Dreyfus and Dreyfus 1986
•Dreyfus, H. L., & Dreyfus, S. E. (1986). Mind over machine: The power of human intuition and expertise in the era of the computer. New York: Free Press.
Novices vs Experts
Read
Analyze
Explore
Plan
Implement
Verify
Novice Pattern Expert Pattern
PowerSimulator with EPRI OTS Background
• EPRI OTS Power System Model (PSM) developed by Arizona State University / Control Data with EPRI Sponsorship. Dr. Anjan Bose and Dr. Ralph Masiello.
• PACE Power Application Computing Environment- Developed by Incremental Systems and PowerData Corporation.
• PACE and EPRI OTS PSM integrated by IncSys and PowerData with EPRI Sponsorship using EPRI CIM. David Becker and Stephen Lee.
• ESB Ireland Project – First Custom Simulator with Hugh Jones and Dr. Charles Hansen of DSI.
• Emergency Operations with PowerSimulator tutorials developed with SOS International partly based on scenarios developed by EPRI and Quality Training.
Generic PowerSimulator Users• Requirement from 2003 US Northeast Blackout –
Each system operator shall have more than 32 hour of emergency training using simulators.– Midwest ISO (100 operators trained each year) – WECC (200)– PJM (500)– ERCOT (500)– SERC (500)– Florida Reliability Coord. (200)– California Electric Training Advisory Comm (500)– SOS International (600)
• Total - more than 3000 NERC certified operators trained each year.
2008 © Copyright Incremental Systems Corporation
EXTERNAL
CHSAN UXBRDG COPMANOR COPLEY STNTON OAKDLEAMUS
POOL
BEAVERASH
LOCHER
MOSES
JENKINHOMERVEXLEY
WYNHAM
ELLS
FARLIE
GRANGE
DAWSON
TANTONRICTER
KINCAID
DOYLE
200
3*200500600
1200
702*400600
1200
BAKER
CRWFRD
AIRPRT NESTLE
230 KV
138 KV
69 KV
33 KV
80
400
PALCO Map
Generic PowerSimulator PJM Room Setup
PowerSimulator can be used to recruit new engineers and operators
Custom PowerSimulator Users
• North American Users :– Alaskan Railbelt
Systems– Alberta– GCPUD– DCPUD– PG&E– Modesto– Turlock– Burbank– NIPSCO– SIPC– ISO-NE
Custom PowerSimulator Users
• International Projects:– Japan– ONS Brazil– Ireland– Iraq
PG&E CETAC Drills (2009)
2009 Copyright Incremental Systems Corporation
• 5 weeks• 500 total operators• All major utilities in California• Generic PSM for principles• Custom PG&E Model and Northern California Area for Daylong drills• Drill based on report of earthquake off the coast of N. California
Alberta Electric System Operator Restoration Drills
2009 Copyright Incremental Systems Corporation
flash 360
http://www.powersimulator.net/?m=psmCENisku.html
Iraq Ministry of Electricity National Dispatch Center
2009 Copyright Incremental Systems Corporation
• Located Al Ameen, Iraq• New and Incumbent Engineers• Generic PSM for principles• Custom Iraq Model• Remedial Action Schemes
Stations May Have a Combination of Custom Breaker and Generic Breaker Configurations
Travelling Care of US Government
Naturalistic Decision Making
• Typical industry training doesn’t match the way people think. Effective training for critical decision making must be informed by cognitive theories and models.
• Naturalistic Decision Making (NDM) is an area of cognitive science that describes how experienced individuals and teams perform in real-time, mission-critical environments.
• First NDM conference held in 1989.• Ninth NDM conference Covent Gardens June
1990
Example Applications of NDM
• Health Care – Surgeons, Nurses• Command and Control• Aviation• Business and Industrial Applications• Process Control• Material Requirements Planning• Nuclear Power Plant Emergencies• Anesthesiology• Naval Officers in Littoral Environments• Skilled Fighter Pilots
The Challenge of Decision MakingThe Challenge of Decision Making
High Stakes
DynamicSettings
Time Stress Uncertainty
Organizational Factors
Vague Goals
Multiple Players
High Task Loading
Critical Decision Making
NDM Thought Leaders
• Gary Klein – Recognition Primed Decision Model• Mica Endsley – Situation Awareness• Marvin Cohen – Consistent, complete, validated
“Story” with Quick Test• Sallie Gordon – Cognitive Task Analysis
Recognition Primed Decision Model Developed by Gary Klein
Situation
Cues
ActionScript
MentalSimulation
MentalModels
That affect the
That let youcreate
That activate
Generates
Which youAssess by
Using your
Patterns
NDM Applications to Power
• RPDM introduced to Power Industry by Doug Harrington of Team Formation
• RPDM extended by Robin Podmore of IncSys, Frank Greitzer of PNNL and Chuck Johansen and Pam Eye of SOS to reflect:– Need for 100% Error Free Decisions – Quick Test– Distinction between Long and Short Term Memory– Endsley’s Three Levels of Situation Awareness
‹29›
Operator Decision ModelSituation
Cues
That affect the
That let youcreate
That activate
Generates
Which youAssess by
Using your
Which youValidate by
Using your
TimeAvailable?N
MentalSimulation
MentalSimulation
ActionScript
Real World Short termmemory
Long termmemory
Story(Patterns)
MentalModels
MentalModels
Y
Completeness of Story
• Levels of Situation Awareness:– Level I – (What) Operator is monitoring correct
cues– Level II – (So What) Operator has developed a
Story using Mental Simulations and Mental Models to explain meaning of the cues.
– Level III – (Now What) Operator can project what will happen in the future using Mental Simulations and Mental Models.
Patterns
• Using cues to recognize key patterns is a critical step.
• Patterns must be recognized to determine appropriate control actions.
• More experience operators recognize a wider range of patterns and will more quickly detect when a new pattern has emerged.
Examples of Patterns
• System vulnerable to single contingency, line/transformer overloads about to cause cascading thermal outages
• Voltages in an area are weak • Neighboring system is drawing MVARs from our
system• System on verge of voltage collapse• Unit in own control area has tripped• Generation reserves are in-sufficient• Generation is bottlenecked• Load demand is bottlenecked• System has formed multiple islands• Discussion of other patterns ……
‹33›
Mental Models
• An expert power system operator has thousands of relevant mental models in LTM.
• Mental models range along a continuum from simple to complex.
• Required for but not limited to:– Physical, mechanical and electrical characterizations
of different system components and how they work together
– The abilities, sensibilities, limitations and motivations of the system operator’s extended team.
– Operating procedures and policies
PNNL – Human Factors Test LaboratoryIncludes PowerSimulator and Areva DTS
Summary – Current Applications• Operator Decision Model (ODM):
– Explains how an expert power system operator makes decisions.– Used to understand and document the Mental Models and Mental
Simulations used under real-time conditions.– Used to capture knowledge from expert system operators– Used to develop range of scenarios from simple to complex– Used to explain how an expert develops situation awareness.– Used to prepare for NERC System Operator Certification– Used to define gaps in training curricula.– Currently used for over 3000 NERC system operators to maintain
Emergency Training Hours.
Summary Future Applications• The ODM can also be used to:
– Develop better applications– Develop better user interfaces
• and better user interfaces.• The ODM can be used to interview and determine ideal
personality profiles for expert system operators.• On-line tutorials and simulation exercises using ODM can
be used to prepare for NERC certification tests.