Upload
others
View
0
Download
0
Embed Size (px)
Citation preview
The Impact of Interface Developments on Operator Consistency
03.10.2013 2
Michael Hildebrandt [email protected]
Industrial Psychology Division Institute for Energy Technology /
OECD Halden Reactor Project, Norway
Example
03.10.2013 3
Traditional process display
03.10.2013 4
Information-rich display
03.10.2013 5
IRD for the Halden BWR LSD
03.10.2013 6
In operation since 2012, developed in cooperation with CR operators
03.10.2013 7
Institute for Energy Technology R&D for the nuclear, oil & gas industry
OECD Halden Reactor Project
03.10.2013 8
OECD Halden Reactor Project • International collaborative research project
• Fuels & Materials • Man, Technology, Organisation (MTO),120 staff
• Halden Agreement first signed in 1958 • Renewed every 3rd year • Current program period from 2012 to 2014
• HRP is jointly funded by its members: • 18 countries • > 100 nuclear organizations world wide
• Participant types • Utilities, Vendors, Licensing Authorities and R&D centers
HRP Members – Signatory Members USA US NRC - United States Nuclear Regulatory Commission Norway IFE - Institutt for energiteknikk Belgium SCK/CEN - Belgian Nuclear Research Centre Denmark Risø DTU Finland MITI - Finnish Ministry of Trade and Industry, VTT France EDF - Electricité de France Germany GRS - Gesellschaft für Anlagen- und Reaktorsicherheit Japan JNES - Japan Nuclear Energy Safety Organization Korea KAERI - Korean Atomic Energy Research Institute Spain CIEMAT - Spanish Centro de Investigaciones Energéticas, Medioambientales y Tecnológicas Sweden SSM - Swedish Radiation Safety Authority Switzerland ENSI - Swiss Federal Nuclear Safety Inspectorate UK NNL – National Nuclear Laboratory
03.10.2013 9
HBWR – Reactor hall MTO-lab Facilities
Facilities MTO-lab
HRA research areas in Halden
Improving HRA
practice
Halden HRA
database
Improving emergency and severe
accident operation
In this presentation • Operator performance, reliability and resilience • Halden interfaces
• Procedure flowchart • State-based alarm systems • Large-screen displays / IRD • Operator screens on iPads for ex-CR staff • Task-based displays • (Highly automated plants)
03.10.2013 13
Operator performance, reliability and resilience
Part 1
03.10.2013 14
Why worry about interfaces? • Information overload • Keyhole effect • Achieving / maintaining situation
awareness • Anticipation / prediction • Human reliability • Knowledge-based decisions, complexity • Resilience
03.10.2013 15
Why is it difficult to draw conclusions about the benefits of innovative interfaces on operator performance in PSA scenarios?
03.10.2013 16
(1) Many criteria for assessing innovative interfaces
• Performance on PSA scenarios / HEP effect • Workload / SA • Operator feedback • Usability • Expert evaluation • Early in the design process:
Assessment of “promisingness”
• It depends partly on your audience / customer
03.10.2013 17
(2) Project drivers • Improving performance on PSA scenarios is
not the direct driver in many of our projects • Beginning to change
• Focusing too much on PSA/HRA type of issues might not always lead to the best interface
03.10.2013 18
(3) Interfaces are part of the larger Emergency Operating System
• Difficult to isolate the effects of interfaces on performance, even in factorial experiments • Operators adapt, strategies change • Compensation, working harder
• Some measures not specific enough to the effect of interfaces
• Some measures too specific to interfaces, not taking into account the overall mission goals and work demands
03.10.2013 19
(4) Effect size • Effect of interface manipulation may not
be large enough to show statistical significance on HRA-type measures
03.10.2013 20
(5) Crew variability • Effect of interfaces can be drowned in
the variability between crews, and even between runs for the same crew
• Usually low sample size • Trade-offs in study design
03.10.2013 21
(6) Baseline performance • EOS performance (operators, procedures,
training, interfaces) is often very high • Even in relatively complex scenarios we see
many crews succeeding • Success on HFE criteria • Not necessarily good performance on other
measures
03.10.2013 22
(7) Baseline CR interface • Baseline control room EOS has proven to be
good enough for many scenarios • In our recent HRA studies, we have not seen
“HMI” show up as a main negative driver (mostly nominal)
03.10.2013 23
(8) Demand • Not clear if there is demand for HRA simulator
studies where factor such as HMI, staffing are systematically manipulated • Does an HRA analyst need / want this information?
• De facto: “Segregation” between HRA studies and interface design studies
• Attempt to change this with new HRA program • Improving emergency and severe accident operation • Test case: 2013/14 study on Resilient Procedure Use
03.10.2013 24
Measurement
Work on Integrated System Validation
03.10.2013 25
Objective/Scope 2012-14 Ø Performance criteria and acceptance criteria issue Ø Development of basis for Criterion-Referenced Approach (CRA) Ø Supplement or alternative to the benchmark approach
Ø Context issue – not sufficient NPP operation and safety relevance of many commonly recommended measures
Ø Causality issue Ø Performance effects of specific control room elements hard to
establish in integrated tests of high reliability systems (redundancy and diversity)
Ø Coherence Issue of performance measures Ø Measures reported in the literature often concentrate on one
performance dimension (for example Performance or Teamwork). Ø Coherent measures important for ISV evaluating the realistic “full”
operation.
03.10.2013 26
Framework Overview
03.10.2013 27
Cognitive Dimensions
03.10.2013 28
Situation Understanding
Plant state and Boundaries
Function / System
Component
Mission
Mission Analysis
Goals
Actions
Control and Verification
Mission Verification
Progress Monitoring
Execution control
Response Verification
Team Cognition
Leadership Decision Making Consultation Communication Back-up Workload Attention Resource Management Team Self-evaluation
Interfaces
Part 2
03.10.2013 29
resilient procedure use simulator study 2013/14
resilience is about… • the ability of a system to adapt to
unforeseen, challenging situations (including but not limited to Fukushima-scale events)
• the ability to mobilize additional resources (cognitive, physical, expertise, staff etc.) when a system approaches its margin of maneuver
• enabling smooth transfer of control
resilient procedure use requires balancing of
heads-down operation
heads-up operation and
How to support this? - Crew roles - Procedure support tools - Overview tools (e.g. LSD) - External support - Others?
Experimental design
03.10.2013 33
Run 1
Run 2
Run 3
Run 4
Shift technical advisor
✔ ✔
✗ ✗ Large-screen display, Procedure flowchart ✔
✗ ✔
✗
Swedish and US crews; four PSA scenarios with complications
new large screen display
status of important
safety systems
status of automatic systems, and other important alarms (e.g. RMS)
sub- cooling
mass balance
increasing/ decreasing
PRZ pressure/ level trends; alarm levels; trip points
new large screen display: primary
SG pressure / level trends
procedure flowchart for STA
Link to background material >
Key decision point >
Critical action steps > < Grouping of procedure steps
< Procedure /step transfers
< Procedure transfers (clickable)
< Short description of step links to full text
Step number >
Link to high-level summary >
CSF status, link to decision tree >
Foldout page, link to appendix >
Some flowcharts contain diagrams with live data >
*
03.10.2013 40
A State Based Alarm System for Halden Boiling Water Reactor (research reactor)
HWR-1065
Christer Nihlwing and Magnhild Kaarstad
General alarm system challenges
03.10.2013 41
Too many alarms Misleading alarms Secondary malfunctions difficult to detect Difficult to get an overview of the situation
NOT USEFUL DURING DISTURBANCES
Purpose and recommendations of the alarm system
The primary function of the alarm system (EEMUA 191, p. 4): • The purpose of an alarm
system is to direct the operator’s attention towards plant conditions requiring timely assessment or action.
03.10.2013 42
The alarm functions shall provide the following features (NUREG-0700/ IEC 62241): • Alert • Inform • Guide • Confirm
Number of alarms first 10 minutes of an upset
Consequence
More than 100 alarms Definitely excessive and very likely to lead to the operator abandoning the use of the system
20-100 Hard to cope with Less than 10 Should be manageable, but may be difficult if several alarms
require a complex operator response
SBA - A different approach to Alarm System Development • Several tests and efforts – difficult to find optimal solution • Basic alarm management - priorities and suppression - not
satisfactory • State Based Alarm (SBA) system developed in multilateral
cooperation • Based on a number of well-defined process states (19) • Continuously adjust itself to actual state • Shows only alarms relevant for the current state of the process • Add-on to an ordinary alarm system • Displayed in a separate list, separate alarm sound
03.10.2013 43
State based alarm presentation
03.10.2013 44
SBA performance - Alarm rate Turbine trip and condensate problem
03.10.2013 47
Ordinary alarm system: 257 State Based alarm system: 26
Turbine trip and condensate problem
FW problem and scram
FW regulation problem and loss of power
Alarm system SBA Ordinary SBA Ordinary SBA Ordinary Alarms 10 first minutes
23 259 33 297 21 424
Alarm rate first 10 minutes into disturbance
Operator performance with the SBA
03.10.2013 49
State based Ordinary Percentage detected 100% 62% Percentage corrected 100% 62%
Detection of secondary disturbance during ongoing disturbance
State based Ordinary Percentage detected 83% 67% Average time for detection
20 sec 33 sec
Average time to detect the return of the 400kV
Usability of SBA
System Usability Scale (SUS) (Brooke, 2001) • Ordinary alarm system: 75 • State based alarm system: 83 Operators´ subjective opinion regarding usability: • Faster control of the situation, easy to notice new relevant alarms • Manageable amount of alarms • Only alarms that need attention were presented • Liked the way alarms were presented in displays and in the list • The colours were clear, and informed about when to enter EOPs
03.10.2013 50
Conclusion • High usability score, and operators expressed that with
the SBA it was easy to take care of a disturbance. • Alarm rate of SBA can be within EEMUA
recommendations. • Operators seemed to detect and correct a secondary
disturbance faster and more often with the SBA. • Promising with regard to usefulness of an advanced
alarm system tool like the SBA. • Provide input to industry with regard to design and
development of advanced alarm system tools.
03.10.2013 51
Large Screen Displays – a Usability Study of Three Different
Designs
HWR-1025
Magnhild Kaarstad Stine Strand
Background • Part of an ongoing Nordic research cooperation since 2000 • Evaluate various large screen design (LSD) solutions
• Traditional LSD • Advanced LSD • Information Rich Design LSD
• Identify how they support operators in their control room tasks
• Operator feedback with regard to usability, use, usefulness and areas of improvements.
53
The evaluated Large screen displays
54
TRADITIONAL - Physical layout of process - Traditional diagrams - System lines indicate flow - Trends (3) - Traditional symbols - Bright colours - Frames around active alarms - Safety systems - Actuated safety system
3
The evaluated Large screen displays
ADVANCED - Physical location of process areas - No system lines - 12 mini-trends and combination trends - Traditional and advanced symbols, bar graphs - Bright colours - Alarms are small squares next to affected system - Safety systems aligned - Actuated safety systems - Frame around selected objects
3
The evaluated Large screen displays
INFORMATION RICH DESIGN (IRD) - Physical layout in areas - A few system lines - Pattern recognition by normalised trends - Traditional and advanced symbols - Dull colours - Alarms: small squares and alarm spot - Safety systems aligned - Actuated safety system
Operator feedback Traditional LSD
Positive • Worked well as a general
overview in normal situation • Simple, systematic display • Reactor and containment • Direction of flow • Symbols, mimics, lines • Background colour, contrast • Yellow alarm frames • Safety system actuation
5
Negative • Too few mini-trends • Unclear design of running
pumps • Design of safety system a bit
confusing • Too much in yellow alarm
colour in disturbance • Need more information to be
able to analyze a disturbance
Positive • Good overview in all situations • Appreciated mini-trends • Pump symbols • Intermediate position of valves • Background colour, contrast • Safety systems • Safety system actuation • Alarm information • Frames around active objects
6
Negative • Direction of flow not clear • Interpretation of some
diagrams • Information represented in bar
graphs • Control rod map • Alarm information • Larger disturbances
Operator feedback Advanced LSD
7
Positive • Good general overview • Normal operation, periodic
tests • Straight line: everything is
normal • Mimics and lines • Background colour • Reactor core • Alarm spot
Negative • Too few numbers • Information too small • Location of some valves • Lack of process lines • Safety system actuation • Mini-trends should have had
frames and scales • Contrast, information too dim • Difficult when too many alarms
Operator feedback IRD LSD
Usability evaluation System usability scale
• Traditional: 75 + Process understanding - Information availability
• Advanced: 60 + Deviation detection - Graphical interpretation • IRD: 59 + Deviation detection - Graphical interpretation
03.10.2013 8
Additional usability evaluations: • All LSDs particularly useful for:
• Colleague cooperation • Shared understanding
Summary of LSD evaluation • Three different LSD designs with
different philosophies and different strengths and weaknesses evaluated • Challenge: Alarm amount • Advantage: Common frame of reference
• The traditional was rated slightly higher than the two others
• None of the designs are completely adequate but all LSD concepts seem useful
9
03.10.2013 62
Information Rich Design (IRD) principles
Alf Ove Braseth
03.10.2013 63
IRD Large screen design philosophy • Provide a stable frame of reference; no pup-ups, new windows
• Shall be easy to spot deviations from normal operating conditions;
ideally before alarm state
• Prevent information overflow to the operators
• Shall complement the operator stations, no intervention or operation in the large screen.
03.10.2013 64
This color scheme leads to information overflow
Dullscreen Bright colors and contrast do not contribute to better design
03.10.2013 65
It is possible to create information rich design without going into “information overflow”
Edward Tufte
Pat. Pending
This is a Dullscreen example
03.10.2013 66
Dullscreen is not enough; enabling pattern recognition is also important
Jens Rasmussen (Risø); the SRK model: Skills :Riding a bike :Effortless, unconscious, high capacity Rules :Soccer :Fun, but can be demanding Knowledge :von Neumann problems :Demanding, conscious low capacity
Knowledge based
Skills based using pattern recognition
03.10.2013 67
What is normalization?
With normalization all six variable can be read as one unit Without normalization the six variables must be read and interpreted separately
03.10.2013 68
IRD component examples
Liquid levelPressureTemperature Flow Liquid levelFlowPressureTemp.
Alarm presentation
With trend information Without trend information
48
H2
130
H3
H4
IRD for the Halden BWR control room
03.10.2013 69
In operation since 2012
03.10.2013 70
03.10.2013 71
Further reading on IRD
JOINT POSTER ON:
HWR-936: Innovative HSI concepts - rationale and design solutions C. Nihlwing, J. Kvalem
HWR-996: Work practices and new technologies – iPad as information source
M. Kaarstad, S. Strand, C. Nihlwing, T. Karlsson
iPad How it Works and
How it can be Used in NPPs
Purpose iPad
73
Present continuously updated process information to staff located outside the main control room
Examples of information on the iPad • All process displays • Overview display • Alarms • Plant status circles • Plant automation • State based alarm list • Process flow charts • Logic flow charts • Procedures
• Updated on process situation before entering control room
• Directly prepared for discussions and decisions with crew members
• Can start immediate actions
How the iPad works
iPad client requests display to server and read it as jpg file from web server
Server selects, capture and put the display as jpg file on web server
iPad operator Process display area: A computer with two 30” displays
74 HWR-936: Innovative HSI concepts, rationale and design solutions
HSI working areas visualized on two 30” displays
75
Plant automation State based alarms
Plant status circles Overview Display
A server program handles the communication between iPad, HAMBO and web server
HWR-936: Innovative HSI concepts, rationale and design solutions
Process displays
iPad connection and dataflow
Client app on iPad in objective-C
1. Sending request for Statebased Alarmlist
2. Capture selected area, put it on the webserver cyclic
76 HWR-936: Innovative HSI concepts, rationale and design solutions
A server program handles the communication between iPad, HAMBO and web server
Area for capturing requested display
Web server
3. Reading from Web server
Controlling simulator from iPad
Start, stop, loading snapshot, set malfunction, operating objects in the HAMBO simulator can be done from a pop-up display with a picker wheel
77 HWR-936: Innovative HSI concepts, rationale and design solutions
Display navigation
78 HWR-936: Innovative HSI concepts, rationale and design solutions
Zooming overview display on iPad
79 HWR-936: Innovative HSI concepts, rationale and design solutions
Zoom
Work practices and new technologies- use of iPad for supervisor
• Background • Identify challenges and opportunities with introduction of new
technologies and computer-based control rooms • Possible changes:
• Communication • Operator tasks • Information needs • Transparency of team member activities
• Current focus on Shift supervisor (SS) • Needs information about the process and team member activities
80 HWR-996: Work practices and new technologies – iPad as information source for shift supervisor
Research goal Explore whether iPad can be used as an information source for the shift supervisor when he/she is located outside the
main control room
81 HWR-996: Work practices and new technologies – iPad as information source for shift supervisor
82
HAMMLAB Location of Crew
Experimenters gallery
”Office/ Meeting area” Location of Shift supervisor
The iPad Study
HWR-996: Work practices and new technologies – iPad as information source for shift supervisor
Data collection • Simulator: HAMMLAB PWR (RIPS) • Five crews from nuclear power plants participated in the
experiment • Three to six licensed operators in each crew
• 4 short test scenarios • In two scenarios, Shift supervisor was equipped
with iPad when located outside the control room • In two scenarios, Shift supervisor was NOT equipped
with iPad when located outside the control room
83 HWR-996: Work practices and new technologies – iPad as information source for shift supervisor
Scenarios & Data collection - overview
84
After EACH scenario Operator ratings Process overview Mental demand CR operator activities Teamwork Shared understanding
After ALL scenarios iPad usability ratings System Usability Scale iPad display usefulness Frustration Satisfaction Interview During ALL scenarios
Process Expert performance evaluation
HWR-996: Work practices and new technologies – iPad as information source for shift supervisor
Start 1 min
Supervisor located outside CR
4 min
Supervisor enters CR
10 min
Scenario end Disturbance
Preliminary findings
Examples of iPad benefits stated by operators • Overall, a very useful tool when out of CR (e.g. meetings) • Supervisor already updated on process situation when
entering the control room, directly prepared for discussions • Only a short turnover is needed
• Decreased communication needs • Control room operators less distracted from their tasks
• Misunderstandings less likely
• Can help avoid tunnel vision • Supervisor do not tie up CR personnel in their procedures
85 HWR-996: Work practices and new technologies – iPad as information source for shift supervisor
Preliminary findings Usefulness personnel groups • Shift supervisor/ Shift
manager • Operations manager • Plant manager • Upper level management • Chemistry personnel • Maintenance • Emergency personnel • Technical support centre • Field operators
86 HWR-996: Work practices and new technologies – iPad as information source for shift supervisor
iPad application areas • Training issues • Procedures • In meetings • Take pictures/videos of
components, send to CR staff for immediate evaluation
• Work orders • Day-to-day monitoring, avoiding
disturbances • Evacuation of control room • Easier for field operator to find
correct objects
Preliminary findings
87
3.8
5.0 4.8 4.8
3.8 4.4 4.4
4.8 4.4 4.6
0
1
2
3
4
5
• Very high usability ratings (High score = Good Usability) • All supervisors would like to use this iPad functionality in home plant
6.4 6.4 6.0 6.2 6.2 6.0
1
2
3
4
5
6
7
System Usability Scale (5-point) Overall Usability ratings (7-point)
88 HWR-996: Work practices and new technologies – iPad as information source for shift supervisor
Preliminary findings When iPad available in the first part of
the scenario, Shift Supervisor experienced Significantly less mental demand Significantly better process overview Significantly better overview of the CR
operator activities at the end of the scenario
When iPad available in the first part of the scenario, Shift Supervisor experienced that obtaining shared understanding of the process state was Significantly easier Significantly less mentally demanding Significantly less time consuming
Preliminary findings
89
When iPad available, Shift Supervisor needs significantly less time to be updated on the process situation when he/she enters the control room
HWR-996: Work practices and new technologies – iPad as information source for shift supervisor
Preliminary findings - Summary • iPad is a very useful tool for the shift supervisor when not located
in the control room • Operators very positive to the iPad, and imagine a broad range of
application areas and use for different personnel • Very high usability ratings • Significantly less mental demand, better process overview, better
overview of the CR operator activities, better shared understanding when iPad available
• Important considerations • Avoid that operators feel monitored by management • The iPad should NOT be used to control the process
• Technical solutions to avoid this are available
90 HWR-996: Work practices and new technologies – iPad as information source for shift supervisor
03.10.2013 91
Procedure and Process Integration Displays
Presentation of the Concept and the HAMMLAB User-test 2006
03.10.2013 92
Presentation of the concept
03.10.2013 93
Design criteria / Goal • Easy to use and in a way that is natural for process operators § Maintain the process overview when performing the procedures § Relevant information provided when performing procedure steps § Relevant feedback from process after step completion § Possible to mark steps with problems - easy to find later
• High transparency of operators work • Keep overview of performed procedures and procedure steps • Minimize navigation. The relevant information shall come to the operator • Minimize the risks for erroneous operations • Design tool: Possible for operators to implement/change procedures
without programming knowledge. • Easy to review • Possible to use for normal-, abnormal- and emergency situations
03.10.2013 94
Procedure and Process Integration Displays. HAMMLAB 2009
2009 the concept was adapted to one 30 inch screen.
03.10.2013 95
Procedure and Process Integration Displays. HAMMLAB 2013
Procedure Selection and Overview Display
Procedure Performance Display
Event-dependent Assistance display
Now the process displays has 30 inch format and by that two 30 inch screens needs for the concept.
03.10.2013 96
Procedure Selection and Overview Display
Reactor procedures Turbine procedures
Emergency Operating Procedures
03.10.2013 97
Push buttons to activate or observe procedures
Procedure Selection and Overview Display
03.10.2013 98
Overview of the selected procedure
Not active Actuated protection signal Active Interrupt without remains Interrupt with remains Completed Observation
Color codes for
procedure status
Displays with procedure steps
Procedure Selection and Overview Display
Reactor run-back
Containment isolation
03.10.2013 99
Close active procedure
Close procedure system
Alternative steps
Procedure Selection and Overview Display
03.10.2013 100
Procedure Performance Display Procedure text
Actual components are marked with a frame. The arrow shows that the valves shall be closed.
Navigation buttons
03.10.2013 101
When the procedure step is performed the operator press the ”Sign” button if the step is OK. If the operator wants to go back to the step later, he/she press the ”Rest” (remaining/postponed) button.
Procedure Performance Display
03.10.2013 102
The remaining/postponed list
”Rest” marked steps.
”Rest” marked steps in all procedures are visible in the Reactor operator or Turbine operator ”Rest” list.
It is possible to select a “Rest” marked step and jump directly to the procedure step.
03.10.2013 103
The 2005/2006 Usability Test
Ø Performed in the HAMBO simulator Ø 3 crews (RO, TO, SS) participated in 4 scenarios Ø The concept was implemented for all operators Ø The purpose of the test was to assess the usability of
the concept - not performance
03.10.2013 104
Results – the overall concept Ø Easy to learn Ø 4 hours of training sufficient Ø Relative comfortable in using the concept in simulator Ø Considered the concept as good and advantageous Ø Positive toward application of the concept for other tasks
• Complexity rated as low • Easy to use • Organization of functions good • Felt confident in using • Considered as safe to use
Usability scale
0
1
2
3
4
5
6
7
1 2 3 4 5 6 7
Response scale
Freq
uenc
y
Complexity (mean = 6.5)
Ease of use (mean = 5.9)
Function organisation (mean =6.4)
Confidence in use (mean = 6)
Safety in use (mean = 6)
03.10.2013 105
Results – display specific All 3 display types Ø Useful Ø Easy to understand Ø Low frustration Ø Did not miss information
in the displays • Clear results for
understanding and frustration for all display types
• Larger variation and lower mean values for information amount (mean value still above 5 on 7-point scale)
Understanding1: very difficult to underst. 7: very easy to underst.
0
2
4
6
8
1 2 3 4 5 6 7
Response scale
Fre
qu
ency
PO (mean = 6.5)
Proc.perf (mean = 6.5)
Eda (mean = 6.1)
Frustration1: very frystrated 7: not frustrated
0
2
4
6
8
10
1 2 3 4 5 6 7
Response scale
Fre
qu
ency
PO (mean = 6.7)
Proc.perf (mean = 6.7)
Eda (mean = 6.8)
03.10.2013 106
Results – display specific (cont.) Procedure Sel./Overview Display
• Good overview of procedures • Rest-list and
observation function appreciated • Intuitive, self-explanatory • Easy to select procedure,
easy to see procedure status • Easy to navigate
Procedure Performance Display • Good illustration of components • Easy, fast, safe • Few navigation problems • Disagreement in the need for
textboxes • Suggestions for improvement
e.g. color, text amount
Event-Dependent Assistance Display • Recognized benefits of a display relevant for the respective procedures • Provided quick overview (level, pressure, pumps, trends most useful) • Generally positive toward use of expert-help • Not all noticed and/or used the display (poor training + additional
information available, i.e. large overview display) • Suggestions for improvement, e.g. training in use of the display
03.10.2013 107
Process Expert (PE) rated performance
Ø PE qualitative ratings related to procedure handling and scenario handling judged to be good (no problems in handling procedures or scenarios)
Ø PE quantitative ratings relatively high
• Communication • Crew Efficiency • Goal achievement • Procedure handling
PE quantitative crew performance ratings (average)
5,17
5,75
6,255,83
0
1
2
3
4
5
6
7
Communication Crew efficiency Scenario goal attainment Procedure handling
03.10.2013 108
Comparison with paper-based proc. (“Mental” judgment) Ø Easier Ø Less time-consuming Ø Less mentally demanding
Ø Easier to recall deviations Ø Easier to recall previous
activities Ø General preference for the
computer based (60%) Ø Pitfall: process overview
Comparison between TBD and paper-based procedures
0
1
2
3
4
5
6
7
1 2 3 4 5 6 7
Response scale
Freq
uenc
y
diff icult/easy (mean = 5.6)
more/less timeconsuming(mean = 6.2)more/less mental demand(mean = 5.9)
more diff icult/easier to recalldev. (mean = 5.7)more diff icult/easier to recallact. (mean = 5.3)
03.10.2013 109
Summary and further plans Ø All measures provided results in the same direction
• The concept well received by the operators • The concept considered as valuable and useful
Ø Input to further refinements provided Ø The initiated work should be continued
• Currently incorporated in HAMMLAB Experiment set-up for Work Practices Experiment
Ø Plans (initiated):
• Continue with refinements of the current design • Further development of the Emergency Operating
Procedure (EOP led by the shift supervisor) • Expand the concept to other tasks
(e.g. start-up/shut-down procedure)
Summary • Halden developed and tested many different interfaces • Most have received positive feedback from crews • Performance benefits are evident in many cases
• But not all
• More difficult to show direct effects on PSA scenarios • Methodology, scenarios, sample size • Highly reliable EOS, low failure rates
• Many of these interfaces may have their strength in supporting knowledge-based emergency operation • Resilience
03.10.2013 110