Embed Size (px)
Citation preview
Proactive safety management in health care : towards abroader view of risk analysis, error recovery, and safetycultureCitation for published version (APA):Habraken, M. M. P. (2010). Proactive safety management in health care : towards a broader view of riskanalysis, error recovery, and safety culture. Technische Universiteit Eindhoven.https://doi.org/10.6100/IR657709
DOI:10.6100/IR657709
Document status and date:Published: 01/01/2010
Document Version:Publisher’s PDF, also known as Version of Record (includes final page, issue and volume numbers)
Please check the document version of this publication:
• A submitted manuscript is the version of the article upon submission and before peer-review. There can beimportant differences between the submitted version and the official published version of record. Peopleinterested in the research are advised to contact the author for the final version of the publication, or visit theDOI to the publisher's website.• The final author version and the galley proof are versions of the publication after peer review.• The final published version features the final layout of the paper including the volume, issue and pagenumbers.Link to publication
General rightsCopyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright ownersand it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.
• Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal.
If the publication is distributed under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license above, pleasefollow below link for the End User Agreement:www.tue.nl/taverne
Take down policyIf you believe that this document breaches copyright please contact us at:[email protected] details and we will investigate your claim.
Download date: 25. Jul. 2020
Proactive Safety Management in Health Care:
Towards a Broader View of Risk Analysis, Error Recovery, and Safety Culture
Marieke M.P. Kessels - Habraken
Proactive safety management in health care: Towards a broader view of risk analysis, error
recovery, and safety culture /
by Marieke M.P. Kessels - Habraken
– Eindhoven: Technische Universiteit Eindhoven, 2009. – Proefschrift. –
A catalogue record is avalaible from the Eindhoven University of Technology Library
ISBN 978-90-386-2095-4
NUR 801
Keywords: Patient safety / Safety Management / Prospective risk analysis / Incident reporting
/ Retrospective incident analysis / Error recovery / Near miss / Safety culture
Printed by Universiteitsdrukkerij Technische Universiteit Eindhoven
Cover design: Oranje Vormgevers
© 2009, Marieke M.P. Kessels - Habraken, Helmond
All rights reserved. No part of this publication may be reproduced or utilised in any form or by any means,
electronic or mechanical, including photocopying, recording, or by any information storage and retrieval system,
without permission in writing from the author.
Proactive Safety Management in Health Care:
Towards a Broader View of Risk Analysis, Error Recovery, and Safety Culture
PROEFSCHRIFT
ter verkrijging van de graad van doctor aan de Technische Universiteit Eindhoven,
op gezag van de rector magnificus, prof.dr.ir. C.J. van Duijn, voor een commissie
aangewezen door het College voor Promoties in het openbaar te verdedigen
op woensdag 20 januari 2010 om 16.00 uur
door
Marieke Maria Petronella Kessels - Habraken
geboren te Mierlo
Dit proefschrift is goedgekeurd door de promotoren:
prof.dr. J. de Jonge
en
prof.dr. C.G. Rutte
Copromotor:
prof.dr. T.W. van der Schaaf
v
Acknowledgements
This is it. With this dissertation, my PhD research is completed. Before I started this project, I
weighed up the pros and cons of doing a PhD. I did see the advantages of acquiring and
enhancing academic skills, conducting field research, and writing a book. Nevertheless, I also
realised that it would be a challenge to complete the project successfully and in time.
Fortunately, the potential risk of an unsuccessful project was minimised thanks to a good
many people.
First, I would like to thank my promotors Jan de Jonge and Christel Rutte, and my
copromotor Tjerk van der Schaaf. Their knowledge, constant support, and confidence has
enabled me to design, conduct, and complete this PhD research. Our co-authorship was a
very valuable and pleasant experience. I have learned a lot from you. Thank you.
I owe great gratitude to Alysis Zorggroep, Infoland, and MERS International for giving me
the opportunity to do this research. In particular, I thank Gert de Bey and Gerard Gerritsen
from Alysis Zorggroep, Jan Stege, Frank Stege and Piet Baudoin from Infoland, and Rinus
Gelijns and Annemarie Eras from MERS International for their willingness to collaborate.
The combination of theory, practice, and software solutions appeared to be a big success.
Moreover, I would like to thank Infoland for the belief in my potential, as demonstrated by
vi
the decision to hire me. I feel very honoured to work for such an innovative and ambitious
organisation. Thank you.
Further, I am grateful to all employees and managers in Alysis Zorggroep who, despite their
pressure of work, participated in the studies, which demonstrates that patient safety is
absolutely a top priority in Alysis Zorggroep. A special thanks to Gerard Gerritsen, Paulien
Ogink, Katja Kerkvliet, Gonda Nienhuis, Hanneke Stoffels, Adriaan van Sorge, the members
from the ―Van MIP naar VIM‖ project group, and my esteemed colleagues from the quality
department. I also thank Ian Leistikow and Petra Reijnders-Thijssen for our collaboration and
co-authorship, Carolien Plaisier and Dorien Zwart for their help in data collection, and all
people of University Medical Centre Utrecht and MAASTRO clinic who participated in the
HFMEA™ analyses. Thanks to the Netherlands Health Care Inspectorate for introducing me
to the field of patient safety and providing access to their incidents database and case files. In
addition, I would like to thank the undergraduate students who assisted me in data collection
and analysis: Jeroen Rutteman, Hanneke Wijers, Frank Rinkens, Onno Kuip, and Zeno
Korsmit. Thank you.
I would like to thank my co-workers (and former co-workers) at the Human Performance
Management Group of Eindhoven University of Technology for their support and friendship.
In particular, I owe gratitude to Anniek van Bemmelen for correcting my manuscripts and for
taking care of all kinds of administrative matters. Further, I am grateful to Ad Kleingeld for
his methodological advices. Thanks to Eric van der Geer for his support and advice,
especially during the final months of my project. Last but not least, I would like to thank my
room-mate, Marieke van den Tooren. I think we perfectly Matched. Thank you.
During my PhD research, I sometimes really needed to take my mind of it. In that case,
exercising or going out appeared to be the best medicine. My Borrel friends and korfball
teammates always helped me to unbend my mind. Your friendship means a lot to me. Thank
you.
In addition, I would like to thank my family and in-laws. Your moral support and belief in
my capacity made me realise that I would make it. A special thanks to my parents and sister. I
am proud of you. Thank you.
vii
Finally, a thousand thanks and lots of love to Maikel. During the last few years, you
mitigated all stressful moments, just by loving me. So Incredible. Thank you.
Marieke
viii
ix
Contents
Chapter 1 Introduction 1
1.1 Definitions 2
1.2 Proactive Safety Management 3
1.3 Risk Analysis 6
1.4 Error Recovery 7
1.5 Safety Culture 8
1.6 Dissertation Outline 8
Chapter 2 Prospective Risk Analysis of Health Care Processes:
A Systematic Evaluation of the Use of HFMEA™
in Dutch Health Care 11
2.1 Methods 13
2.2 Results 19
2.3 Discussion 29
Chapter 3 Integration of Prospective and Retrospective Methods
for Risk Analysis in Hospitals 33
3.1 Methods 36
3.2 Results 38
3.3 Discussion 42
x
Chapter 4 Prospective Risk Analysis Prior to Retrospective Incident Reporting
and Analysis as a Means to Enhance Incident Reporting Behaviour:
A Quasi-experimental Field Study 45
4.1 Methods 49
4.2 Results 53
4.3 Discussion 58
Chapter 5 Defining Near Misses:
Towards a Sharpened Definition Based on Empirical Data 63
5.1 Methods 67
5.2 Results 72
5.3 Discussion 76
Chapter 6 If Only….: Failed, Missed and Absent Error Recovery Opportunities
in Medication Errors 81
6.1 Methods 85
6.2 Results 87
6.3 Discussion 92
Chapter 7 Trends in Safety Culture in Three Dutch Hospitals:
A Longitudinal Panel Survey 95
7.1 Methods 99
7.2 Results 105
7.3 Discussion 112
Chapter 8 General Discussion 119
8.1 Methodological Considerations 121
8.2 Theoretical Implications 123
8.3 Practical Implications 129
8.4 Future Research 132
8.5 Concluding Remarks 134
References 135
Appendix: Safety Culture Dimensions and Corresponding Survey Items 151
Summary 155
Samenvatting 159
List of Publications 163
About the Author 165
1
Chapter 1
Introduction
Risk is part of everyday life. People face risks in their working environments, private lives,
and leisure activities. Obviously, particular industries and environments or certain kinds of
sports and hobbies are considered more hazardous than others. Risk management is core
business for managers and operators in chemical plants and nuclear power stations. People
think about risks before setting up a company, taking out a mortgage, or making a parachute
jump. However, do they also consider risks prior to a hospital visit or when consulting a
family doctor? Do they realise that they run the risk of being harmed by medical errors?
Errors are made in health care organisations, just like in any other organisation.
However, in contrast with most other industries, in health care human lives are at risk rather
than products or processes. Unfortunately, fatal medical errors happen frequently. In fact,
fewer people die from airplane crashes, road traffic accidents, or natural disasters, such as
earthquakes and tsunamis, than from medical errors in acute care (Runciman, Merry, &
Walton, 2007). In the United States for instance, annually tens of thousands of people die in
hospitals due to medical errors (Kohn, Corrigan, & Donaldson, 2000). Record review showed
that medical errors cause about 1,700 deaths in Dutch hospitals every year (Wagner & De
Bruijne, 2007). A systematic review revealed that nearly 1 out of 10 patients (9.2%)
experiences an unintended injury or complication during hospital admission (De Vries,
Ramrattan, Smorenburg, Gouma, & Boermeester, 2008). In their evaluation of the frequency
Chapter 1
2
and nature of medical errors in primary care, Sandars and Esmail (2003) found an incidence
of 5 to 80 errors per 100,000 consultations. Leape (1994) even argued that in the United
States the annual number of deaths caused by health care itself, instead of the injury or
disease (i.e. iatrogenic harm), equates to three jumbo jet crashes every two days.
Errors in health care are not always related to complex treatments or sophisticated
surgeries. Instead, many medical errors are related to routine acts and caused by a catalogue
of failures. Think about Wayne Jowett, an 18 years old cancer patient, who died after a
cytotoxic drug mistakenly had been injected into his spine (Dyer, 2001). Or consider the 18-
month old Josie King, who died because of dehydration. A lack of communication between
doctors and nurses and their failure to respond to the parents‘ concerns caused the little girl‘s
death (King, 2006).
Medical errors can result in various negative consequences. First, patients, their
relatives, and even health care employees themselves can be harmed physically, mentally,
and emotionally. Often, additional care and extended length of stay are necessary to mitigate
those adverse effects. Diminished satisfaction of patients and their relatives could damage
health care organisations‘ reputations and even result in liability claims. Moreover, health
care employees themselves could get frustrated, which might negatively affect their
performance. Further, medical errors can cause damage to medical equipment, devices, and
buildings. In the end, those negative consequences could all result in financial losses for
health care organisations, households, and even for society in terms of decreased productivity
and diminished population health status (Cohen, 2001; Kohn et al., 2000). For adverse drug
events only, Bates et al. (1997) and Classen, Pestotnik, Evans, Lloyd, & Burke (1997)
calculated an additional length of hospital stay of 2.2 and 1.9 days on average, which resulted
in increased costs of at least $2595 (i.e. €1851) and $2262 (i.e. €1613) per incident,
respectively. Total costs would probably even exceed those figures because costs of
malpractice claims were not taken into account in those studies. In the United States, it was
estimated that the total amount of additional costs caused by medical errors that resulted in
harm would be $37.6 (i.e. €26.8) billion, annually (Thomas et al., 1999). Before elaborating
on the efforts that health care organisations could make to reduce the number of medical
errors, we first introduce some important terms and definitions.
1.1 Definitions
In theory and practice, multiple terms are used with regard to patient safety (Runciman et al.,
2009; Yu, Nation, & Dooley, 2005). This section presents three important terms, together
Introduction
3
with their working definitions as used in this dissertation: incident, accident, and near miss.
An incident can be defined as ―an event where a failure or combination of failures has
occurred with the potential to lead to negative … consequences, irrespective of whether in the
end these negative consequences became manifest, at least to some extent, or were avoided
completely‖ (Kanse, 2004, p. 193). In this dissertation, the terms incident and (medical) error
are used interchangeably. The foregoing definition of incidents encompasses both accidents
and near misses. So-called accidents1 did have negative consequences for patients, whereas in
case of so-called near misses adverse consequences were prevented (Kanse, 2004; Van der
Schaaf, 1992).
1.2 Proactive Safety Management
The large number of medical errors and the harm, costs, and other negative consequences
involved express the need for effective safety management. However, in spite of media
coverage, which largely resulted from the Institute of Medicine report ―To Err Is Human”
(Kohn et al., 2000) progress in improving patient safety appears to be slow (Coiera &
Braithwaite, 2009; Leape & Berwick, 2005; Patel & Cohen, 2008). This might be related to
the facts that, traditionally, medical culture considers errors unavoidable and an evident
feature of medical care, and that particularly doctors tend to normalise errors (Quick, 2006;
Waring, 2005). Moreover, until recently health care organisations in particular used band-aid
approaches to deal with medical errors after they occurred (Karsh, Escoto, Beasley, &
Holden, 2006; Pronovost et al., 2003). An example of such a reactive approach towards
safety management is the Radboud hospital affair. In the Dutch Radboud hospital in 2006, the
cardiac surgery unit was closed for several months due to unusually high mortality and
morbidity rates (Netherlands Health Care Inspectorate, 2006). Early warning signals had been
ignored, and not until a whistle-blower openly brought the quality and safety of the cardiac
surgeries up for discussion, did the Netherlands Health Care Inspectorate investigate the
problems.
Apparently, such a reactive safety management approach is not sufficient. The vision
of safety management efforts in health care should be zero patient harm and therefore, the
—————————————
1In USA-based patient safety literature, incidents that resulted in patient harm (i.e. accidents) are commonly
referred to as adverse events. However, we decided to use the term accident to be more consistent with safety
literature in other industries.
Chapter 1
4
objective is minimal patient harm (Battles & Lilford, 2003). In line with this objective, a
proactive approach towards safety management is essential. It is necessary to foresee risks,
and to eliminate or at least minimise them before harm is done (Battles, Dixon, Borotkanics,
Rabin-Fastman, & Kaplan, 2006; Hollnagel, 2008; Rath, 2008). The question arising from
this objective, which is central to the present dissertation, is:
How could health care organisations apply proactive safety management to prevent
patient harm and minimise costs of poor safety?
This dissertation proposes that proactive safety management could be implemented
via three distinct but complementary approaches (see Figure 1.1).
Proactive
Safety
Management
Organisational
Context:
Safety Culture
Methods:
Risk Analysis
Data:
Error Recovery
Figure 1.1: Three approaches towards proactive safety management.
First, health care organisations can use more prospective methods to identify and
assess risks before errors may occur (Hollnagel, 2008). Health care organisations can use
prospective and/or retrospective methods to identify risks. Prospective methods aim to
foresee risks, while retrospective methods attempt to derive lessons from medical errors that
have actually happened. In a prospective analysis, multiple health care employees together
determine and assess potential risks and propose actions to eliminate or reduce those risks.
Prospectively developed failure scenarios can be used to reveal and solve latent problems that
could some day have resulted in incidents with severe consequences for patients (Reason,
Introduction
5
2004). This in contrast with retrospective methods, which are used to identify and analyse
medical errors that have actually occurred. Retrospective methods are applied to facilitate
learning, and measures are taken to prevent recurrence of the errors. Logically, prospective
methods are most appropriate for proactive safety management because they concentrate on
potential risks and enable health care organisations to come into action before harm is done.
Second, certain data can be used to improve patient safety in a more proactive way.
Near misses, which by definition did not result in patient harm, can yield information about
error recovery, that is, the way errors are detected and corrected. This information could be
used to promote effective error recovery strategies, which is important since errors cannot be
completely prevented (Aspden, Corrigan, Wolcott, & Erickson, 2004; Hollnagel, 2008;
Kanse, Van der Schaaf, Vrijland, & Van Mierlo, 2006). Moreover, reporting and analysis of
near misses offers opportunities to eliminate risks before they may result in actual accidents
with adverse consequences for patients (Aspden et al., 2004; Barach & Small, 2000; Kaplan
& Rabin Fastman, 2003; Van der Schaaf & Wright, 2005).
Third, besides those analytic approaches, advances in organisational context (i.e.
safety culture) can be important for proactive safety management in health care (Aspden et
al., 2004; Hudson, 2001; Nieva & Sorra, 2003; Pronovost & Sexton, 2005). Safety culture
can be defined as ―the product of individual and group values, attitudes, perceptions,
competencies, and patterns of behaviour that determine the commitment to, and the style and
proficiency of, an organisation‘s health and safety management.‖ (Advisory Committee on
the Safety of Nuclear Installations, 1993, p. 23). In an advanced safety culture, health care
employees at all levels constantly consider safety a top priority (Hale, 2003; Nieva & Sorra,
2003; Pronovost et al., 2003) and aim to minimise patient harm. A positive safety culture in
which safety is an important goal, could enhance safety behaviour and performance (Aspden
et al., 2004; Clarke, 2006b; Flin, 2007; Flin, Burns, Mearns, Yule, & Robertson, 2006; Neal,
Griffin, & Hart, 2000). Besides, safety culture could be considered ―the motor that makes the
structure of the SMS [safety management system] work‖ (Hale, 2003, p. 194).
The three approaches (i.e. risk analysis, error recovery, and safety culture) could
enable health care organisations to improve patient safety more proactively (see Figure 1.1).
Consider, for instance, the implementation of a bar coding system in a health care
organisation. A bar coding system could assist nurses in making sure that the right drug is
administered to the right patient, at the right dose, and at the right time (Bates, 2000;
Hampton, 2004). However, such a system might also induce problems, like degraded
coordination (Patterson, Cook, & Render, 2002). In a paper-based system, doctors and nurses
Chapter 1
6
have quick access to current medication orders at the patient‘s bedside and discuss those. In
an electronic bar coding system, such as the one evaluated by Patterson et al., it is often
impossible to gain a clear and instant view of the pending medication orders, which might,
for instance, prevent doctors and nurses from recognising errors. True proactive safety
management would imply that the health care organisation conducts a prospective analysis
prior to the implementation of the bar coding system in order to identify and reduce possible
risks and to raise risk awareness among the people involved. Afterwards, near miss reporting
and analysis could facilitate learning. Through a dual approach of error reduction and error
recovery promotion strategies, patient harm could be averted, whereby safety-related costs
could be decreased. Unfortunately, such proactive safety management in health care is still in
its infancy.
1.3 Risk Analysis
Although both prospective and retrospective methods can be used to improve patient safety,
health care so far has particularly used retrospective methods, such as incident reporting
(Karsh et al., 2006; Pronovost et al., 2003). However, since retrospective methods focus on
actual errors, which might already have caused harm to patients, this focus seems not to be
adequate enough. It is also important to foresee risks by identifying and assessing risks before
incidents may occur (Battles et al., 2006; Hollnagel, 2008; Rath, 2008). Unfortunately,
prospective methods, such as Healthcare Failure Mode and Effect Analysis (HFMEA™),
have been applied only limitedly in health care. In an HFMEA™ analysis, a multidisciplinary
team identifies and prioritises potential risks in a selected health care process, and
subsequently identifies actions to eliminate or reduce those risks (DeRosier, Stalhandske,
Bagian, & Nudell, 2002). Though several studies report about the application and evaluation
of such prospective methods in health care (e.g., Jeon, Hyland, Burns, & Momtahan, 2007;
Kunac & Reith, 2005; Wetterneck et al., 2006), hardly any systematic research has yet been
conducted that evaluates and discusses the benefits and drawbacks of the use of those
methods in health care.
Actually, complete and reliable prospective analyses would anticipate all risks and
consequently render retrospective analyses superfluous (Senders, 2004). However, both
prospective and retrospective methods are subject to biases, such as inaccurate risk
assessment, incomplete data, and hindsight and recall bias. Therefore, triangulation of those
methods seems to be necessary to obtain a more complete and reliable overview of patient-
safety-related risks (Battles & Lilford, 2003; Herzer, Mark, Michelson, Saletnik, &
Introduction
7
Lundquist, 2008; Runciman et al., 2006; Senders, 2004). In addition, integration of
prospective and retrospective methods enables direct comparison of the results of the
analyses. This might limit the additional resources required and support health care
management in making sense of patient safety data and setting priorities for appropriate
interventions (Battles et al., 2006; Hogan et al., 2008). Although several studies have
explored possibilities for the integration of prospective and retrospective methods (e.g.,
Trucco & Cavallin, 2006; Van der Hoeff, 2003; Wetterneck et al., 2006), until now no
research has concentrated on the perceived usefulness of this integration. Moreover, due to
limited resources, like available funds or staff, it could be impossible for health care
organisations to implement prospective and retrospective methods simultaneously (Akins &
Cole, 2005; Devers, Pham, & Liu, 2004). However, it still remains to be explored which
order of implementation is most preferable (Hale, 2003).
1.4 Error Recovery
Although safety can be defined as the absence of risk (Hollnagel, 2008), errors will always
occur. Therefore, the ultimate objective of safety management in health care should not be
zero risk or zero errors; instead, one should strive for zero or at least minimal patient harm
(Battles & Lilford, 2003). Hence, health care organisations can focus on error reduction as
well as error recovery promotion (Aspden et al., 2004; Hollnagel, 2008; Kanse et al., 2006).
While error reduction strategies intervene between contributing factors and the error, error
recovery promotion strategies intervene between the error and negative consequences
(Kontogiannis, 1997). Near misses enable health care organisations to acquire insight into
error recovery. Moreover, since the causal pattern of near misses and accidents is likely to be
similar, analysis of near misses might prevent actual accidents from occurring, thereby
proactively averting patient harm (Aspden et al., 2004; Barach & Small, 2000; Kaplan &
Rabin Fastman, 2003; Van der Schaaf & Wright, 2005). Nevertheless, both in theory and in
practice, there is a lack of a clear and consistent definition of near misses (Affonso & Jeffs,
2004; Aspden et al., 2004; Yu et al., 2005). This causes underreporting of near misses and
analysis problems (Affonso & Jeffs, 2004; Etchegaray, Thomas, Geraci, Simmons, & Martin,
2005; Tamuz, Thomas, & Franchois, 2004). Because of this, health care has still failed to
make the most of near misses and information about error recovery (Aspden et al., 2004;
Parnes et al., 2007; Patel & Cohen, 2008).
Chapter 1
8
1.5 Safety Culture
By triangulation of prospective and retrospective methods and by obtaining information
about error recovery, health care organisations can make progress on the analytical pathway
to improve patient safety. In addition, health care organisations can advance on the cultural
pathway (Aspden et al., 2004; Hudson, 2001; Nieva & Sorra, 2003; Pronovost & Sexton,
2005). Hudson (2003) and Reason (1998) stated that in an advanced safety culture, health
care employees and management are (1) informed about quality, safety, and risks, (2) trust
each other; that is, they openly speak about errors without being blamed or punished, (3) are
adaptable to change through learning, and (4) worry about safety, that is, they are
preoccupied with risks. Advances in safety culture could change health care employees‘
behaviour, thereby indirectly reducing the number of medical errors (Aspden et al., 2004;
Clarke, 2006b; Flin, 2007; Flin et al., 2006; Neal et al., 2000). In a positive safety culture,
health care employees could probably better observe safety regulations and procedures (Neal
et al., 2000), which could reduce the number of errors that happen. Further, a safety culture of
alertness and vigilance might enhance error recognition and correction (Kontogiannis &
Malakis, 2009), as a result of which incidents could be prevented from developing into
accidents with actual patient harm.
The two other approaches towards proactive safety management (i.e. risk analysis and
error recovery) and safety culture are interrelated. On the one hand, a safety culture in which
health care employees are aware of risks and openly discuss errors is essential for prospective
and retrospective methods to be applied successfully (Cannon & Edmondson, 2005; Hudson,
2001; Nieva & Sorra, 2003). On the other hand, conducting a prospective analysis or
introducing an incident reporting and analysis system that facilitates learning might, in turn,
positively influence safety culture (Aspden et al., 2004; Carroll, Rudolph, & Hatakenaka,
2002; Kaplan & Barach, 2002; Pronovost et al., 2007). In line with the latter assumption,
Nieva and Sorra (2003) claimed that safety culture change could be viewed as an indirect
outcome measure of patient safety interventions.
1.6 Dissertation Outline
This dissertation deals with three approaches towards proactive safety management as
depicted in Figure 1.1: risk analysis (methods), error recovery (data), and safety culture
(organisational context). Six studies were carried out, which are presented in Chapters 2 to 7
(see Figure 1.2). Together, those studies address important gaps in current knowledge
Introduction
9
regarding safety management. Although the six studies are related, each chapter can be read
independently from the others.
Proactive
Safety
Management
Organisational
Context:
Safety CultureCh. 4 / 5 / 7
Methods:
Risk AnalysisCh. 2 / 3 / 4 / 7
Data:
Error RecoveryCh. 5 / 6
Figure 1.2: Dissertation outline: Overview of chapters.
Chapters 2 to 4 mainly concentrate on appropriate methods for proactive safety
management. More specifically, Chapter 2 presents a qualitative field study that evaluates the
application of a prospective risk analysis method (HFMEA™) in Dutch health care by means
of user feedback. The qualitative field study presented in Chapter 3 addresses any biases
underlying prospective and retrospective methods and deals with the triangulation and
integration of both methods on two units of a Dutch general hospital. The quasi-experimental
field study reported in Chapter 4 concentrates on the relation between the order of
implementation of prospective and retrospective methods and incident reporting behaviour on
12 units of two Dutch general hospitals.
The qualitative field studies presented in Chapters 5 and 6 both focus on proactive
data, that is, information about error recovery. In Chapter 5, empirical data from four units of
two Dutch general hospitals are used to sharpen the definition of near misses in order to
stimulate their reporting and to gather information about effective error recovery strategies.
In Chapter 6, accidents are used as a supplementary source of information about error
recovery.
Though organisational context also comes up in Chapters 2 to 6, Chapter 7 lists some
important findings regarding safety culture, and presents a longitudinal panel survey on
Chapter 1
10
trends in safety culture in three Dutch hospitals after an extensive safety management
programme had been implemented. Chapter 7 furthermore explores which safety culture
dimensions predict incident reporting behaviour.
The studies presented in Chapters 3, 4, 5, and 7 were all conducted in the same health
care foundation, which comprises three hospitals: a teaching hospital that offers basic and
specialised care (750 beds), a hospital that offers basic care (250 beds), and a hospital for
outpatient treatment (50 beds). For each of the studies described in Chapters 3 to 5, we
selected different units from those hospitals; in the panel survey on safety culture (Chapter 7),
we included all units from the three hospitals (see Figure 1.3).
In Chapter 8, the main findings of the six studies are summarised and reflected upon,
the strengths and limitations inclusive. Theoretical and practical implications are discussed,
and suggestions for future research are put forward.
Health care foundation
Hospital A:
Teaching hospital
Basic and specialised care
Hospital B:
Basic care
Hospital C:
Outpatient treatment
Hospital units Hospital units Hospital units
Ch. 4
Ch. 5
Ch. 3
. . . . . . . . .Ch. 7 Ch. 7
Figure 1.3: Sub samples to be used in Chapters 3, 4, 5, and 7. The health care foundation
comprises three hospitals, which, in turn, each consist of multiple units. Chapters 3, 4, and 5
each concerned different units; Chapter 7 concerned all units from the three hospitals.
11
Chapter 2
Prospective Risk Analysis of Health Care Processes:
A Systematic Evaluation of the Use of HFMEA™
in Dutch Health Care*
This chapter evaluates the use of the prospective risk analysis method Healthcare
Failure Mode and Effect Analysis (HFMEA™) in Dutch health care. Thirteen
HFMEA™ analyses of various health care processes were carried out. User feedback
uncovered perceived benefits and drawbacks regarding HFMEA™ and showed there
is room for improvement. Several suggestions are put forward to improve the
perceived utility and acceptance of this prospective method.
Safety management in health care is still in its infancy compared to other sectors, such as the
chemical and nuclear industries, and civil aviation. Health care organisations so far have
particularly concentrated on retrospective incident reporting and analysis, while prospective
risk analysis has been applied less frequently. However, when one considers the objective of
safety management, this retrospective focus does not seem to be sufficient enough. According
to the definition of patient safety, the objective of safety management should be to prevent
—————————————
*This chapter is largely based on: Habraken, M. M. P., Van der Schaaf, T. W., Leistikow, I. P., & Reijnders-
Thijssen, P. M. J. (2009). Prospective risk analysis of health care processes: A systematic evaluation of the use
of HFMEA™ in Dutch health care. Ergonomics, 52, 809-819.
This study was funded by ZonMw – the Netherlands Organisation for Health Research and Development.
Chapter 2
12
patient harm (Battles & Lilford, 2003). Hence, one should foresee risks in health care
processes instead of reactively taking action after incidents have occurred (Battles et al.,
2006; Hollnagel, 2008; Rath, 2008).
Failure Mode and Effect Analysis (FMEA) is a systematic method for prospective
analysis that can be used to identify and assess potential failure modes in products, processes,
and systems. FMEA has a long history in the technical design of work settings. In subsequent
applications, the human and organisational components of work settings have also been taken
into account. FMEA is mainly used in manufacturing. However, it has also been applied in
health care to improve patient safety in processes such as drug administration and blood
transfusion (e.g., Adachi & Lodolce, 2005; Apkon, Leonard, Probst, DeLizio, & Vitale, 2004;
Burgmeier, 2002; Day, Dalto, Fox, Allen, & Ilstrup, 2007; Dhillon, 2003; Jeon et al., 2007;
Kunac & Reith, 2005; Paparella, 2007). In 2002, Healthcare Failure Mode and Effect
Analysis (HFMEA™) was developed by the United States Department of Veterans Affairs'
National Center for Patient Safety (NCPS) by combining concepts, components, and
definitions from Failure Mode and Effect Analysis (FMEA), Hazard Analysis and Critical
Control Points (HACCP), and Root Cause Analysis (RCA) (DeRosier et al., 2002). This
method was designed to enable health care organisations to evaluate and improve health care
processes before actual incidents may occur.
In both FMEA and HFMEA™, a multidisciplinary team graphically describes a
selected process and subsequently identifies and assesses all potential failure modes. In
FMEA, the team calculates a so-called risk priority number for each identified failure mode
by multiplying its potential severity, frequency, and detectability. In HFMEA however, each
identified failure mode is assessed with respect to its potential severity and frequency only,
while a decision tree is used to consider the detectability of the failure mode and the
availability of existing control measures. After having identified the failure mode causes, the
FMEA or HFMEA™ team determines actions, barriers, and controls that either eliminate the
failure mode causes or mitigate their effects.
Since its introduction in 2002, HFMEA™ has been applied on several health care
processes, such as drug ordering and administration, and the sterilisation and use of surgical
instruments (e.g., Esmail et al., 2005; Linkin et al., 2005; Van Tilburg, Leistikow,
Rademaker, Bierings, & Van Dijk, 2006; Wetterneck, Skibinski, Schroeder, Roberts, &
Carayon, 2004; Wetterneck et al., 2006). In the United States in 2004, the Joint Commission
on Accreditation of Healthcare Organizations (JCAHO) began requiring accredited health
Prospective Risk Analysis of Health Care Processes
13
care organisations to conduct one prospective analysis every year (The Joint Commission:
Standard PI.3.20).
Despite reported successful FMEA and HFMEA™ applications in several health care
settings, the use of those prospective methods in health care still needs to be thoroughly
evaluated and discussed. In some studies a single FMEA or HFMEA™ analysis has been
conducted and critically evaluated (e.g., Jeon et al., 2007; Kunac & Reith, 2005; Wetterneck
et al., 2006). Unfortunately, a systematic evaluation of a larger set of HFMEA™ analyses
has, to our knowledge, not taken place yet. The need for a profound evaluation of HFMEA™
applications is endorsed by the fact that The Joint Commission has found that health care
organisations are not always conducting their prospective analyses consistently or well (N.
Kupka, The Joint Commission, personal communication, April 24, 2008). In this study, we
carried out multiple HFMEA™ analyses at MAASTRO clinic, a radiotherapy institute in
Maastricht, and at University Medical Center Utrecht (UMC Utrecht) to systematically
evaluate HFMEA™ by means of user feedback. The clustered positive and negative
comments resulted in several suggestions for change to improve the perceived utility and
acceptance of HFMEA™.
2.1 Methods
Setting
A total of 13 HFMEA™ analyses were carried out to obtain insight into the perceived
benefits and drawbacks of the application of HFMEA™ in Dutch health care. MAASTRO
clinic provided us with a single focus, a high volume health care environment, while UMC
Utrecht represented the general and academic hospitals.
Selection of Health Care Processes
At MAASTRO clinic, four HFMEA™ analyses were conducted on topics which were
selected by the management team and the patient safety manager. At MAASTRO clinic,
actual accidents, near misses, and process deviations are registered in a database and analysed
in a systematic way. The processes that were selected for the HFMEA™ analyses of this
study were high risk processes (according to the MAASTRO clinic incidents database) and/or
new and innovative processes.
At UMC Utrecht all 12 divisions were asked to define three high risk processes.
Subsequently, the patient safety coordinator and the division management involved jointly
selected one of these processes. Criteria for this decision were: a direct connection with
Chapter 2
14
patient care, high risk, availability of clear process boundaries, and feasibility. Finally, nine
health care processes were selected to be included in this study. In three cases, two divisions
both identified identical high risk processes. In those cases the divisions involved carried out
a single HFMEA™ analysis collectively.
The 13 selected processes were quite diverse. Both acute and non-acute care were
included, and scheduled as well as unscheduled tasks were considered. Moreover, technology
played an important role in some selected processes, while it played a minor role in others.
Finally, processes in both inpatient and outpatient settings were selected.
HFMEA™ Analysis
For each HFMEA™ analysis a multidisciplinary team was composed that consisted of at least
two employees who were involved in the investigated health care process (e.g., nurses,
doctors, technicians, or clerical staff) and a facilitator. In three teams, a patient or a patient's
relative participated in the analysis. At MAASTRO clinic, in two teams, the patient safety
manager (PR) was also present during the meetings; in one of those two teams, a student (JR)
was present to learn more about how to facilitate an HFMEA™ analysis. In those two teams,
the patient safety manager and the student were only indirectly involved in the HFMEA™
analysis. At MAASTRO clinic, the number of team members ranged from 4 to 8; on average
a team consisted of 5.5 persons (SD = 1.7). At UMC Utrecht, the number of team members
ranged from 6 to 13; on average a team consisted of 7.9 persons (SD = 2.1). This difference
in the average number of team members can be explained by the fact that at UMC Utrecht
sometimes multiple units were involved in a single HFMEA™ analysis, while in all
HFMEA™ analyses at MAASTRO clinic only one unit was involved.
In all teams the facilitator concentrated on the correct use of HFMEA™ and the
progress of the analysis. In 12 of the 13 teams the facilitator also took the minutes. In nine
teams the facilitator was not involved in the selected process at all (MH and JR); in fact,
those two facilitators are non-health care workers. In four teams the facilitator was either
employed at the organisation and familiar with the investigated health care process (PR and
CP) or directly involved in the investigated health care process (DZ). All facilitators gathered
specific knowledge about HFMEA™ by means of the NCPS toolkit. One facilitator (PR) had
conducted HFMEA™ analyses before; two facilitators (MH and JR) had been taught FMEA
at university. All facilitators had experience in conducting incident analyses and were
familiar with the system approach (Reason, 2000). The other team members received a
concise draft manual about HFMEA™ (translated into Dutch). Furthermore, at the start of the
Prospective Risk Analysis of Health Care Processes
15
first meeting the facilitator gave a short presentation about the objective and the contents of
HFMEA™.
Each multidisciplinary team met several times and each meeting took one and a half
hours. The teams first reached an understanding about the exact definition of the selected
process. Subsequently, the selected process was mapped. Then, the teams made a decision
about the focus of the analysis. Sometimes, the complete process was analysed, while in other
cases only a particular part of the selected process was analysed due to time constraints. Next,
the team determined all possible ways in which the process could fail (i.e. not produce the
anticipated result). Those identified failure modes were all assessed on their potential severity
(i.e. catastrophic, major, moderate, or minor outcomes) and frequency (i.e. frequent,
occasional, uncommon, or remote). For each failure mode a decision was made about the
extent to which the risk was sufficiently covered in the health care system. In case the system
did not take care of the failure mode effectively, the team identified the causes of the failure
mode. After the team had assigned priorities to the failure mode causes, the team described
actions, barriers, and controls to either reduce the chance of occurrence of the failure modes
or to mitigate their effects. All information and decisions were (mostly) on site recorded in a
worksheet. As an example, the results of a single HFMEA™ analysis are summarised in Box
2.1.
User Feedback on HFMEA™: Evaluation Forms
At the end of an HFMEA™ analysis all team members (apart from the facilitators) were
asked to fill out an evaluation form about their experiences with HFMEA™. The evaluation
forms consisted of both multiple choice questions and open-ended questions. At MAASTRO
clinic, the patient safety manager (PR) and the student (JR) were not asked to fill out an
evaluation form because they were only indirectly involved in the two HFMEA™ analyses in
question. Hence, none of the facilitators and none of the members of the research group filled
out an evaluation form. The evaluation form for patients or their relatives slightly differed
from the evaluation form for employees. The evaluation forms were anonymous with respect
to person, but not with respect to team.
Chapter 2
16
Box 2.1
Example of an HFMEA™ analysis.
Step 1. Define the HFMEA™ topic
Medication administration by means of infusion pumps at an Intensive Care Unit
Step 2. Assemble the team
- Two nurses
- A member of the quality department
- An internal medicine specialist
- An external facilitator (MH)
Step 3. Graphically describe the process
The selected process was divided into the following process steps:
- Prescribing the medication
- Entering the prescription in the computer system
- Dispensing the medication
- Conducting a double check
- Adjusting the drip speed
Step 4. Conduct a hazard analysis
The team identified several potential failure modes such as:
- Wrong prescription or wrong entry of the medication or its concentration
- Making use of the wrong fluid when dispensing the medication
- Not conducting a double check
- Adjusting the drip speed wrongly
The causes underlying the failure modes were technical, organisational and human in nature.
Examples of failure mode causes were:
- Wrong computation
- Lack of communication about a modified layout or the medication cupboard
- Incorrect or incomplete protocols
- Health care employees being unfamiliar with certain types of infusion pumps
Step 5. Identify actions and outcome measures
The team proposed several actions to eliminate or control the failure modes.
The most important actions were:
- Use of generic drug names when prescribing medication
- Use of weighing beds
- Communication of modifications via e-mail and advice
- Revision of the double check protocol
- Computation as part of Intensive Care Unit education
- Specific instructions in case of new equipment
Data Coding
On the evaluation forms, the respondents were asked to write down in free text comments
regarding HFMEA™ and its application. Subsequently, those comments were categorised.
Two independent coders were involved in the coding process (MH and HW). The fact that
one of the two coders (MH) was the facilitator of nine teams could have biased the results of
Prospective Risk Analysis of Health Care Processes
17
the coding process. However, this potential bias was minimised because the second coder
was an Industrial Engineering student (HW), who, apart from the coding process, was not
involved in the study at all. HW had been taught FMEA and HFMEA™ at university and as
part of her master project she had used patient safety tools such as incident analysis
techniques. Moreover, the potential bias was lessened because the two coders discussed until
a consensus was reached. The two coders first independently classified the comments into
four categories: positive (single, positive statements; e.g., "a constructive attitude of the
participants"), negative (single, negative statements; e.g., "the analysis was time-
consuming"), plural (multiple statements; e.g., "a thorough approach, enthusiastic guidance,
cooperation"), and irrelevant (single statements without any relation to the contents of
HFMEA™ and/or its application; e.g., "good luck!"). The percentage agreement between the
two coders was 71.8%. The corresponding Cohen's kappa of .61 indicated substantial
agreement (Landis & Koch, 1977). For the comments that were classified as plural, the
coders also determined which separate statements could be distinguished and to which
category (i.e. positive, negative, or irrelevant) those statements could be assigned. The
percentage agreement between the two coders regarding the classification of the plural
statements was 58.3%. During a consensus meeting the two coders reached an agreement
about the categorisation of all statements.
Subsequently, the two coders jointly defined nine codes that referred to the separate
steps and aspects of HFMEA™ (such as the multidisciplinary team, the facilitator, and the
identification of failure modes and failure mode causes). In addition, the coders used open
coding (Babbie, 2005) to develop codes for the exact opinion the respondents had on the
various steps and aspects of HFMEA™ (e.g., "difficult" or "time-consuming"). While
assigning the statements to the positive, negative, and irrelevant categories, the coders gained
a first understanding of the exact opinion of the respondents. Together, the two coders
decided upon six codes for type of opinion. Those codes completely emerged from the data,
which is in accordance with the open coding principle. Each of the six codes for type of
opinion was formulated in both positive and negative terms (e.g., "easy" and "difficult" or
"clear" and "unclear"). Both coders then independently assigned all positive and negative
statements to one of the nine codes for the steps and aspects of HFMEA™ and to one or more
of the six codes for type of opinion. The percentage agreement between the two coders on the
classification of the statements into both steps / aspects of HFMEA™ and type of opinion
was 58.4%. Because the coders were allowed to classify one statement into multiple types of
opinions, it was only possible to calculate kappa for the assignment of the statements to the
Chapter 2
18
nine steps / aspects of HFMEA™. The percentage agreement between the two coders on the
assignment of the statements to the steps / aspects of HFMEA™ was 77.3%. The
corresponding Cohen's kappa of .72 again indicated substantial agreement. During a second
meeting the two coders reached a consensus about the classification of all statements.
Moreover, the coders decided to accentuate the definitions of some types of opinions and to
add an additional code referring to HFMEA™ in general. The final classification scheme for
positive and negative statements regarding HFMEA™ thus consisted of ten codes for steps /
aspects of HFMEA™ and six codes for type of opinion (see Table 2.1).
Table 2.1
Classification scheme for positive and negative statements regarding HFMEA™.
Step / aspect of HFMEA™
Process selection and scope
Multidisciplinary team
Facilitator
Process description
Identification of failure modes and failure mode causes
Risk assessment
Identification of actions and outcome measures
Implementation of actions
HFMEA™ in general
Other
Type of opinion (positively stated) Type of opinion (negatively stated)
Pleasant Unpleasant
Easy Difficult
Clear Unclear
High output Low output
Small time investment Large time investment
Other Other
Facilitator's Feedback on HFMEA™: Discussions
During the project, the research group (MH, TS, IL, and PR) also consulted the facilitators
(MH, PR, JR, DZ, and CP) to evaluate the application of HFMEA™. The research group and
facilitators met several times and exchanged experiences. After all HFMEA™ analyses had
been finalised but before the quantitative and qualitative data analysis of the evaluation
Prospective Risk Analysis of Health Care Processes
19
forms, the research group and facilitators collectively drew conclusions regarding the
application of HFMEA™ in Dutch health care.
2.2 Results
Descriptive Statistics
All 13 HFMEA™ analyses were successfully concluded. Table 2.2 presents some important
descriptive statistics for each selected health care process and the accompanying health care
setting: the initials of the facilitator, the team size, the number of meetings, the total number
of person-hours needed for the analysis, the number of identified failure modes, and the
number of proposed actions. Every meeting took one and a half hours, as scheduled
beforehand. The number of meetings needed to carry out the analysis ranged from 4 to 8; on
average the teams needed 6.3 meetings (SD = 1.3). The average number of meetings at
MAASTRO clinic was lower than that at UMC Utrecht (5.8 and 6.6, respectively). This
difference can partly be attributed to the fact that at MAASTRO clinic the processes had
already been mapped before the formal start of the HFMEA™ analyses and the graphical
process descriptions only needed to be verified by the team members involved. The number
of person-hours needed to conduct the analysis ranged from 30.0 to 136.5 (excluding
reporting on the meetings and reporting on the results of the HFMEA™ analysis); on average
the HFMEA™ analyses took 69.1 person-hours excluding reporting (SD = 28.7) and 78.0
person-hours including reporting. The differences between the teams with respect to time
investment can largely be attributed to differences in team size and scope. In earlier studies,
HFMEA™ analyses on vincristine prescription and administration and the sterilisation and
use of surgical instruments took a total of 140 and 250 person-hours, respectively (Linkin et
al., 2005; Van Tilburg et al., 2006). The average number of identified failure modes was 51.8
(SD = 30.6) and the average number of proposed actions was 16.2 (SD = 8.8). Again,
differences in scope contributed to team differences regarding the number of identified failure
modes and the number of proposed actions.
Table 2.2
Selected health care processes, accompanying health care settings and descriptive statistics.
ID Health care process Health care setting Facilitatora Team
sizeb
No. of
meetings
No. of
person-hoursc
No. of
failure modes
No. of
actions
1 Documentation of treatment Radiotherapy PR 5 4 30.0 32 17
2 Electronic Portal Imaging
(EPI)
Radiotherapy MH 8 6 72.0 109 33
3 Treatment on
linear accelerator
Radiotherapy JR 5 8 60.0 70 30
4 Release of accelerator
after maintenance
Radiotherapy PR 4 5 30.0 50 22
5 Communication of
unexpected findings
Radiology
Cardiology
MH 7 5 52.5 19 7
6 Diet food process Children's Hospital MH 13 7 136.5 39 18
7 Physically restraining
patients
Neurosurgery MH 7 7 73.5 31 17
8 Ordering repeat
prescriptions
Primary care DZ 8 8 96.0 50 12
9 Patients with hip fractures Emergency Room
Radiology
Nursing ward
Operating Room
MH 8 6 72.0 120 7
10 Medication administration
(pumps)
Intensive Care Unit MH 6 6 54.0 46 22
Table 2.2 continued
Selected health care processes, accompanying health care settings and descriptive statistics.
ID Health care process Health care setting Facilitatora Team
sizeb
No. of
meetings
No. of
person-hoursc
No. of
failure modes
No. of
actions
11 Admission of
cardiac patients
Emergency Room
Cardiac Cath Room
Coronary Care Unit
CP 6 6 54.0 44 6
12 Use of a PICC line
(catheter)
Neonatal Intensive
Care Unit
MH 8 8 96.0 37 8
13 Administration of
blood products
Laboratory
Haematology ward
MH 8 6 72.0 27 11
M
(SD)
7.2
(2.2)
6.3
(1.3)
69.1
(28.7)
51.8
(30.6)
16.2
(8.8) aMH and JR Eindhoven University of Technology; PR MAASTRO clinic; DZ and CP UMC Utrecht.
bA patient was included in teams 1, 6, and
8. cReporting on the meetings and the results of the HFMEA™ analysis are excluded.
Chapter 2
22
User Feedback on HFMEA™: Results from Evaluation Forms
All team members apart from the facilitators and team members (if any) who were only
indirectly involved in the HFMEA™ analysis (i.e. 77 people) were asked to fill out the
evaluation form. In total 62 evaluation forms were filled out and returned to the researchers;
59 by employees and 3 by patients or their relatives. The overall response rate was 80.5%.
The response rates of MAASTRO clinic and UMC Utrecht were almost equal (80.0% and
80.6%, respectively). Table 2.3 presents the contents and results of the multiple choice
questions of the evaluation forms.
About 90% of the employees and patients who filled out the evaluation form thought
that the HFMEA™ analysis was meaningful (90.3%). The majority of the respondents
(87.1%) expected the investigated health care process to become more safe as a result of the
HFMEA™ analysis that had been carried out. Also about 90% of the respondents would
recommend others to participate in an HFMEA™ analysis (90.3%). The evaluation form for
the employees also included some questions about patient involvement in the HFMEA™
analysis. Of all respondents who participated in an HFMEA™ analysis in which a patient was
involved, over 90% (93.3%) thought that this patient involvement was useful. Interestingly,
only a minority of all respondents who participated in an HFMEA™ analysis in which no
patient had been involved (9.1%) thought patient involvement would have been useful.
Table 2.3
Contents and results of evaluation forms: Multiple choice questions.
Health care employees
(n = 59)
Patients
(n = 3)
Health care employees + Patients
(N = 62)
Question Yes No No answer Yes No No answer Yes No No answer
Did the manual provide you
with sufficient information
about conducting an
HFMEA™ analysis?
94.9% 1.7% 3.4% 100.0% 0.0% 0.0% 95.2% 1.6% 3.2%
Were all relevant disciplines
represented in the team?
88.1% 11.9% 0.0% 66.7% 33.3% 0.0% 87.1% 12.9% 0.0%
Was a patient represented in
the team?
25.4% 74.6% 0.0%
- If yes, do you think this was
useful?
93.3% 6.7% 0.0%
- If no, do you think this
would have been useful?
9.1% 72.7% 18.2%
Were all meetings useful for
you?
74.6% 20.3% 5.1% 100.0% 0.0% 0.0% 75.8% 19.4% 4.8%
Do you think the HFMEA™
analysis was meaningful?
91.5% 1.7% 6.8% 66.7% 0.0% 33.3% 90.3% 1.6% 8.1%
Do you think the investigated
process will be safer thanks
to the HFMEA™ analysis?
88.1% 1.7% 10.2% 66.7% 33.3% 0.0% 87.1% 3.2% 9.7%
Table 2.3 continued
Contents and results of evaluation forms: Multiple choice questions.
Health care employees
(n = 59)
Patients
(n = 3)
Health care employees + Patients
(N = 62)
Question Yes No No answer Yes No No answer Yes No No answer
Did you obtain another
insight into your own work
process thanks to the
HFMEA™ analysis?
45.8% 45.8% 8.5%
Would you recommend
others to participate in an
HFMEA™ analysis?
93.2% 1.7% 5.1% 33.3% 33.3% 33.3% 90.3% 3.2% 6.5%
Are you more willing to
report incidents since you
have conducted the
HFMEA™ analysis?
23.7% 62.7% 13.6%
Are you more assured about
safety in the institution since
you have conducted the
HFMEA™ analysis?
33.3% 0.0% 66.7%
Fine Too
long
Too
short
No
answer Fine
Too
long
Too
short
No
answer Fine
Too
long
Too
short
No
answer
What did you think of the
duration of the meetings?
83.1% 10.2% 6.8% 0.0% 66.7% 0.0% 0.0% 33.3% 82.3% 9.7% 6.5% 1.6%
Note. Empty cells indicate that the particular question is not applicable for the particular group of respondents.
Prospective Risk Analysis of Health Care Processes
25
The classification of the comments of the respondents into steps / aspects of
HFMEA™ and types of opinions shows the perceived benefits and drawbacks of HFMEA™.
In addition to the percentage of respondents that made a particular comment, the results show
the number of teams in which that particular comment was made by at least one team
member. In both the results section and the tables the percentage of respondents is directly
followed by the number of teams, which is presented between parentheses. Table 2.4 presents
the resulting classification of the positive statements. Since the respondents were allowed to
write down multiple (positive) comments and because some respondents did not answer the
open-ended questions, the totals do not equal 100%. According to 36.4% of the respondents
(10 teams) the HFMEA™ analysis resulted in high output in terms of the insight obtained
into the health care process in general, in other employees' tasks, and in the possible risks
(e.g., "HFMEA™ makes failure modes apparent" or "by means of HFMEA™ I gained a clear
insight into processes and relations"). Positive remarks with respect to HFMEA™ in general,
such as the fact that HFMEA™ is a systematic, stepwise approach, were made by 28.6% of
the respondents (9 teams) (e.g., "HFMEA™ is a clear method" or "it is a structural
approach"). Furthermore, 22.1% of the respondents (8 teams) thought the multidisciplinary
nature of the analysis was pleasant and useful (e.g., "the multidisciplinary approach was
useful").
Table 2.4
Positive user feedback on HFMEA™: Percentage of respondents (no. of teams) per combination of step / aspect of HFMEA™ and type of opinion.
Type of opinion
Pleasant Easy Clear High output Small time
investment Other Total
Step / aspect of HFMEA™
Process selection and scope 1.3% (1) 1.3% (1)
Multidisciplinary team 2.6% (1) 3.9% (2) 16.9% (8) 22.1% (8)
Facilitator 5.2% (3) 2.6% (2) 1.3% (1) 6.5% (4) 13.0% (5)
Process description 10.4% (5) 1.3% (1) 11.7% (6)
Identification of failure mode (causes) 7.8% (5) 7.8% (5)
Risk assessment 2.6% (2) 1.3% (1) 3.9% (3)
Identification of actions 0.0% (0)
Implementation of actions 2.6% (2) 1.3% (1) 3.9% (2)
HFMEA™ in general 1.3% (1) 1.3% (1) 2.6% (2) 13.0% (7) 2.6% (2) 15.6% (6) 28.6% (9)
Other 1.3% (1) 1.3% (1)
Total 9.1% (4) 1.3% (1) 5.2% (4) 36.4% (10) 2.6% (2) 37.7% (11)
Note. An empty cell indicates that no comment referred to that particular combination of step / aspect of HFMEA™ and type of opinion. Since the
respondents were allowed to write down multiple (positive) comments and because some respondents did not answer the open questions, the totals do
not equal 100%.
Prospective Risk Analysis of Health Care Processes
27
Table 2.5 presents the resulting classification of the negative statements. Since the
respondents were allowed to write down multiple (negative) comments and because some
respondents did not answer the open questions, the totals do not equal 100%. Negative
remarks of 20.8% of the respondents (9 teams) concerned the notion that the time investment
necessary to conduct the HFMEA™ analysis was large (e.g., "it takes a lot of time").
Although the positive remarks indicated that the HFMEA™ analysis resulted in high output
in terms of the (additional) insight into processes, tasks, and risks, 20.8% of the respondents
(7 teams) felt that the analysis did not yield (significant) results, that is, that the output was
low (e.g., "many aspects lead to useless discussions"). According to 13.0% of the respondents
(6 teams) the HFMEA™ analysis was difficult to carry out. From the negative remarks of
7.8% of the respondents (5 teams), it can be concluded that especially the risk assessment
part of HFMEA™ (i.e. determining the hazard score and using the decision tree) was
perceived to be difficult (e.g., "the decision tree was difficult for me" or "it is difficult to
score the risks"). In general, the risk assessment part of HFMEA™ was subject of the
negative comments of 15.6% of the respondents (8 teams). Although the multidisciplinary
nature of the team was perceived to be beneficial, 13.0% of the respondents (6 teams) faced
problems within the team such as planning problems and problems regarding the frequent
absence of certain team members (e.g., "often people were absent").
As can be concluded from the positive and negative remarks, the facilitator's role is
perceived to be crucial. Respondents from 5 teams mentioned that the facilitator's presence
had been of great value (e.g., "pleasant and clear guidance"), while respondents from 4 teams
even claimed that the facilitator's role had been essential and that the analysis would not have
been possible without the facilitator (e.g., "a good facilitator is necessary" or "we needed
quite a lot of guidance").
Table 2.5
Negative user feedback on HFMEA™: Percentage of respondents (no. of teams) per combination of step / aspect of HFMEA™ and type of opinion.
Type of opinion
Unpleasant Difficult Unclear Low output Large time
investment Other Total
Step / aspect of HFMEA™
Process selection and scope 1.3% (1) 1.3% (1)
Multidisciplinary team 5.2% (4) 7.8% (5) 13.0% (6)
Facilitator 6.5% (4) 6.5% (4)
Process description 1.3% (1) 5.2% (2) 5.2% (4) 11.7% (6)
Identification of failure mode (causes) 2.6% (2) 1.3% (1) 3.9% (2)
Risk assessment 7.8% (5) 1.3% (1) 2.6% (1) 1.3% (1) 5.2% (4) 15.6% (8)
Identification of actions 1.3% (1) 1.3% (1) 2.6% (2)
Implementation of actions 6.5% (4) 6.5% (4)
HFMEA™ in general 1.3% (1) 2.6% (2) 6.5% (3) 16.9% (8) 9.1% (5) 27.3% (10)
Other 0.0% (0)
Total 0.0% (0) 13.0% (6) 3.9% (3) 20.8% (7) 20.8% (9) 37.7% (11)
Note. An empty cell indicates that no comment referred to that particular combination of step / aspect of HFMEA™ and type of opinion. Since the
respondents were allowed to write down multiple (negative) comments and because some respondents did not answer the open questions, the totals do
not equal 100%.
Prospective Risk Analysis of Health Care Processes
29
Facilitators' Feedback on HFMEA™: Results from Discussions
In addition to the problems mentioned before, the facilitators and the research group
identified two possible threats to the quality of the outcomes of an HFMEA™ analysis. First,
HFMEA™ itself provides no guidelines for the identification of failure mode causes. This
might result in a biased analysis if the team members would use a person approach instead of
a system approach. If one applies a system approach during the causal analysis part of
HFMEA™, one concentrates on the conditions under which health care employees work. On
the other hand, if one applies a person approach, one especially blames individuals for their
errors, inattention, or forgetfulness (Reason, 2000). The second problem that the research
group and the facilitators recognised is the fact that HFMEA™ itself does not include
guidelines for the translation of any identified failure mode cause into an appropriate
countermeasure. Therefore, the countermeasures that team members come up with might not
be the most effective ones, for instance because again a person approach might be
predominant.
2.3 Discussion
Although all 13 HFMEA™ analyses were successfully concluded, the user feedback revealed
both positive and negative comments with regard to HFMEA™. Interestingly, most positive
comments of the participants were not exclusively related to HFMEA™. For example, the
multidisciplinary nature of the team seemed to be an important strength, which was also
concluded in the evaluation of other, single HFMEA™ analyses (Esmail et al., 2005;
Wetterneck et al., 2004; Wetterneck et al., 2006). However, this is not an aspect of the
method that is unique for HFMEA™. On the other hand, many negative comments that were
put forward by the participants were indeed related to the aspects of HFMEA™ that
distinguish it from some other prospective methods, like the rating scales and the use of the
decision tree. As concluded in earlier studies, one of the most important problems regarding
FMEA and HFMEA™ is the fact that the analysis is very resource intensive (Carstens, 2006;
Kunac & Reith, 2005; Linkin et al., 2005; Wetterneck et al., 2004; Wetterneck et al., 2006).
From the user feedback it can be concluded that the role of the facilitator is crucial for
the successful application of HFMEA™ (Rath, 2008). The conclusions of the research group
and the facilitators stress the importance of team guidance as well. Besides the facilitator's
task to explain the HFMEA™ steps and to control the progress of the analysis, the facilitator
could assist the team in applying a system approach when identifying failure mode causes
and describing appropriate actions (Kunac & Reith, 2005; Wetterneck et al., 2004;
Chapter 2
30
Wetterneck et al., 2006). This system approach is important because only when one
concentrates on the work settings and the conditions under which people have to work, the
system failures can be revealed and effective interventions to improve patient safety can be
determined (Carayon, Alvarado, & Hundt, 2007; Reason, 2000). Therefore, the facilitators
should be selected carefully and, if necessary, they should be trained in using a system and
human factors approach. When applying FMEA on the medication use process of a neonatal
intensive care unit, Kunac and Reith (2005) already successfully combined FMEA with a
system approach.
Based on the comments of the team members and the experiences of the facilitators
we put forward several suggestions to improve the perceived utility and acceptance of
HFMEA™. First, we recommend changing the categories for frequency of occurrence into
more defined and reliable categories to prevent team members from placing their own
interpretation on the categories. For instance, one could use categories such as "weekly,
monthly, yearly, and less than yearly" instead of the current HFMEA™ categories (i.e.
frequent, occasional, uncommon, and remote, respectively). In accordance with
recommendations that resulted from earlier single-case studies, we advise health care
organisations to verify whether the HFMEA™ rating scales are applicable to the process
under investigation. If necessary, one could customise the rating scales (Jeon et al., 2007;
Wetterneck et al., 2004). Such a modification will probably prevent lengthy discussions about
the exact meaning of the categories for severity and frequency (Israelski & Muto, 2007).
Second, we suggest replacing the numbers in the HFMEA™ hazard scoring matrix.
According to the facilitators, some participants (wrongly) assumed that the numbers in the
hazard scoring matrix represented a ratio scale. Therefore, we propose to replace those
numbers by ordinal scale categories like "very high risk, high risk, low risk, and very low
risk" with accompanying red or green shades of colour.
Because of the fact that the time investment for an HFMEA™ analysis might be too
large, we also put forward several suggestions to decrease the amount of time necessary to
carry out the analysis. One could, for instance, ask a sub group of the team to map the
selected process in advance. During the first meeting with the entire team the other team
members could then be asked to verify the graphical process description. However, as a
result, team members might obtain less insight into each other's tasks because the team might
discuss the graphical process description in less detail. Another possible way to save time
could be to conduct HFMEA™ Step 4 and Step 5 in direct succession for each process step.
In other words, one could first carry out both a hazard analysis and determine appropriate
Prospective Risk Analysis of Health Care Processes
31
actions for one process step before investigating the next one. By doing so, the team members
will probably master the different steps of HFMEA™ more quickly, allowing a faster
handling of the other process steps. Moreover, if time constraints would force the team to
stop the analysis, a complete HFMEA™ analysis would be conducted for at least one or more
entire process steps. A possible downside of this linear approach is the fact that the actions
that the team comes up with might only optimise a particular process step, while the solutions
might be suboptimal for the entire process. Therefore, the facilitator should assist the team in
applying a comprehensive system view during and after the HFMEA™ analysis. For some
teams it might be tempting to omit the decision tree for the failure mode causes. In that case
the team determines the risk scores for a set of failure mode causes and subsequently the team
decides directly which failure mode causes warrant further action, that is, without using the
decision tree. Because answering the detailed questions in the decision tree is omitted, the
analysis will probably take less time. However, since the assessment of the detectability of
the failure mode causes is an important aspect of prospective analysis, those teams should
actually consider to use FMEA and the original risk priority numbers of FMEA (in which
detectability is built into) instead of omitting the HFMEA™ decision tree. But still, this
approach should only be considered by teams with sufficient analytical capabilities because it
might be too difficult for the majority of the health care employees to take into account
detectability without using the predefined questions of the decision tree. Again, the facilitator
could determine the best approach for the team without compromising the quality of the
FMEA or HFMEA™ analysis.
Irrespective of the type of modification, we emphasise the importance of applying the
basic principles of HFMEA™ as well as of other (similar) prospective methods. One should
first describe the process, then identify the risks within this process (and their causes), and
finally determine actions to either eliminate or control the risks or to mitigate their effects,
instead of the observed tendency to directly jump from identified problems to
countermeasures. This structural approach seems to be effective and highly appreciated by
the users as well.
Although the above-mentioned recommendations seem to be plausible, we did not
systematically test those recommendations, let alone measure their effects. Therefore,
additional research is necessary to find out whether those recommendations really improve
the perceived utility and acceptance of HFMEA™. Moreover, similar studies should be
carried out in other health care settings and other countries to verify whether the conclusions
and recommendations are valid for those settings as well. It should be noted that this study
Chapter 2
32
concentrated on the perceived utility and acceptance of HFMEA™. Therefore, future
research could also focus on the effectiveness of HFMEA™ and similar prospective methods.
By means of longitudinal research designs one could, for instance, examine if proposed
actions are indeed implemented and whether those actions do actually improve patient safety.
In contrast with the user feedback, the experiences of the facilitators have not been
studied in a very systematic way. In future studies, the facilitators could be asked to provide
feedback on each HFMEA™ session. Structured evaluation forms might, for example,
consist of questions regarding: the steps of HFMEA™ that have been dealt with in a
particular meeting; the duration of the steps; the problems that occurred during the session;
and the impression the facilitator got from the meeting. Such studies might provide a more
detailed insight into both the opportunities and problems with respect to the application of
HFMEA™ in health care.
An interesting finding of our study is the fact that the respondents had differing views
on the benefits of patient involvement. The respondents of teams in which a patient
participated nearly all experienced the involvement of the patient as useful. On the other
hand, only a few respondents who participated in an HFMEA™ analysis without patient
involvement thought that patient participation would have been valuable. The usefulness of
patient involvement probably depends on the type of process to be analysed. However, our
results might also indicate that health care employees do not recognise the merits of patient
involvement in (prospective) risk analysis until they actually see it happen. Since patient
involvement in health care (and safety management in particular) is becoming increasingly
important, future studies could focus on the benefits and drawbacks of patient participation in
(prospective) risk analysis (Coulter, 2006; Entwistle, 2007; Lyons, 2007).
The results of our study may also be useful for other sectors in which prospective
methods are used that are similar to HFMEA™, such as FMEA and HACCP. For instance,
the engineering and manufacturing industries and the food industry could consider the user
feedback and the suggestions that have been put forward in their FMEA and HACCP
applications, respectively. Future research will probably result in modifications to existing
prospective methods such as HFMEA™, or even in new methods. Such developments might
improve the perceived utility and acceptance, and even the effectiveness of those methods.
Nevertheless, health care organisations should not wait for a perfect method, but continue or
start conducting prospective analyses in their current form to improve patient safety
proactively, that is, to really prevent patient harm.
33
Chapter 3
Integration of Prospective and Retrospective Methods
for Risk Analysis in Hospitals*
This chapter deals with the combined application of prospective and retrospective
methods for risk analysis on two units of a Dutch general hospital. In the prospective
analyses, employees identified and assessed possible risks in selected processes. In
the retrospective analyses, incidents were reported by employees and subsequently
investigated. We integrated the methods by using information from retrospective
incident reports for prospective risk identification and assessment, and by matching
their categorisation schemes. The results showed that the two analyses yielded
divergent overviews of risks and that triangulation can provide a better picture. An
integrative approach might be advantageous in terms of efficiency of analysis, setting
priorities, and improving the methods themselves.
Hospitals use retrospective methods to analyse errors and to prevent their recurrence.
However, the objective of minimal patient harm (Battles & Lilford, 2003) stresses the need to
—————————————
*This chapter is largely based on: Kessels-Habraken, M., Van der Schaaf, T., De Jonge, J., Rutte, C., &
Kerkvliet, K. (2009). Integration of prospective and retrospective methods for risk analysis in hospitals.
International Journal for Quality in Health Care, doi:10.1093/intqhc/mzp043.
Chapter 3
34
identify risks prospectively and to foresee errors (Hollnagel, 2008). This is endorsed by the
requirement of the Joint Commission on Accreditation of Healthcare Organizations (JCAHO)
to conduct one prospective analysis every 18 months (The Joint Commission, 2009: Standard
LD.04.04.05). Several methods for prospective analysis are available, like (Healthcare)
Failure Mode and Effect Analysis ((H)FMEA), Hazard Analysis and Critical Control Points
(HACCP), and Probabilistic Risk Assessment (PRA). Despite differences between these
methods, such as the consideration of combinatorial events in PRA and the use of a decision
tree in HFMEA™ and HACCP, they all aim to identify, assess, and eliminate or reduce risks
before errors may occur (Battles et al., 2006; Marx & Slonim, 2003; McDonough, Solomon,
& Petosa, 2004).
Perfect prospective analyses would anticipate all errors and therefore make
retrospective analyses redundant (Senders, 2004). However, both methods are subject to
biases (see Table 3.1). For instance, judgement variability could influence the reliability of
risk identification in prospective analyses (Bonnabry et al., 2006), and prospective risk
assessments might be inaccurate due to a lack of insight into error rates (Israelski & Muto,
2007; Marx & Slonim, 2003; Trucco & Cavallin, 2006). Retrospective incident reporting and
analysis is susceptible to problems such as underreporting (Aspden et al., 2004; Barach &
Small, 2000; Evans et al., 2006; Hogan et al., 2008; Johnson, 2003; Kingston, Evans, Smith,
& Berry, 2004; Olsen et al., 2007; Shojania, 2008; Waring, 2005), incomplete data (Barach &
Small, 2000; Cannon & Edmondson, 2005), hindsight and recall bias (Henriksen & Kaplan,
2003), and unreliable classifications (Evans et al., 2006; Johnson, 2003). In case of an
exclusively prospective approach, hospitals would not be able to compare their results with
actual data. Conversely, an exclusively retrospective approach could yield an incomplete
overview of the nature of risks as well as an underestimation of their magnitude.
The question arises how to overcome those biases. Since a ―golden standard‖ is still
lacking, triangulation could be the answer for now. By triangulation of prospective and
retrospective methods their strengths could be combined and their weaknesses minimised,
which could yield a better picture of risks (Battles & Lilford, 2003; Herzer et al., 2008;
Runciman et al., 2006; Senders, 2004). For instance, a broad prospective analysis could
complement the limited scope of retrospective incident reports, while retrospective causal
trees (Aspden et al., 2004) might compensate for the fact that most prospective methods fail
to consider combinatorial events. Recently, the National Quality Forum recommended such a
combined approach to improve patient safety (National Quality Forum, 2009).
Integration of Prospective and Retrospective Methods for Risk Analysis
35
Table 3.1
Possible biases of prospective risk analysis and retrospective incident reporting and analysis.
Prospective risk analysis Retrospective incident reporting and analysis
Unreliable risk identification due to
judgement variability during brainstorming
(Bonnabry et al., 2006)
Limited number of reported incidents
(Aspden et al., 2004; Barach & Small, 2000;
Evans et al., 2006), for instance due to a lack
of error recognition, a tendency to keep
errors in-house, feelings of fear or shame,
time pressure, and a lack of feedback
(Evans et al., 2006; Johnson, 2003; Kingston
et al., 2004; Shojania, 2008; Waring, 2005)
Inaccurate risk assessment due to a lack of
insight into error rates
(Israelski & Muto, 2007; Marx & Slonim,
2003; Trucco & Cavallin, 2006)
Limited spectrum of reported incidents,
partly due to the lack of incident reports
from doctors
(Evans et al., 2006; Hogan et al., 2008;
Johnson, 2003; Kingston et al., 2004; Olsen
et al., 2007; Shojania, 2008)
Failure to consider combinatorial eventsa
(Israelski & Muto, 2007; Marx & Slonim,
2003)
Incomplete data for instance due to
anonymity, confidentiality, shame, and fear
(Barach & Small, 2000; Cannon &
Edmondson, 2005)
Hindsight and recall bias
(Henriksen & Kaplan, 2003)
Poor quality of classifications
(Evans et al., 2006; Johnson, 2003)
aProbabilistic Risk Assessment (PRA) does explicitly consider combinatorial events.
But will the advantages outweigh the additional resources required to conduct two
analyses instead of just one? Probably yes, because the extra efforts could be limited if the
methods are integrated in terms of matching categorisations for risk identification and
assessment. Then, efficiency of analysis might be increased, for instance by making use of
retrospective data for the development of prospective failure scenarios (Aspden et al., 2004;
Harms-Ringdahl, 2004). An additional advantage of such integration is related to the fact that
hospital management must reflect on the outcomes of risk analyses to allocate resources to
appropriate actions (Battles et al., 2006; Hogan et al., 2008). Through integration of
prospective and retrospective methods, the analysis results will be directly comparable,
thereby facilitating the process of making sense of risks and determining interventions
(Battles et al., 2006).
Chapter 3
36
In only a few studies researchers have concentrated on such integration of methods,
for instance by using retrospective error rates for prospective analyses (Trucco & Cavallin,
2006; Wetterneck et al., 2006), or by comparing prospectively and retrospectively identified
causes of risks (Van der Hoeff, 2003). Although those studies have demonstrated several
possibilities for the integration of prospective and retrospective methods, those studies did
not consider the perceived usefulness of such integration. In the present study, we answered
the questions (1) how a relatively simple form of integration of prospective and retrospective
methods could be realised and (2) whether this integration would be perceived useful, taking
into account the additional resources required. We integrated the methods by using
information from retrospective incident reports for prospective risk identification and
assessment, and by matching their categorisation schemes. User feedback provided insight
into the perceived usefulness of the methods and their integration. We furthermore wondered
whether integration of prospective and retrospective methods would only be useful for
hospital management, or also for frontline staff, and whether the perceived usefulness
depends on participation in the analyses.
3.1 Methods
Setting
The study was conducted at two units of a Dutch general hospital. At the pharmacy a project
called RISC (Risk analysis by Incident reporting and Scenario analysis in the Cytostatics
dispensing process) concentrated on the process from ordering up to and including delivering
chemotherapy drugs and archiving. A project at the nuclear medicine unit called NUSAFE
(NUclear medicine SAFE) included the complete process from planning an examination or
treatment up to and including archiving.
Study Design
The projects comprised both prospective risk analyses and retrospective incident reporting
and analysis. For both units, the quality coordinators constructed flowcharts of the selected
processes by means of process mapping (Barach & Johnson, 2006); all process steps were
sequential. In the prospective analyses, employees identified and assessed possible risks for
each process step; in the retrospective analyses, all process steps that had contributed to the
occurrence of reported incidents were registered. During feedback sessions, the employees
were informed about preliminary results.
Integration of Prospective and Retrospective Methods for Risk Analysis
37
For a 4-month period, all 46 employees that were involved in the selected processes
were asked to report any deviation from normal patient care. At the pharmacy, employees
used a hardcopy reporting form, while an electronic form was used at the nuclear medicine
unit. Moreover, clerical staff from the latter unit scored each occurrence of a predefined set of
minor deviations in the sub process of planning. For both units, the first author together with
one or more employees analysed the reported incidents. Information about the incidents and
the process steps involved was registered in special databases.
Two months after the start of the incident reporting, 22 of the 46 employees
participated in prospective analyses. For each unit, two teams were composed, which were
comparable in terms of disciplines involved and participants‘ work experience. Each team
conducted a condensed version of an HFMEA™ analysis (Habraken, Van der Schaaf,
Leistikow, & Reijnders-Thijssen, 2009, see also Chapter 2). We decided to use HFMEA™
because the suggested components of a prospective analysis as proposed by JCAHO are all
part of HFMEA™ (The Joint Commission, 2009: Standard LD.04.04.05), because
HFMEA™ has been applied in a diverse range of hospital settings, and because a manual and
DVD are available. The analysis consisted of the identification of risks in the selected
processes and the assessment of their frequencies. The estimated frequencies were corrected
for the 4-month study period to enable direct comparison with the incident analyses. At each
unit, one team was provided information from the incidents database, such as the type and
frequency of reported incidents, while the other team had to rely completely on the expertise
and judgement of its team members.
We used two self-developed evaluation forms to examine the perceived usefulness of
the prospective and retrospective methods and their integration. After the prospective
analyses had been finalised, the 22 participants received an evaluation form (Form 1); 19
(86.4%) were completed and returned. At the end of the project, all 46 employees received
another evaluation form (Form 2); 34 (73.9%) were completed and returned. The evaluation
forms included the following statements:
- ―Since the prospective analysis, I am more willing to report incidents‖ (Form 1);
- ―Information about incidents and their frequencies was or would have been useful for
the prospective analysis‖ (Form 1);
- ―Information about causes of incidents was or would have been useful for the
prospective analysis‖ (Form 1);
- ―Retrospective incident reporting and analysis is useful for improving patient safety‖
(Form 2);
Chapter 3
38
- ―Retrospective incident reporting and analysis is useful for optimising processes‖
(Form 2);
- ―Thanks to the retrospective reporting and analysis of incidents, I have obtained
insight into new risks‖ (Form 2);
- ―Prospective analysis is useful for improving patient safety‖ (Form 2);
- ―Prospective analysis is useful for optimising processes‖ (Form 2);
- ―Thanks to the prospective analysis, I have obtained insight into new risks‖ (Form 2).
Five-point rating scales ranged from ―agree strongly‖ to ―disagree strongly‖. In
addition, both evaluation forms included the following question: ―Which analysis did provide
you most insight into risks?‖. Response categories were: (1) retrospective analysis, (2)
prospective analysis, (3) combination of prospective and retrospective analyses, and (4) no
idea.
Data Analysis
To explore the benefit of the integration, we used chi-square tests to compare the prospective
and retrospective evaluations of risks per process step. Since some expected cell counts did
not exceed the minimum level (Siegel & Castellan, 1988), Pareto analyses were used to
identify those process steps that accounted for the majority of the risks. The remaining
process steps were combined into a single category, called ―other‖. For setting priorities and
determining appropriate interventions, exact frequencies might be not that important
(Bonnabry et al., 2006), as opposed to rankings of risks. Therefore, for each analysis we
ranked the process steps in terms of the identified frequencies of risks. Next, Spearman rank
correlation coefficients (rs) were calculated to explore differences between the analyses
regarding the rankings of the ten highest risk process steps. For all statistical analyses, an
alpha level of .05 was used.
3.2 Results
We integrated prospective and retrospective methods by using similar categorisation
schemes. This enabled us to compare the analysis results directly. Tables 3.2 and 3.3 present
the results of the analyses in terms of the identified frequencies of risks per process step and
accompanying rankings. For both units, the results clearly showed a lack of congruence
between prospective and retrospective analyses. For instance, Table 3.2 shows that the
prospective analysis teams estimated that in a period of four months about 700 process
Integration of Prospective and Retrospective Methods for Risk Analysis
39
deviations would occur in the process step ―check labels and dispensing protocol‖, while in
the 4-month study period only 119 of such process deviations had been actually identified by
the retrospective analysis of reported incidents.
At the hospital pharmacy (RISC), 503 incident reports were analysed, which revealed
1,421 process deviations. When corrected for the study period, the prospective analysis teams
predicted that risks would have resulted in 7,062 and 12,654 process deviations, respectively.
The frequencies of risks were significantly different, χ2(24, N = 21,137) = 3,443.00, p < .001.
At the nuclear medicine unit (NUSAFE), 552 incident reports were analysed, which showed
1,169 process deviations. After correction for the study period, the prospective analysis teams
estimated that risks would have caused 8,677 and 4,756 process deviations to occur,
respectively. Assessment of differences in those overviews yielded a significant result, χ2(30,
N = 14,602) = 6,925.00, p < .001. The significant results for RISC and NUSAFE indicate that
prospective and retrospective analyses can result in divergent overviews of the nature and
magnitude of risks.
This finding might make it difficult for hospital management to determine
interventions to improve patient safety. However, for priority setting the relative magnitude
of risks might be more important than their exact frequencies (Bonnabry et al., 2006).
Therefore, we calculated the correlations between the rankings of the ten process steps that
were provided with the highest frequencies of risks. For RISC, significant positive
correlations were found between the retrospective incident analyses and the two prospective
analyses (rs = .59, p < .05; rs = .79, p < .01). No significant correlation was found between the
two prospective analyses (rs = .35, p = .24). For NUSAFE, no significant correlations were
found at all (rs = .16, p = .54; rs = .18; p = .48; rs = .22, p = .39).
Although the prospective and retrospective analyses showed a lack of congruence
regarding the frequencies of risks, the analysis of risk rankings yielded a somewhat different
conclusion. For NUSAFE, hospital management might still feel uncertain about resource
allocation, due to the lack of substantial consensus on risk rankings. Conversely, for RISC,
predictions were supported by actual data (as reflected by the two significant correlations).
This might convince management to allocate resources to the process step ―enter data and
print labels‖, which was identified as a high risk process step by all analyses.
Chapter 3
40
Table 3.2
RISC: Identified frequencies (freq.) of risks per process step and accompanying rankings
(rank) by analysis.
Analysis
RIA PRA RISC 1 PRA RISC 2
Process step Freq. Rank Freq. Rank Freq. Rank
Ordering
Fill in prescription forma 207 2 600 1250 5
Pre-check prescription form 11 366 33
Sending
Fax prescription form to pharmacy 114 5 649 5 704
Processing
Fill in dispensing protocol 140 3 917 3 1758 4
Enter data and print labels 255 1 1109 1 2100 2
Check labels and dispensing protocol 119 4 675 4 700
Add prescription form 77 350 1834 3
Sort prescription form by date 83 284 2516 1
Dispensing
Put medication ready 74 944 2 433
Dispense chemotherapy drugs 53 375 272
Release chemotherapy drugs 56 8 333
Delivering
Transport chemotherapy drugs 66 176 167
Other 166 609 554
Total 1421 7062 12654
Note. Frequencies (freq.) have been corrected for the study period of four months. Rankings
(rank) are only presented for the five highest risk process steps; all other cells are left empty.
RIA = Retrospective Incident reporting and Analysis. PRA RISC 1 = Prospective Risk
Analysis without information from the retrospective incidents database. PRA RISC 2 =
Prospective Risk Analysis with information from the retrospective incidents database. aDiagnosis errors have been excluded.
Integration of Prospective and Retrospective Methods for Risk Analysis
41
Table 3.3
NUSAFE: Identified frequencies (freq.) of risks per process step and accompanying rankings
(rank) by analysis.
Analysis
RIA PRA NUSAFE 1 PRA NUSAFE 2
Process step Freq. Rank Freq. Rank Freq. Rank
Planning
Receive order 168 2 383 1202 1
Code order 69 291 217
Plan examination or treatment 247 1 584 5 333
Inform or instruct patient 61 180 184
Execution
Refer patient to waiting room 42 160 175
Prepare examination or treatment 90 5 95 100
Call patient and check patient data 69 2673 1 120
Select protocol and equipment 71 48 171
Carry out examination or treatment 121 3 861 4 348 3
Assess, edit and provide images 21 1814 2 50
Execution – other process steps 99 4 0 258
Reporting
Type report 10 1012 3 350 2
Archiving
Correct report 5 0 337 4.5
Send report and hardcopy 19 6 220
Archiving – other process steps 4 0 337 4.5
Other 73 570 354
Total 1169 8677 4756
Note. Frequencies (freq.) have been corrected for the study period of four months. Rankings
(rank) are only presented for the five highest risk process steps; all other cells are left empty.
Ties have been assigned the average value of the associated ranks (Siegel & Castellan, 1988).
RIA = Retrospective Incident reporting and Analysis. PRA NUSAFE 1 = Prospective Risk
Analysis without information from the retrospective incidents database. PRA NUSAFE 2 =
Prospective Risk Analysis with information from the retrospective incidents database.
Chapter 3
42
Evaluation Forms
The evaluation form of the entire project (Form 2) revealed that 33 respondents (97.1%)
agreed that incident reporting and analysis was useful for improving patient safety and
optimising processes. Also, most respondents felt that prospective analysis was useful for
improving patient safety (n = 26; 76.5%) and optimising processes (n = 27; 79.4%).
Furthermore, prospective and retrospective analyses provided insight into new risks
according to 16 (47.1%) and 20 (58.8%) respondents, respectively.
Form 2 also showed that 16 respondents (47.1%) thought it was the combination of
prospective and retrospective analyses that provided most insight into risks. Others felt it was
either the prospective (n = 3; 8.8%) or retrospective (n = 7; 20.6%) analysis that yielded most
insight. In those numbers, the participants in the prospective analyses are included, but they
also answered this question on Form 1. Interestingly, on Form 1 a much higher percentage of
the respondents (n = 14; 73.7%) thought it was the combination of prospective and
retrospective analyses that provided most insight into risks.
Regarding the integration of the methods, ten participants in the prospective analyses
(52.6%) felt that information about incidents and their frequencies was or would have been
useful; information about causes of incidents was or would have been useful according to 11
participants (57.9%). Form 1 also revealed that 7 participants (50%, excluding management)
were more willing to report incidents after participation in the prospective analysis. This
could imply that participation in a prospective analysis could enhance incident reporting
behaviour. For both units, follow-up chi-square tests indicated that, after the start of the
prospective analyses, participants reported other incident types than non-participants in terms
of the sub processes that contributed to the occurrence of the reported incidents (p < .01 and p
< .05, respectively). This endorses the assumption that participation in a prospective analysis
is positively associated with incident reporting behaviour.
3.3 Discussion
In this study, we examined how prospective and retrospective methods for risk analysis could
be integrated and whether this integration is perceived to be useful. Our findings show that
both methods are considered valuable in terms of improving patient safety and optimising
processes. Our study supports earlier findings that prospective and retrospective analyses are
partly complementary because both can yield divergent overviews of risks in terms of nature
and magnitude (Runciman et al., 2006; Senders, 2004). Hence, our study empirically
endorses the theoretical contention that thanks to convergent evidence, triangulation of the
Integration of Prospective and Retrospective Methods for Risk Analysis
43
methods can provide hospital management and frontline staff with a more complete and less
biased picture of risks (Battles & Lilford, 2003; Herzer et al., 2008; Runciman et al., 2006;
Senders, 2004).
Provided that risks are categorised similarly, integration of prospective and
retrospective methods enables direct comparison of the analysis results. Then, follow-up
research could reveal biases, whereby the methods could be further improved (Aspden et al.,
2004). Moreover, integration might limit the additional resources that could be required due
to the application of two methods instead of just one. As we proposed, information about
incidents and their retrospectively reported frequencies could be used as a reference point in
prospective analyses, which might facilitate frontline staff in risk assessment. Conversely,
prospectively developed failure scenarios could be used as guideline for retrospective
incident analyses. Such an approach is consistent with control theory paradigms, which state
that unreliable models and analyses need feedback (Berden, Brombacher, & Sander, 2000).
However, such integration might mainly be advantageous for frontline staff members who
actively participate in both analyses, and the benefits might be less visible for non-
participants. Besides the likely consequential increase in efficiency of analysis, integration of
the methods could also support hospital management in making sense of risks and justifying
their decisions regarding interventions (Battles et al., 2006).
Our study has several limitations. We did not test all possibilities for integration.
However, we purposely selected those possibilities that could be easily applied by hospitals
themselves to gain a better picture of risks. In future studies, more possibilities could be
tested and one could establish whether integration actually increases efficiency of analysis.
Moreover, future research could aim to develop a ―golden standard‖ to assess the actual
validity of prospective and retrospective methods, for instance by means of direct
observation.
The results of our retrospective analyses might have been affected by hindsight bias;
that is, the tendency for people to overstate the extent to which they would have predicted
events beforehand (Henriksen & Kaplan, 2003). We have tried to limit this by analysing
incidents as soon as possible after they had been reported and by interviewing the people
involved (Carthey, De Leval, & Reason, 2001).
The perceived usefulness of the integration of prospective and retrospective methods
could be influenced by respondents logically tending to evaluate the triangulation better than
the application of only one method; conversely, respondents could tend to evaluate the
triangulation negatively because of the extra effort required. Since the former positively
Chapter 3
44
affects the perceived usefulness, while the latter negatively affects it, future studies could
examine whether the perceived benefits of combining the methods actually outweigh the
perceived drawbacks.
Similar studies should be carried out in other health care settings to assess the external
validity of our results. However, independent-samples t tests and ANOVA did not reveal any
significant differences between the two units or the four prospective analysis teams, which
confirms our findings. Further, our results could suggest that participation in a prospective
analysis positively influences health care employees‘ willingness to report incidents.
Therefore, future studies could focus on the effects of participation in and taking notice of a
prospective analysis on incident reporting behaviour, thereby contributing to the vastly
growing literature on barriers to incident reporting (see Chapter 4).
In conclusion, notwithstanding the fact that either prospective or retrospective
methods can be used to improve patient safety, hospital management should seriously
consider their integration. Such an integrative approach might increase efficiency of analysis
and can yield a better picture of risks, which could support hospital management in setting
priorities for patient safety and allocating resources to the most important problems.
Moreover, integration of the methods could bring about advances in safety research by
improving the methods themselves. Together, such progress in theory and practice could
make health care safer and reduce patient harm accordingly.
45
Chapter 4
Prospective Risk Analysis Prior to
Retrospective Incident Reporting and Analysis
as a Means to Enhance Incident Reporting Behaviour:
A Quasi-experimental Field Study*
This chapter questions whether the order of implementation of prospective and
retrospective methods for risk analysis influences the resultant impact on incident
reporting behaviour. Twelve units of two Dutch general hospitals participated in a
quasi-experimental reversed-treatment non-equivalent control group design. The six
units of Hospital 1 first conducted a prospective analysis, after which a sophisticated
incident reporting and analysis system was implemented. On the six units of Hospital
2, the two methods were implemented in reverse order. The results revealed that
carrying out a prospective analysis first can yield a wider spectrum of reported
incident types and a larger proportion of incidents reported by doctors. This order of
implementation could enable hospitals to advance on the cultural pathway.
Nowadays, harm caused by health care itself instead of an injury or disease (i.e. iatrogenic
harm) is one of the main causes of death. Worldwide, more people die as a consequence of
—————————————
*This chapter is largely based on: Kessels-Habraken, M., De Jonge, J., Van der Schaaf, T., & Rutte, C. (2009).
Prospective risk analysis prior to retrospective incident reporting and analysis as a means to enhance incident
reporting behaviour: A quasi-experimental field study. Manuscript under revision.
Chapter 4
46
medical errors in acute care than of road traffic accidents or natural disasters, such as
earthquakes or tsunamis (Runciman et al., 2007). This alarming fact necessitates hospitals to
identify risks and implement effective interventions, so-called safety management
programmes. In this context, hospitals can use retrospective and/or prospective methods to
improve patient safety. Retrospective methods, such as record review and incident reporting,
are used to identify medical errors. Subsequent causal analysis can reveal systematic
problems and facilitate learning. Next, measures could be taken to prevent recurrence of the
errors. In contrast to retrospective methods, prospective methods aim to determine and assess
risks before incidents may occur. In a prospective analysis, a multidisciplinary team lists and
prioritises potential risks in a health care process. Ultimately, the team describes actions to
eliminate or reduce the risks, thereby preventing patient harm more proactively. Since the
vision of safety management efforts should be to achieve zero patient harm, and their
objective is to at least minimise patient harm (Battles & Lilford, 2003), both prospective and
retrospective approaches are necessary (Hollnagel, 2008). Moreover, since both methods
have advantages and disadvantages, triangulation can result in a better picture of risks
(Battles & Lilford, 2003; Herzer et al., 2008; Runciman et al., 2006; Senders, 2004). Such
analytical insight could support hospital management in prioritising patient safety
interventions (Battles et al., 2006; Hogan et al., 2008).
Besides this analytical pathway, hospitals can also make progress on the, more
indirect, cultural pathway to improve patient safety, for instance by enhancing incident
reporting behaviour. Each time health care employees decide to report incidents and receive
feedback, it might positively change their risk perceptions, their attitudes towards safety, and
ultimately their behaviour as well (Aspden et al., 2004; Kaplan & Barach, 2002; Pronovost et
al., 2007). However, the majority of the hospitals seem to fail to learn from errors due to
limited error recognition and analysis (Cannon & Edmondson, 2005). Generally, incident
reporting behaviour in hospitals often leaves much to be improved (Hudson, 2003). Far too
many medical errors go unreported (Aspden et al., 2004; Barach & Small, 2000; Evans et al.,
2006). Further, health care employees habitually report particular types of incidents, like
those with serious consequences (Hogan et al., 2008; Ligi et al., 2008; Moss, Embleton, &
Fenton, 2005) or incidents without a direct relation with staff action, like falls (Hogan et al.,
2008). While falls and certain medication errors seem to be over reported, other types of
incidents appear to be underreported, such as those related to clinical treatment (Evans et al.,
2006; Nuckols, Bell, Liu, Paddock, & Hilborne, 2007; Olsen et al., 2007). Additionally,
doctors are less willing to disclose errors than nurses are (Johnson, 2003; Kingston et al.,
Prospective Risk Analysis to Enhance Incident Reporting Behaviour
47
2004; Shojania, 2008). Research has revealed a number of reasons for those problems, such
as lack of error recognition, feelings of fear or shame, doctors‘ attitudes of errors being
unavoidable and their inclination to keep errors in-house, unfamiliarity with the incident
reporting system and analysis process, lack of feedback and follow-up, and time pressure
(Evans et al., 2006; Holden & Karsch, 2007; Johnson, 2003; Kingston et al., 2004; Shojania,
2008; Waring, 2005).
Prompted by regulations (Devers et al., 2004) and the safety objective of preventing
patient harm, hospitals recognise the need for proactive safety management. However, a lack
of financial and nonfinancial resources, like staff, might hinder hospitals from implementing
the necessary elements of a safety management system simultaneously (Akins & Cole, 2005;
Devers et al., 2004). Unfortunately, little is known about the optimal order in which
prospective and retrospective methods should be implemented (Hale, 2003). To our
knowledge, no research has concentrated on the question of whether the order of conducting
a prospective analysis and implementing a sophisticated incident reporting and analysis
system influences the resultant impact on incident reporting behaviour.
Apparently, a sophisticated incident reporting and analysis system can improve
incident reporting behaviour because of clear definitions, limited time needed to fill out the
reporting form, short feedback loops, and clearly visible improvement efforts (Aspden et al.,
2004; Shojania, 2008). Nevertheless, retrospective analyses are probably still perceived as
more threatening than prospective ones. After a health care employee has reported an actual
error that might have produced patient harm, he or she is confronted with questions about
what has happened and what has caused the error. This might cause feelings of
embarrassment or fear, which impedes openness and limits learning (Cannon & Edmondson,
2005). On the other hand, prospective analyses, such as Healthcare Failure Mode and Effect
Analysis (HFMEA™), are less threatening (Senders, 2004), thanks to open and active
multidisciplinary discussions about possible risks. A process model, which is the starting
point for the prospective analysis, provides insight into other health care employees‘ tasks
(Habraken et al., 2009, see also Chapter 2) and might increase health care employees‘
abilities to identify errors (Pronovost et al., 2007). The multidisciplinary discussions could
create a shared vision (Bonnabry et al., 2006) and growing understanding of potential risks
(Battles et al., 2006). This enlarged understanding might enhance error recognition through
increased alertness and vigilance (Kontogiannis & Malakis, 2009). Moreover, the open and
positive atmosphere might remove specific social barriers for incident reporting, such as
shame or fear (Cannon & Edmondson, 2005).
Chapter 4
48
Together, the facts that many errors go unreported, that reports do not cover the full
spectrum of incident types, and that particularly doctors are reluctant to disclose errors,
indicate that incident reporting in hospitals is still in its infancy. Because a prospective
analysis might enhance error recognition and remove social barriers for incident reporting,
one might assume that it is advantageous to conduct a prospective analysis before the
introduction of a sophisticated incident reporting and analysis system. On the basis of this
assumption, we formulated a first hypothesis:
Hypothesis 1: If a prospective risk analysis is carried out prior to, instead of after, the
implementation of a sophisticated retrospective incident reporting and analysis
system, the resultant positive impact on incident reporting behaviour will be enlarged
in terms of:
a. the number of reported incidents;
b. the spectrum of reported incident types;
c. the proportion of incidents reported by doctors.
Practically speaking, this hypothesis is only valuable for those hospitals that have not
yet implemented a sophisticated incident reporting and analysis system. Although this holds
true for many hospitals, several hospitals are already using a sophisticated incident reporting
and analysis system that promotes learning. Since those hospitals do not start from scratch, it
is also interesting to explore whether a prospective analysis could be used to boost existing
incident reporting behaviour. Therefore, we formulated a second hypothesis:
Hypothesis 2: Conducting a prospective risk analysis has a positive effect on existing
incident reporting behaviour in terms of:
a. the number of reported incidents;
b. the spectrum of reported incident types;
c. the proportion of incidents reported by doctors.
Because advances in incident reporting increase hospitals‘ possibilities to learn from
errors, it would be valuable if the anticipated positive effect on incident reporting behaviour
not only holds true for the participants of the prospective analysis but also for their direct
colleagues. Moreover, because carrying out a prospective analysis such as HFMEA™ takes a
lot of time (Habraken et al., 2009, see also Chapter 2; Linkin et al., 2005), hospital
Prospective Risk Analysis to Enhance Incident Reporting Behaviour
49
management probably will not even allow all health care employees to participate in a
prospective analysis. Theories about social contagion support the diffusion of beliefs and
perceptions among individuals. According to the network theory of social contagion,
individuals adopt attitudes and behaviours from others, just by communicating with them; an
intention to influence is unnecessary (Scherer & Cho, 2003). Research has shown that this
theory can explain the creation of risk perceptions within social networks (Scherer & Cho,
2003). More specifically, in a social network, such as a nursing ward, individuals
communicate about their own risk perceptions with their colleagues. Beliefs about error and
risk are thus shared in groups, enabling organisational learning to take place (Cannon &
Edmondson, 2001; Edmondson, 2004). Consequently, if participation in a prospective
analysis would actually change participants‘ risk perceptions and incident reporting
behaviour, mere communication with colleagues might bring about dissemination. On the
basis of this assumption, we formulated a final hypothesis:
Hypothesis 3: A positive effect of conducting a prospective risk analysis on incident
reporting behaviour holds true both for participants and non-participants, provided
that the latter are informed about the results of the analysis.
4.1 Methods
Setting
A quasi-experimental study was carried out in two Dutch general hospitals, both belonging to
the same health care foundation. At the start of the study, both hospitals used a simple
procedure for reporting (major) incidents. However, both hospitals had not yet implemented a
sophisticated incident reporting and analysis system that facilitates learning, nor had they
applied prospective analyses at unit level. Twelve units were included in the study. The units
represented a diverse range of specialties, inpatient and outpatient settings, and acute and
non-acute care: two internal medicine and two obstetrics nursing wards, two clinical chemical
laboratories, two operating rooms, two policlinics of pulmonary diseases, and two surgery
policlinics. The units were divided into two non-equivalent groups, representing the two
hospitals. The six units of Hospital 1 matched the six units of Hospital 2.
Design
In this field study, a quasi-experimental reversed-treatment non-equivalent control group
design was used (Cook & Campbell, 1979). The six units of Hospital 1 first conducted a
Chapter 4
50
prospective analysis, after which a sophisticated incident reporting and analysis system was
implemented. On the six units of Hospital 2 the two methods were implemented in reverse
order (see Figure 4.1). In the remainder of this chapter, Hospital 1 will be referred to as
―prospective first‖, while Hospital 2 will be labelled ―retrospective first‖. Further, the period
before the implementation of the sophisticated incident reporting and analysis system is
referred to as ―Period 1‖, while the period after its introduction will be labelled ―Period 2‖.
Finally, for Hospital 2, Period 2 is subdivided. The period before the prospective analyses is
referred to as ―Period 2a‖; the period during and after the prospective analyses is referred to
as ―Period 2b‖ (see Figure 4.1). In the new incident reporting and analysis system, all
employees can report incidents electronically. Special unit-based committees analyse the
reported incidents regarding their causes. The committee members were trained beforehand.
Moreover, all units were offered a meeting to discuss the importance of incident reporting
and to inform the employees about the new system. For the prospective analysis, each
participating unit selected a process to be investigated. Then, a multidisciplinary team was
composed, which conducted an adapted version of HFMEA™ (Habraken et al., 2009, see
also Chapter 2). It consisted of the identification of risks in the selected process, assessment
of those risks in terms of severity of consequences and frequency of occurrence,
identification of their causes, and description of actions to eliminate or reduce them. The
HFMEA™ decision tree was omitted.
To test the hypotheses, we used data from the incident reporting and analysis system
and from evaluation forms. For each unit, data had been extracted from the incident reporting
and analysis system about the number of reported incidents, the spectrum of reported incident
types, and the profession of reporters. The number of reported incidents was averaged per
month and corrected for the number of employees per unit (excluding doctors). The spectrum
of reported incident types consisted of six categories, which were defined by a special study
group: ―equipment / materials / devices / ICT‖, ―blood / medication / nutrition‖, ―examination
/ treatment‖, ―organisation / communication / documentation‖, ―falls‖, and ―other‖. When
filling out the reporting form, the reporters had assigned the incident to one or more of those
categories. Subsequently, the unit-based committees verified, and if necessary corrected, this
classification. Regarding the profession of reporters, the proportion of incidents reported by
doctors was calculated, including both medical specialists and junior doctors. Incidents that
had been reported anonymously were excluded.
Retrospective incident reporting and analysis system
Prospective
risk analysis
Month →
Hospital 1 (prospective first) →
Hospital 2 (retrospective first) →
02≤01 03 04 05 06 07 08 09
Retrospective incident reporting and analysis system
Prospective
risk analysis
(6 units)
(6 units)
Period 1
Period 2bPeriod 2a
Period 1 Period 2
Period 2
Figure 4.1: Study design: A quasi-experimental reversed-treatment non-equivalent control group design.
Chapter 4
52
A self-developed evaluation form was used to assess self-reported changes in incident
reporting behaviour by means of five items such as ―since the implementation of the new
reporting system, I am more willing to report incidents‖ and ―since the implementation of the
new reporting system, I have started to report other incident types, too‖. Five-point rating
scales ranged from (1) ―agree strongly‖ to (5) ―disagree strongly‖. The form was distributed
among all employees involved in the processes that had been selected for the prospective
analyses of the 12 units. It was distributed at least three months after both interventions had
been implemented; 199 evaluation forms were completed and returned (84 from Hospital 1;
115 from Hospital 2). Due to a lack of reliable information about the precise number of
employees having received an evaluation form, an exact response rate could not be
calculated. The fact that the forms were not distributed by the researchers but by the unit
managers explains this lack of information. However, the average response rate was
estimated to be 61% (range 34% - 88%), based on the maximum number of employees that
could have received a form.
Data Analysis
A Wilcoxon signed ranks test was conducted to evaluate whether the newly implemented
sophisticated incident reporting and analysis system resulted in a larger number of reported
incidents. We used a Mann-Whitney U test to assess between-hospital differences regarding
the number of reported incidents in Periods 1 and 2 (Hypothesis 1a). Similarly, chi-square
tests were applied to evaluate between-hospital differences regarding the spectrum of
reported incident types and the proportions of incidents reported by doctors (Hypotheses 1b
and 1c, respectively). Independent-samples t tests were used to see whether the two hospitals
differed regarding self-reported changes in incident reporting behaviour (Hypotheses 1a and
1b). For Hospital 2, we used a Wilcoxon signed ranks test and chi-square tests to assess
differences between Periods 2a and 2b regarding the number of reported incidents, the
spectrum of reported incident types, and the proportion of incidents reported by doctors
(Hypotheses 2a, 2b, and 2c, respectively). For all analyses with 2 x 2 contingency tables, we
used χ2
corrected for continuity since N > 40 (Siegel & Castellan, 1988). For Hospital 1,
independent-samples t tests were used to find out whether non-participants who had been
informed about the results of their unit‘s prospective analysis, differed in self-reported
changes in incident reporting behaviour from non-participants who had not been notified
(Hypothesis 3). For all statistical analyses, an alpha level of .05 was used.
Prospective Risk Analysis to Enhance Incident Reporting Behaviour
53
4.2 Results
When compared to the old procedure for incident reporting (Period 1), the sophisticated
incident reporting and analysis system (Period 2) resulted in a significant increase of the
overall average number of reported incidents per month per employee from 0.04 to 0.19 (see
Table 4.1), that is, a 400% increase (z = -2.90, p < .01). In some units, like the surgery
policlinic in Hospital 2, the number of reported incidents increased tremendously.
Table 4.1
Average number of incidents reported by unit per month per employee for Period 1(baseline)
and Period 2 (study period).
Unit Hospitala
Period 1b
Period 2c
Clinical chemical laboratory 1 0.002 0.22
2 0.003 0.05
Internal medicine nursing ward 1 0.04 0.06
2 0.15 1.03
Obstetrics nursing ward 1 0.02 0.09
2 0.01 0.07
Operating room 1 0.14 0.29
2 0.04 0.03
Policlinic of pulmonary diseases 1 0 0.03
2 0.002 0.01
Policlinic of surgery 1 0.04 0.10
2 0.001 0.30
M 0.04 0.19
aHospital 1 = Prospective first; Hospital 2 = Retrospective first.
bAverage per month,
corrected for the number of employees per unit (excluding doctors). Averages were
calculated on the basis of the figures of the last 38 months (Hospital 1) or 37 months
(Hospital 2) prior to the implementation of the sophisticated incident reporting and analysis
system. cAverage per month, corrected for the number of employees per unit (excluding
doctors). Averages were calculated on the basis of the figures of the first seven months
(Hospital 1) or eight months (Hospital 2) since the implementation of the sophisticated
incident reporting and analysis system.
Number of Reported Incidents
A Mann-Whitney U test showed no significant between-hospital differences regarding the
number of reported incidents in Period 1 (z = -0.40, p = .69) and Period 2 (z = -0.32, p = .75).
However, respondents from Hospital 1 (prospective first) (M = 2.60, SD = 1.09) agreed
significantly more often with the statement that since the introduction of the new reporting
Chapter 4
54
system, they were more willing to report incidents than respondents from Hospital 2
(retrospective first) (M = 3.03, SD = 1.19), t(175) = 2.49, p < .05 (see Table 4.2). Despite this
significant result, the nonsignificant result of the objective measure prevents us from
confirming Hypothesis 1a. Conducting a prospective analysis before implementing a
sophisticated incident reporting and analysis system did not necessarily result in a larger
number of reported incidents.
Spectrum of Reported Incident Types
To test whether the proposed order of implementing prospective and retrospective methods
does positively influence the spectrum of reported incident types (Hypothesis 1b), we
compared the two hospitals regarding their distributions of reported incidents across six
incident types (see Table 4.3). Chi-square tests showed significant results for both Period 1,
χ2(3, N = 727) = 60.50, p < .001 and Period 2, χ
2(5, N = 678) = 23.24, p < .001. Although in
Period 1 the two hospitals differed regarding the spectrum of reported incident types,
employees from both hospitals routinely reported certain types of incidents, while other types
were hardly reported, such as incidents related to equipment and materials, or problems
related to organisation, communication, and documentation. In Period 1 the two hospitals
were thus fairly similar with regard to incident reporting behaviour. Conversely, in Period 2
in Hospital 1 (prospective first) the reported incidents were more evenly distributed over the
categories of incident types than in Hospital 2 (retrospective first). Moreover, respondents
from Hospital 1 (prospective first) (M = 2.88, SD = 1.27) reported significantly more often
than respondents from Hospital 2 (retrospective first) (M = 3.28, SD = 1.17) that since the
introduction of the new reporting system they had started to report other incident types, too,
t(171) = 2.12, p < .05 (see Table 4.2). Based on those results Hypothesis 1b is supported.
Table 4.2
Means (Standard Deviations) of items included in the evaluation form (N =199).
Hospital 1 Hospital 2
Item M (SD) M (SD) p
Since the implementation of the new reporting system, I am more willing to report incidents 2.60 (1.09) 3.03 (1.19) .01*
Since the implementation of the new reporting system, I have started to report other incident types, too 2.88 (1.27) 3.28 (1.17) .04*
Since the prospective analysis, I am more willing to report incidentsa
2.73 (1.01) 2.83 (1.03) .76
Since the prospective analysis, I have started to report other incident types (too)a
3.16 (1.11) 3.75 (0.87) .10
Thanks to the prospective analysis, I have obtained insight into new risksa
2.27 (1.01) 2.42 (1.08) .68
Note. Hospital 1 = Prospective first (n = 84). Hospital 2 = Retrospective first (n = 115). aItems were filled out only by health care employees that had been informed about the results of the prospective analysis.
*p < .05, independent-samples t test.
Chapter 4
56
Table 4.3
Percentage of reported incidents per incident type and profession of reporter for Hospital 1
and Hospital 2.
Period 1a
Period 2b
Hosp. 1
Hosp. 2
Hosp. 1
Hosp. 2
Incident typec
Equipment / materials / devices / ICT 15.7% 7.0%
Blood / medication / nutrition 25.6% 33.2% 25.3% 36.6%
Examination / treatment 44.0% 20.7% 14.6% 12.8%
Organisation / communication / documentation 20.8% 26.2%
Falls 11.3% 11.3% 3.9% 5.2%
Other 19.1% 19.1% 19.7% 12.2%
Profession of reporterd
Doctors 17.4% 16.2% 17.6% 4.1%
Other professions 82.6% 83.8% 82.4% 95.9%
Note. Empty cells indicate the incident type was not a distinct category. Hospital (hosp.) 1 =
Prospective first. Hospital 2 (hosp.) = Retrospective first. aPercentages were calculated on the basis of the figures of the last 38 months (Hospital 1) or
37 months (Hospital 2) prior to the implementation of the sophisticated incident reporting
and analysis system. bPercentages were calculated on the basis of the figures of the first seven
months (Hospital 1) or eight months (Hospital 2) since the implementation of the
sophisticated incident reporting and analysis system. cSignificant between-hospital
differences for Period 1 (baseline) and Period 2 (study period), χ2, α = .05.
dNo significant
between-hospital difference for Period 1 (baseline); significant between-hospital difference
for Period 2 (study period), χ2, α = .05.
Proportion of Incidents Reported by Doctors
A chi-square test was conducted to explore whether the implementation of a sophisticated
incident reporting and analysis system that is preceded by a prospective analysis would result
in an increase in the proportion of incidents reported by doctors (Hypothesis 1c). In Period 1
no significant between-hospital difference was identified, χ2(1, N = 720) = 0.12, p = .73,
whereas in Period 2, the test result was significant, χ2(1, N = 606) = 27.38, p < .001. In Period
2, the proportion of incidents reported by doctors was much larger in Hospital 1 (prospective
first) (P = .18) than in Hospital 2 (retrospective first) (P = .04) (see Table 4.3). Therefore,
Hypothesis 1c is confirmed. It should be noted that for both hospitals the proportions of
incidents reported by doctors in Period 2 were equal to or even lower than those in Period 1,
yet the absolute numbers have risen significantly.
Prospective Risk Analysis to Enhance Incident Reporting Behaviour
57
Effects of Prospective Risk Analysis on Existing Incident Reporting Behaviour
For Hospital 2, a Wilcoxon signed ranks test was used to evaluate differences in the number
of reported incidents between the period before (Period 2a) and the period during and after
the prospective analyses (Period 2b). The test showed a nonsignificant result (z = -1.57, p =
.12). Accordingly, Hypothesis 2a is rejected. On the other hand, a chi-square test supports
Hypothesis 2b. The distribution of the reported incidents over six incident types in Period 2a
significantly differed from the distribution in Period 2b, χ2(5, N = 500) = 26.31, p < .001.
Moreover, the differences mirror the topics of the prospective analyses. For instance, two of
the six analyses particularly concentrated on medical examination and treatment (such as the
admission, diagnosis, and treatment of a patient with pneumonia on an internal medicine
nursing ward). This might be reflected by the percentage increase of reported incidents
classified as ―examination / treatment‖. In addition, the proportion of incidents reported by
doctors had increased significantly since the start of the prospective analysis (Period 2b) (P =
.09), compared to the months before (Period 2a) (P = .01), χ2(1, N = 458) = 13.05, p < .001.
Therefore, Hypothesis 2c is also confirmed. In sum, carrying out a prospective analysis (as
part of a safety management programme that also consists of a sophisticated incident
reporting and analysis system) positively influenced existing incident reporting behaviour in
terms of the spectrum of reported incident types and the proportion of incidents reported by
doctors, whereas it did not influence the number of reported incidents. Interestingly, on the
evaluation form respondents from both hospitals did not really agree with the statements that
since the prospective analysis, they were more willing to report incidents (M = 2.73, SD =
1.01 and M = 2.83, SD = 1.03) and they had started to report other incident types, too (M =
3.16, SD = 1.11 and M = 3.75, SD = 0.87). However, respondents from both hospitals did
report that the prospective analysis provided them with insight into new risks (M = 2.27, SD
= 1.01 and M = 2.42, SD = 1.08) (see Table 4.2).
Social Contagion
For Hospital 1, we evaluated whether the positive effects of conducting a prospective analysis
on incident reporting behaviour also held true for employees who had not participated in the
analysis, but who had been notified about its results. The test results support Hypothesis 3
and are in the expected direction, t(67.17) = -5.26, p < .001 and t(66) = -2.34, p < .05. Non-
participants who had been notified reported significantly more often that since the
implementation of the new reporting system, they were more willing to report incidents (M =
1.92, SD = 0.56), and had started to report other incident types, too (M = 2.38, SD = 1.21),
Chapter 4
58
than non-participants who had not been informed (M = 2.95, SD = 1.08 and M = 3.11, SD =
1.26, respectively).
4.3 Discussion
According to the safety objective of reducing patient harm, both prospective analysis and
retrospective incident reporting and analysis are necessary to uncover risks and to raise risk
awareness. This chapter dealt with the question of whether the order of implementation of
those two methods influences the resultant impact on incident reporting behaviour. Three
hypotheses were formulated accordingly. We hypothesised that if a prospective analysis is
conducted prior to, instead of after, the implementation of a sophisticated incident reporting
and analysis system, the resultant positive impact on incident reporting behaviour will be
enlarged (Hypothesis 1) and that conducting a prospective analysis has a positive effect on
existing incident reporting behaviour (Hypothesis 2). Further, we formulated the hypothesis
that a positive effect of a prospective analysis on incident reporting behaviour holds true both
for participants and non-participants, provided that the latter are informed about the results of
the analysis (Hypothesis 3).
Theoretical Implications
Our quasi-experiment fills an important gap in safety management research, that is, the order
of implementation of prospective and retrospective methods, as indicated by Hale (2003). To
our knowledge, this is the first study that has examined this order issue. Moreover, this study
contributes to the vastly growing literature on incident reporting. Equal to earlier findings
(Evans et al., 2007), our study has demonstrated that it is possible to expand the range of
reported incidents. Our results indicate that conducting a prospective analysis before the
introduction of a sophisticated incident reporting and analysis system can enhance incident
reporting behaviour in terms of a wider spectrum of reported incident types and a larger
proportion of incidents reported by doctors. Most likely, those effects are interrelated. Since
incidents reported by doctors and those reported by other professions are complementary,
improved rates of error disclosure by doctors will probably result in a more diverse range of
reported incidents (Evans et al., 2006; Ligi et al., 2008; Nuckols et al., 2007).
Apparently, a prospective analysis could improve health care employees‘
understanding of possible risks (Battles et al., 2006). Such improvements in risk awareness
could, in turn, enhance error recognition, and might even stimulate recovery from errors
(Kontogiannis & Malakis, 2009). The fact that the improvements regarding the spectrum of
Prospective Risk Analysis to Enhance Incident Reporting Behaviour
59
reported incidents particularly reflect the topics of the prospective analyses might endorse the
assumption that a prospective analysis produces a change in risk perceptions. For instance, in
Hospital 1 employees started to report incidents regarding medical equipment and devices
after the start of the prospective analyses which partly concentrated on technical failures,
while such incidents had hardly been reported in the past. Besides the progress related to
error recognition, the seemingly open and non-threatening atmosphere in which risks are
discussed in a prospective analysis could remove certain social barriers for incident reporting
such as shame or fear (Cannon & Edmondson, 2005). Together, those advances in error
recognition and incident reporting facilitate learning (Cannon & Edmondson, 2005).
Our results did not indicate any positive influence of a prospective analysis (as part of
a safety management programme that also consists of a sophisticated incident reporting and
analysis system) on the number of reported incidents. This might suggest that, despite their
increased willingness to report incidents (as a result of the extensive safety management
programme, see also Chapter 7), employees had reached their saturation point regarding
incident reporting efforts. Time pressure might limit the number of incidents that can be
reported (Akins & Cole, 2005; Evans et al., 2007). Besides, the acquired insight and changes
in risk perceptions, which can be attributed to carrying out a prospective analysis, could make
health care employees decide to report other incident types at the expense of incidents that
used to be reported. This would reflect earlier findings that health care employees are
reluctant to report known problems because of a lack of learning opportunities (Van der
Schaaf & Kanse, 2004) and that especially new incidents require extensive reporting (Hale,
2003). Moreover, the emphasis on the spectrum as opposed to the number of reported
incidents is in line with the idea that incident reporting is useful for risk identification and not
for determining exact error rates (Battles & Lilford, 2003; Helmreich, 2000; Pronovost et al.,
2007).
Our findings also support the basic principle of learning through sharing perceptions
and beliefs in groups (Cannon & Edmondson, 2001; Edmondson, 2004). Evidently, the
positive effects of conducting a prospective analysis prior to the implementation of a
sophisticated incident reporting and analysis system not only held true for those employees
that participated in the analysis, but also for those direct colleagues that had been notified
about its outcomes. Building on the network theory of social contagion (Scherer & Cho,
2003) our results might indicate that health care employees can disseminate their own risk
perceptions by mere communication with others.
Chapter 4
60
Practical Implications
Assuming that both prospective and retrospective approaches are required to improve patient
safety proactively (Hollnagel, 2008), the introduction of a sophisticated incident reporting
and analysis system can well be preceded by a prospective analysis to increase the resultant
positive impact on incident reporting behaviour. Besides, in case of an existing sophisticated
incident reporting and analysis system, hospitals can use a prospective analysis to boost
present incident reporting behaviour. If, for instance, a certain incident type seems to be
underreported, conducting a prospective analysis that concentrates on that specific topic will
probably bring about such changes in risk perceptions and awareness that, from then on, that
particular incident type will be included in the spectrum of reported incident types. This is
important since a diverse range of reported incident types is essential for risk identification
and learning (Evans et al., 2007). Moreover, carrying out a prospective analysis with doctors
as participants could enhance their incident reporting behaviour, which is necessary to cover
the full spectrum of incident types (Evans et al., 2006; Ligi et al., 2008; Nuckols et al., 2007).
The established contagion effect could enable hospitals to take full advantage of the
positive influence of a prospective analysis on incident reporting behaviour. Participants
could talk about the analysis itself and its outcomes with their colleagues to distribute their
new insights. The possibilities for learning facilitated by such dissemination of perceptions
and beliefs might outweigh the time investment that is required to conduct a prospective
analysis.
The fact that during the study period the number of reported incidents per month per
employee significantly increased compared to the old procedure for incident reporting might
indicate that employees started to report all minor deviations or even frustrations. This could
also explain the explosive rise in the number of reported incidents for the surgery policlinic in
Hospital 2 and other units. Though valuable, such a large number of reported incidents might
cause problems due to a lack of resources available for incident analysis. Hence, selection of
incidents that are eligible for causal analysis could be necessary (Aspden et al., 2004; Van der
Schaaf & Wright, 2005).
Study Strengths, Limitations, and Future Research
Despite the strengths of this study, that is, its quasi-experimental design and the simultaneous
use of observations and evaluation forms, our study has several limitations. First, history
might be a threat to internal validity (Cook & Campbell, 1979). We did not take into account
whether the units organised a discussion meeting about the importance of incident reporting
Prospective Risk Analysis to Enhance Incident Reporting Behaviour
61
and if so, whether employees attended this meeting. In the six units of Hospital 1 (prospective
first) those meetings were all organised after the implementation of the sophisticated incident
reporting and analysis system, while in the six units of Hospital 2 (retrospective first) those
meetings were all organised before or at the very start of its introduction. Because it can be
assumed that the discussion meetings certainly did not negatively influence incident reporting
behaviour, the between-hospital differences would probably only have been greater if we had
considered those meetings. Second, the number of units participating in this study was
limited. Nevertheless, our results could also hold true for other hospital units and health care
settings since we purposely selected units that together represented a diverse range of
specialties and settings. On the other hand, due to cultural differences between health care
and other industries (Hudson, 2003), future studies should further explore the external
validity of our findings.
Although our results contribute to the literature on incident reporting, future research
could focus on the exact relation between conducting a prospective analysis and incident
reporting behaviour. This is especially important because in our study employees themselves
did not really attribute the positive changes in incident reporting behaviour to the prospective
analysis. For instance, one could explore whether carrying out a prospective analysis
particularly enhances error recognition or whether it primarily removes social barriers for
incident reporting. Furthermore, a diary study could concentrate on the assumption that time
pressure makes health care employees decide not to report more incidents, but other incident
types instead. To conclude, future studies could explore what kinds of communication enable
dissemination of risk perceptions within social networks such as hospital units.
Conclusions
Conducting a prospective risk analysis prior to the implementation of a sophisticated incident
reporting and analysis system can enhance incident reporting behaviour in terms of a wider
spectrum of reported incident types and a larger proportion of incidents reported by doctors.
This proposed order of implementation could enable hospitals to advance on the cultural
pathway through changed risk perceptions and dissemination of those perceptions through
social contagion. Together with the progress on the analytical pathway resulting from the use
of both prospective and retrospective approaches, this provides hospitals with a set of
instruments to actually improve patient safety proactively.
62
63
Chapter 5
Defining Near Misses:
Towards a Sharpened Definition
Based on Empirical Data*
This chapter establishes the need for a clearer and more consistent definition of near
misses to enable their large-scale reporting and analysis in order to obtain
information about error recovery. Qualitative incident reports and interviews were
collected on four units of two Dutch general hospitals. Analysis of the 143
accompanying error handling processes demonstrated that different incident types
each provide unique information about error handling. The results enabled us to put
forward two possible definitions of near misses.
Although "first, do not harm" is one of the principal precepts in medicine, patients still can be
harmed by errors. Research has revealed that harmful medical errors during hospital
admission affect 9.2% of the patients (De Vries et al., 2008). Nowadays, many health care
organisations have implemented incident reporting systems to manage those errors. After an
error has happened, a health care employee can disclose it by filling out a reporting form.
Subsequent causal analysis can bring about learning to enhance the safety and quality of care
—————————————
*This chapter is largely based on: Kessels-Habraken, M., Van der Schaaf, T., De Jonge, J., & Rutte, C. (2009).
Defining near misses: Towards a sharpened definition based on empirical data. Manuscript under revision.
Chapter 5
64
(Aspden et al., 2004; Evans et al., 2007). The ultimate objective of safety management is no
or minimal patient harm (Battles & Lilford, 2003). However, 100% safety cannot be achieved
because errors will surely arise. Therefore, the current, limited focus on error reduction is
insufficient. In addition, there is a need for strategies that aim to promote error recovery, that
is, people‘s abilities to intercept errors and avert patient harm (Aspden et al., 2004; Hollnagel,
2008; Kanse et al., 2006).
If an error is observed, this can trigger off a so-called error handling process, which
has been defined as the complete process from error recognition to error correction (if
possible) and consists of three possible phases: detection, explanation, and countermeasures
(e.g., Kanse, 2004; Kanse & Van der Schaaf, 2001; Zapf & Reason, 1994). In the detection
phase, someone first finds out that an error has occurred. The explanation phase refers to the
attempts that people make to explore what exactly happened. In the countermeasures phase,
people take corrective measures to return the situation to normal or to limit negative
consequences. In case of so-called near misses, adverse consequences for patients were
prevented. Hence, in the error handling processes of near misses the countermeasures phase
should be present more often than in those of so-called accidents, which did have negative
consequences for patients. This information about effective countermeasures could enable
health care organisations to develop or boost strategies that promote timely error correction,
that is, before patients are harmed.
Near misses can thus provide information about successful error recovery. Besides,
near miss reporting and analysis offers several other advantages. First, near misses occur far
more frequently than actual accidents, which implies that more data may be collected in less
time. Secondly, the causal path of near misses and accidents is likely to be similar. Hence, by
eliminating the causes of near misses, one could prevent actual accidents. Thirdly, because in
the case of near misses patients were not harmed, health care employees might be less
ashamed of what happened and have less fear of litigation, which might positively influence
their willingness to report near misses (Aspden et al., 2004; Barach & Small, 2000; Kaplan &
Rabin Fastman, 2003; Van der Schaaf & Wright, 2005).
Despite those advantages, so far near misses have been underutilised as a source of
information to improve the safety and quality of care (Aspden et al., 2004; Parnes et al.,
2007; Patel & Cohen, 2008). This can partly be ascribed to a lack of consensus about the
definition of near misses (Affonso & Jeffs, 2004; Aspden et al., 2004; Yu et al., 2005). When
reviewing the literature, we identified two factors that are used when distinguishing different
incident types: ―patient reached‖ and ―patient harmed‖. A combination of these factors results
Defining Near Misses
65
in a matrix that can be used to classify incidents (see Figure 5.1). Because the combination of
―patient not reached‖ and ―patient harmed‖ is logically impossible, three incident types can
be differentiated:
1. Incidents that did not reach the patient (e.g., a nurse rightly questioned a drug
prescription and asked the doctor to adjust it before administering the drug);
2. Incidents that reached the patient but did not cause harm (e.g., a patient who was
administered blood that actually was intended for another patient, but fortunately
both patients had the same blood group);
3. Incidents that reached the patient and caused harm (e.g., the administration of an
overdose of a high-blood-pressure drug which resulted in brain damage).
No Yes
No
Yes
1
2 3
Patient harmed?
Pa
tie
nt re
ach
ed
?
Figure 5.1: Classification matrix for incident types.
Some researchers use the term near miss exclusively for incidents in which effective
countermeasures (i.e. successful error recovery) prevented the incident from reaching the
patient (e.g., Barnard, Dumkee, Bains, & Gallivan, 2006; Kaplan & Rabin Fastman, 2003).
This definition only includes incident type 1 of the matrix. Others define near misses as
incidents that did not cause patient harm, irrespective of the reasons why. Such definitions
thus encompass both cases in which the incident did not reach the patient because of
successful error recovery and cases in which harm was averted by coincidence or patient
robustness (e.g., Barach, Small, & Kaplan, 1999; Gurwitz et al., 2000). Hence, such
definitions of near misses include both incident types 1 and 2. The definition that was used by
Aspden et al. (2004) even partly included incident type 3 because they defined near misses as
incidents that did not cause serious harm. Table 5.1 presents some examples of definitions of
Chapter 5
66
near misses and the classification according to the matrix. This list is by no means
exhaustive; rather it demonstrates the diversity in definitions.
Table 5.1
Examples of definitions of near misses.
Definition Classification
An event or circumstance that has the potential to cause an incident or
critical incident but that did not actually occur due to corrective action
and/or timely intervention.
(Barnard et al., 2006)
1
An act of commission or omission that could have harmed the patient but
was prevented from completion through a planned or unplanned recovery.
(Kaplan & Rabin Fastman, 2003)
1
Any event that could have had adverse consequences but did not and was
indistinguishable from fully fledged adverse events in all but outcome.
(Barach & Small, 2000; Barach et al., 1999)
1 & 2
Errors that had the capacity to cause injury but failed to do so, either by
chance or because they were intercepted.
(Gurwitz et al., 2000)
1 & 2
An error of commission or omission that could have harmed the patient, but
serious harm did not occur as a result of chance, prevention, or mitigation.
(Aspden et al., 2004)
1 & 2 & (3)
Due to the lack of a clear and consistent definition, people attribute different
meanings to near misses and conceptual misunderstandings arise. This might result in
underreporting of near misses and problems with data analysis (Affonso & Jeffs, 2004;
Etchegaray et al., 2005; Tamuz et al., 2004). Consequently, valuable safety-related
information about successful error recovery mechanisms remains unavailable or gets lost
(Ramanujam & Rousseau, 2006). Moreover, health care organisations do not yet take
advantage of the fact that near miss reporting offers an indirect, cultural pathway to
improving patient safety by changing health care employees' risk perceptions, their attitudes
towards safety, and ultimately their behaviour as well (Aspden et al., 2004).
The present chapter questions whether it is useful to make a distinction between
incidents that did not reach the patient (type 1) and incidents that reached the patient but did
not cause harm (type 2) when defining near misses, or whether they may just as well be
lumped into one category. To investigate this question, we concentrated on the error handling
Defining Near Misses
67
processes that underlie the three incident types. Kanse (2004) developed a model that shows
that an error handling process starts with a deviation that is detected, followed by any
combination of explanation and countermeasures phases (see Figure 5.2). We used this model
to describe the error handling processes underlying the three incident types and to examine
which factor of the matrix is leading with respect to any differences in error handling
processes: ―patient reached‖ or ―patient harmed‖. This insight resulted in suggestions for the
definition of near misses in order to stimulate near miss reporting, and to obtain information
about effective strategies for error recovery.
Failure(s)
Deviation Detection
Explanation
Countermeasures
Outcomes
Figure 5.2: Error handling process model (Kanse, 2004).
The grey-coloured boxes represent the error handling process.
5.1 Methods
Data Collection
We used qualitative incident reports and interviews to collect empirical data about error
handling processes. Data were collected in two Dutch general hospitals of the same
foundation: a hospital offering basic care and a teaching hospital offering basic and
specialised care. Four units were selected: an intensive care unit, an emergency department,
an internal medicine nursing ward, and a haemodialysis ward. The participating hospitals and
units thus represented a variety of hospital settings. During a 4-month period, all employees
Chapter 5
68
of the units could report incidents electronically. An incident was defined as any deviation
from normal patient care, irrespective of the presence of harm. In total, 107 incidents were
reported. Typically, the initial reports consisted of a few lines of text describing what had
happened and which actions had been taken to prevent patient harm. See Table 5.2 for an
example.
Table 5.2
Example of an incident description and corresponding coding.
Incident description:
―The cardiology assistant physician had written a drug prescription without indicating how
often the drug should be administered. I showed the prescription to the internal medicine
assistant physician, who completed it.‖
Incident type: Type 1 patient not reached
Error handling process phases:
1. The nurse finding out that the prescription was incomplete Detection
2. The nurse asking the assistant physician to complete the prescription Countermeasures
Variables:
Second_phase: Countermeasures
Last_phase: Countermeasures
Presence_explanation: No
Presence_countermeasures: Yes
The first author together with employees from the four units analysed all incidents. If
possible, they interviewed the people involved to obtain more information about the error and
the way it had been dealt with. In addition to the incident reports, the first author also
interviewed employees from the four units. Eighteen employees (4 doctors and 14 nurses)
were selected by the unit managers. A semi-structured interview scheme was used. First, each
interviewee was asked to describe at least two incidents that had not been reported to the
voluntary incident reporting system. Next, specific questions were asked, such as: ―how was
the error recognised?‖ and ―which countermeasures were taken?‖. The interviews resulted in
44 incident descriptions. The total data set thus consisted of 151 separate cases. For each
case, the first author composed a brief description of the incident and an extensive description
of the accompanying error handling process.
Defining Near Misses
69
Data Coding
Based on the outcomes of the incident, each case was classified into one of the three incident
types depicted in Figure 5.1. Further, for each case, the accompanying error handling process
was mapped on Kanse‘s model by (1) subdividing the complete error handling process into
phases, each representing one or more actions and/or cognitive processes and (2) classifying
each identified phase according to its goal, that is, detection, explanation, or countermeasures
(see Table 5.2 for an example). Two authors acted as coders and jointly developed
instructions, which were first tested by coding 25 cases. This sample was representative in
terms of the unit on which the error had been observed, the type of error, and the data source.
During consensus meetings, the results were discussed and the coding instructions revised
(Miles & Huberman, 1994; Weston et al., 2001). Subsequently, the first author subdivided the
error handling processes of the remainder of the cases into separate phases and next both
coders independently classified those identified phases (Cohen's kappa = .91). They also
classified each case into one of the three incident types (Cohen's kappa = .62). After a
consensus meeting, the instructions were revised once more and subsequently both coders
again classified all cases into one of the three incident types (Miles & Huberman, 1994;
Weston et al., 2001). The corresponding value of Cohen's kappa increased from .62 to .82,
which indicated substantial agreement (Landis & Koch, 1977). The coders reached full
agreement before proceeding in further analysis. The final coding instructions are presented
in Table 5.3.
Chapter 5
70
Table 5.3
Coding instructions.
Code Definition
Error handling process phase A separate phase consisting of any action(s) and/or cognitive
process(es) is distinguished when:
(1) it had a different goal (i.e. detection, explanation, or
countermeasures)
(2) it was performed at a different moment in time (e.g., an
unrelated action was taken in between, or at least for a while
no efforts were taken)
(3) new people were involved and/or new equipment was
used
Detection (D) A phase in which a deviation is first detected
Example: “the nurse heard the machine alarms” or “the
patient doubted the prescribed dose”
Explanation (E) A phase in which people look for further information to draw
conclusions about the deviation, its causes and its
importance, and in which people decide whether
countermeasures are necessary (including extra observation
and diagnostic tests to examine the (potential) consequences
and to determine whether action is necessary)
Example: “the nurse consulted the doctor about the dose” or
“the nurse checked the patient’s file”
Countermeasures A phase involving the planning and implementation of
actions to return the situation to normal or to limit the
consequences (including consulting others to discuss what
needs to be done to avoid or limit the consequences)
Example: “the doctor corrected the dose” or “the nurse used
another haemodialyser to treat the patient”
Incident type 1 The incident did not reach the patient and his/her treatment
by timely and effective error recovery
Example: “before administering the drug to the patient, the
nurse consulted the doctor who corrected the dose” or “the
doctor found out that the data of two patients had been mixed
up and corrected the error before treating them”
Incident type 2 The incident reached the patient and his/her treatment, but
the treatment was not altered due to the incident and the
patient was not harmed (by patient robustness, sheer luck, or
coincidence)
Example: “the nurse forgot to administer the drug to the
patient, but without any consequences” or “the patient fell,
but was not injured”
Defining Near Misses
71
Table 5.3 continued
Coding instructions.
Code Definition
Incident type 3 The incident caused patient harm in terms of physical injury
and/or due to the incident the patient's treatment was altered
(both additional treatment and advancing, postponing, or
cancelling treatment)
Example: “due to a drug overdose the patient had eye
problems” or “the patient slipped and fell, and was in severe
pain”
Data Analysis
By combining the various phases of Kanse‘s model, five types of error handling processes
can be distinguished (Kanse, 2004), starting with detection and:
- No further phases (DetOnly): “When distributing medication, the nurse found out that
she had forgotten to distribute the former dose (detection).”;
- Followed by one or more explanation phases (DetExp): “After the nurse had
administered the drug, she immediately realised that she had administered the wrong
dose (detection). Next, she checked whether the patient’s haemoglobin was too high
(explanation).”;
- Followed by one or more countermeasures phases (DetCount): “During a double
check the nurse detected that another nurse had used the wrong haemodialysis acid
concentrate (detection). Then, she changed the concentrate (countermeasures).”;
- Followed by explanation, and a combination of explanation and countermeasures
phases (DetExpCount): “When connecting the haemodialyser, the nurse discovered
that the patient’s haemoglobin was low (detection). Subsequently, she found out that
the dose of a certain drug was wrong (explanation). She stopped the medication order
in the computer system (countermeasures). After she had consulted the assistant
physician, the patient received packed red blood cells (countermeasures).”;
- Followed by countermeasures, and a combination of explanation and countermeasures
phases (DetCountExp): “The pharmacist detected that potassium chloride (KCl) had
been prescribed via the wrong route (detection). On the prescription he commented
that KCl should not be administered via drip-feed (countermeasures). Two days later,
a nurse read the comment and consulted the doctor. Together, they discussed the
Chapter 5
72
results of the laboratory tests (explanation). Next, the doctor asked the nurse to stop
the administration of KCl (countermeasures).”.
A chi-square test would be appropriate to investigate whether the three incident types
differed with regard to their underlying error handling process types. However, several
expected cell counts did not exceed the minimum level (Siegel & Castellan, 1988). Therefore,
the coded data were regrouped into four variables that together characterise an error handling
process (see Table 5.2 for an example):
- second_phase: the goal of the first phase that follows detection (i.e. explanation or
countermeasures);
- last_phase: the goal of the last phase of the error handling process (i.e. explanation or
countermeasures);
- presence_explanation: whether at least one explanation phase is present;
- presence_countermeasures: whether at least one countermeasures phase is present.
Chi-square tests (α = .05) were conducted to evaluate the overall differences between
the three incident types regarding the four variables. Follow-up chi-square tests were used to
assess pair-wise differences (including a Bonferroni correction) (Green & Salkind, 2003).
Since in all analyses N > 40, we used chi-square corrected for continuity (Siegel & Castellan,
1988). We also used chi-square tests (α = .05) to establish which factor of the matrix is
predominant: ―patient reached (types 2&3 combined) or not (type 1)‖, or ―patient harmed
(type 3) or not (types 1&2 combined)‖.
Analyses were conducted to check for any systematic differences between the cases
obtained from the voluntary incident reporting system and those retrieved from the
interviews. No significant differences were identified regarding type of error, incident type,
and the four variables that characterise an error handling process. Therefore, for data analysis,
cases from both sources were lumped together.
5.2 Results
For all 151 cases, the error handling processes could be described according to Kanse's
model. However, since for eight cases the incident type was unknown, only 143 were
included in the analysis. Table 5.4 shows the distribution of the cases over the error handling
process types by incident type.
Defining Near Misses
73
Table 5.4
Number (and percentage) of cases per error handling process type by incident type.
Incident type
Error handling process type 1 2 3
DetOnly 0 (0.0%) 4 (6.2%) 0 (0.0%)
DetExp 2 (3.1%) 14 (21.5%) 2 (14.3%)
DetCount 30 (46.9%) 14 (21.5%) 2 (14.3%)
DetExpCount 31 (48.4%) 24 (36.9%) 10 (71.4%)
DetCountExp 1 (1.6%) 9 (13.8%) 0 (0.0%)
Total 64 (100.0%) 65 (100.0%) 14 (100.0%)
Note. DetOnly = detection only; DetExp = detection and one or more explanation phases;
DetCount = detection and one or more countermeasures phases; DetExpCount = detection,
followed by explanation and a combination of explanation and countermeasures phases;
DetCountExp = detection, followed by countermeasures and a combination of explanation
and countermeasures phases.
Although statistical testing by means of chi-square was not possible due to low
expected cell counts, closer inspection of Table 5.4 did reveal some insights. For instance,
processes in which, after detection of the error, only countermeasures had been taken
(DetCount) were identified in 30 out of 64 cases (46.9%) for incident type 1, while this error
handling process type was only identified in 14 out of 65 cases (21.5%) for type 2 and in 2
out of 14 cases (14.3%) for type 3. Furthermore, for incident type 3, many processes were
observed in which, after detection and explanation, a combination of explanation and
countermeasures phases had occurred (DetExpCount). In those processes, after realising that
an error had occurred, one first looked for further information about the error and its causes
before taking corrective measures. Such processes were identified in 10 out of 14 cases
(71.4%) for incident type 3, while for types 1 and 2 such processes were only observed in 31
out of 64 cases (48.4%) and 24 out of 65 cases (36.9%), respectively. Table 5.5 shows the
distribution of the cases over the four variables that are related to the goals of the second and
last phases of the error handling process and the presence of explanation and
countermeasures phases.
Chapter 5
74
Table 5.5
Number (and percentages) of cases per variable, by incident type (inc. type) and analysis.
Overall differences between incident types
Variable Value Inc. type 1 Inc. type 2 Inc. type 3
Second_phasea Explanation 33 (51.6%) 38 (62.3%) 12 (85.7%)
Countermeasures 31 (48.4%) 23 (37.7%) 2 (14.3%)
Last_phasea
Explanation 2 (3.1%) 20 (32.8%) 4 (28.6%)
Countermeasures 62 (96.9%) 41 (67.2%) 10 (71.4%)
Presence_
explanation
Yes 34 (53.1%) 47 (72.3%) 12 (85.7%)
No 30 (46.9%) 18 (27.7%) 2 (14.3%)
Presence_
countermeasures
Yes 62 (96.9%) 47 (72.3%) 12 (85.7%)
No 2 (3.1%) 18 (27.7%) 2 (14.3%)
Patient not reached (incident type 1) versus patient reached (incident types 2&3)
Variable Value Inc. type 1 Inc. type 2&3
Second_phasea
Explanation 33 (51.6%) 50 (66.7%)
Countermeasures 31 (48.4%) 25 (33.3%)
Last_phasea
Explanation 2 (3.1%) 24 (32.0%)
Countermeasures 62 (96.9%) 51 (68.0%)
Presence_
explanation
Yes 34 (53.1%) 59 (74.7%)
No 30 (46.9%) 20 (25.3%)
Presence_
countermeasures
Yes 62 (96.9%) 59 (74.7%)
No 2 (3.1%) 20 (25.3%)
Patient not harmed (incident types 1&2) versus patient harmed (incident type 3)
Variable Value Inc. type 1&2 Inc. type 3
Second_phasea
Explanation 71 (56.8%) 12 (85.7%)
Countermeasures 54 (43.2%) 2 (14.3%)
Last_phasea
Explanation 22 (17.6%) 4 (28.6%)
Countermeasures 103 (82.4%) 10 (71.4%)
Presence_
explanation
Yes 81 (62.8%) 12 (85.7%)
No 48 (37.2%) 2 (14.3%)
Presence_
countermeasures
Yes 109 (84.5%) 12 (85.7%)
No 20 (15.5%) 2 (14.3%)
aFour cases of incident type 2 were excluded because their error handling processes only
comprised a detection phase.
Defining Near Misses
75
Overall Differences between Incident Types
Analysis revealed that people had engaged in different error handling processes for the three
incident types. More specifically, the error handling processes underlying the three incident
types significantly differed with respect to the presence of phases in which people had
explored the error and its causes (i.e. explanation), χ2(2, N = 143) = 8.14, p < .05, and the
presence of phases in which corrective measures had been taken to return the situation to
normal or to limit adverse consequences for the patient (i.e. countermeasures), χ2(2, N = 143)
= 14.97, p < .01. Moreover, the goal of the last phase of the error handling process (i.e.
explanation or countermeasures) was significantly different for the three incident types, χ2(2,
N = 139) = 19.07, p < .001.
Pairwise Differences between Incident Types
Follow-up chi-square tests demonstrated that the last phase of the error handling processes
that underlay incidents that did not reach the patient (type 1) was significantly different from
those of incidents that reached the patient (types 2 and 3). If the incident did not reach the
patient (type 1), the last error handling process phase was significantly more often a
countermeasures phase than an explanation phase, compared to incidents that reached the
patient but did not cause harm (type 2), and incidents that reached the patient and caused
harm (type 3), χ2(1, N = 125) = 16.96, p < .001, and χ
2(1, N = 78) = 7.20, p < .01,
respectively. For incidents that did not reach the patient (type 1), the error handling processes
of 96.9% of the cases had been ended by people taking corrective actions. On the other hand,
the error handling processes underlying incidents that reached the patient but did not cause
harm (type 2) and incidents that reached the patient and caused harm (type 3) had been
stopped with countermeasures in only 67.2% and 71.4% of the cases, respectively. Actually,
the error handling processes that underlay the latter two incident types had also frequently
been concluded by people looking for further information about the error and its impact, for
instance by consulting others or conducting diagnostic tests (i.e. explanation).
In general, if the incident did not reach the patient (type 1), a countermeasures phase
was significantly more often present than if the incident reached the patient but did not cause
harm (type 2), χ2(1, N = 129) = 13.04, p < .001. In other words, in case the incident did not
reach the patient (type 1), people had more often been able to take effective actions to
prevent the incident from reaching the patient, either directly or after localising the error and
its causes. With respect to the four variables, chi-square tests showed no significant
Chapter 5
76
differences between the error handling processes of incidents that reached the patient but did
not cause harm (type 2) and incidents that reached the patient and caused harm (type 3).
Patient Not Reached (Incident Type 1) versus Patient Reached (Incident Types 2&3)
Countermeasures phases were significantly more often identified for incidents that did not
reach the patient (type 1) than for incidents that reached the patient (types 2&3), χ2(1, N =
143) = 11.73, p < .01. Furthermore, compared to incidents that reached the patient (types
2&3), the error handling processes of incidents that did not reach the patient (type 1) had
significantly more often been completed with a countermeasures phase than an explanation
phase, χ2(1, N = 139) = 17.08, p < .001. So, especially in cases where the incident did not
reach the patient (type 1), error handling processes were identified in which people had
recognised the error in time and subsequently had taken successful corrective measures.
Patient Not Harmed (Incident Types 1&2) versus Patient Harmed (Incident Type 3)
With respect to the four variables, no significant differences were identified between the error
handling processes underlying incidents that did not cause patient harm (types 1&2) and
incidents that caused patient harm (type 3).
5.3 Discussion
Health care organisations do not yet take full advantage of near misses to improve the safety
and quality of care (Aspden et al., 2004; Parnes et al., 2007; Patel & Cohen, 2008). This is
partly explained by the lack of a clear and consistent definition of near misses (Affonso &
Jeffs, 2004; Aspden et al., 2004; Yu et al., 2005). The present chapter addressed this problem
by questioning whether it is useful to differentiate between incidents that did not reach the
patient and incidents that reached the patient but did not cause harm, when defining near
misses.
Theoretical Implications
This study demonstrated that Kanse‘s model (2004) can be used to describe the way medical
errors are recognised and dealt with. Our empirical results showed that people engage in
different error handling processes for different types of incidents. The statistical tests
revealed that the error handling processes underlying incidents that did not reach the patient
significantly differed from those of incidents that reached the patient, irrespective of harm
because of successful countermeasures that had been taken after detection of the error. This
Defining Near Misses
77
finding could suggest defining near misses as incidents in which timely error recovery
prevented the incident from reaching the patient. Then, no-harm incidents could be defined
as incidents that reached the patient but did not cause harm, and accidents could be defined
as incidents that reached the patient and caused harm. Those definitions would endorse the
definitions as suggested by the drafting group for an International Classification for Patient
Safety (Runciman et al., 2009) and are important to guide patient safety research (Affonso &
Jeffs, 2004). A comparison of the suggested definition of near misses with other definitions
shows that some researchers indeed use the term near miss exclusively for incidents that did
not reach the patient (e.g., Barnard et al., 2006; Kaplan & Rabin Fastman, 2003).
Statistical tests showed no significant differences between the error handling
processes that underlay incidents that reached the patient but did not cause harm and
incidents that reached the patient and caused harm. Taken into account that patient reached
seems to be the predominant factor of the proposed matrix for the classification of incident
types, this result might suggest to lump those incidents into one category when analysing
error handling processes. However, since a descriptive observation of the error handling
process types indicated that particularly incidents that reached the patient and caused harm
were associated with processes in which people first localised the error and then took actions
to avert or minimise negative consequences, the former conclusion might be unjustifiable. In
fact, the descriptive observation suggests that the three incident types should indeed be
differentiated because they all provide unique information about error handling.
Practical Implications
The proposed definition of near misses as incidents in which successful error recovery
prevented the incident from reaching the patient has several advantages. First, this definition
is positively stated because incidents that reached the patient but did not cause harm are
excluded. This might result in an increased willingness to report near misses because of
diminished feelings of shame and fear (Lawton & Parker, 2002; Waring, 2005) by the
emphasis on positive behaviour, that is, successful error recovery (Affonso & Jeffs, 2004).
Another advantage of the proposed definition of near misses is the fact that the resulting large
dataset will only comprise incidents with successful active error recovery because incidents
that did not cause harm by sheer luck or patient robustness would be excluded. By analysing
this dataset, health care organisations could identify effective error recovery strategies and
stimulate the use of those strategies to improve the safety and quality of care.
Chapter 5
78
However, some researchers also include incidents that reached the patient but did not
cause harm in their definitions of near misses (e.g., Barach et al., 1999; Gurwitz et al., 2000).
This might be advantageous because such a broad definition of near misses enlarges the
incident types that are eligible for reporting, which could result in a larger dataset. Moreover,
the difference between the presence and absence of patient harm might be more clear-cut than
determining whether the incident reached the patient. Defining near misses as incidents that
did not cause patient harm might thus yield a better understanding of what should be
reported, which could bring about larger numbers of reported near misses (Affonso & Jeffs,
2004; Etchegaray et al., 2005). Besides, incidents that reached the patient but did not cause
harm might contain important information about failed or missed error recovery
opportunities. Health care organisations could use this information to redesign error recovery
mechanisms to promote successful error recognition and correction (Habraken & Van der
Schaaf, in press, see also Chapter 6; Kanse et al., 2006).
To summarise, from a practical point of view, the optimal definition of near misses
may be contingent on organisational context. This assumption endorses the suggestion from
Tamuz and Thomas (2006) that standardised definitions probably still may be interpreted
differently within various health care organisations. For instance, in a safety culture in which
health care employees feel ashamed or are punished in case of errors, a narrow but positively
stated definition of near misses as incidents that did not reach the patient might be most
appropriate. On the other hand, in a more advanced safety culture in which trust and learning
predominate, near misses might well be defined more broadly as incidents that did not cause
patient harm.
Study Strengths and Limitations
The use of incident reports and interviews is an important strength of this empirical study,
because non-reported data are often overlooked (Hogan et al., 2008). Another strength of our
research is the use of Kanse‘s model (2004) for the description of error handling processes,
because this model is theoretically driven and empirically validated.
However, our study has also some limitations. First, a reporting bias might have
affected the results, which would imply that the cases resulting from the voluntary incident
reporting system were different in nature than those obtained from the interviews. For
instance, interviewees might only have disclosed those errors that were prevented from
reaching the patient. However, analysis ruled out this potential bias because no systematic
differences were identified between the cases from the two data sources.
Defining Near Misses
79
A limitation of any retrospective incident analysis is the potential for two biases:
recall bias and hindsight bias; that is, the tendency of people who are aware of the outcome of
an incident to exaggerate the extent to which they would have predicted the incident
beforehand (Henriksen & Kaplan, 2003). With respect to the incident reports, we have tried
to limit these biases by collecting information as soon as possible after an incident had been
reported, by cross-validating the information by interviewing the people involved (if
possible), and by focussing on positive error recovery behaviour (Carthey, De Leval, &
Reason, 2001; Kaplan & Barach, 2002). With respect to the interviews, we asked the
interviewees to recall incidents that had occurred recently, and during an interview, we
concentrated on the way the error had been detected and corrected (Kaplan & Barach, 2002).
However, we did not cross-validate the information that was obtained during the interviews.
Another limitation of this study is the relatively small number of incidents that
reached the patient and caused harm. This small dataset made it impossible to test whether
the three incident types were characterised by different error handling process types and
forced us to regroup the coded data into four variables in order to explore differences.
However, it should be noted that those variables were not independent, nor mutually
exclusive. Hence, this study should be conducted with larger datasets and in other hospitals,
and even in other health care settings such as nursing and mental homes, to establish external
validity and allow robust statistical testing. Although this study proposes a way forward for
the definition of near misses, we encourage future research with larger datasets to further
sharpen the definition of near misses.
Conclusions
This study has put forward two suggestions for the definition of near misses. Their
application in incident reporting systems could result in larger numbers of reported near
misses. Subsequent analysis enables health care organisations to advocate a more proactive
approach towards patient safety. Health care organisations could (1) eliminate failure factors
before real accidents may occur, (2) enhance their ability to recover from errors, and (3)
improve their safety culture, thereby indirectly improving safety performance. Health care
organisations that have implemented a safety management system in which near misses are
registered, analysed, and interpreted do consider both (important) strategies towards patient
safety: error reduction and error recovery promotion. This might result in significant
advances in patient safety in terms of reduced numbers of medical errors.
80
81
Chapter 6
If Only….: Failed, Missed and Absent
Error Recovery Opportunities
in Medication Errors*
This chapter explores whether accidents could be used as an alternative data source
to near misses for the analysis and understanding of error recovery. Failed, missed
and absent error recovery opportunities were identified in 52 medication errors that
all resulted in severe patient harm or patient death. For all identified error recovery
opportunities the underlying failure factors were identified and classified. Those
failure factors represent negative influences on error recovery, which could be
reduced to enhance error detection and correction. We concluded that hospital can
use both near misses and accidents to understand and promote error recovery.
Medical errors can be characterised by one or more initial errors that are either detected and
corrected in time or not. Until recently, retrospective incident analysis particularly
concentrated on the identification of failure factors underlying medical errors. However,
errors cannot be completely prevented. Therefore, the importance of the analysis of error
recovery is increasingly being recognised in health care (Aspden et al., 2004; Kanse et al.,
—————————————
*This chapter is largely based on: Habraken, M. M. P., & Van der Schaaf, T. W. (in press). If only….: Failed,
missed and absent error recovery opportunities in medication errors. Quality and Safety in Health Care.
Chapter 6
82
2006; Parnes et al., 2007). In case of a near miss, timely and effective error recovery did
prevent patient harm (Van der Schaaf, 1991). Systematic analysis of near misses is important
because, in comparison with actual accidents, near misses provide information about error
recovery factors. Error recovery factors explain why developing incidents did not result in
actual accidents, that is, why adverse consequences were prevented (Kanse et al., 2006).
Those factors thus provide insight into the extent to which hospitals are capable of detecting
and correcting initial errors. This provides hospitals with an additional strategy to improve
patient safety, that is, the enhancement of their resilience (Van der Schaaf & Wright, 2005).
Information about error recovery can be obtained in two ways: by focussing on both
successful and unsuccessful error recovery. Usually, near misses are collected and analysed
to find out how patient harm was prevented. This approach concentrates on successful error
recovery. However, failed or missed error recovery opportunities can also provide us with
important safety-related information. In a field study on near misses in a hospital pharmacy, it
was demonstrated that often multiple error recovery opportunities are missed or fail before
successful error recovery takes place. In addition to the factors that contributed to successful
error recovery, Kanse et al. (2006) identified the factors that contributed to unsuccessful error
recovery. Subsequently, the hospital pharmacy was advised to enhance the positive
influences on error recovery and to reduce the negative ones. Kanse et al. only concentrated
on error recovery in relation to near misses. However, one might assume that, in addition to
near misses, accidents could also provide us with information about negative influences on
error recovery.
In another study that was carried out in 2005, we had already analysed 52 medication
errors of the Netherlands Health Care Inspectorate's incidents database that all resulted in
severe patient harm or patient death. The initial errors in this set consisted of prescription,
transcription, dispensing, and administration errors (see Figure 6.1). In-depth causal analysis
had identified on average 7.3 failure factors per error, which had all been classified according
to the Eindhoven Classification Model (ECM). This model considers technical,
organisational, human, patient-related, and other failure factors (see Table 6.1). Inter-rater
reliability checks showed satisfactory results (Habraken, 2005; Habraken & Van der Schaaf,
2005). In the present exploratory study, we conducted secondary analyses on the same 52
medication errors. We identified and categorised failed, missed and absent error recovery
opportunities to find out what factors negatively influence error recovery. Moreover, we tried
to answer the question whether accidents could be used as an alternative data source to near
misses for the analysis and understanding of error recovery.
Failed, Missed and Absent Error Recovery Opportunities
83
Figure 6.1: Distribution of initial errors over prescribing, transcription, dispensing, and
administration errors. One incident consisted of two independent types of initial errors and
therefore, the total number equals 53.
Chapter 6
84
Table 6.1
Eindhoven Classification Model (ECM) for the medical domain
(Aspden et al., 2004; Battles, Kaplan, Van der Schaaf, & Shea, 1998).
Category (code) Description
Technical-External
(TEX)
Technical failures beyond the control and responsibility of
the investigating organisation
Technical-Design
(TD)
Failures due to poor design of equipment, software, labels, or
forms
Technical-Construction
(TC)
Correct design was not followed accurately during
construction
Technical-Materials
(TM)
Material defects not classified under TD or TC
Organisational-External
(OEX)
Failures at an organisational level beyond the control and
responsibility of the investigating organisation
Organisational-Knowledge
transfer
(OK)
Failures resulting from inadequate measures taken to ensure
that situational or domain-specific knowledge or information
is transferred to all new or inexperienced staff
Organisational-Protocols
(OP)
Failures related to the quality and availability of the protocols
within the department (too complicated, inaccurate,
unrealistic, absent, or poorly presented)
Organisational-Management
priorities
(OM)
Internal management decisions in which safety is relegated to
an inferior position in the face of conflicting demands or
objectives; a conflict between production needs and safety
Organisational-Culture
(OC)
Failures resulting from a collective approach to risk and
attendant modes of behaviour in the investigating
organisation
Human-External
(HEX)
Human failures beyond the control and responsibility of the
investigating organisation
Human-Knowledge
(HKK)
The inability of an individual to apply existing knowledge to
a novel situation
Human-Qualifications
(HRQ)
Incorrect fit between an individual's qualifications, training,
or education and a particular task
Human-Coordination
(HRC)
Lack of task coordination within a health care team in an
organisation
Human-Verification
(HRV)
Failures in the correct and complete assessment of a
situation, including relevant conditions of the patient and
materials to be used, before starting the intervention
Human-Intervention
(HRI)
Failures that result from faulty task planning (selecting the
wrong protocol) and/or execution (selecting the right protocol
but carrying it out incorrectly)
Failed, Missed and Absent Error Recovery Opportunities
85
Table 6.1 continued
Eindhoven Classification Model (ECM) for the medical domain
(Aspden et al., 2004; Battles, Kaplan, Van der Schaaf, & Shea, 1998).
Category (code) Description
Human-Monitoring
(HRM)
Failures during monitoring of process or patient status during
or after an intervention
Slips
(HSS)
Failures in performing of fine motor skills
Tripping
(HST)
Failures in whole-body movements
Patient-related factor
(PRF)
Failures related to patient characteristics or conditions that
influence treatment and are beyond the control of staff
Unclassifiable
(X)
Failures that cannot be classified in any other category
6.1 Methods
To develop the procedure for the identification and categorisation of error recovery
opportunities, we selected ten medication errors out of the total set of 52. This sample was
representative for the complete set in terms of type of initial error and complexity. In the
earlier study, the first author had composed a causal tree for each medication error
(Habraken, 2005; Habraken & Van der Schaaf, 2005). In the present study, both authors
independently identified error recovery opportunities in those existing causal trees. During a
consensus meeting, the results for the ten selected errors were compared and we agreed upon
the identified error recovery opportunities, which we subsequently independently categorised.
We distinguished between planned and unplanned error recovery and between failed, missed
and absent error recovery opportunities. Accordingly, we distinguished six categories of error
recovery opportunities. Planned error recovery opportunities involve organisational and
technical defences and barriers that are built into the health care system to avoid safety-
related consequences (Hollnagel, 1999; Kanse et al., 2006; Svenson, 2001). Unplanned error
recovery opportunities are ad hoc solutions that are not formally required and supported by
procedures or instructions but instead largely depend on the problem solving abilities of the
people involved (Kanse et al., 2006). Table 6.2 presents the coding scheme together with
some examples.
Chapter 6
86
Table 6.2
Categories of error recovery opportunities.
Category Description Example
Planned-
failed
Formalised barriers that were utilised,
but that failed
Prescribing or dispensing errors not
noticed during formalised checks
Planned-
missed
Formalised barriers that could have
been utilised but were not
Required check not performed;
prescription not authorized;
required check just before
medication administration not
carried out
Planned-
absent
Formalised barriers that could not be
utilised because they were absent, but
that should have been in place
according to the state of the art or
expert opinion
Vital or important check not
performed due to absent protocol
Unplanned-
failed
A person is aware of the initial error
and wants to correct it, but does not
succeed
Prescription not verified, despite
other employees' doubts; ignoring
patients' reminders
Unplanned-
missed
A person does not detect (a very
obvious) error that should have been
detected due to professional expertise
Unusual combination of age and
dose not noticed; deviating colour
of fluid not noticed
Unplanned-
absent
A person should detect the error, but is
lacking the necessary resources or
abilities
Lack of knowledge or experience to
detect (extreme) overdose
Subsequently, we independently identified and categorised error recovery
opportunities in the total set of 52 medication errors. If we could not determine which of two
categories should be assigned to a particular error recovery opportunity, we decided to assign
both categories, which each counted for half. Consensus was achieved on all categories.
Finally, we linked the six categories for error recovery opportunities to their
underlying failure factors to determine the negative influences on error recovery. In the
earlier study, all failure factors had already been classified according to the ECM. For each
error recovery opportunity, we registered the related underlying failure factors. Thus, we
were able to create a profile of underlying failure factors for each category of error recovery
opportunities.
Failed, Missed and Absent Error Recovery Opportunities
87
6.2 Results
In total, 127 error recovery opportunities were identified that had been absent, missed, or that
had failed. The number of error recovery opportunities per error ranged from 0 to 11; on
average 2.4 error recovery opportunities were identified. In only four accidents, no error
recovery opportunities were identified at all.
Table 6.3 shows the distribution of error recovery opportunities across the six
categories. It should be noted that some categories show partial frequencies because in a few
cases two categories were assigned to a single error recovery opportunity, which each
counted for half. Of the 127 error recovery opportunities, 94 were planned and 33 were
unplanned. Failure to detect and correct initial errors was thus more related to problems with
formalised barriers than to difficulties with ad hoc problem solving. In contrast with the
planned error recovery opportunities, which were almost equally distributed among the three
categories, most unplanned error recovery opportunities were categorised as unplanned-
failed. This indicates that with respect to the ad hoc problem solving cases employees
frequently noticed that something was wrong, but were not able to solve it (unplanned-
failed). However, it occurred less frequently that employees did not detect an error that in fact
should have been detected due to professional expertise (unplanned-missed).
Table 6.3
Distribution of medication error recovery opportunities across categories.
Category No. of cases (%)
Planned-failed 32.0 (25.2)
Planned-missed 29.5 (23.2)
Planned-absent 32.5 (25.6)
Unplanned-failed 17.0 (13.4)
Unplanned-missed 10.5 (8.3)
Unplanned-absent 5.5 (4.3)
Total 127.0 (100.0)
Note. Some categories show partial frequencies because in a few cases two categories were
assigned to a single error recovery opportunity, which each counted for half.
Chapter 6
88
Negative Influences on Planned Medication Error Recovery Opportunities
Table 6.4 shows how often particular failure factors contributed to unsuccessful planned error
recovery. The dominant failure factor is organisational protocols (OP). Absent, incomplete,
or unclear protocols prevented employees from detecting errors in drug prescriptions or
performing double checks after dispensing or before administering the drug. Failure to
recover from errors could also frequently be attributed to incorrect or incomplete assessment
and verification of the prescription, the drug, and the patient before drug dispensing or
administration (HRV). Nurses did not read drug labels, or failed to notice a difference in dose
between the drug prescription and the dispensed drug. Other factors that made it impossible
for employees to recover from initial errors were heavy workload due to management
decisions related to staffing (OM) and an organisational culture in which compliance with
safety-related procedures was low, and consequently required checks were not always carried
out (OC). Regarding the main categories of failure factors, the organisational failure factors
contributed the most to unsuccessful planned error recovery.
Failed, Missed and Absent Error Recovery Opportunities
89
Table 6.4
Failure factors and main categories of failure factors underlying failed, missed and absent
planned medication error recovery opportunities.
Failure factor
(Eindhoven
Classification Model)
No. of times
failure factor
was involved
No. of times
main category of
failure factors
was involved
Example
Technical 11
Technical-External
(T-EX)
1
No indication of drug dose
on phial
Technical-Design
(TD)
7
Bad readability of label
Technical-Construction
(TC)
2
Upper limits not adjusted in
computer system
Technical-Materials
(TM)
1
Problems with patient's
alarm bell
Organisational 74
Organisational-
Knowledge transfer
(OK)
1 Employees not informed
about specific drug
Organisational-Protocols
(OP)
42
Lack of protocols for
required checks or lack of
clear standards
Organisational-
Management priorities
(OM)
16
No presence of supervisors
due to understaffing
Organisational-Culture
(OC)
15
Lack of compliance with
regulations and protocols
(with respect to checks)
Human 45
Human-External
(H-EX)
1
Error not corrected by
employees in external
pharmacies
Human-Knowledge
(HKK)
5
Tasks or checks carried out
without specific knowledge
Human-Qualifications
(HRQ)
1
Administrative worker
distributing medication to
patients
Human-Coordination
(HRC)
6
Lack of agreement on
checking procedure
(misunderstanding)
Chapter 6
90
Table 6.4 continued
Failure factors and main categories of failure factors underlying failed, missed and absent
planned medication error recovery opportunities.
Failure factor
(Eindhoven
Classification Model)
No. of times
failure factor
was involved
No. of times
main category of
failure factors
was involved
Example
Human (cont.)
Human-Verification
(HRV)
25
Insufficient checks; not
asking other employees to
check medication
Human-Intervention
(HRI)
6
Incomplete information
written down on drug labels
Human-Monitoring
(HRM)
1
Drips not monitored
Other 4
Patient Related Factor
(PRF)
3
Sudden resuscitation of
patient
Unclassifiable
(X)
1
No supervision because of
sudden absence of supervisor
(call)
Total 134 134
Note. Multiple failure factors can underlie a single error recovery opportunity. Failure factors
with a frequency of zero are omitted.
Negative Influences on Unplanned Medication Error Recovery Opportunities
Table 6.5 shows how often particular failure factors were underlying unsuccessful unplanned
error recovery. No dominant failure factor was identified. Several failure factors to some
extent contributed to unsuccessful unplanned error recovery. In several cases, suspicion was
present. In those cases, an employee or patient was more or less aware of the initial error, but
lack of verification (HRV), coordination with colleagues (HRC), or in-depth knowledge or
routine (HKK) prevented them from successful error correction. For example, sometimes
nurses or patients did suspect an overdose, but when the doctors were notified, the doctors
held to their decision and the wrong dose was still administered. In other cases, the
employees involved were not able to solve the problem because of absent, erroneous, or
unclear (treatment) protocols (OP). In contrast with unsuccessful planned error recovery,
human failure factors contributed the most to unsuccessful unplanned error recovery.
Failed, Missed and Absent Error Recovery Opportunities
91
Table 6.5
Failure factors and main categories of failure factors underlying failed, missed and absent
unplanned medication error recovery opportunities.
Failure factor
(Eindhoven Classification
Model)
No. of times
failure factor
was involved
No. of times
main category of
failure factors was
involved
Example
Technical 1
Technical-Design
(TD)
1
Check impossible due
to layout of forms
Organisational 17
Organisational-
Knowledge transfer
(OK)
4
Employees not
informed about specific
drug and its stock
Organisational-Protocols
(OP)
7
No clear protocol
about therapy
Organisational-
Management priorities
(OM)
5
Lack of supervision
due to understaffing
Organisational-Culture
(OC)
1
Negligence in response
to ambiguous
comments
Human 28
Human-Knowledge
(HKK)
6
Practical experience
from other hospital
wrongfully used
Human-Coordination
(HRC)
6
Drug administration
not reported to other
employees
Human-Verification
(HRV)
10
Dose not verified,
despite other
employees' doubts
Human-Intervention
(HRI)
4
Route of administration
not recorded on drug
prescription
Human-Monitoring
(HRM)
2
Failure to monitor
patients while taking
medication
Other 3
Patient Related Factor
(PRF)
2
Taking medication
despite doubts
Chapter 6
92
Table 6.5 continued
Failure factors and main categories of failure factors underlying failed, missed and absent
unplanned medication error recovery opportunities.
Failure factor
(Eindhoven Classification
Model)
No of times
failure factor
was involved
No. of times
main category of
failure factors was
involved
Example
Unclassifiable
(X)
1
Because of expiration
dates, two forms had to
be filled out
Total 49 49
Note. Multiple failure factors can underlie a single error recovery opportunity. Failure factors
with a frequency of zero are omitted.
6.3 Discussion
Theoretical Implications
Since the ultimate objective of zero errors is unreachable, the current, limited focus of many
error reduction methods on failure factors is insufficient. Besides those traditional methods
there is a need for methods that explore why errors are detected and corrected accurately and
in time or why not, that is, methods that discover successful and unsuccessful error recovery
strategies (Aspden et al., 2004; Kanse et al., 2006; Parnes et al., 2007). This study has shown
that such error recovery methods can use accidents as a data source in addition to near
misses.
Practical Implications
To gain an in-depth understanding of error recovery, hospitals can conduct two kinds of
analysis. For near misses, the steps that lead up to successful error recovery can be identified
to reveal positive influences on error recovery. For both near misses and accidents, hospitals
can identify unsuccessful error recovery opportunities that arose after the initial errors. The
underlying failure factors represent negative influences on error recovery. Together, those
two sources of information could enable hospitals to enhance their resilience by reinforcing
the positive influences on error recovery and reducing the negative ones. This study also
shows that hospitals do not have to wait for actual accidents to occur to implement this novel
approach, because existing case files and incidents databases can already be reused to obtain
information about unsuccessful error recovery.
Failed, Missed and Absent Error Recovery Opportunities
93
An additional practical advantage of considering failed, missed and absent error
recovery opportunities when analysing accidents relates to the fact that positively intended
behaviour (albeit failed) is elucidated. Concentrating on the positive mechanisms of error
detection and correction might result in larger numbers of reported incidents (Affonso &
Jeffs, 2004; Tamuz et al., 2004).
Take-home Lessons for Enhancing Medication Error Recovery in Hospitals
For medication safety, hospitals can mainly reduce negative influences on planned error
recovery by adding and improving formalised protocols that health care employees need to
detect and correct errors, and by giving management priority to safety in terms of adequate
staffing levels. Furthermore, hospitals could improve existing organisational cultures by
increasing risk awareness, for instance by educating staff on safety science and by enabling
voluntary and nonpunitive error reporting (Pronovost et al., 2003). Focussed training and
instructions can reduce negative influences on unplanned error recovery. Hospitals should
ensure that the knowledge and skills of their employees are up to date to enable them to
detect and correct errors. Such training could concentrate on both standard (checking)
procedures and problem solving abilities, and could (if possible) be simulation based
(Henriksen & Dayton, 2006; Shapiro et al., 2004). Because these guidelines are based on data
from multiple Dutch hospitals, they will probably also be applicable for other hospitals, and
possibly for other countries as well. However, hospitals should always verify to what extent
the recommendations can be applied. For instance, adding protocols might not always be
appropriate, depending on organisational culture and the way protocols are generally
perceived and interpreted (Katz-Navon, Naveh, & Stern, 2005).
Limitations and Future Research
A limitation of our study is the fact that we conducted a secondary analysis of data that had
already been collected in an earlier study. In the present study, we were not able to ask
additional questions to the inspectors or the hospitals involved. Therefore, this study should
be replicated in a setting in which it is possible to gather recent additional information.
Another limitation of our approach is the potential for hindsight bias; that is, the tendency for
people who are aware of the outcome to exaggerate the extent to which the incident could
have been predicted beforehand (Henriksen & Kaplan, 2003). We tried to limit this bias by
only using information that had been agreed upon by inspectors of the Netherlands Health
Care Inspectorate in the earlier study. Because our study is explorative in nature, no formal
Chapter 6
94
inter-rater reliability checks were conducted. Future studies should concentrate on the extent
to which multiple raters agree on the categorisations. Furthermore, intervention studies could
be carried out to discover best practices related to the promotion of error recovery in
hospitals, like the study on error recovery strategies in the emergency department by
Henneman, Blank, Gawlinski, & Henneman (2006).
General Conclusion
This study has shown that accidents can be used as an alternative data source to near misses
for the analysis and understanding of error recovery. By using both sources, hospitals could
enhance their resilience by reinforcing the positive influences on error recovery and reducing
the negative ones. Although this is a very important safety strategy, traditional error reduction
methods, which concentrate on eliminating failure factors, are equally important. In other
words, triangulating information is necessary to provide a complete and comprehensive
picture (Hogan et al., 2008; Runciman et al., 2006). Hence, only by applying the two
complementary safety strategies of error reduction and error recovery promotion, hospitals
can significantly improve patient safety.
95
Chapter 7
Trends in Safety Culture
in Three Dutch Hospitals:
A Longitudinal Panel Survey
This chapter presents a longitudinal panel survey among 701 health care employees
that explored the changes in safety culture in three Dutch hospitals after an extensive
safety management programme had been implemented. Significant positive trends
were observed regarding incident reporting behaviour, response to errors, and
management support. Logistic and multiple regression analyses revealed feedback
about and learning from errors, hospital handoffs and transitions, as well as
teamwork within hospital units to be positively associated with incident reporting
behaviour. Due to the limited number of significant changes in safety culture, the use
of self-reported safety culture surveys as an evaluation instrument could be
questioned.
A vast majority of health care organisations implement safety management programmes to
reduce the large number of medical errors. These programmes often consist of structural
—————————————
*This chapter is largely based on: Kessels-Habraken, M., De Jonge, J., Van der Schaaf, T., Rutte, C., &
Gerritsen, G. (2009). Trends in safety culture in three Dutch hospitals: A longitudinal panel survey. Manuscript
in preparation.
Chapter 7
96
components to identify and analyse problems, such as retrospective and prospective methods
for risk analysis. Health care organisations use incident reporting systems to detect and
investigate medical errors after they occurred and to take measures to prevent them from
happening again. As opposed to such retrospective methods, which facilitate learning from
actual errors, prospective methods concentrate on potential risks in health care processes. In
a prospective analysis, a multidisciplinary team openly discusses and assesses problems that
could crop up. Subsequently, the team proposes actions to reduce those risks.
Obviously, such analytical approaches enable health care organisations to minimise
patient harm. However, it is widely recognised that health care organisations can also follow
a cultural pathway to improve patient safety (Aspden et al., 2004; Hudson, 2001; Nieva &
Sorra, 2003; Pronovost & Sexton, 2005). Safety culture is commonly defined as ―the product
of individual and group values, attitudes, perceptions, competencies, and patterns of
behaviour that determine the commitment to, and the style and proficiency of, an
organisation‘s health and safety management.‖ (Advisory Committee on the Safety of
Nuclear Installations, 1993, p. 23). Health care organisations should pursue a culture in which
safety is the first priority (Hale, 2003; Nieva & Sorra, 2003; Pronovost et al., 2003) and
health care employees at all levels aim to avert patient harm. According to Hudson (2003)
and Reason (1998), an advanced safety culture is characterised by four aspects:
1. Health care employees and managers are notified about actual errors and potential
risks. They are informed about relevant quality and safety issues in their
organisation; they know what is going on.
2. People trust one another and are willing to share lessons regarding medical errors,
without the fear of punishment.
3. A sophisticated safety culture is adaptable to change through learning and
flexibility.
4. In advanced safety cultures people worry about safety. They are aware that health
care is hazardous and are constantly anticipating problems.
Despite the fact that researchers hold different views on the strength of the relation
between safety culture and safety performance (Colla, Bracken, Kinney, & Weeks, 2005;
Cooper & Philips. 2004; Clarke, 2006a, 2006b), it is generally argued that safety culture
could positively influence safety performance through its effects on safety behaviour (Aspden
et al., 2004; Clarke, 2006b; Flin, 2007; Flin et al., 2006; Neal et al., 2000). Moreover, Flin
(2007) claims that a weak safety culture in itself could contribute to medical errors.
Trends in Safety Culture
97
Therefore, safety culture can provide health care organisations with an indirect pathway to
improve patient safety. An advanced safety culture could result in better compliance with
safety regulations and procedures (Neal et al., 2000), as a result of which fewer errors might
occur in the first place. Furthermore, a safety culture in which people are preoccupied with
risks could enhance the opportunities for error recognition and correction (Kontogiannis &
Malakis, 2009), whereby patient harm could be prevented, or at least minimised.
Safety culture could thus form a proactive and leading indicator of patient safety
(Flin, Mearns, O‘Connor, & Bryden, 2000; Itoh, Andersen, & Madsen, 2007). Its assessment
could identify weaknesses in safety culture and facilitate organisational learning (Itoh et al.,
2007; Nieva & Sorra, 2003), thus enabling health care organisations to improve patient safety
proactively. Such a proactive approach is preferable to traditional, reactive approaches, which
use lagging indicators, such as injury rates (Flin et al., 2000), to determine and reduce risks
after errors have occurred and patients have been harmed.
Moreover, safety culture and the structural components of a safety management
system (i.e. prospective and retrospective methods for risk analysis) are interrelated (see
Figure 7.1).
Proactive
Safety
Management
Organisational
Context:
Safety Culture
Methods:
Risk Analysis
Data:
Error Recovery
+
+
Figure 7.1: Risk analysis and safety culture: Mutual influence.
Safety culture could be seen as ―the motor that makes the structure of the SMS [safety
management system] work‖ (Hale, 2003, p. 194). Safety cultures in which people consider
the health care system infallible and reproach each other for making errors impede health care
Chapter 7
98
employees from openly discussing risks in a prospective analysis or from learning from
retrospectively reported incidents (if they are reported at all) (Nieva & Sorra, 2003).
Conversely, a culture in which safety is given priority and health care employees are aware of
risks and willing to share lessons can promote the success of prospective and retrospective
methods (Cannon & Edmondson, 2005; Hudson, 2001; Nieva & Sorra, 2003). Alternatively,
prospective and retrospective methods could positively influence safety culture (Aspden et
al., 2004; Carroll et al., 2002; Kaplan & Barach, 2002; Pronovost et al., 2007), which has also
been demonstrated in this dissertation (see Chapter 4). The present chapter addresses the
latter association by evaluating and discussing changes in safety culture in three Dutch
hospitals after an extensive safety management programme had been implemented. Such an
evaluation is one of the possible objectives of safety culture assessment as proposed by Nieva
and Sorra (2003). In addition, we explore which safety culture dimensions predict incident
reporting behaviour.
Dimensions of Safety Culture
In line with the idea that assessment precedes improvement and advancement (Nieva &
Sorra, 2003), researchers and health care organisations currently pay much attention to
measurement of safety culture. Assessment of safety culture is often conducted by means of
questionnaire-based surveys, for which many instruments are available (Flin et al., 2006; Flin
et al., 2000). Despite an ongoing debate regarding the fundamental dimensions of safety
culture, researchers seem to agree about the importance of management and supervisor
commitment to safety for both health care and industry (Firth-Cozens, 2003; Flin, 2007; Flin,
et al., 2006; Zohar, 2003). In other words, if managers at all levels give priority to safety
(instead of other organisational goals such as production or costs), this could positively
influence safety culture (Flin, 2007; Flin & Yule, 2004). In their study on barriers to the
implementation of patient safety interventions, Akins and Cole (2005) even found that
management commitment could be a prerequisite for the effective implementation of safety
management programmes.
In addition, safety system and work pressure appear to be important dimensions of
safety culture in health care (Flin, 2007; Flin et al., 2006). Safety system includes safety
training and the availability of personal protective equipment; work pressure indicates
whether employees can manage the workload and have sufficient time to work according to
safety procedures (Flin, 2007). In their review of measurements of patient safety culture,
Singla, Kitch, Weissman, and Campbell (2006) identified three core dimensions of safety
Trends in Safety Culture
99
culture: management commitment to safety, communication openness, and teamwork.
Communication openness can be defined as the extent to which issues about patient care are
communicated within the health care institution or unit; teamwork refers to the degree of
cooperation between health care employees within the institution or a particular unit (Singla
et al., 2006). Colla et al. (2005) consider incident reporting to be a key dimension of safety
culture. This is in line with the idea that greater rates of incident reporting equate to an
increased willingness to share lessons, which, in turn, is an important aspect of an advanced
safety culture (Hudson, 2003; Reason, 1998). To summarise, management commitment,
safety system (or policies and procedures), work pressure (or staffing), communication
openness, teamwork, and incident reporting are perceived to be important dimensions of
safety culture.
7.1 Methods
Procedure and Participants
A longitudinal panel survey was conducted in three Dutch hospitals, all belonging to the
same health care foundation: a teaching hospital that offers basic and specialised care (750
beds), a hospital that offers basic care (250 beds), and a hospital for outpatient treatment (50
beds). All units of the three hospitals participated in the study, ranging from intensive care
units, emergency departments, operating rooms, and nursing wards to hospital pharmacies,
various laboratories, and numerous outpatient departments.
In the period from November 2007 until June 2008, these hospitals implemented a
large-scale safety management programme. In the first quarter of 2008, a sophisticated
retrospective incident reporting and analysis system was introduced in all three hospitals (see
also Chapter 4). From November 2007 until June 2008, prospective risk analyses were
carried out at 14 selected units (see also Chapters 3 and 4). Further, from November 2007
until April 2008, discussion meetings were organised (see also Chapter 4). In those meetings,
a special film was shown in which a patient talks about a medical error in a very touching
way. This film was used to stimulate a debate, in which employees openly talked about errors
and incident reporting.
In October/November 2007 and September/October 2008, the same survey was
distributed among all employees of the three hospitals who had direct or indirect contact or
interaction with patients, such as doctors, nurses, laboratory assistants and technicians,
secretaries, and unit managers. This sampling strategy was in accordance with the procedure
proposed by Sorra and Nieva (2004). The procedure of distribution was similar in both years.
Chapter 7
100
Doctors received the surveys and cover letters by internal mail and were asked to return the
completed surveys by means of return envelopes. Unit managers were responsible for the
distribution of the surveys and cover letters among the other employees. Those employees
were asked to put the completed surveys in a box. After the deadline had passed, the central
investigator collected the completed surveys. At baseline, each employee was given a unique
identification number, which was retained and re-used at follow-up. After the surveys had
been distributed, the identification numbers were only available for the central investigator
and only used for analysis purposes.
At baseline and follow-up, 1359 and 1372 surveys were filled out and returned,
respectively. Because for the greater part the unit managers were responsible for the
distribution of the surveys, an exact response rate could not be calculated for all units.
Therefore, a minimum response rate was computed by making use of the maximum number
of employees that could have received a survey as the denominator. Those estimates were
similar for both assessments: 49.1% at baseline and 49.6% at follow-up, which appeared to
be average response rates in organisational research (Baruch & Holtom, 2008). Respondents
who filled out less than half of the survey items were excluded (Sorra & Nieva, 2004). At
baseline, data from 13 respondents were removed and at follow-up, the same was done with
data from 21 respondents. Finally, a panel of 701 respondents filled out and returned the
surveys in both years and were thus eligible for further analysis. Of those respondents, 76
(10.8%) were doctors.
Our panel survey yielded an attrition rate of 47.9%. In other words, 47.9% of the
respondents who participated at baseline, dropped out at follow-up. Analysis according to the
guidelines from Goodman and Blum (1996) was conducted to assess any attrition effects.
Logistic regression analysis revealed that attrition led to non-random sampling. More
specifically, independent-samples t tests showed that attrition influenced the average scores
of two safety culture dimensions, namely ―teamwork within hospital units‖ and
―communication openness‖. However, comparison of variances (with the normal
approximation to the chi-square distribution) demonstrated that attrition did not affect the
variances of the measures. Moreover, multiple regression analysis showed that attrition did
not influence the relations between the measures. In sum, those analyses indicated that
attrition did hardly affect the results.
Trends in Safety Culture
101
Survey and Measures
On behalf of the Agency for Healthcare Research and Quality (AHRQ), the Hospital Survey
on Patient Safety Culture was developed (Sorra & Nieva, 2004). This survey is regarded as a
valid and reliable instrument to assess safety culture in hospitals, because it is based on
literature and sound psychometric tests (Castle & Sonon, 2006; Colla, et al., 2005; Flin et al.,
2006). Translation and validation of the AHRQ survey finally resulted in a Dutch version
(COMPaZ), in which two original survey items had been left out (Smits, Christiaans-
Dingelhoff, Wagner, Van der Wal, & Groenewegen, 2007). Moreover, the final COMPaZ
survey uses a slightly different factor structure and consists of 11 safety culture dimensions,
while the AHRQ survey comprises 12 dimensions (Smits et al., 2007; Sorra & Nieva, 2004).
For our study, we used the first Dutch translation of the AHRQ survey, consisting of all 51
original survey items. However, for data analysis the COMPaZ factorial model was used to
enable benchmarking with other Dutch hospitals and to comply with national directives. The
survey contained demographic characteristics, such as unit and profession, outcome
measures, and items that can be grouped into 11 safety culture dimensions.
Together the 11 dimensions represented the core dimensions of safety culture (i.e.
management commitment, safety system, work pressure, communication openness,
teamwork, and incident reporting), although safety system was only partially covered (Colla
et al., 2005; Flin et al., 2006). The safety culture dimensions consisted of the average score of
two to six items (after reversing the scores of the negatively worded items). All items were
scored on 5-point Likert scales that either ranged from (1) ―strongly disagree‖ to (5) ―strongly
agree‖ or from (1) ―never‖ to (5) ―always‖. See the Appendix for a complete overview of the
11 safety culture dimensions and their corresponding items.
Teamwork across hospital units was measured with a five-item scale. The items
addressed the coordination and cooperation between hospital units. In particular, they
covered the exchange of information and transfer of patients. For instance, ―Things ‗fall
between the cracks‘ when transferring patients from one unit to another‖ (reverse coded).
Teamwork within hospital units was assessed with a four-item scale consisting of
statements about the cooperation between employees within hospital units. A sample item is:
―When one area in this unit gets really busy, others help out‖.
Hospital handoffs and transitions was assessed with two items related to the
consequences of shift changes within hospital units. For example, ―Important patient care
information is often lost during shift changes‖ (reverse coded).
Chapter 7
102
Frequency of incident reporting was measured with a three-item scale. The items
concerned the extent to which people reported errors, such as ―When a mistake is made that
could harm the patient, but does not, how often is this reported?‖.
Nonpunitive response to error was measured with a three-item scale that examined
whether the response to error was ―blame free‖. A sample item is: ―When an incident is
reported, it feels like the person is being written up, not the problem‖ (reverse coded).
Communication openness was assessed with a three-item scale. The items
concentrated on the extent to which people felt free to discuss issues related to patient safety.
For instance, ―Staff will freely speak up if they see something that may negatively affect
patient care‖.
Feedback about and learning from errors was measured with a six-item scale. The
items inquired whether people were informed about errors and whether interventions were
implemented to prevent their recurrence. For example: ―We are given feedback about
changes put into place based on incident reports‖.
Supervisor/manager expectations and actions promoting safety was adapted from
Zohar (2000) and assessed with a four-item scale. The items considered the expectations of
supervisors and the actions they took to promote patient safety. A sample item is: ―My
supervisor/manager seriously considers staff suggestions for improving patient safety‖.
Hospital management support for patient safety was measured with a three-item scale.
The items concerned perceptions about the importance that hospital management attached to
patient safety. For instance, ―The actions of hospital management show that patient safety is a
top priority‖.
Staffing was assessed with a three-item scale consisting of items related to work
pressure, such as: ―Staff in this unit work longer hours than is best for patient care‖ (reverse
coded).
Overall perceptions of safety was assessed with a four-item scale that included items
that did not address specific aspects of patient safety, but patient safety in general. A sample
item is: ―We have patient safety problems in this unit‖ (reverse coded).
Confirmatory factor analyses (CFA) (LISREL 8.72) were used to justify the COMPaZ
factorial model. To test the overall fit, we used several fit indices as proposed by Hair, Black,
Babin, Anderson, and Tatham (2006): the chi-square test (χ2), the root mean square error of
approximation (RMSEA), the non-normed fit index (NNFI), and the comparative fit index
(CFI). The COMPaZ factorial model consisting of 11 safety culture dimensions yielded the
Trends in Safety Culture
103
best solution: χ2(685) = 4444.51, p < .001, RMSEA = .09, NNFI = .86, and CFI = .87. It
should be noted that the large numbers of items and factors complicated CFA replication of
the COMPaZ factorial model. Moreover, the model‘s complexity and a perfect large sample
size imply that a non-significant chi-square test and cut-off values of .90 for NNFI and CFI
are not realistic and that RMSEA is a more reliable fit index because it corrects for model
complexity and sample size (cf. Hair et al., 2006; see also De Jonge, Van der Linden,
Schaufeli, Peter, & Siegrist, 2008). Based on this line of reasoning, the COMPaZ factorial
model shows reasonable fit, though there seems to be room for improvement, which can also
be concluded from the internal consistency coefficients as depicted below.
Table 7.1 presents the internal consistency coefficients for all safety culture
dimensions at baseline and follow-up, and the test-retest reliability scores. The test-retest
reliability scores indicated that the safety culture dimensions were relatively stable. Because
safety culture interventions could best be aimed at group or unit level (Smits et al., 2007;
Huang et al., 2007; Pronovost & Sexton, 2005), internal consistency coefficients (Cronbach‘s
α) of .60 could be considered acceptable in the present study (Evers, Van Vliet-Mulder, &
Groot, 2000).
Table 7.1
Internal consistency coefficients (Cronbach’s α) and test-retest reliabilities (rt) for safety
culture dimensions.
Cronbach‘s α
Safety culture dimension 2007 2008 rt
Teamwork across hospital units .76 .75 .48**
Teamwork within hospital units .63 .69 .36**
Hospital handoffs and transitions .56a
.58a
.48**
Frequency of incident reporting .82 .85 .38**
Nonpunitive response to error .60 .64 .41**
Communication openness .65 .67 .41**
Feedback about and learning from errors .71 .76 .48**
Supervisor/manager expectations and actions promoting safety .66 .71 .37**
Hospital management support for patient safety .69 .69 .44**
Staffing .57 .57 .49**
Overall perceptions of safety .64 .64 .40**
aPearson correlation (p < .01), indicating acceptable inter-item correlation and reliability. A
Pearson correlation was calculated, because Cronbach‘s α could not be calculated for this
two-item measure.
**p < .01.
Chapter 7
104
Except for the staffing dimension, the internal consistencies of the safety culture
dimensions were all acceptable (α > .60) and comparable to earlier studies in which the same
dimensions had been used (Smits et al., 2007; Snijders, Kollen, Van Lingen, Fetter, &
Molendijk, 2009). Several safety culture dimensions, such as ―nonpunitive response to error‖
and ―teamwork within hospital units‖, at baseline yielded internal consistencies that were just
above the cut-off level of .60. However, their internal consistencies increased at follow-up.
The internal consistency of the staffing dimension in our study (α = .57) was comparable to
its consistency in the studies by Smits et al. (2007) (α = .58) and Snijders et al. (2009) (α =
.54). However, due to the poor internal consistency of the staffing dimension at baseline and
follow-up, we decided to exclude this dimension from further analyses.
In addition to the safety culture dimensions, two outcome measures were included in
the survey: ―patient safety grade‖ and ―number of incidents reported‖.
Patient safety grade was a single-item measure (―Please give your work area / unit in
this hospital an overall grade on patient safety‖) scored on a 5-point Likert scale ranging from
(1) ―excellent‖ to (5) ―failing‖ (reverse coded).
Number of incidents reported was a single-item measure (―In the past 12 months, how
many incident reports have you filled out and submitted‖) with an ordinal 6-point scale
ranging from ―0‖ to ―21 or more‖ incident reports.
Because in the three hospitals of this study, all incidents could be reported,
irrespective of any presence of harm, a large number of reported incidents indicates a high
―detection sensitivity level‖ (Battles & Lilford, 2003; Kaplan, Battles, Van der Schaaf, Shea,
& Mercer, 1998). Moreover, an increased willingness to report errors is accompanied by trust
and readiness to share lessons, which, in turn, are important aspects of an advanced safety
culture (Hudson, 2003; Reason, 1998). Therefore, a higher score on the outcome measure
―number of incidents reported‖ equates to a more positive safety culture. After reversing the
scores of the negatively worded items and the outcome measure ―patient safety grade‖, a
higher score was thus associated with a more positive safety culture for all measures.
Data Analysis
Descriptive statistics (means and standard deviations) are presented for the safety culture
dimensions and the outcome measure ―patient safety grade‖ for baseline and follow-up.
Paired-samples t tests were used to evaluate changes over time. The outcome measure
―number of incidents reported‖ was dichotomised distinguishing between ―0‖ and ―1 or
Trends in Safety Culture
105
more‖ incidents reported in the last 12 months, coded as (0) and (1), respectively. For this
outcome measure, we used a McNemar test (with continuity correction) to explore
differences between baseline and follow-up. Additional descriptive statistics provided more
detailed information about the exact changes regarding the number of incidents reported. Due
to the large sample size, a conservative alpha level of .01 was used for the paired-samples t
tests and McNemar test.
Logistic regression analysis was carried out to explore which safety culture
dimensions predicted the number of incidents reported. The dichotomised outcome measure
―number of incidents reported‖ at follow-up was used as the criterion variable, which is a
self-reported measure indicating whether an employee filled out at least one incident
reporting form. Odds ratios (ORs) and 95% confidence intervals (CIs) were derived. In
addition, a multiple regression analysis was conducted to determine which safety culture
dimensions were associated with frequency of incident reporting. The criterion variable was
―frequency of incident reporting‖ at follow-up, referring to the extent to which errors were
generally reported. Beta coefficients (βs) and 95% confidence intervals (CIs) were obtained.
In the logistic and multiple regression analyses, an alpha level of .05 was used.
7.2 Results
Table 7.2 presents the means and standard deviations for the outcome measure ―patient safety
grade‖ and the safety culture dimensions for baseline and follow-up, as well as the results of
the paired-samples t tests. Despite the fact that after the baseline assessment an extensive
safety management programme had been implemented, only a few significant changes were
in fact identified with regard to safety culture. Paired-samples t tests only showed significant
positive trends for the safety culture dimensions ―frequency of incident reporting‖, t(577) =
2.63, p ≤ .01, ―nonpunitive response to error‖, t(663) = 2.55, p ≤ .01, and ―hospital
management support for patient safety‖, t(628) = 2.99, p ≤ .01.
Table 7.2
Means, standard deviations, and paired-samples t tests for safety culture measures (N = 701).
2007 2008
Safety culture measure M SD M SD Mean difference t df p
Patient safety grade 3.23 0.62 3.22 0.59 -0.01 -0.54 645 .59
Teamwork across hospital units 2.89 0.54 2.87 0.53 -0.02 -0.77 639 .44
Teamwork within hospital units 3.91 0.40 3.87 0.45 -0.04 -2.17 694 .03
Hospital handoffs and transitions 3.39 0.65 3.37 0.65 -0.02 -0.82 619 .41
Frequency of incident reporting 3.04 0.95 3.15 0.90 0.11 2.63 577 .01**
Nonpunitive response to error 3.49 0.59 3.56 0.58 0.07 2.55 663 .01**
Communication openness 3.80 0.56 3.79 0.56 -0.01 -0.60 655 .55
Feedback about and learning from errors 3.37 0.53 3.38 0.57 0.01 0.50 681 .62
Supervisor/manager expectations and actions promoting safety 3.47 0.53 3.47 0.55 0.00 -0.29 632 .77
Hospital management support for patient safety 3.07 0.62 3.14 0.64 0.07 2.99 628 .00**
Overall perceptions of safety 3.38 0.57 3.34 0.58 -0.04 -1.76 683 .08
Note. Higher scores were associated with a more positive safety culture for all measures.
**p ≤ .01.
Trends in Safety Culture
107
A significant positive change was found for the dichotomised outcome measure
―number of incidents reported‖, χ2(N = 669) = 44.78, p < .001. At follow-up, significantly
more employees filled out at least one incident reporting form than at baseline. Table 7.3
provides more detailed information about the distribution of the respondents over the six
response categories of the outcome measure ―number of incidents reported‖. As shown by the
McNemar test, at follow-up significantly more respondents (n = 418; 62.5%) claimed that
they had reported at least one incident than at baseline (n = 325; 48.6%). This positive change
is reflected in all response categories. For instance, at baseline, only 3 respondents (0.4%)
stated that they had reported 11 incidents or more, while at follow-up, 25 respondents (3.7%)
made such a claim. Figure 7.2 shows the percentage increase for each response category
when comparing the results of the two assessments. In accordance with the result of the
McNemar test, the chart shows a decrease for the response category ―none‖. The chart shows
furthermore an increase for all other response categories, demonstrating that the increased
willingness to report incidents held true both for people who used to report incidents
occasionally and for people who had already been filling out incident reports on a more
regular basis. This finding is consistent with the results presented in Chapter 4, which showed
that the newly implemented incident reporting and analysis system yielded a 400% increase
in the overall average number of reported incidents per employee.
Table 7.3
Number of incidents reported at baseline (2007) and follow-up (2008).
2007 2008
Response category Frequency (%) Frequency (%)
None 344 (51.4) 251 (37.5)
1 to 2 227 (33.9) 246 (36.8)
3 to 5 71 (10.6) 108 (16.1)
6 to 10 24 (3.6) 39 (5.8)
11 to 20 1 (0.1) 16 (2.4)
21 or more 2 (0.3) 9 (1.3)
Total 669 (100.0) 669 (100.0)
Chapter 7
108
Figure 7.2: Percentage increase of the number of incidents reported when comparing
the baseline results in 2007 to those of follow-up in 2008.
Table 7.4 presents the results of the logistic regression analysis. In Model 1, we
controlled for the number of incidents reported at baseline. In Model 2, by using stepwise
regression, we explored which safety culture dimensions predicted the number of incidents
reported. Model 2 showed that feedback about and learning from errors explained an
additional amount of variance in the number of incidents reported, over and above the
amount of variance that was explained by the baseline measure (Δ-2 Log likelihood = 9.37, df
= 1, p < .01). The accompanying values of the pseudo R2 measures were .20 and .27,
indicating that Model 2 accounted for 20 to 27% of the variation in number of incidents
reported, χ2(2) = 116.87, p < .001. More specifically, Model 2 showed that the number of
incidents reported at baseline significantly predicted the number of incidents reported at
follow-up (OR = 8.10, p < .001). In other words, employees who had filled out at least one
incident reporting form in 2007 were eight times more likely to report at least one incident in
2008 than people who had not reported incidents in 2007. Further, the model demonstrated a
significant positive association between feedback about and learning from errors and the
number of incidents reported (OR = 1.81, p < .01). Employees who felt that they received
feedback about errors and that learning took place were about two times more likely to fill
out at least one incident reporting form than those who were less positive about feedback and
learning mechanisms.
Table 7.4
Summary of logistic regression analysis for variables predicting the number of incidents reported 2008 (N = 701).
95% CI OR
Model and variable(s) B SE B OR Lower Upper -2 Log likelihood χ2 Δ-2 Log likelihood
Model 1a
577.13 107.50***
Number of incidents reported 2007 2.04*** 0.21 7.65 5.04 11.61
Model 2b 567.76 116.87*** 9.37**
Number of incidents reported 2007 2.09*** 0.22 8.10 5.29 12.40
Feedback about and learning from errors 2007 0.59** 0.20 1.81 1.23 2.65
Note. B = unstandardised coefficient; OR = odds ratio; CI = confidence interval. aCox & Snell R
2 = .19; Nagelkerke R
2 = .25.
bCox & Snell R
2 = .20; Nagelkerke R
2 = .27.
**p < .01, ***p < .001.
Chapter 7
110
The logistic regression model showed which safety culture dimensions predicted
whether employees would fill out at least one incident reporting form. In addition, a multiple
regression analysis was conducted to explore the predictors of frequency of incident
reporting, that is, the extent to which errors were generally reported. The results of this
analysis are summarised in Table 7.5. In Model 1, we included frequency of incident
reporting at baseline as a control variable. Next, stepwise regression was used to identify the
predictors of frequency of incident reporting. For purposes of conciseness, only the results of
the final regression model (i.e. Model 4) are presented and discussed. The details involved in
adding the separate variables (i.e. Models 2 and 3) have been omitted. Model 4 revealed that
a regression variate including three other safety culture dimensions explained a greater
proportion of variance in frequency of incident reporting than a variate including only the
baseline measure (ΔR2 = .06). Model 4 accounted for 22% of the variance in frequency of
incident reporting, F(4) = 33.87, p < .001. Besides frequency of incident reporting at baseline
(β = .32, p < .001), other significant predictors of frequency of incident reporting were:
feedback about and learning from errors (β = .16, p < .001), hospital handoffs and transitions
(β = .11, p < .01), and teamwork within hospital units (β = .08, p < .05). Consistent with the
findings from the logistic regression analysis, feedback about and learning from errors
seemed to be positively associated with frequency of incident reporting. In addition, shift
changes and teamwork were positively associated with the extent to which errors were
generally reported.
Table 7.5
Summary of multiple regression analysis for variables predicting frequency of incident reporting 2008 (N = 701).
95% CI β
Model and variable(s) B SE B β Lower Upper R2
F ΔR2
Model 1
.16 96.65***
Frequency of incident reporting 2007 0.38*** 0.04 .40 .30 .45
Model 4 .22 33.87*** .06
Frequency of incident reporting 2007 0.30*** 0.04 .32 .22 .38
Feedback about and learning from errors 2007 0.27*** 0.07 .16 .12 .41
Hospital handoffs and transitions 2007 0.15** 0.05 .11 .05 .26
Teamwork within hospital units 2007 0.19* 0.09 .08 .00 .37
Note. Results are only provided for the final regression model (i.e. Model 4) that resulted from the stepwise regression. Models 2 and 3 have
been omitted. B = unstandardised coefficient; CI = confidence interval.
*p < .05, **p < .01, ***p < .001.
Chapter 7
112
7.3 Discussion
It is widely argued that safety management in health care will not reach its full potential
without achievements in safety culture (Aspden et al., 2004; Hudson, 2001; Nieva & Sorra,
2003; Pronovost & Sexton, 2005). A more positive safety culture could enhance safety
behaviour and safety performance, thereby indirectly improving patient safety (Aspden et al.,
2004; Clarke, 2006b; Flin, 2007; Flin et al., 2006; Neal et al., 2000). For those reasons, safety
culture has recently been put forward as one of the top priorities for patient safety research
(Bates, Larizgoitia, Prasopa-Plaizier, & Jha, 2009). The present study responded to this need
by evaluating changes in safety culture in three Dutch hospitals after a large-scale safety
management programme had been implemented. In addition, we examined which safety
culture dimensions predicted incident reporting behaviour.
Theoretical Implications
The most obvious result of our study is the fact that at follow-up respondents self-reported a
significant larger number of reported incidents than at baseline. Moreover, a significant
positive trend was observed regarding the safety culture dimension ―frequency of incident
reporting‖, that is, the extent to which errors were generally reported. Those advances in
incident reporting behaviour can probably be largely ascribed to the extensive safety
management programme consisting of prospective and retrospective methods for risk
analysis, in which learning is the principal objective (see also Chapter 4). This assumption is
supported by our logistic and multiple regression analyses, which both showed that incident
reporting behaviour was positively associated with feedback about and learning from errors.
Research has revealed a lack of feedback as a possible barrier to incident reporting (Holden
& Karsh, 2007; Kingston et al., 2004; Shojania, 2008). Evans et al. (2006) even found that
about two thirds of all respondents in their study mentioned poor feedback as the major
impediment to report incidents. In the incident reporting and analysis system that has been
implemented in the three hospitals of our study, all employees can report incidents
electronically. Special unit-based committees deal with the incident reports. They gather
supplementary information and uncover the causes of the incidents. If necessary, they
propose or implement actions for improvement. This decentralised approach brings about
short feedback loops and subsequent learning, which are considered important to promote
incident reporting (Benn et al., 2009; Kaplan & Rabin Fastman, 2003).
The positive change regarding the outcome measure ―number of incidents reported‖
appeared to be twofold. At follow-up, significantly more respondents than at baseline claimed
Trends in Safety Culture
113
that they had reported at least one incident. Moreover, at follow-up more respondents than at
baseline claimed that they had reported several (or even many) incidents. To put it
differently, at follow-up (1) more employees had been involved in incident reporting and (2)
in general, employees had reported more incidents. This might indicate that, in addition to the
increased feedback, the safety management efforts in the three hospitals have reduced or
eliminated both social and technical barriers to incident reporting, as distinguished by
Cannon and Edmondson (2005). Possibly, the interventions did remove certain social barriers
to incident reporting, such as shame or fear of punishment (see also Chapter 4), as a result of
which employees decided to start reporting incidents. The significant positive change
regarding the safety culture dimension ―nonpunitive response to error‖ might support this
assumption. Although our regression analyses did not reveal this safety culture dimension as
a significant predictor of incident reporting behaviour, Snijders et al. (2009) did find a
positive association between a nonpunitive response to error and the number of incidents
reported. Yet, the safety management programme might also have removed impediments to
error recognition (see also Chapter 4) or more technical barriers to incident reporting, like the
complexity of the reporting form, whereby employees who had already reported incidents
before, became willing to report even more incidents.
In addition to the positive influence of feedback and learning mechanisms on incident
reporting behaviour, the multiple regression analysis showed that hospital handoffs and
transitions as well as teamwork within hospital units also were positively associated with
frequency of incident reporting. This finding is consistent with the idea that teamwork is one
of the most important dimensions of safety culture (Singla et al., 2006). As argued by other
researchers, good teamwork in terms of coordination, cooperation, communication, and
personal relationships could encourage health care employees‘ willingness to report errors
(Edmondson, 1996; Wilson, Burke, Priest, & Salas, 2005).
Reason (1998) stated that a positive safety culture is a culture in which people are
informed about risks. He claimed that to achieve this, people should be willing to disclose
errors and to share lessons, that is, a reporting culture should be effected. He stated that such
a reporting culture, in turn, requires a so-called just culture; a culture in which people trust
each other and are not blamed or punished in case of errors. Our findings could indicate that
in the three hospitals under investigation progress has been made in terms of the reporting
culture and the underlying just culture. Evidently, the reporting culture has improved, as
indicated by the advancements regarding the safety culture dimension ―frequency of incident
reporting‖, the outcome measure ―number of incidents reported‖ and the observed increase in
Chapter 7
114
the overall average number of reported incidents per employee (see Chapter 4). Besides, the
fact that at follow-up people perceived the response to error to be more blame free than at
baseline might suggest that the safety culture has evolved towards a just culture.
Further, a significant positive trend was observed with regard to hospital management
support for patient safety. At follow-up, more than at baseline, employees perceived that
hospital management considered patient safety as a top priority. Most likely, this could be
ascribed to the large-scale safety management programme that had been implemented. This
finding is promising, since researchers claim that management support is a key element of a
positive safety culture (Firth-Cozens, 2003; Flin, 2007; Flin et al., 2006; Zohar, 2003). More
specifically, it is argued that managers at all levels have an important role in creating a
learning culture (Cannon & Edmondson, 2001; Carroll & Edmondson, 2002; Edmondson,
2004; Mohr, Abelson, & Barach, 2002). Akins and Cole (2005) even found that a lack of
senior leadership could constitute a major barrier to the implementation of patient safety
interventions in health care organisations.
Although the observed positive trends are promising, only a few significant changes
regarding safety culture were identified, whereby the use of safety culture as an outcome
measure could be questioned. The lack of significant changes in safety culture might be
related to the statement that safety culture measures do not, by definition, mirror actual safety
behaviour (Cooper & Philips, 2004). As has been concluded in Chapter 4, the safety
management programme that had been implemented in the three hospitals of our study has
resulted in actual positive changes in incident reporting behaviour and the extent to which
lessons about risks and errors are shared. Those advances could have been reflected by
significant positive changes in certain safety culture dimensions such as ―feedback about and
learning from errors‖ and ―communication openness‖. The lack of such changes could
indicate that a safety management programme like the one in our research might indeed lead
to progress in safety behaviour without any perceptible changes in safety culture (Cooper &
Philips, 2004). On the other hand, safety culture might be such long-lasting (Guldenmund,
2000) that a larger time interval between baseline and follow-up might be necessary to be
able to underpin changes in safety behaviour through comparable changes in safety culture.
Alternatively, the lack of significant changes in safety culture might partly be
explained by a so-called ―ceiling‖ effect. This would imply that respondents who at baseline
scored high on a particular measure, would show only limited (or no) response to an
intervention aimed at increasing respondents‘ scores on that measure at follow-up (Taris,
2000; Wilder, 1967). In our study, this ceiling effect could possibly explain the lack of a
Trends in Safety Culture
115
positive change with respect to communication openness. A final explanation for the stability
of safety culture could be that the assessment of safety culture itself might be considered an
intervention, as suggested by Nieva and Sorra (2003). In an evaluation of several patient
safety interventions in four Belgian hospitals, Hellings (2009) also hardly found any
significant changes regarding safety culture. He suggested that a baseline assessment itself
could bring about changes in awareness of patient safety and arouse high expectations of
change. The fact that subsequently interventions fall short of those high expectations might
explain constant or even diminished scores on safety culture measures at follow-up.
Practical Implications
Our results suggest that safety management programmes could be effective in terms of
improved safety behaviour and still not yield any significant positive changes in safety
culture (Cooper & Philips, 2004). This might imply that it is problematic to use self-reported
safety culture surveys as an evaluation instrument. However, other outcome measures, such
as injury rates, also face measurement problems (Flin, 2007; Ginsburg, Norton, Casebeer, &
Lewis, 2005). Therefore, health care organisations can apply self-reported safety culture
surveys for the evaluation of safety management efforts, but preferably, other measures
should be used as well (Pronovost et al., 2006). To put it differently, triangulation of
measures can yield a more reliable insight into the effectiveness of patient safety
interventions thanks to convergent evidence.
In spite of the possible problems related to the use of self-reported safety culture
surveys as an evaluation instrument, such surveys can reveal important weaknesses regarding
safety culture, such as punitive responses to errors or limited management support for patient
safety. Health care organisations could use such insights to implement interventions to
improve safety culture (Nieva & Sorra, 2003; Itoh et al., 2007). In addition to safety culture
assessment through surveys, follow-up qualitative research could support health care
organisations in identifying the underlying organisational problems and determining
appropriate interventions (Flin et al., 2006; Hellings, 2009; Nieva & Sorra, 2003). It should
be noted that the group or unit level appears to be dominating for safety culture, and
therefore, interventions can best be aimed at groups or units (Smits et al., 2007; Huang et al.,
2007; Pronovost & Sexton, 2005).
The multivariate regression analysis of our study has demonstrated the importance of
feedback and learning mechanisms as well as teamwork to enhance reporting culture, which,
in turn, is an important aspect of safety culture. These findings endorse the viewpoints of
Chapter 7
116
Edmondson (1996) and Wilson et al. (2005) that teams in health care organisations can
develop into so-called self-correcting teams (Hackman, 1993) to foster incident reporting. In
such self-correcting teams, team members are preoccupied with risks, aim to detect and
correct errors in time, and are willing to report and discuss errors to share lessons. In general,
team training could concentrate on coordination and cooperation between team members, for
instance to realise smooth shift changes. Moreover, crew resource management trainings as
developed in aviation could be used to enhance cognitive and social skills, such as situation
awareness and error management (Helmreich, 2000, Flin & Maran, 2004; Musson &
Helmreich, 2004; Sexton, Thomas, & Helmreich, 2000).
Health care organisations can thus enhance local feedback within teams by promoting
self-correction mechanisms within these teams. However, health care organisations are
recommended to implement feedback mechanisms at the organisational level as well (Benn et
al., 2009). In their recent review on feedback from incident reporting, Benn et al.
distinguished several important types of feedback. For instance, health care organisations
could inform reporters about the progress of their incident reports to ensure them that the
reports are acted upon. Furthermore, Benn et al. suggested that organisation-wide feedback
about identified problems and actions that are taken is important. The former could increase
risk awareness among health care employees, while the latter could convince health care
employees of the usefulness of incident reporting and could create support for improvements.
According to Benn et al., feedback can best be provided in a variety of forms, such as e-mail
notifications and individual debriefings about the status and progress of incident reports, team
briefings to inform people about observed risks and corrective actions, and targeted
campaigns aimed at specific incident types.
It should be noted that creating a learning culture requires effective leadership
(Cannon & Edmondson, 2001; Carroll & Edmondson, 2002; Edmondson, 2004; Mohr et al.,
2002). Hence, management commitment at all levels is important for both the development of
self-correcting teams (Edmondson, 1996; Wilson et al., 2005) and the implementation of
feedback and learning mechanisms (Benn et al., 2009). Executive managers and team leaders
should give evidence of the priority of safety and show that problems are dealt with. This
emphasis on management commitment corresponds to the viewpoint that changes in safety
culture can best be produced through advances in teamwork, communication, and leadership
(Nieva & Sorra, 2003; Singla et al., 2006).
Trends in Safety Culture
117
Study Strengths, Limitations, and Future Research
Our study has some important strengths. First, its longitudinal design has made it possible to
evaluate changes in safety culture instead of only diagnosing its state. Second, we have
assessed safety culture across all units in the three participating hospitals, which is necessary
to obtain a complete picture, because safety culture varies across units (Huang et al., 2007;
Pronovost & Sexton, 2005). Notwithstanding those strengths, this study has also some
limitations. Exact response rates could not be calculated due to our distribution procedure.
However, the estimated minimal response rates appeared to be acceptable. Further, attrition
was present, for instance because employees had left the hospital after the baseline
assessment, or employees were on holiday in the period of the follow-up assessment. Despite
the fact that analysis has shown that attrition hardly affected the results, attrition and non-
response might still limit the external validity of our findings. The somewhat disappointing
results of the confirmatory factor analysis and reliability analysis ask for follow-up research
on the factorial model and psychometric quality of the AHRQ and COMPaZ surveys.
Despite the fact that we observed several statistically significant changes in safety
culture, the practical relevance of those trends might be questioned. On the other hand, the
fact that a few significant trends were identified is promising since safety culture is
considered to be long-lasting (Guldenmund, 2000). Further, the regression analyses revealed
feedback about and learning from errors as an important predictor of incident reporting
behaviour, after we controlled for the stability of the criterion variables. All measures were
based on self-reports, which might have inflated the correlations between certain variables
due to so-called common method variance. However, this potential bias was refuted by
Spector (2006), who stated that self-report measures are not necessarily subject to such bias
and that researchers should instead consider specific biases underlying their studies. In the
present study, the observed positive trend regarding the self-reported outcome measure
―number of incidents reported‖ was consistent with the actual increase in the number of
incident reports (see Chapter 4), which triangulates our findings.
Nevertheless, future research could concentrate on the use of more objective measures
to determine the extent to which an advanced safety culture produces positive changes in
safety performance, that is, fewer medical errors and less patient harm. Such research should
take into account that only prospective study designs, in which safety culture is assessed prior
to determining error rates, appear to be valid (Clarke, 2006b). Further, future studies could
model the exact relations between safety culture and the structural components of a safety
management system, that is, prospective and retrospective methods for risk analysis. In
Chapter 7
118
contrast with the present study, in which safety culture was assessed by means of two distinct
snapshots, we encourage other researchers to use multiple waves of measurement and a
longer study period.
Conclusions
The present study has revealed several positive trends regarding safety culture. However, the
limited number of significant changes endorses the stability of safety culture and questions
the use of self-reported safety culture surveys to evaluate the effects of safety management
programmes. Taken into account that improvements in safety culture thus appear to be
difficult to achieve, this would imply that the observed advances regarding incident reporting
behaviour, response to errors, and management support are promising. Health care
organisations can foster incident reporting behaviour by providing feedback about errors and
enhancing teamwork. Such interventions could improve the reporting culture, the overarching
safety culture, and ultimately, patient safety as well.
119
Chapter 8
General Discussion
Health care is a high-risk environment, in which medical errors seem to be unavoidable and
occur frequently. The harm and additional costs involved with those errors ask for effective
safety management. The ultimate goal of safety management efforts in health care should be
zero or at least minimal patient harm (Battles & Lilford, 2003) and therefore, a proactive
approach is essential. Health care organisations should aim to anticipate and reduce risks
before harm is caused (Battles et al., 2006; Hollnagel, 2008; Rath, 2008). However, so far,
health care organisations have particularly used reactive approaches to improve patient safety
(Karsh et al., 2006; Pronovost et al., 2003) and proactive safety management is still
immature. On the basis of this finding, we formulated the main research question of this
dissertation:
How could health care organisations apply proactive safety management to prevent
patient harm and minimise costs of poor safety?
This dissertation has proposed three distinct but complementary approaches towards
proactive safety management (see Figure 8.1): (1) conducting and integrating prospective and
retrospective risk analyses (methods), (2) obtaining information about error recovery (data),
and (3) improving on safety culture (organisational context).
Chapter 8
120
Proactive
Safety
Management
Organisational
Context:
Safety CultureCh. 4 / 5 / 7
Methods:
Risk AnalysisCh. 2 / 3 / 4 / 7
Data:
Error RecoveryCh. 5 / 6
Figure 8.1: Three approaches towards proactive safety management.
To minimise patient harm, it is essential to foresee risks before harm is done and
hence, to conduct prospective analyses (Battles et al., 2006; Hollnagel, 2008; Rath, 2008).
We have systematically evaluated the benefits and drawbacks of a prospective risk analysis
method (Healthcare Failure Mode and Effect Analysis, HFMEA™) by applying it to 13
processes in Dutch health care (Chapter 2). Though participants perceived HFMEA™ to be
valuable, this study has shown that HFMEA™ can be improved. Notwithstanding the
usefulness of prospective methods, just like retrospective methods for risk analysis, they are
subject to biases. The qualitative field study presented in Chapter 3 has demonstrated the
added value of triangulating and integrating prospective and retrospective methods on two
units of a Dutch general hospital. This combined application appears to be beneficial in terms
of more complete and reliable risk identification and assessment. Further, a quasi-
experimental field study on 12 units of two Dutch general hospitals has indicated that
conducting a prospective analysis before the introduction of a sophisticated incident reporting
and analysis system can enhance incident reporting behaviour (Chapter 4).
Information about the way errors are discovered and corrected (i.e. error recovery)
can be useful to improve patient safety proactively. Analysis of near misses could anticipate
errors (Aspden et al., 2004; Barach & Small, 2000; Kaplan & Rabin Fastman, 2003; Van der
Schaaf & Wright, 2005) and induce health care organisations to implement or enhance
effective error recovery strategies, which is vital due to the unavoidability of errors (Aspden
et al., 2004; Hollnagel, 2008; Kanse et al., 2006). The qualitative field study presented in
General Discussion
121
Chapter 5 addressed the importance of a clearer and more consistent definition of near misses
to enable their large-scale reporting and analysis. On the basis of retrospective analysis of
143 error handling processes of four units of two Dutch general hospitals, we have suggested
two possible definitions of near misses. The qualitative field study on 52 medication errors
presented in Chapter 6 has recognised accidents as a supplementary source of information
about error recovery, because they can provide insight into failed, missed and absent error
recovery opportunities.
A positive safety culture can be essential for proactive safety management (Aspden et
al., 2004; Hudson, 2001; Nieva & Sorra, 2003; Pronovost & Sexton, 2005). If health care
employees constantly strive for safety (Hale, 2003; Nieva & Sorra, 2003; Pronovost et al.,
2003) and minimal patient harm, this might improve their safety behaviour and performance
(Aspden et al., 2004; Clarke, 2006b; Flin, 2007; Flin et al., 2006; Neal et al., 2000).
Moreover, safety culture could be regarded as the foundation of effective safety management,
since a positive safety culture can contribute to the successful application of prospective and
retrospective methods (Cannon & Edmondson, 2005; Hudson, 2001; Nieva & Sorra, 2003).
On the other hand, the structural components of a safety management system (i.e. prospective
and retrospective methods for risk analysis) can positively influence safety culture (Aspden et
al., 2004; Carroll et al., 2002; Kaplan & Barach, 2002; Pronovost et al., 2007), which to some
extent has been demonstrated in our longitudinal panel survey among 701 health care
employees of three Dutch hospitals (Chapter 7).
8.1 Methodological Considerations
One of the major strengths of the research presented in this dissertation is the multifaceted
approach. The combined use of three approaches towards proactive safety management (i.e.
risk analysis, error recovery, and safety culture) has enabled us to add to various subfields of
safety management research. Moreover, this wide scope may well have practical relevance
since health care organisations differ greatly regarding their achievements in safety
management. Hence, each health care organisation might have a different point of departure
and might thus prefer a tailor-made approach towards proactive safety management.
A strong point of our research is the use of multiple methods for data collection. In
our quasi-experimental field study, we used data from the incident reporting and analysis
system and from evaluation forms to test our hypotheses regarding incident reporting
behaviour. This triangulation of data sources has strengthened our findings. Similarly, we
used two data sources to describe error handling processes. By the simultaneous use of
Chapter 8
122
incident reports and interviews we have obtained a more complete picture of the way people
recognise and correct medical errors. Moreover, the quasi-experimental design of the study
on the order of implementation of prospective and retrospective methods and the longitudinal
design of our survey on safety culture are important strengths as well.
Further, the fact that some of the studies were carried out successively made it
possible to build on our own findings. On the basis of our systematic evaluation of
HFMEA™, we have put forward practical recommendations regarding its application. Some
of those suggestions were tried out during the prospective analyses that were conducted as
part of subsequent studies. Further, on the basis of the results of the study on the integration
of prospective and retrospective methods, we assumed that participation in a prospective
analysis might positively influence incident reporting behaviour. In our study on the order of
implementation of prospective and retrospective methods we explicitly tested this
assumption.
A final strength of the current research is its practical relevance. All studies were
field studies, facing us with the possibilities and impossibilities of safety management efforts
in health care, and forcing us to propose feasible suggestions, such as the relatively simple
guidelines for the integration of prospective and retrospective methods. Given the social
impact of the patient safety issue, this practical relevance of our research is of the utmost
importance.
Despite those strengths, our research has also some limitations. First, nearly all
studies concentrated on hospitals only, rather than on health care organisations in general.
Although this focus might be obvious since hospitals lead the way with regard to safety
management efforts (e.g., Castle & Sonon, 2006; Sandars & Esmail, 2003), it might still limit
the external validity of our findings. However, in our study on the evaluation of the
application of HFMEA™ in Dutch health care, primary care was included as well.
Furthermore, in other studies we purposely selected those hospital units that represented a
variety of settings; that is, different specialties, inpatient and outpatient departments, and
acute and non-acute care. In our panel survey on safety culture, we even included all hospital
units.
We used HFMEA™ (or an adapted version) for all prospective analyses. This seems
to be a sound decision because the suggested components of a prospective analysis as
proposed by organisations such as the Joint Commission on Accreditation of Healthcare
Organizations (JCAHO) are all part of HFMEA™ (The Joint Commission, 2009: Standard
LD.04.04.05). However, the decision to solely use HFMEA™ might prevent us from
General Discussion
123
generalising our results to other methods, like Hazard Analysis and Critical Control Points
(HACCP). Notwithstanding any differences between prospective methods, nearly all methods
are based on the same underlying approach. One first maps the process under investigation;
then, one identifies and assesses possible risks, followed by the determination of appropriate
actions for improvement. Therefore, our findings and suggestions might (to a large extent)
hold true for other prospective methods as well.
In several studies, retrospective incident analyses were conducted. A limitation of
such analyses is the potential for hindsight and recall bias. Hindsight bias refers to the
inclination of people to (wrongly) assume that they could have foreseen the incident
beforehand (Henriksen & Kaplan, 2003). Recall bias refers to the fact that people might have
difficulties remembering what exactly happened. In general, we tried to limit those biases by
gathering information as soon as possible after an incident occurred, by cross-validating
findings, as well as by considering positive behaviour (Carthey et al., 2001; Kaplan &
Barach, 2002).
Finally, in the studies on the order of implementation of prospective and retrospective
methods and the definition of near misses, robust statistical testing was not always possible
due to a limited number of participating units and/or a relatively small dataset. However,
most research questions could still be answered by making use of non-parametric tests or
regrouping data.
8.2 Theoretical Implications
In this section, the theoretical contributions of this dissertation are clustered by the three
approaches towards proactive safety management; that is, risk analysis (methods), error
recovery (data), and safety culture (organisational context).
Risk Analysis
Earlier studies have explored and assessed the application of prospective analyses, such as
FMEA or HFMEA™, in health care (e.g., Jeon et al., 2007; Kunac & Reith, 2005;
Wetterneck et al., 2006). In contrast with those single-case studies, we systematically
evaluated the use of HFMEA™ by means of a larger set of HFMEA™ analyses, which
strengthens our findings. Several researchers cited the multidisciplinary nature of an
HFMEA™ analysis as an important benefit (Esmail et al., 2005; Wetterneck et al., 2004;
Wetterneck et al., 2006), which is consistent with our findings. Others mentioned that
carrying out an HFMEA™ analysis can yield an increased understanding of processes, tasks,
Chapter 8
124
and risks (Linkin et al., 2005), which has also appeared to be an important strength in our
study. On the basis of single-case studies, many researchers argued that the time investment
that is associated with a FMEA or an HFMEA™ analysis is large (Carstens, 2006; Kunac &
Reith, 2005; Linkin et al. 2005; Wetterneck et al., 2004; Wetterneck et al., 2006). Our
evaluation of 13 HFMEA™ analyses endorses this claim. Further, in line with the findings of
Jeon et al. (2007), Wetterneck et al. (2004) and Wetterneck et al. (2006), our results have
indicated that the use of rating scales to assess identified risks is perceived to be difficult.
On the basis of our systematic evaluation of multiple HFMEA™ analyses we suggest
that HFMEA™ can be improved. We have put forward several suggestions to improve its
perceived utility and acceptance in health care. Some of these recommendations were
successfully applied in subsequent studies. For instance, we customised the rating scales as
also proposed by Jeon et al. (2007) and Wetterneck et al. (2004) and we replaced the numbers
in the hazard scoring matrix by ordinal scale categories. We further omitted the decision tree
for the failure mode causes and instead, directly labelled the most important ones. Although
not tested systematically, those modifications seemed to simplify risk evaluation and reduced
the time necessary to complete an HFMEA™ analysis.
Our research has supported the common viewpoint that both prospective and
retrospective methods can be useful for improving patient safety and optimising processes
(Battles et al., 2006; Hollnagel, 2008; Rath, 2008). However, both methods are subject to
specific biases such as inaccurate risk assessment, incomplete data, and hindsight and recall
bias. Therefore, several researchers proposed triangulation of prospective and retrospective
methods to overcome those biases and to obtain a more complete and reliable picture of risks
(Battles & Lilford, 2003; Herzer et al., 2008; Runciman et al., 2006; Senders, 2004). Our
research has endorsed this theoretical contention by means of empirical data. As suggested by
Runciman et al. (2006) and Senders (2004), we have demonstrated that prospective and
retrospective analyses can yield different insights into the scale and nature of risks. The
combined application of the two methods can thus provide a more complete and less biased
overview of risks because of convergent evidence.
In line with earlier studies (e.g., Trucco & Cavallin, 2006; Van der Hoeff, 2003;
Wetterneck et al., 2006), we explored several possibilities for the integration of prospective
and retrospective methods. Similar to Trucco and Cavallin (2006) and Wetterneck et al.
(2006) we used retrospective information about incidents to develop prospective failure
scenarios. Moreover, we have established the perceived usefulness of this integration, which
was not explicitly assessed in the studies mentioned before. Further, we have demonstrated
General Discussion
125
that integration of prospective and retrospective methods can facilitate direct comparison of
the outcomes of the analyses, as long as risks are classified similarly in both methods. Such a
comparison of results could uncover biases, whereby prospective and retrospective methods
could be improved, as suggested by Aspden et al. (2004).
Our quasi-experimental study addressed the order of implementation of prospective
and retrospective methods. This is an underdeveloped issue in safety management research
and so far, it has received little attention (Hale, 2003). Our results have indicated that it is
preferred to carry out a prospective analysis before the introduction of a sophisticated
incident reporting and analysis system. This order of implementation could yield maximum
results regarding incident reporting behaviour. Our research supports the finding of Evans et
al. (2007) that it is possible to expand the range of reported incidents, which is essential to
obtain a complete overview of risks. Conducting a prospective analysis first can enlarge the
spectrum of reported incident types, both directly through increased understanding of
possible risks (Battles et al., 2006) and increased error recognition (Kontogiannis & Malakis,
2009), and indirectly through the increased willingness of doctors to report incidents as
proposed by other researchers (Evans et al., 2006; Ligi et al., 2008; Nuckols et al., 2007).
Possibly, the open and positive atmosphere during a prospective analysis causes doctors to
stop normalising errors and instead, convinces them to disclose errors.
Error Recovery
Prospective and retrospective methods can be used to identify risks and take measures to
eliminate or reduce those risks. However, since errors are unavoidable, it is sensible to
promote error recovery, too (Aspden et al., 2004; Hollnagel, 2008; Kanse et al., 2006). In this
respect, Affonso and Jeffs (2004) emphasise the importance of research on error recovery
patterns and the use of insights from other safety critical industries, such as aviation and the
chemical industry. By means of empirical data of 143 error handling processes, we have
demonstrated that the error handling process model developed by Kanse (2004) on the basis
of data from the chemical industry is also appropriate to describe error handling in health
care. Our study has furthermore shown that different incident types (i.e. near misses, no-harm
incidents, and accidents) each provide unique information about the way errors are
recognised and dealt with.
Aspden et al. (2004) proposed that accidents can yield information about unsuccessful
error recovery; that is, opportunities for error recognition or correction that had failed, or that
had been missed or absent. Our exploratory study on 52 medication errors has empirically
Chapter 8
126
confirmed this viewpoint by demonstrating that accidents can indeed be a treasure of
information regarding error recovery. Our analysis of failed, missed and absent error recovery
opportunities in actual accidents largely corresponds to barrier analysis (e.g., Hollnagel,
2008; Johnson, 2007; Svenson, 2001). However, contrary to barrier analysis, which only
concentrates on failures in planned system defences, we also explicitly examined failures
related to unplanned ad hoc problem solving, as suggested by Parnes et al. (2007). Moreover,
our approach does not consider failed, missed and absent error recovery opportunities as
contributors to error. Instead, we agree with Aspden et al. (2004) that unsuccessful error
recovery opportunities could be used to understand the predictors of successful error
recovery. This focus on effective safety mechanisms is in line with the general viewpoint of
Wagenaar and Hudson (1998, p. 66) that safety research and practice should move from an
“in search of misery” tradition towards an “in search of safety” approach.
Apparently, a clear and consistent definition of near misses is essential to make the
most of near misses and error recovery. Unfortunately, such a universal definition is still
lacking and consequently, a diversity of definitions is used (Affonso & Jeffs, 2004; Aspden et
al., 2004; Yu et al., 2005). Near misses are sometimes defined as incidents that did not reach
the patient (e.g., Barnard et al., 2006; Kaplan & Rabin Fastman, 2003). Conversely, several
researchers define near misses as incidents that did not cause patient harm (e.g., Barach et al.,
1999; Gurwitz et al., 2000). On the basis of empirical data about error handling processes, we
prefer to define near misses as incidents that did not reach the patient. However, we also
argue that both definitions of near misses could be valuable and that the optimal definition
may well be dependent on organisational context, as stated by Tamuz and Thomas (2006).
The findings of our research on error recovery and definitions of near misses can facilitate
researchers and health care organisations in making use of information about error recovery,
which so far has been underutilised (Aspden et al., 2004; Parnes et al., 2007; Patel & Cohen,
2008).
Safety Culture
In our panel survey among 701 health care employees of three Dutch hospitals, we found
significant positive changes regarding the self-reported number of reported incidents and the
safety culture dimension ―frequency of incident reporting‖. This suggests that after the
implementation of a safety management programme including prospective and retrospective
methods for risk analysis as well as discussion meetings, employees were more willing to
report incidents. This was validated by a 400% increase in the actual number of reported
General Discussion
127
incidents as indicated by our quasi-experimental field study. Evidently, the reporting culture
has improved, which has been mentioned as an important aspect of an advanced safety
culture (Reason, 1998). Logistic and multiple regression analyses have indicated that incident
reporting behaviour is positively associated with feedback about and learning from errors,
shift changes, as well as teamwork. Findings from our quasi-experimental field study confirm
the importance of learning and teamwork for incident reporting, since they have shown that
carrying out a prospective analysis can enhance incident reporting behaviour. More
specifically, a prospective analysis could change people‘s risk perceptions (Battles et al.,
2006), whereby they learn to recognise errors (Kontogiannis & Malakis, 2009). Moreover, a
prospective analysis might take away social barriers to incident reporting, such as shame or
fear of punishment, thanks to its open atmosphere (Cannon & Edmondson, 2005). The latter
assumption might be supported by the observed significant positive change in people‘s
perceptions about the blame free response to error.
Our quasi-experimental field study has shown that the positive effects of conducting a
prospective analysis held true for participants and for those colleagues who had been notified
about the results of the analysis. This confirms the contention of Cannon and Edmondson
(2001) that believes about errors and risks are shared within units, which is important for
learning to take place (Edmondson, 2004). In line with this, Carroll et al. (2002) claimed that
health care employees who had participated in a root cause analysis spread their new attitudes
when they returned to their units. Building on those thoughts, we assume that employees who
participated in the prospective analyses of our study have shared their knowledge, beliefs,
and risk perceptions with their colleagues, simply by talking about their experiences and the
results. Our findings might thus support the network theory of social contagion (Scherer &
Cho, 2003) that argues that team members could spread their own risk perceptions, just by
communicating.
Although carrying out a prospective analysis as part of a large-scale safety
management programme can yield a wider spectrum of reported incident types and a larger
proportion of incidents reported by doctors, our research has not revealed any relation
between conducting a prospective analysis and the number of incidents reported. This could
suggest that safety management efforts could indeed increase health care employees‘
willingness to report incidents, but that health care employees might reach a stage in which,
for instance, time constraints hinder them from reporting even more incidents. Our findings
might endorse the contention of Van der Schaaf and Kanse (2004) that health care employees
are unwilling to report known problems due to limited opportunities for learning and instead
Chapter 8
128
decide to report ―new‖ problems. This scenario might be particularly true in case of time
pressure (Evans et al., 2007).
Notwithstanding those positive results, our panel survey on safety culture has shown
only a few significant trends and therefore, the use of self-reported safety culture surveys as
an evaluation instrument could be doubted. The limited number of significant trends in safety
culture might be related to the fact that changes in safety behaviour (as demonstrated by the
actual advances in incident reporting behaviour) are not always reflected by comparable
changes in safety culture (Cooper & Philips, 2004). Alternative explanations could be that (1)
safety culture change takes more time (Guldenmund, 2000), (2) some respondents already
showed extremely positive scores at baseline (Taris, 2000; Wilder, 1967), or (3) the safety
management programme did not meet the high expectations that were raised during baseline
assessment (Hellings, 2009). Nevertheless, the positive trends that were observed are
promising. Moreover, the regression analyses have revealed feedback and learning
mechanisms as an important predictor of incident reporting behaviour, even after we
controlled for the stability of the criterion variables.
Often, error rates are used to evaluate improvements regarding patient safety.
However, researchers and health care organisations actually use this outcome measure to
assess advancements related to two different goals: (1) decreasing the number of errors as an
indication of reduced risk and (2) increasing error reporting as an indication of a positive
reporting culture (Edmondson, 1996; Itoh et al., 2007; Ramanujam, Keyser, & Sirio, 2005).
Hence, error rates are confounded by different goals and might thus be inappropriate for
measuring advances in patient safety. Therefore, future research could aim to develop better
measures. For instance, Pronovost et al. (2006) made a good attempt by proposing the
consolidated use of (1) harm rates (e.g., the number of infections), (2) process measures (e.g.,
how often needed interventions are actually provided), (3) structural measures (e.g., how
often units learn from problems), and (4) safety culture measures.
Patient Involvement
An interesting result of our systematic evaluation of the application of HFMEA™ is the
debate regarding patient involvement in patient safety, which has been started by other
researchers (Coulter, 2006; Entwistle, 2007; Lyons, 2007). At first sight, our study might
endorse the value of patient involvement, since their participation was considered to be useful
by the respondents of those teams in which a patient had actually been involved. However,
our results might also indicate that the usefulness of patient participation is contingent on the
General Discussion
129
health care process under investigation. Alternatively, we suggest that the value of patient
involvement is not evident for health care employees until they experience it themselves.
Those assumptions might contribute to research on patient involvement in patient safety,
which is regarded as one of the priorities for patient safety research (Bates et al., 2009).
8.3 Practical Implications
This section presents the practical implications and recommendations that have resulted from
our research. These suggestions enable health care organisations to develop their own tailor-
made approach towards proactive safety management, and to minimise medical errors and
associated costs accordingly. Anyhow, we encourage health care organisations to make a
deliberate and fundamental decision regarding safety management. Generally, we promote a
change from a reactive attitude, focussing on error reduction and actual accidents only, to a
proactive attitude, concentrating on both error reduction and error recovery, and
acknowledging the added value of a positive safety culture.
Risk Analysis
Based on the safety objective of minimal patient harm (Battles & Lilford, 2003) and our
findings, health care organisations are recommended to apply both prospective and
retrospective methods to improve patient safety. This combined approach can facilitate
management and frontline staff in acquiring a more complete and reliable picture of risks
(Battles & Lilford, 2003; Herzer et al., 2008; Runciman et al., 2006; Senders, 2004).
However, triangulation of methods might also necessitate additional resources. Akins and
Cole (2005) reported that a lack of available resources due to staffing problems and work
overloads could be an important barrier to safety management efforts in health care. They
found that limited resources might result in health care employees being reluctant to adopt
new analysis methods. Integration of prospective and retrospective methods could partly
solve this problem. For instance, multidisciplinary teams could use information about
numbers and types of retrospectively reported incidents as a starting point for a prospective
analysis. On the other hand, prospectively developed failure scenarios could guide
information gathering after incidents have retrospectively been reported. Such integration
might save time. Moreover, integration of prospective and retrospective methods could also
facilitate management in making sense of patient safety data and prioritising possible
interventions (Battles et al., 2006).
Chapter 8
130
Regarding prospective analyses, we encourage health care organisations to actually
consider our recommendations regarding HFMEA™, such as customising the rating scales
and mapping the selected process prior to the start of the analysis. Such modifications could
increase the perceived utility and acceptance of the method by team members and limit the
time investment needed to conduct the analysis. Moreover, we agree with Rath (2008) that
the selection of a facilitator for an HFMEA™ analysis seems to be highly important. Health
care organisations are advised to give preference to facilitators who advocate a nonpunitive
system approach towards safety (Kunac & Reith, 2005; Wetterneck et al., 2004; Wetterneck
et al., 2006). Such persons concentrate on problems and deficiencies in the working
environment and do not blame persons for their errors, which seems to be necessary to
achieve actual improvements in patient safety (Carayon et al., 2007; Reason, 2000).
Furthermore, according to the principles of proactive safety management it would be valuable
if a facilitator could pay attention to both error reduction and error recovery promotion
strategies when supporting the team members in determining appropriate actions to improve
patient safety.
Error Recovery
At first, most health care organisations focus on error reduction to improve patient safety.
However, they can also employ strategies to promote error recovery (Aspden et al., 2004;
Hollnagel, 2008; Kanse et al., 2006). Our studies on error recovery have shown that
information about error detection and correction can be obtained by analysing various
incident types. On the one hand, health care organisations can use near misses to reveal
positive influences on error recovery by analysing the steps that led up to successful error
recognition and correction. On the other hand, analysis of near misses, no-harm incidents,
and accidents can uncover negative influences on error recovery by scrutinising the steps that
resulted in unsuccessful error recovery, that is, failed, missed and absent error recovery
opportunities. Then, health care organisations could promote error recovery by reinforcing
the positive and reducing the negative influences on error detection and correction.
We emphasised the importance of a clear and consistent definition of near misses to
enable their large-scale reporting and analysis. We have proposed two possible definitions of
near misses: (1) incidents that did not reach the patient and (2) incidents that did not cause
patient harm. Health care organisations are encouraged to apply that definition that fits their
organisational context best in order to get the most out of near misses. Defining near misses
as incidents that did not cause patient harm might be appropriate in case a learning focus is
General Discussion
131
present and trust is prevalent. Conversely, a health care organisation in which there is a
tendency to reproach people for their errors could better use the more positive definition of
near misses that only includes incidents that did not reach the patient.
Safety Culture
In our panel survey on the effects of a safety management programme on safety culture, we
have identified only a few positive trends. This endorses the contention of Guldenmund
(2000) that safety culture appears to be relatively stable and thus might be difficult to change.
Further, our results are in line with the earlier finding that progress in safety behaviour and
performance may not, by definition, be reflected by similar positive changes in safety culture
(Cooper & Philips, 2004). On the basis of those findings, we question the use of self-reported
safety culture surveys to assess the effects of safety management efforts. Instead, we agree
with Pronovost et al. (2006) that health care organisations can better use multiple methods for
such evaluations. One could, for instance, use vignettes describing different types of medical
errors and associated consequences to examine trends in incident reporting behaviour and
simultaneously validate the results of survey scales that are related to reporting culture
(Bognár et al., 2008).
To make progress with regard to reporting culture, which is an important aspect of
safety culture (Reason, 1998), health care organisations can conduct prospective analyses.
Health care organisations that have not yet implemented a sophisticated incident reporting
and analysis system are advised to make teams conduct prospective analyses in advance. This
order of implementation will probably enhance the resultant positive impact on incident
reporting behaviour and enlarges the potential for learning. Health care organisations that
have already introduced a sophisticated incident reporting and analysis system that facilitates
learning, can ask teams to carry out prospective analyses to boost existing incident reporting
behaviour. Further, health care employees that participate in a prospective analysis could
share their new insights by communication with their colleagues. This distribution of risk
perceptions might bring about organisational learning and compensate for the time
investment that is required to conduct a prospective analysis.
In general, as shown by our panel survey on safety culture, feedback and learning
mechanisms and good teamwork (including smooth shift changes) are important to enhance
incident reporting behaviour. Health care organisations can develop and train so-called self-
correcting teams in which team members coordinate their tasks, cooperate closely, and
anticipate and discuss problems, whereby they will be more willing to report errors and share
Chapter 8
132
lessons (Edmondson, 1996; Wilson et al., 2005). In addition to such local feedback,
organisation-wide feedback about the way incident reports are dealt with, about identified
problems, and about preventive and corrective actions, can assure health care employees of
the value of incident reporting, which, in turn, can enhance the reporting culture (Benn et al.,
2009). A factor that should not be underestimated in this respect is leadership at all levels,
ranging from the executive to the unit level, because this appears to be crucial for a learning
culture to evolve (Cannon & Edmondson, 2001; Carroll & Edmondson, 2002; Edmondson,
2004; Mohr et al., 2002).
8.4 Future Research
The findings presented in this dissertation contribute to safety management research and
practice and should be replicated in other health care settings, such as nursing homes,
psychiatric hospitals, and primary care to examine the external validity. It would be
interesting, for instance, to explore whether conducting a prospective analysis could remove
social impediments to incident reporting in other health care settings. Such a positive effect
of a prospective analysis would be very valuable since for example in nursing homes the
perceived response to error is considered to be fairly punitive (Handler et al., 2006). Further,
similar studies could be carried out in other industries, such as the chemical and nuclear
industries, because some results like the insights regarding the order of implementation of
prospective and retrospective methods and the definitions of near misses might also be
valuable for these industries. Moreover, the findings of the research presented in this
dissertation should be replicated by making use of larger datasets and time series analysis to
allow robust statistical testing of (1) the relation between prospective and retrospective
methods and incident reporting behaviour and (2) the differences between near misses, no-
harm incidents, and accidents regarding underlying error handling process types. Future
studies should furthermore reproduce our findings by means of prospective methods other
than HFMEA™. Regarding HFMEA™ we invite other researchers to evaluate the
effectiveness of both HFMEA™ in general and our recommendations for improvement.
Although we have demonstrated the possibility and perceived usefulness of the
integration of prospective and retrospective methods, future research could explicitly assess
the benefits and drawbacks of such integration for both hospital management and frontline
staff. For instance, one could investigate whether integration of prospective and retrospective
methods could really increase the efficiency of analysis and restrict the number of resources
required.
General Discussion
133
To enable large-scale reporting of near misses, future studies could explore whether a
standardised definition is feasible or whether their definition is indeed contingent on
organisational context. Although our research has shown several relations between
prospective and retrospective methods on the one hand and safety culture on the other hand,
future studies could attempt to establish the exact relations between those approaches towards
proactive safety management. In particular, future research could concentrate on the exact
influence of prospective methods, feedback and learning mechanisms, as well as teamwork
(including shift changes) on reporting culture in terms of the number and types of reported
incidents. Such research could consider the separate as well as the combined impact of those
predictors.
In general, we encourage future studies on error provoking factors. Researchers could
investigate whether prospective and retrospective methods could also be used to reveal
specific determinants of risks, such as those related to job stress and job (re-)design. This
approach is in line with the general principles that safety behaviour is an interaction between
psychological and situational factors (Bogner, 2007; Reason, 1997) and that job (re-)design
should consider many aspects of work (Carayon et al., 2007). In addition, we also invite
researchers to explore the associations between teamwork and patient safety. Besides the fact
that the development of self-correcting teams could enhance health care employees‘
willingness to report errors, such teamwork might, for instance, also compensate for
problems caused by stressed or fatigued individuals (Edmondson, 1996). This could be
valuable since workload and fatigue can affect performance negatively and contribute to
errors (e.g., Bognár et al., 2008). Good teamwork might promote backup behaviour: team
members might assist colleagues, for example in case of fatigue, thereby preventing or
correcting errors and improving patient safety accordingly (Sexton et al., 2000; Wilson et al.,
2005). In line with this, we encourage research on training and selection processes. Research
could concentrate on the requirements for (crew resource management) training of team
members and leaders on non-technical skills, such as situation awareness, error management,
and the recognition of human performance limiters like fatigue and stress (Helmreich, 2000;
Flin & Maran, 2004; Musson & Helmreich, 2004; Sexton et al., 2000). Moreover, one could
examine whether it is possible to include such cognitive and social skills in recruitment and
selection procedures.
A finding that inspired us to come up with opportunities for future research is our
interesting, but ambiguous result regarding patient involvement in prospective analyses.
Participants of the HFMEA™ analyses differed in their opinions about the usefulness of
Chapter 8
134
patient involvement. Hence, future research could concentrate on the value of patient
participation in improving patient safety, as suggested by Bates et al. (2009). For instance,
future studies could explore the advantages of the use of input from patients in retrospective
analyses. This could enable cross-validation of existing insights or, more likely, yield
different overviews of problems. Patients pass through the complete process of care, are
nearly always available (Lyons, 2007), and can thus shed a different light on patient safety.
Moreover, patients might constitute an additional barrier by contributing to error detection
and correction themselves (e.g., Lyons, 2007; Vincent & Coulter, 2002). However, it could
be questioned whether it is sensible to provide patients with responsibilities for their own care
and safety (Lyons, 2007). In line with this scepticism, Entwistle (2007) stressed the
importance of distinguishing between relying on patients to ensure their safety and involving
patients to improve their safety. Future studies should anyhow reveal the benefits and
drawbacks of patient involvement, so health care organisations could make well-considered
decisions regarding the role of patients in patient safety. A final direction for future research
could even be to explore the possibility to rely on health care employees who become
patients themselves for information about quality and safety from a patient‘s perspective.
8.5 Concluding Remarks
Health care is hazardous and calls for proactive safety management. Although we have
proposed three distinct approaches towards proactive safety management, we suppose that
prospective analysis of health care processes could, in fact, embody all three of them.
Prospective methods can be used to identify and eliminate risks before errors may occur.
Further, a prospective analysis could enhance health care employees‘ understanding and
awareness of possible risks. Such a vigilant attitude could enable these employees to
recognise errors if they do crop up and to avert patient harm by timely error recovery.
Moreover, a multidisciplinary prospective analysis with both doctors and other professions
involved might promote teamwork and remove social barriers to incident reporting and
learning from errors, thereby developing a more positive safety culture. Such a proactive
approach towards safety management could improve patient safety, minimise patient harm,
and limit costs of poor safety. In conclusion, proactive safety management is important for
those people who find that patient safety is a moving target.
135
References
Adachi, W., & Lodolce, A. E. (2005). Use of Failure Mode and Effects Analysis in
improving the safety of i.v. drug administration. American Journal of Health-System
Pharmacy, 62, 917-920.
Advisory Committee on the Safety of Nuclear Installations. (1993). ACSNI Study Group on
Human Factors: Third report - Organising for Safety. Sheffield, UK: HSE Books.
Affonso, D., & Jeffs, L. (2004). Near misses: Lessons learned in the process of care.
International Nursing Perspectives, 4, 115-122.
Akins, R. B., & Cole, B. R. (2005). Barriers to implementation of patient safety systems in
healthcare institutions: Leadership and policy implications. Journal of Patient Safety,
1, 9-16.
Apkon, M., Leonard, J., Probst, L., DeLizio, L., & Vitale, R. (2004). Design of a safer
approach to intravenous drug infusions: Failure Mode Effects Analysis. Quality and
Safety in Health Care, 13, 265-271.
Aspden, P., Corrigan, J. M., Wolcott, J., & Erickson, S. M. (Eds.). (2004). Patient safety:
Achieving a new standard for care. Washington, DC: National Academies Press.
Babbie, E. R. (2005). The basics of social research (3rd ed.). Toronto, Canada:
Thomson/Wadsworth.
References
136
Barach, P., & Johnson, J. K. (2006). Understanding the complexity of redesigning care
around the clinical microsystem. Quality and Safety in Health Care, 15(Suppl. 1), i10-
i16.
Barach, P., & Small, S. D. (2000). Reporting and preventing medical mishaps: Lessons from
non-medical near miss reporting systems. British Medical Journal, 320, 759-763.
Barach, P., Small, S. D., & Kaplan, H. (1999). Designing a confidential safety reporting
system: In depth review of thirty major medical incident reporting systems, and near-
miss safety reporting systems in the nuclear, aviation, and petrochemical industries.
Anesthesiology, 91, A1209.
Barnard, D., Dumkee, M., Bains, B., & Gallivan, B. (2006). Implementing a good catch
program in an integrated health system. Healthcare Quarterly, 9(Sp), 22-27.
Baruch, Y., & Holtom, B. C. (2008). Survey response rate levels and trends in organizational
research. Human Relations, 61, 1139-1160.
Bates, D. W. (2000). Using information technology to reduce rates of medication errors in
hospitals. British Medical Journal, 320, 788-791.
Bates, D. W., Larizgoitia, I., Prasopa-Plaizier, N., & Jha, A. K. (2009). Global priorities for
patient safety research. British Medical Journal, 338, 1242-1244.
Bates, D. W., Spell, N., Cullen, D. J., Burdick, E., Laird, N., Petersen, L. A., et al. (1997).
The costs of adverse drug events in hospitalized patients. Journal of the American
Medical Association, 277, 307-311.
Battles, J. B., Dixon, N. M., Borotkanics, R. J., Rabin-Fastman, B., & Kaplan, H. S. (2006).
Sensemaking of patient safety risks and hazards. Health Services Research, 41, 1555-
1575.
Battles, J. B., Kaplan, H. S., Van der Schaaf, T. W., & Shea, C. E. (1998). The attributes of
medical event-reporting systems: Experience with a prototype medical event-reporting
system for transfusion medicine. Archives of Pathology and Laboratory Medicine,
122, 231-238.
Battles, J. B., & Lilford, R. J. (2003). Organizing patient safety research to identify risks and
hazards. Quality and Safety in Health Care, 12(Suppl. 2), ii2-ii7.
Benn, J., Koutantji, M., Wallace, L., Spurgeon, P., Rejman, M., Healey, A., et al. (2009).
Feedback from incident reporting: Information and action to improve patient safety.
Quality and Safety in Health Care, 18, 11-21.
References
137
Berden, T. P. J., Brombacher, A. C., & Sander, P. C. (2000). The building bricks of product
quality: An overview of some basic concepts and principles. International Journal of
Production Economics, 67, 3-15.
Bognár, A., Barach, P., Johnson, J. K., Duncan, R. C., Birnbach, D., Woods, D., et al. (2008).
Errors and the burden of errors: Attitudes, perceptions, and the culture of safety in
pediatric cardiac surgical teams. The Annals of Thoracic Surgery, 85, 1374-1381.
Bogner, M. S. (2007). The artichoke systems approach for identifying the why of error. In P.
Carayon (Ed.), Handbook of human factors and ergonomics in health care and patient
safety (pp. 109-126). Mahwah, NJ: Lawrence Erlbaum Associates.
Bonnabry, P., Cingria, L., Ackermann, M., Sadeghipour, F., Bigler, L., & Mach, N. (2006).
Use of a prospective risk analysis method to improve the safety of the cancer
chemotherapy process. International Journal for Quality in Health Care, 18, 9-16.
Burgmeier, J. (2002). Failure Mode and Effect Analysis: An application in reducing risk in
blood transfusion. Joint Commission Journal on Quality Improvement, 28, 331-339.
Cannon, M. D., & Edmondson, A. C. (2001). Confronting failure: Antecedents and
consequences of shared beliefs about failure in organizational work groups. Journal of
Organizational Behavior, 22, 161-177.
Cannon, M. D., & Edmondson, A. C. (2005). Failing to learn and learning to fail
(intelligently): How great organizations put failure to work to innovate and improve.
Long Range Planning, 38, 299-319.
Carayon, P., Alvarado, C. J., & Hundt, A. Schoofs. (2007). Work design and patient safety.
Theoretical Issues in Ergonomics Science, 8, 395-428.
Carroll, J. S., & Edmondson, A. C. (2002). Leading organisational learning in health care.
Quality and Safety in Health Care, 11, 51-56.
Carroll, J. S., Rudolph, J. W., & Hatakenaka, S. (2002). Lessons learned from non-medical
industries: Root cause analysis as culture change at a chemical plant. Quality and
Safety in Health Care, 11, 266-269.
Carstens, D. S. (2006). Error analysis leading technology development. Theoretical Issues in
Ergonomics Science, 7, 525-549.
Carthey, J., De Leval, M. R., & Reason, J. T. (2001). The human factor in cardiac surgery:
Errors and near misses in a high technology domain. The Annals of Thoracic Surgery,
72, 300-305.
Castle, N. G., & Sonon, K. E. (2006). A culture of patient safety in nursing homes. Quality
and Safety in Health Care, 15, 405-408.
References
138
Clarke, S. (2006a). Contrasting perceptual, attitudinal and dispositional approaches to
accident involvement in the workplace. Safety Science, 44, 537-550.
Clarke, S. (2006b). The relationship between safety climate and safety performance: A meta-
analytic review. Journal of Occupational Health Psychology, 11, 315-327.
Classen, D. C., Pestotnik, S. L., Evans, R. S., Lloyd, J. F., & Burke, J. P. (1997). Adverse
drug events in hospitalized patients: Excess length of stay, extra costs, and attributable
mortality. Journal of the American Medical Association, 277, 301-306.
Cohen, H. (2001). Shrinking medication errors down to size: What you can do about this far-
reaching problem. Nursing Management, 32(10), 25-31.
Coiera, E., & Braithwaite, J. (2009). Market-based control mechanisms for patient safety.
Quality and Safety in Health Care, 18, 99-103.
Colla, J. B., Bracken, A. C., Kinney, L. M., & Weeks, W. B. (2005). Measuring patient safety
climate: A review of surveys. Quality and Safety in Health Care, 14, 364-366.
Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis issues
for field settings. Chicago: Rand McNally.
Cooper, M. D., & Philips, R. A. (2004). Exploratory analysis of the safety climate and safety
behavior relationship. Journal of Safety Research, 35, 497-512.
Coulter, A. (2006). Patient safety: What role can patients play? Health Expectations, 9, 205-
206.
Day, S., Dalto, J., Fox, J., Allen, A., & Ilstrup, S. (2007). Utilization of Failure Mode Effects
Analysis in trauma patient registration. Quality Management in Health Care, 16, 342-
348.
De Jonge, J., Van der Linden, S., Schaufeli, W., Peter, R., & Siegrist, J. (2008). Factorial
invariance and stability of the effort-reward imbalance scales: A longitudinal analysis
of two samples with different time lags. International Journal of Behavioral
Medicine, 15, 62-72.
DeRosier, J., Stalhandske, E., Bagian, J. P., & Nudell, T. (2002). Using Health care Failure
Mode and Effect Analysis: The VA National Center for Patient Safety‘s prospective
risk analysis system. Joint Commission Journal on Quality Improvement, 28, 248-
267.
Devers, K. J., Pham, H. H., & Liu, G. (2004). What is driving hospitals‘ patient-safety
efforts? Health Affairs, 23(2), 103-115.
References
139
De Vries, E. N., Ramrattan, M. A., Smorenburg, S. M., Gouma, D. J., & Boermeester, M. A.
(2008). The incidence and nature of in-hospital adverse events: A systematic review.
Quality and Safety in Health Care, 17, 216-223.
Dhillon, B. S. (2003). Methods for performing human reliability and error analysis in health
care. International Journal of Health Care Quality Assurance, 16, 306-317.
Dyer, C. (2001). Government to introduce safer administration of cancer drugs after fatal
error. British Medical Journal, 322, 1013.
Edmondson, A. C. (1996). Learning from mistakes is easier said than done: Group and
organizational influences on the detection and correction of human error. Journal of
Applied Behavioral Science, 32, 5-28.
Edmondson, A. C. (2004). Learning from failure in health care: Frequent opportunities,
pervasive barriers. Quality and Safety in Health Care, 13(Suppl. 2), ii3-ii9.
Entwistle, V. A. (2007). Differing perspectives on patient involvement in patient safety.
Quality and Safety in Health Care, 16, 82-83.
Esmail, R., Cummings, C., Dersch, D., Duchscherer, G., Glowa, J., Liggett, G., et al. (2005).
Using Healthcare Failure Mode and Effect Analysis tool to review the process of
ordering and administrating potassium chloride and potassium phosphate. Healthcare
Quarterly, 8(Sp), 73-80.
Etchegaray, J. M., Thomas, E. J., Geraci, J. M., Simmons, D., & Martin, S. K. (2005).
Differentiating close calls from errors: A multidisciplinary perspective. Journal of
Patient Safety, 1, 133-137.
Evans, S. M., Berry, J. G., Smith, B. J., Esterman, A., Selim, P., O‘Shaughnessy, J., et al.
(2006). Attitudes and barriers to incident reporting: A collaborative hospital study.
Quality and Safety in Health Care, 15, 39-43.
Evans, S. M., Smith, B. J., Esterman, A., Runciman, W. B., Maddern, G., Stead, K., et al.
(2007). Evaluation of an intervention aimed at improving voluntary incident reporting
in hospitals. Quality and Safety in Health Care, 16, 169-175.
Evers, A., Van Vliet-Mulder, J. C., & Groot, C. J. (2000). Documentatie van tests and test
research in Nederland: Deel 2 Testresearch (p. 1411). Assen, The Netherlands: Van
Gorcum.
Firth-Cozens, J. (2003). Evaluating the culture of safety. Quality and Safety in Health Care,
12, 401.
Flin, R. (2007). Measuring safety culture in health care: A case for accurate diagnosis. Safety
Science, 45, 653-667.
References
140
Flin, R., Burns, C., Mearns, K., Yule, S., & Robertson, E. M. (2006). Measuring safety
climate in health care. Quality and Safety in Health Care, 15, 109-115.
Flin, R., & Maran, N. (2004). Identifying and training non-technical skills for teams in acute
medicine. Quality and Safety in Health Care, 13(Suppl. 1), i80-i84.
Flin, R., Mearns, K., O‘Connor, P., & Bryden, R. (2000). Measuring safety climate:
Identifying the common features. Safety Science, 34, 177-192.
Flin, R., & Yule, S. (2004). Leadership for safety: Industrial experience. Quality and Safety
in Health Care, 13(Suppl. 2), ii45-ii51.
Ginsburg, L., Norton, P. G., Casebeer, A., & Lewis, S. (2005). An educational intervention to
enhance nurse leaders‘ perceptions of patient safety culture. Health Services
Research, 40, 997-1020.
Goodman, J. S., & Blum, T. C. (1996). Assessing the non-random sampling effects of subject
attrition in longitudinal research. Journal of Management, 22, 627-652.
Green, S. B., & Salkind, N. J. (2003). Using SPSS for Windows and Macintosh: Analyzing
and understanding data (3rd ed.). Upper Saddle River, NJ: Prentice Hall.
Guldenmund, F. W. (2000). The nature of safety culture: A review of theory and research.
Safety Science, 34, 215-257.
Gurwitz, J. H., Field, T. S., Avorn, J., McCormick, D., Jain, S., Eckler, M., et al. (2000).
Incidence and preventability of adverse drug events in nursing homes. The American
Journal of Medicine, 109, 87-94.
Habraken, M. M. P. (2005). Better care for incidents in health care: Redesign oriented
evaluation of the incidents database of the IGZ. Unpublished master‘s thesis,
Eindhoven University of Technology, Eindhoven, The Netherlands.
Habraken, M., & Van der Schaaf, T. (2005). Biases in a medical incident causation database:
A quantitative evaluation using PRISMA-Medical. In N. Marmaras, T. Kontogiannis,
& D. Nathanael (Eds.), ACM International Conference Proceeding Series: Vol. 132.
Proceedings of the 2005 Annual Conference on European Association of Cognitive
Ergonomics (pp. 167-173). Chania, Greece.
Habraken, M. M. P., & Van der Schaaf, T. W. (in press). If only….: Failed, missed and
absent error recovery opportunities in medication errors. Quality and Safety in Health
Care.
Habraken, M. M. P., Van der Schaaf, T. W., Leistikow, I. P., & Reijnders-Thijssen, P. M. J.
(2009). Prospective risk analysis of health care processes: A systematic evaluation of
the use of HFMEA™ in Dutch health care. Ergonomics, 52, 809-819.
References
141
Hackman, J. R. (1993). Teams, leaders, and organizations: New directions for crew-oriented
flight training. In E. L. Wiener, B. G. Kanki, & R. L. Helmreich (Eds.), Cockpit
resource management (pp. 47-69). Orlando, FL: Academic Press.
Hair, J. F., Black, W. C., Babin, B. J., Anderson, R. E., & Tatham, R. L. (2006). Multivariate
data analysis (6th ed.). Upper Saddle River, NJ: Prentice Hall.
Hale, A. R. (2003). Safety management in production. Human Factors and Ergonomics in
Manufacturing, 13, 185-201.
Hampton, T. (2004). Bar codes mandated for hospital meds. Journal of the American Medical
Association, 291, 1685-1686.
Handler, S. M., Castle, N. G., Studenski, S. A., Perera, S., Fridsma, D. B., Nace, D. A., et al.,
(2006). Patient safety culture assessment in the nursing home. Quality and Safety in
Health Care, 15, 400-404.
Harms-Ringdahl, L. (2004). Relationships between accident investigations, risk analysis, and
safety management. Journal of Hazardous Materials, 111, 13-19.
Hellings, J. (2009). Patiëntveiligheid in het ziekenhuis: Een evaluatieonderzoek over
medicatieveiligheid. Unpublished doctoral dissertation, Katholieke Universiteit
Leuven, Leuven, Belgium.
Helmreich, R. L. (2000). On error management: Lessons from aviation. British Medical
Journal, 320, 781-785.
Henneman, E. A., Blank, F. S. J., Gawlinski, A., & Henneman, P. L. (2006). Strategies used
by nurses to recover medical errors in an academic emergency department setting.
Applied Nursing Research, 19, 70-77.
Henriksen, K., & Dayton, E. (2006). Issues in the design of training for quality and safety.
Quality and Safety in Health Care, 15(Suppl. 1), i17-i24.
Henriksen, K., & Kaplan, H. (2003). Hindsight bias, outcome knowledge and adaptive
learning. Quality and Safety in Health Care, 12(Suppl. 2), ii46-ii50.
Herzer, K. R., Mark, L. J., Michelson, J. D., Saletnik, L. A., & Lundquist, C. A. (2008).
Designing and implementing a comprehensive quality and patient safety management
model: A paradigm for perioperative improvement. Journal of Patient Safety, 4, 84-
92.
Hogan, H., Olsen, S., Scobie, S., Chapman, E., Sachs, R., McKee, M., et al. (2008). What can
we learn about patient safety from information sources within an acute hospital: A
step on the ladder of integrated risk management? Quality and Safety in Health Care,
17, 209-215.
References
142
Holden, R. J., & Karsh, B.-T. (2007). A review of medical error reporting system design
considerations and a proposed cross-level systems research framework. Human
Factors, 49, 257-276.
Hollnagel, E. (1999). Accidents and barriers. In J. M. Hoc, P. Millot, E. Hollnagel, & P.C.
Cacciabue (Eds.), Proceedings of CSAPC'99, Seventh European Conference on
Cognitive Science Approaches to Process Control (pp. 175-180). Villeneuve d'Asq,
France: Presses Universitaires de Valenciennes.
Hollnagel, E. (2008). Risk + barriers = safety? Safety Science, 46, 221-229.
Huang, D. T., Clermont, G., Sexton, J. B., Karlo, C. A., Miller, R. G., Weissfeld, L. A., et al.
(2007). Perceptions of safety culture vary across the intensive care units of a single
institution. Critical Care Medicine, 35, 165-176.
Hudson, P. T. W. (2001). Safety management and safety culture: The long, hard and winding
road. In: W. Pearse, C. Gallagher, & L. Bluff (Eds.), Occupational health and safety
management systems (pp. 3-32). Melbourne: Crown Content.
Hudson, P. (2003). Applying the lessons of high risk industries to health care. Quality and
Safety in Health Care, 12(Suppl. 1), i7-i12.
Israelski, E. W., & Muto, W. H. (2007). Human factors risk management in medical
products. In P. Carayon (Ed.), Handbook of human factors and ergonomics in health
care and patient safety (pp. 615-647). Mahwah, NJ: Lawrence Erlbaum Associates.
Itoh, K., Andersen, H. B., & Madsen, M. D. (2007). Safety culture in health care. In P.
Carayon (Ed.), Handbook of human factors and ergonomics in health care and patient
safety (pp. 199-216). Mahwah, NJ: Lawrence Erlbaum Associates.
Jeon, J., Hyland, S., Burns, C. M., & Momtahan, K. (2007). Challenges with applying FMEA
to the process for reading labels on injectable drug containers. In Proceedings of the
Human Factors and Ergonomics Society 51st Annual Meeting (pp. 735-739).
Baltimore, Maryland.
Johnson, C. W. (2003). How will we get the data and what will we do with it then? Issues in
the reporting of adverse healthcare events. Quality and Safety in Health Care,
12(Suppl. 2), ii64-ii67.
Johnson, C. (2007). Incident analysis in health care. In P. Carayon (Ed.), Handbook of human
factors and ergonomics in health care and patient safety (pp. 715-750). Mahwah, NJ:
Lawrence Erlbaum Associates.
References
143
Kanse, L. (2004). Recovery uncovered: How people in the chemical process industry recover
from failures. Unpublished doctoral dissertation, Eindhoven University of
Technology, Eindhoven, The Netherlands.
Kanse, L., & Van der Schaaf, T. (2001). Recovery from failures in the chemical process
industry. International Journal of Cognitive Ergonomics, 5, 199-211.
Kanse, L., Van der Schaaf, T. W., Vrijland, N. D., & Van Mierlo, H. (2006). Error recovery
in a hospital pharmacy. Ergonomics, 49, 503-516.
Kaplan, H., & Barach, P. (2002). Incident reporting: Science or protoscience? Ten years later.
Quality and Safety in Health Care, 11, 144-145.
Kaplan, H. S., Battles, J. B., Van der Schaaf, T. W., Shea, C. E., & Mercer, S. Q. (1998).
Identification and classification of the causes of events in transfusion medicine.
Transfusion, 38, 1071-1081.
Kaplan, H. S., & Rabin Fastman, B. (2003). Organization of event reporting data for sense
making and system improvement. Quality and Safety in Health Care, 12(Suppl. 2),
ii68-ii72.
Karsh, B.-T., Escoto, K. H., Beasley, J. W., & Holden, R. J. (2006). Toward a theoretical
approach to medical error reporting system research and design. Applied Ergonomics,
37, 283-295.
Katz-Navon, T., Naveh, E., & Stern, Z. (2005). Safety climate in health care organizations: A
multidimensional approach. Academy of Management Journal, 48, 1075-1089.
King, S. (2006). Our story. Clinical Pediatric Emergency Medicine, 7, 268-270.
Kingston, M. J., Evans, S. M., Smith, B. J., & Berry, J. G. (2004). Attitudes of doctors and
nurses towards incident reporting: A qualitative analysis. Medical Journal of
Australia, 181, 36-39.
Kohn, L. T., Corrigan, J. M., & Donaldson, M. S. (Eds.). (2000). To err is human: Building a
safer health system. Washington, DC: National Academies Press.
Kontogiannis, T. (1997). A framework for the analysis of cognitive reliability in complex
systems: A recovery centred approach. Reliability Engineering and System Safety, 58,
233-248.
Kontogiannis, T., & Malakis, S. (2009). A proactive approach to human error detection and
identification in aviation and air traffic control. Safety Science, 47, 693-706.
Kunac, D. L., & Reith, D. M. (2005). Identification of priorities for medication safety in
neonatal intensive care. Drug Safety, 28, 251-261.
References
144
Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical
data. Biometrics, 33, 159-174.
Lawton, R., & Parker, D. (2002). Barriers to incident reporting in a healthcare system.
Quality and Safety in Health Care, 11, 15-18.
Leape, L. L. (1994). Error in medicine. Journal of the American Medical Association, 272,
1851-1857.
Leape, L. L., & Berwick, D. M. (2005). Five years after to err is human: What have we
learned? Journal of the American Medical Association, 293, 2384-2390.
Ligi, I., Arnaud, F., Jouve, E., Tardieu, S., Sambuc, R., & Simeoni, U. (2008). Iatrogenic
events in admitted neonates: A prospective cohort study. The Lancet, 371, 404-410.
Linkin, D. R., Sausman, C., Santos, L., Lyons, C., Fox, C., Aumiller, L., et al. (2005).
Applicability of Healthcare Failure Mode and Effects Analysis to healthcare
epidemiology: Evaluation of the sterilization and use of surgical instruments. Clinical
Infectious Diseases, 41, 1014-1019.
Lyons, M. (2007). Should patients have a role in patient safety? A safety engineering view.
Quality and Safety in Health Care, 16, 140-142.
Marx, D. A., & Slonim, A. D. (2003). Assessing patient safety risk before the injury occurs:
An introduction to sociotechnical probabilistic risk modelling in health care. Quality
and Safety in Health Care, 12(Suppl. 2), ii33-ii38.
McDonough, J. E., Solomon, R., & Petosa, L. (2004). Quality improvement and proactive
hazard analysis models: Deciphering a new tower of Babel. In: P. Aspden, J. M.
Corrigan, J. Wolcott, & S. M. Erickson (Eds.), Patient safety: Achieving a new
standard for care (pp. 471-508). Washington, DC: National Academies Press.
Miles, M. B., & Huberman, A. M. (1994). Qualitative data analysis: An expanded
sourcebook (2nd ed.). London: Sage.
Mohr, J. J., Abelson, H. T., & Barach, P. (2002). Creating effective leadership for improving
patient safety. Quality Management in Health Care, 11, 69-78.
Moss, S. J., Embleton, N. D., & Fenton, A. C. (2005). Towards safer neonatal transfer: The
importance of critical incident review. Archives of Disease in Childhood, 90, 729-732.
Musson, D. M., & Helmreich, R. L. (2004). Team training and resource management in
health care: Current issues and future directions. Harvard Health Policy Review, 5(1),
25-35.
National Quality Forum. (2009). Safe practices for better healthcare – 2009 update: A
consensus report. Available from http://www.qualityforum.org
References
145
Neal, A., Griffin, M. A., & Hart, P. M. (2000). The impact of organizational climate on safety
climate and individual behavior. Safety Science, 34, 99-109.
Netherlands Health Care Inspectorate. (2006). An inadequate care process: An investigation
into the quality and safety of the cardio-surgical care chain for adults at St Radboud
UMC, Nijmegen. Retrieved June 23, 2009, from
http://www.igz.nl/15451/106463/060727_Eindrapport_Radboudz1.pdf
Nieva, V. F., & Sorra, J. (2003). Safety culture assessment: A tool for improving patient
safety in healthcare organizations. Quality and Safety in Health Care, 12(Suppl. 2),
ii17-ii23.
Nuckols, T. K., Bell, D. S., Liu, H., Paddock, S. M., & Hilborne, L. H. (2007). Rates and
types of events reported to established incident reporting systems in two US hospitals.
Quality and Safety in Health Care, 16, 164-168.
Olsen, S., Neale, G., Schwab, K., Psaila, B., Patel, T., Chapman, E. J., et al. (2007). Hospital
staff should use more than one method to detect adverse events and potential adverse
events: Incident reporting, pharmacist surveillance and local real-time record review
may all have a place. Quality and Safety in Health Care, 16, 40-44.
Paparella, S. (2007). Failure Mode and Effects Analysis: A useful tool for risk identification
and injury prevention. Journal of Emergency Nursing, 33, 367-371.
Parnes, B., Fernald, D., Quintela, J., Araya-Guerra, R., Westfall, J., Harris, D., et al. (2007).
Stopping the error cascade: A report on ameliorators from the ASIPS collaborative.
Quality and Safety in Health Care, 16, 12-16.
Patel, V. L., & Cohen, T. (2008). New perspectives on error in critical care. Current Opinion
in Critical Care, 14, 456-459.
Patterson, E. S., Cook, R. I., & Render, M. L. (2002). Improving patient safety by identifying
side effects from introducing bar coding in medication administration. Journal of the
American Medical Informatics Association, 9, 540-553.
Pronovost, P., Holzmueller, C. G., Needham, D. M., Sexton, J. B., Miller, M., Berenholtz, S.,
et al. (2006). How will we know patients are safer? An organization-wide approach to
measuring and improving safety. Critical Care Medicine, 34, 1988-1995.
Pronovost, P. J., Holzmueller, C. G., Young, J., Whitney, P., Wu, A. W., Thompson, D. A., et
al. (2007). Using incident reporting to improve patient safety: A conceptual model.
Journal of Patient Safety, 3, 27-33.
Pronovost, P., & Sexton, B. (2005). Assessing safety culture: Guidelines and
recommendations. Quality and Safety in Health Care, 14, 231-233.
References
146
Pronovost, P. J., Weast, B., Holzmueller, C. G., Rosenstein, B. J., Kidwell, R. P., Haller,
K.B., et al. (2003). Evaluation of the culture of safety: Survey of clinicians and
managers in an academic medical center. Quality and Safety in Health Care, 12, 405-
410.
Quick, O. (2006). Outing medical errors: Questions of trust and responsibility. Medical Law
Review, 14, 22-43.
Ramanujam, R., Keyser, D. J., & Sirio, C. A. (2005). Making a case for organizational
change in patient safety initiatives. Advances in Patient Safety: From Research to
Implementation, Agency for Healthcare Research and Quality. Retrieved July 13,
2009, from http://www.ahrq.gov/downloads/pub/advances/vol2/Ramanujam.pdf
Ramanujam, R., & Rousseau, D. M. (2006). The challenges are organizational not just
clinical. Journal of Organizational Behavior, 27, 811-827.
Rath, F. (2008). Tools for developing a quality management program: Proactive tools
(Process Mapping, Value Stream Mapping, Fault Tree Analysis, and Failure Mode
and Effects Analysis). International Journal of Radiation Oncology Biology Physics,
71(Suppl. 1), S187-S190.
Reason, J. (1997). Managing the risks of organizational accidents. Aldershot, UK: Ashgate.
Reason, J. (1998). Achieving a safe culture: Theory and practice. Work & Stress, 12, 293-
306.
Reason, J. (2000). Human error: Models and management. British Medical Journal, 320,
768-770.
Reason, J. (2004). Beyond the organisational accident: The need for ―error wisdom‖ on the
frontline. Quality and Safety in Health Care, 13(Suppl. 2), ii28-ii33.
Runciman, W., Hibbert, P., Thomson, R., Van der Schaaf, T., Sherman, H., & Lewalle, P.
(2009). Towards an International Classification for Patient Safety: Key concepts and
terms. International Journal for Quality in Health Care, 21, 18-26.
Runciman, B., Merry, A., & Walton, M. (2007). Safety and ethics in healthcare: A guide to
getting it right. Aldershot, UK: Ashgate.
Runciman, W. B., Williamson, J. A. H., Deakin, A., Benveniste, K. A., Bannon, K., &
Hibbert, P. D. (2006). An integrated framework for safety, quality and risk
management: An information and incident management system based on a universal
patient safety classification. Quality and Safety in Health Care, 15(Suppl. 1), i82-i90.
Sandars, J., & Esmail, A. (2003). The frequency and nature of medical error in primary care:
Understanding the diversity across studies. Family Practice, 20, 231-236.
References
147
Scherer, C. W., & Cho, H. (2003). A social network contagion theory of risk perception. Risk
Analysis, 23, 261-267.
Senders, J. W. (2004). FMEA and RCA: The mantras of modern risk management. Quality
and Safety in Health Care, 13, 249-250.
Sexton, J. B., Thomas, E. J., & Helmreich, R. L. (2000). Error, stress, and teamwork in
medicine and aviation: Cross sectional surveys. British Medical Journal, 320, 745-
749.
Shapiro, M. J., Morey, J. C., Small, S. D., Langford, V., Kaylor, C. J., Jagminas, L., et al.
(2004). Simulation based teamwork training for emergency department staff: Does it
improve clinical team performance when added to an existing didactic teamwork
curriculum? Quality and Safety in Health Care, 13, 417-421.
Shojania, K. G. (2008). The frustrating case of incident-reporting systems. Quality and Safety
in Health Care, 17, 400-402.
Siegel, S., & Castellan, N. J. (1988). Nonparametric statistics for the behavioral sciences
(2nd ed.). New York: McGraw-Hill.
Singla, A. K., Kitch, B. T., Weissman, J. S., & Campbell, E. G. (2006). Assessing patient
safety culture: A review and synthesis of the measurement tools. Journal of Patient
Safety, 2, 105-115.
Smits, M., Christiaans-Dingelhoff, I., Wagner, C., Van der Wal, G., & Groenewegen, P. P.
(2007). The psychometric properties of the ‗hospital survey on patient safety culture‘
in Dutch hospitals. BMC Health Services Research, 8, 230-238.
Snijders, C., Kollen, B. J., Van Lingen, R. A., Fetter, W. P. F., & Molendijk, H. (2009).
Which aspects of safety culture predict incident reporting behavior in neonatal
intensive care units? A multilevel analysis. Critical Care Medicine, 37, 61-67.
Sorra, J., & Nieva, V. (2004). Hospital survey on safety culture. Agency for Healthcare
Research and Quality (AHRQ) Publication No. 04-0041. Retrieved July 13, 2009,
from http://www.ahrq.gov/qual/patientsafetyculture/hospcult.pdf
Spector, P. E. (2006). Method variance in organizational research: Truth or urban legend?
Organizational Research Methods, 9, 221-232.
Svenson, O. (2001). Accident and incident analysis based on the Accident Evolution and
Barrier function (AEB) model. Cognition, Technology & Work, 3, 42-52.
Tamuz, M., & Thomas, E. J. (2006). Classifying and interpreting threats to patient safety in
hospitals: Insights from aviation. Journal of Organizational Behavior, 27, 919-940.
References
148
Tamuz, M., Thomas, E. J., & Franchois, K. E. (2004). Defining and classifying medical error:
Lessons for patient safety reporting systems. Quality and Safety in Health Care, 13,
13-20.
Taris, T. (2000). A primer in longitudinal analysis. London: Sage.
Thomas, E. J., Studdert, D. M., Newhouse, J. P., Zbar, B. I. W., Howard, K. M., Williams, E.
J., et al. (1999). Costs of medical injuries in Utah and Colorado. Inquiry, 36, 255-264.
Trucco, P., & Cavallin, M. (2006). A quantitative approach to clinical risk assessment: The
CREA method. Safety Science, 44, 491-513.
Van der Hoeff, N. W. S. (2003). Theory and practice of in-hospital patient risk management.
Unpublished doctoral dissertation, Delft University of Technology. Veenendaal, The
Netherlands: Universal Press.
Van der Schaaf, T. W. (1991). Introduction. In T. W. Van der Schaaf, D. A. Lucas, & A. R.
Hale (Eds.), Near miss reporting as a safety tool (pp. 1-8). Oxford, England:
Butterworth-Heinemann Ltd.
Van der Schaaf, T. W. (1992). Near miss reporting in the chemical process industry.
Unpublished doctoral dissertation, Eindhoven University of Technology, Eindhoven,
The Netherlands.
Van der Schaaf, T., & Kanse, L. (2004). Biases in incident reporting databases: An empirical
study in the chemical process industry. Safety Science, 42, 57-67.
Van der Schaaf, T. W., & Wright, L. B. (2005). Systems for near miss reporting and analysis.
In: J. R. Wilson, & E. N. Corlett (Eds.), Evaluation of human work (3rd ed., pp. 877-
899). Boca Raton, FL: CRC Press.
Van Tilburg, C. M., Leistikow, I. P., Rademaker, C. M. A., Bierings, M. B., & Van Dijk, A.
T. H. (2006). Health care Failure Mode and Effect Analysis: A useful proactive risk
analysis in a pediatric oncology ward. Quality and Safety in Health Care, 15, 58-64.
Vincent, C. A., & Coulter, A. (2002). Patient safety: What about the patient? Quality and
Safety in Health Care, 11, 76-80.
Wagenaar, W. A., & Hudson, P. T. W. (1998). Industrial safety. In P. J. D. Drenth, H.
Thierry, & C. J. De Wolff (Eds.), Handbook of work and organizational psychology
(2nd ed.): Vol. 2 Work psychology (pp. 65-87). Hove, UK: Psychology Press.
Wagner, C., & De Bruijne, M. (2007). Onbedoelde schade in Nederlandse ziekenhuizen.
Retrieved July 13, 2009, from http://www.nivel.nl/pdf/onbedoelde-schade-in-
nederlandse-ziekenhuizen-publieksamenvatting-2007.pdf
References
149
Waring, J. J. (2005). Beyond blame: Cultural barriers to medical incident reporting. Social
Science & Medicine, 60, 1927-1935.
Weston, C., Gandell, T., Beauchamp, J., McAlpine, L., Wiseman, C., & Beauchamp, C.
(2001). Analyzing interview data: The development and evolution of a coding system.
Qualitative Sociology, 24, 381-400.
Wetterneck, T. B., Skibinski, K. A., Roberts, T. L., Kleppin, S. M., Schroeder, M. E., Enloe,
M., et al. (2006). Using Failure Mode and Effects Analysis to plan implementation of
smart i.v. pump technology. American Journal of Health-System Pharmacy, 63, 1528-
1538.
Wetterneck, T. B., Skibinski, K., Schroeder, M., Roberts, T. L., & Carayon, P. (2004).
Challenges with the performance of Failure Mode and Effects Analysis in healthcare
organizations: An iv medication administration HFMEA™. In Proceedings of the
Human Factors and Ergonomics Society 48th
Annual Meeting (pp. 1708–1712). New
Orleans, Louisiana.
Wilder, J. F. (1967). Stimulus and response: The law of initial value. Bristol: Wright.
Wilson, K. A., Burke, C. S., Priest, H. A., & Salas, E. (2005). Promoting health care safety
through training high reliability teams. Quality and Safety in Health Care, 14, 303-
309.
Yu, K. H., Nation, R. L., & Dooley, M. J. (2005). Multiplicity of medication safety terms,
definitions and functional meanings: When is enough enough? Quality and Safety in
Health Care, 14, 358-363.
Zapf, D., & Reason, J. T. (1994). Introduction: Human error and error handling. Applied
Psychology, 43, 427-432.
Zohar, D. (2000). A group-level model of safety climate: Testing the effect of group climate
on microaccidents in manufacturing jobs. Journal of Applied Psychology, 85, 587-
596.
Zohar, D. (2003). Safety climate: Conceptual and measurement issues. In J. C. Quick, & L. E.
Tetrick (Eds.), Handbook of occupational and health psychology (pp. 123-142).
Washington, DC: American Psychological Association.
150
151
Appendix
Safety Culture Dimensions
and Corresponding Survey Items
Appendix
152
Table A.1
Safety culture dimensions and corresponding survey items
(Smits et al., 2007; Sorra & Nieva, 2004).
Safety culture dimension Survey items
Teamwork across hospital units Hospital units do not coordinate well with each other (r)
Things ‗fall between the cracks‘ when transferring
patients from one unit to another (r)
There is good cooperation among hospital units that
need to work together
Problems often occur in the exchange of information
across hospital units (r)
Hospital units work well together to provide the best
care for patients
Teamwork within hospital units People support one another in this unit
When a lot of work needs to be done quickly, we work
together as a team to get the work done
In this unit, people treat each other with respect
When one area in this unit gets really busy, others help
out
Hospital handoffs and transitions Important patient care information is often lost during
shift changes (r)
Shift changes are problematic for patients in this
hospital (r)
Frequency of incident reporting When a mistake is made, but is caught and corrected
before affecting the patient, how often is this
reported?
When a mistake is made, but has no potential to harm
the patient, how often is this reported?
When a mistake is made that could harm the patient,
but does not, how often is this reported?
Nonpunitive response to error Staff feel like their mistakes are held against them (r)
When an incident is reported, it feels like the person is
being written up, not the problem (r)
Staff worry that mistakes they make are kept in their
personnel file (r)
Communication openness Staff will freely speak up if they see something that
may negatively affect patient care
Staff feel free to question the decisions or actions of
those with more authority
Staff are afraid to ask questions when something does
not seem right (r)
Safety Culture Dimensions and Corresponding Survey Items
153
Table A.1 continued
Safety culture dimensions and corresponding survey items
(Smits et al., 2007; Sorra & Nieva, 2004).
Safety culture dimension Survey items
Feedback about and learning from
errors
We are actively doing things to improve patient safety
Mistakes have led to positive changes here
After we make changes to improve patient safety, we
evaluate their effectiveness
We are given feedback about changes put into place
based on incident reports
We are informed about errors that happen in this unit
In this unit, we discuss ways to prevent errors from
happening again
Supervisor/manager expectations
and actions promoting safety
My supervisor/manager says a good word when he/she
sees a job done according to established patient safety
procedures
My supervisor/manager seriously considers staff
suggestions for improving patient safety
Whenever pressure builds up, my supervisor/manager
wants us to work faster, even if it means taking
shortcuts (r)
My supervisor/manager overlooks patient safety
problems that happen over en over (r)
Hospital management support
for patient safety
Hospital management provides a work climate that
promotes patient safety
The actions of hospital management show that patient
safety is a top priority
Hospital management seems interested in patient safety
only after an incident happens (r)
Staffing We have enough staff to handle the workload
Staff in this unit work longer hours than is best for
patient care (r)
We use more agency/temporary staff than is best for
patient care (r)
Overall perceptions of patient
safety
It is just by chance that more serious mistakes don‘t
happen around here (r)
We work in ‗crisis mode‘ trying to do too much, too
quickly (r)
We have patient safety problems in this unit (r)
Our procedures and systems are good at preventing
errors from happening
Note. (r) = reverse coded item.
154
155
Summary
Proactive Safety Management in Health Care
Towards a Broader View of
Risk Analysis, Error Recovery, and Safety Culture
Medical errors occur frequently. The harm and additional costs associated with those errors
ask for effective safety management. According to the objective of minimal patient harm,
safety management in health care should be proactive; that is, risks should be anticipated and
reduced before patients are harmed. However, until recently, health care organisations
particularly used reactive approaches. Not until errors happened and harm was caused, did
they conduct risk analyses. Because such a reactive safety management approach is
insufficient, the research reported in this dissertation aimed to contribute to the understanding
of proactive safety management. This dissertation presents six studies that dealt with the main
research question: “How could health care organisations apply proactive safety management
to prevent patient harm and minimise costs of poor safety?”. Together, the studies addressed
three distinct but complementary approaches towards proactive safety management: (1)
conducting and integrating prospective and retrospective risk analyses (methods), (2)
obtaining information about error recovery (data), and (3) improving on safety culture
(organisational context).
In the first study (Chapter 2), a qualitative field study, the application of the
prospective risk analysis method Healthcare Failure Mode and Effect Analysis (HFMEA™)
was evaluated in Dutch health care. In an HFMEA™ analysis, a multidisciplinary team
identifies and assesses potential risks in a selected health care process and determines actions
Summary
156
to eliminate or reduce those risks. In total, 13 HFMEA™ analyses were conducted. User
feedback has revealed benefits and drawbacks regarding HFMEA™. Benefits were the
systematic approach, the multidisciplinary nature of the analysis, and the fact that the analysis
yielded a clear understanding of the process itself, the accompanying tasks, as well as the
potential risks. Drawbacks were related to the risk assessment part of HFMEA™ (i.e. the
rating scales and the decision tree) and the time investment needed to conduct the analysis. In
sum, this study has shown that HFMEA™ can successfully be applied in health care, but also
that the method can be improved, for instance by customising the rating scales.
The second study (Chapter 3), a qualitative field study on two units of a Dutch general
hospital, concentrated on the triangulation and integration of prospective and retrospective
methods for risk analysis, which is important because both methods are subject to biases. In
the prospective analyses, a condensed version of HFMEA™ was used for the identification
and assessment of risks in selected processes. In the retrospective analyses, incidents were
reported by employees and subsequently investigated. The methods were integrated by
making use of information from retrospective incident reports for prospective risk
identification and assessment, and by matching their categorisation schemes. Results
indicated that the two analyses yielded divergent overviews of risks. Two evaluation forms,
filled out by employees, showed that the combination of prospective and retrospective
analyses provided additional insight into risks. Thus, this study has demonstrated that
triangulation of prospective and retrospective methods can provide a more complete and
reliable picture of risks. Furthermore, integration of the two methods could be advantageous
in terms of efficiency of analysis, setting priorities, and improving the methods themselves.
The third study (Chapter 4) addressed the order of implementation of prospective and
retrospective methods and its influence on incident reporting behaviour. A quasi-
experimental field study was conducted on 12 units of two Dutch general hospitals. A
reversed-treatment non-equivalent control group design was used to test the hypotheses that
had been formulated. The six units of Hospital 1 first carried out a prospective risk analysis
(an adapted version of HFMEA™), after which a sophisticated retrospective incident
reporting and analysis system was introduced. On the six units of Hospital 2, the two methods
were implemented in reverse order. Data from the incident reporting and analysis system and
from evaluation forms showed that carrying out a prospective analysis first (i.e. before
introducing a sophisticated incident reporting and analysis system) did improve incident
reporting behaviour in terms of a wider spectrum of reported incident types and a larger
proportion of incidents reported by doctors. However, the proposed order did not necessarily
Summary
157
yield a larger number of reported incidents. Overall, this study has shown that health care
organisations can use prospective methods to enhance incident reporting behaviour.
The fourth study (Chapter 5), a qualitative field study, concentrated on error recovery,
which is important since errors are unavoidable and cannot be completely prevented by error
reduction strategies. There appeared to be a need for a clearer and more consistent definition
of near misses to enable their large-scale reporting and analysis. By means of incident reports
and interviews on four units of two Dutch general hospitals, information about error handling
was collected. Analysis of 143 error handling processes has revealed that different incident
types each provide unique information about the way errors are detected and dealt with. Two
possible definitions of near misses have been proposed and it has been argued that the
optimal definition may well be contingent on organisational context.
In the fifth study (Chapter 6), also a qualitative field study, it was argued that besides
information about successful error recovery, information about unsuccessful error recovery
can also be used to develop strategies that promote people‘s abilities to recognise and
intercept errors in time. In total, 52 medication errors (that all resulted in severe patient harm
or patient death, i.e. accidents) were analysed to reveal failed, missed and absent error
recovery opportunities. The results have indicated that, in addition to near misses, accidents
can be used as a data source to obtain information about error recovery as well.
The sixth study (Chapter 7) was a longitudinal panel survey, in which it was proposed
that a positive safety culture can be essential for proactive safety management. A culture in
which safety is considered a top priority, could enhance safety behaviour and performance
and can promote the success of prospective and retrospective methods for risk analysis. In a
panel survey among 701 health care employees of three Dutch hospitals, the trends in safety
culture were evaluated after an extensive safety management programme had been
implemented. The use of self-reported safety culture surveys as an evaluation instrument
could be questioned because only a few significant changes were identified. Nevertheless, the
observed positive trends regarding incident reporting behaviour, response to errors, and
management support are promising. Further, the results have shown that incident reporting
behaviour is positively associated with feedback about and learning from errors, handoffs and
transitions, as well as teamwork. In case of effective feedback and learning mechanisms and
good teamwork (including smooth shift changes), health care employees are thus more likely
to report errors.
To conclude, three distinct but complementary approaches to proactive safety
management have been proposed in this dissertation. A culture in which safety is deemed of
Summary
158
utmost importance and people are preoccupied with risks is essential for proactive safety
management. Critical assessment of processes can be useful to identify and eliminate risks
before errors may occur. If errors do crop up, a vigilant attitude of health care employees can
enhance timely error recognition and correction, as a result of which patient harm can still be
averted. In case of errors, people should anyhow be willing to report them to share lessons
and facilitate organisational learning. Such a proactive approach towards safety management
could improve patient safety, minimise patient harm, and limit costs of poor safety. In
conclusion, proactive safety management is important for those people who find that patient
safety is a moving target.
159
Samenvatting
(Summary in Dutch)
Proactive Safety Management in Health Care
Towards a Broader View of
Risk Analysis, Error Recovery, and Safety Culture
Medische fouten vinden met enige regelmaat plaats. De schade en extra kosten die hiermee
gepaard gaan vragen om effectief veiligheidsmanagement. Het doel van
veiligheidsmanagement in de gezondheidszorg is het minimaliseren van schade voor
patiënten, wat pleit voor een proactieve aanpak. Men zou risico‘s moeten voorzien en
reduceren, voordat patiënten schade oplopen. Echter, tot voor kort gebruikten
zorginstellingen vooral reactieve benaderingen. Pas nadat fouten zich hadden voorgedaan en
schade was veroorzaakt, voerden zij risicoanalyses uit. Omdat een dergelijke reactieve
benadering van veiligheidsmanagement ontoereikend is, had het in deze dissertatie
beschreven onderzoek tot doel om een bijdrage te leveren aan het begrip van proactief
veiligheidsmanagement. In deze dissertatie worden zes studies gepresenteerd die ingingen op
de belangrijkste onderzoeksvraag: “Hoe zouden zorginstellingen proactief
veiligheidsmanagement in de praktijk kunnen brengen om schade aan patiënten te voorkomen
en kosten van gebrekkige veiligheid te minimaliseren?”. Gezamenlijk hadden de studies
betrekking op drie verschillende, maar complementaire, benaderingen voor proactief
veiligheidsmanagement: (1) het uitvoeren en integreren van prospectieve en retrospectieve
risicoanalyses (methoden), (2) het verkrijgen van informatie over het herstellen van fouten
(data), en (3) het verbeteren van de veiligheidscultuur (organisatorische context).
Samenvatting
160
In de eerste studie (Hoofdstuk 2), een kwalitatief veldonderzoek, werd de toepassing
van de prospectieve risicoanalyse methode Healthcare Failure Mode and Effect Analysis
(HFMEA™) geëvalueerd in de Nederlandse gezondheidszorg. In een HFMEA™ analyse
heeft een multidisciplinair team als taak mogelijke risico‘s in een bepaald zorgproces te
identificeren en te beoordelen. Vervolgens beschrijft het team acties om die risico‘s te
elimineren of te reduceren. In totaal zijn 13 HFMEA™ analyses uitgevoerd. Feedback van de
gebruikers heeft de voor- en nadelen van HFMEA™ blootgelegd. Voordelen waren de
systematische aanpak, het multidisciplinaire karakter van de analyse en het feit dat de analyse
een duidelijk inzicht opleverde in het proces zelf, de bijbehorende taken en de mogelijke
risico‘s. Nadelen waren gerelateerd aan het gedeelte van HFMEA™ dat zich richt op het
beoordelen van de risico‘s (d.w.z. de beoordelingsschalen en de beslisboom) en aan de
tijdsinvestering die nodig is om de analyse uit te voeren. Kortom, deze studie heeft
aangetoond dat HFMEA™ succesvol kan worden toegepast in de gezondheidszorg, maar ook
dat de methode kan worden verbeterd, bijvoorbeeld door de beoordelingsschalen aan te
passen.
De tweede studie (Hoofdstuk 3), een kwalitatief veldonderzoek op twee afdelingen
van een Nederlands algemeen ziekenhuis, was gericht op het combineren en integreren van
prospectieve en retrospectieve methoden voor risicoanalyse. Deze geïntegreerde aanpak is
belangrijk, omdat beide methoden aan vertekeningen onderhevig zijn. In de prospectieve
analyses werd een ingekorte versie van HFMEA™ gebruikt om risico‘s in bepaalde
processen te identificeren en te beoordelen. In de retrospectieve analyses werden incidenten
gemeld door medewerkers en vervolgens onderzocht. De methoden werden geïntegreerd door
informatie van de retrospectieve incident meldingen te gebruiken voor de prospectieve risico
identificatie en beoordeling, en door gebruik te maken van vergelijkbare categorisatie
schema‘s. De resultaten lieten zien dat de twee analyses uiteenlopende overzichten van
risico‘s opleverden. Twee evaluatieformulieren, ingevuld door medewerkers, toonden aan dat
de combinatie van prospectieve en retrospectieve analyses aanvullend inzicht gaf in risico‘s.
Dus, deze studie heeft aangetoond dat triangulatie van prospectieve en retrospectieve
methoden een completer en betrouwbaarder beeld van risico‘s kan genereren. Verder zou
integratie van de twee methoden voordelig kunnen zijn in termen van efficiëntie van analyse,
het stellen van prioriteiten en het verbeteren van de methoden zelf.
De derde studie (Hoofdstuk 4) was gericht op de volgorde van implementatie van
prospectieve en retrospectieve methoden en de invloed daarvan op incident meldingsgedrag.
Een quasi-experimenteel veldonderzoek is uitgevoerd op 12 afdelingen van twee Nederlandse
Samenvatting
161
algemene ziekenhuizen. Een ―reversed-treatment non-equivalent control group design‖ is
gebruikt om de opgestelde hypotheses te testen. De zes afdelingen van Ziekenhuis 1 voerden
eerst een prospectieve risicoanalyse uit (een aangepaste versie van HFMEA™), waarna een
sophisticated incident melding- en analysesysteem werd geïntroduceerd. Op de zes
afdelingen van Ziekenhuis 2 werden de twee methoden in omgekeerde volgorde
geïmplementeerd. Gegevens van het incident melding- en analysesysteem en van
evaluatieformulieren toonden aan dat het eerst uitvoeren van een prospectieve analyse (d.w.z.
alvorens een sophisticated incident melding- en analysesysteem te introduceren) incident
meldingsgedrag verbeterde in termen van een breder spectrum van gemelde incidenten en een
grotere proportie van incidenten die gemeld worden door artsen. Echter, de voorgestelde
volgorde leek niet per definitie een groter aantal gemelde incidenten op te leveren. In het
algemeen heeft deze studie aangetoond dat zorginstellingen prospectieve methoden kunnen
gebruiken om incident meldingsgedrag te stimuleren.
De vierde studie (Hoofdstuk 5), een kwalitatief veldonderzoek, had betrekking op het
herstellen van fouten. Het corrigeren van fouten is belangrijk, omdat fouten onvermijdelijk
zijn en niet volledig voorkomen kunnen worden door strategieën die gericht zijn op het
reduceren van fouten. Er bleek een behoefte te bestaan aan een duidelijkere en meer
consistente definitie van bijna incidenten om ervoor te zorgen dat deze op grote schaal
gemeld en geanalyseerd kunnen worden. Door middel van incident meldingen en interviews
op vier afdelingen van twee Nederlandse algemene ziekenhuizen, werd informatie verzameld
over het aanpakken van fouten. Analyse van 143 ―error handling‖ processen heeft laten zien
dat verschillende incident types elk unieke informatie opleveren over de manier waarop
fouten ontdekt en aangepakt worden. Er zijn twee mogelijke definities van bijna incidenten
voorgesteld en er is gesuggereerd dat de optimale definitie best afhankelijk kan zijn van
organisatorische context.
In de vijfde studie (Hoofdstuk 6), ook een kwalitatief veldonderzoek, werd beweerd
dat naast informatie over het succesvol herstellen van fouten, informatie over het niet
succesvol herstellen van fouten ook kan worden gebruikt om strategieën te ontwikkelen die
de mogelijkheden van mensen bevorderen om fouten tijdig te herkennen en te
onderscheppen. In totaal werden 52 medicatiefouten (die allemaal hadden geleid tot ernstige
schade voor de patiënt of zelfs het overlijden van de patiënt, d.w.z. ongelukken) geanalyseerd
om gefaalde, gemiste en afwezige herstelmogelijkheden te identificeren. De resultaten
hebben aangetoond dat, naast bijna incidenten, ook ongelukken gebruikt kunnen worden als
databron om informatie te verkrijgen over het herstellen van fouten.
Samenvatting
162
De zesde studie (Hoofdstuk 7) was een longitudinaal vragenlijstonderzoek, waarin
gesteld werd dat een positieve veiligheidscultuur essentieel kan zijn voor proactief
veiligheidsmanagement. Een cultuur waarin veiligheid als een top prioriteit gezien wordt zou
veiligheidsgedrag en -prestaties kunnen bevorderen en kan het succes van prospectieve en
retrospectieve methoden voor risicoanalyse vergroten. In een panelonderzoek onder 701
zorgprofessionals van drie Nederlandse ziekenhuizen werden de trends in veiligheidscultuur
geëvalueerd nadat een grootschalig veiligheidsmanagement programma was
geïmplementeerd. Het gebruik van veiligheidscultuur vragenlijsten als evaluatie instrument
zou betwijfeld kunnen worden, aangezien er slechts een klein aantal significante
veranderingen geïdentificeerd werd. Toch zijn de waargenomen positieve trends met
betrekking tot incident meldingsgedrag, de reactie op fouten en de support vanuit het
management veelbelovend. Verder hebben de resultaten aangetoond dat incident
meldingsgedrag positief samenhangt met feedback over en leren van fouten, wisseling van
diensten, en teamwork. In geval van effectieve mechanismen voor feedback en leren en goed
teamwork (inclusief goede wisseling van diensten) is de kans groter dat zorgprofessionals
fouten zullen melden.
Concluderend, zijn in deze dissertatie drie verschillende maar complementaire
benaderingen voor proactief veiligheidsmanagement beschreven. Een cultuur waarin
veiligheid van het grootste belang wordt geacht en mensen bedacht zijn op risico‘s is
essentieel voor proactief veiligheidsmanagement. Kritische beoordeling van processen is
bruikbaar om risico‘s te identificeren en te elimineren voordat fouten zich voordoen. Als
fouten optreden, kan een alerte houding van zorgprofessionals het tijdig herkennen en
corrigeren van fouten bevorderen, waardoor schade aan patiënten nog steeds voorkomen kan
worden. Mensen moeten hoe dan ook bereid zijn om fouten te melden en om lessen te delen
om zo organisatorisch leren mogelijk te maken. Een dergelijke proactieve benadering van
veiligheidsmanagement zou patiëntveiligheid kunnen verbeteren, schade aan patiënten
kunnen minimaliseren en kosten voor gebrekkige veiligheid kunnen beperken. Tot besluit,
proactief veiligheidsmanagement is belangrijk voor die mensen die van mening zijn dat
patiëntveiligheid een ―moving target‖ is.
163
List of Publications
Gerritsen, G., De Bey, G., & Kessels-Habraken, M. (2009). Risicoprofiel maakt ziekenhuis
veiliger. Medisch Contact, 64, 1818-1821.
Habraken, M., & Van der Schaaf, T. (2005). Biases in a medical incident causation database:
A quantitative evaluation using PRISMA-Medical. In N. Marmaras, T. Kontogiannis,
& D. Nathanael (Eds.), ACM International Conference Proceeding Series: Vol. 132.
Proceedings of the 2005 Annual Conference on European Association of Cognitive
Ergonomics (pp. 167-173). Chania, Greece.
Habraken, M. M. P., & Van der Schaaf, T. W. (in press). If only….: Failed, missed and
absent error recovery opportunities in medication errors. Quality and Safety in Health
Care.
Habraken, M. M. P., Van der Schaaf, T. W., Leistikow, I. P., & Reijnders-Thijssen, P. M. J.
(2009). Prospective risk analysis of health care processes: A systematic evaluation of
the use of HFMEA™ in Dutch health care. Ergonomics, 52, 809-819.
Habraken, M. M. P., Van der Schaaf, T. W., Van Beusekom, B. R., & Huygelen, C. (2005).
Beter analyseren van incidenten. Medisch Contact, 60, 940-943.
List of Publications
164
Kessels-Habraken, M., De Jonge, J., Van der Schaaf, T., & Rutte, C. (2009). Prospective risk
analysis prior to retrospective incident reporting and analysis as a means to enhance
incident reporting behaviour: A quasi-experimental field study. Manuscript under
revision.
Kessels-Habraken, M., Van der Schaaf, T., De Jonge, J., & Rutte, C. (2009). Defining near
misses: Towards a sharpened definition based on empirical data. Manuscript under
revision.
Kessels-Habraken, M., Van der Schaaf, T., De Jonge, J., Rutte, C., & Kerkvliet, K. (2009).
Integration of prospective and retrospective methods for risk analysis in hospitals.
International Journal for Quality in Health Care, doi:10.1093/intqhc/mzp043.
Leistikow, I. P., Kessels-Habraken, M. M. P., & De Bruijn, J. A. (2009). Risicoanalyse loont
de moeite. Medisch Contact, 64, 1634-1639.
165
About the Author
Marieke Kessels - Habraken was born on November 5th, 1981 in Mierlo in The Netherlands.
She obtained her Master's degree in Industrial Engineering and Management Science at
Eindhoven University of Technology in 2005. She completed the programme cum laude. For
her master thesis on incident reporting and analysis she received the Dutch Risk Management
Study Award 2005. In May 2005, she started her research at the department of Industrial
Engineering & Innovation Sciences at Eindhoven University of Technology. This dissertation
is the result of her PhD research on proactive safety management in health care. Currently,
Marieke works at Infoland in Veldhoven as a consultant. She advises and guides
organisations regarding the design and implementation of software solutions for quality
management.
