View
4
Download
0
Category
Preview:
Citation preview
For peer review only
A stepped-wedge cluster randomised controlled trial to assess the effectiveness of an electronic medication
management system to reduce medication errors, adverse drug events, and average length of stay at two paediatric
hospitals
Journal: BMJ Open
Manuscript ID bmjopen-2016-011811
Article Type: Protocol
Date Submitted by the Author: 07-Mar-2016
Complete List of Authors: Wesbrook, Johanna; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Li, Ling; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Raban, Magdalena; Macquarie University, Centre for Health Systems and
Safety Research, Australian Institute of Health Innovation Baysari, Melissa; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Mumford, Virginia; Macquarie University, Australian Institute of Health Innovation Prgomet, Mirela; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Lake, Rebecca; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University McCullagh, Cheryl; The Sydney Childrens Hospitals Network Dalla Pozza, Luciano; The Sydney Childrens Hospitals Network Karnon, Jon; The University of Adelaide
O'Brien, Tracey; The Sydney Childrens Hospitals Network Ambler, Geoff; The Children’s Hospital at Westmead, Institute of Endocrinology and Diabetes Day, Richard; St Vincents Hospital Sydney and UNSW, Clinical Pharmacology; University of New South Wales, Pharmacology Cowell, Christopher; University of Sydney, Sydney Medical School; Sydney Children's Hospitals Network, Institute of Endocrinology and Diabetes, Children’s Hospital at Westmead Gazarian, Madlen; University of New South Wales, School of Women's and Children's Health; Sydney Children's Hospital, Department of Immunology and Infectious Diseases Worthington, Rachael; The Sydney Childrens Hospitals Network
Lehmann, Christoph; Vanderbilt University White, Les; NSW Health , Office of Kids and Families Barbaric, Draga; The Sydney Childrens Hospitals Network Gardo, Alan; The Sydney Childrens Hospitals Network Kelly, Margaret; NSW Health , Office of Kids and Families Kennedy, Peter; New South Wales Ministry of Health, eHealth
<b>Primary Subject Heading</b>:
Health informatics
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open on D
ecember 26, 2020 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2016-011811 on 21 O
ctober 2016. Dow
nloaded from
For peer review only
Secondary Subject Heading: Paediatrics
Keywords: PAEDIATRICS, eMM Systems, Medication errors, Adverse drug events, Electronic Prescribing
Page 1 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 1
Version 2 – 28 March 2016
A stepped-wedge cluster randomised controlled trial to assess the effectiveness
of an electronic medication management system to reduce medication errors,
adverse drug events, and average length of stay at two paediatric hospitals
Westbrook JI1, Li L
1, Raban MZ
1, Baysari MT
1, Mumford V
1, Prgomet M
1, Georgiou A
1, Kim T
1,
Lake R1, McCullagh C
2, Dalla-Pozza L
2, Karnon J
4, O’Brien TA
2, Ambler G
3, Day R
5, Cowell CT
2,
Gazarian M5
,Worthington R2, Lehmann CU
6, White L
7, Barbaric D
2, Gardo A
2, Kelly M
7, Kennedy
P 8
1Centre for Health Systems and Safety Research, Australian Institute of Health Innovation,
Macquarie University, Sydney, Australia, 2The Sydney Children’s Hospitals Network,
3The Sydney
Children’s Hospitals Network and The University of Sydney, 4University of Adelaide,
5School of
Medical Sciences, Faculty of Medicine, University of New South Wales, 6Vanderbilt University,
USA, 7Office of Kids and Families NSW Health,
8eHealth NSW Health Ministry
Corresponding Author: Professor Johanna Westbrook, Centre for Health Systems and Safety
Research, Australian Institute of Health Innovation, Macquarie University, level 6, 75 Talavera Rd,
Macquarie Park, 2109 johanna.westbrook@mq.edu.au (+61 2) 9850 2402
Keywords: Paediatrics; eMM systems; medication errors; adverse drug events (ADEs); electronic
prescribing
Word count: 4225
Page 2 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 2
ABSTRACT
Introduction Medication errors are the most frequent cause of preventable harm in hospitals.
Medication management in paediatric patients is particularly complex and consequently potential
for harms are greater than in adults. Electronic medication management (eMM) systems are
heralded as a highly effective intervention to reduce adverse drug events (ADEs), yet internationally
evidence of their effectiveness in paediatric populations is limited. This study will assess the
effectiveness of an eMM system to reduce medication errors, ADEs, and length of stay (LOS). The
study will also investigate system impact on clinical work processes.
Methods and Analysis A stepped-wedge cluster randomised control trial (SWCRCT) will measure
changes pre and post eMM system implementation in prescribing and medication administration
error (MAE) rates, potential and actual ADEs, and average ward LOS. In stage 1, 8 wards within
the first paediatric hospital will be randomised to receive the eMM system one week apart. In stage
2, the second paediatric hospital will randomise implementation of a modified eMM and outcomes
will be assessed.
Prescribing errors will be identified through record reviews, and MAEs through direct observation
of nurses and record reviews. Actual and potential severity will be assigned. Outcomes will be
assessed at the patient-level using mixed models, taking into account correlation of admissions
within wards and multiple admissions for the same patient, with adjustment for potential
confounders. Interviews and direct observation of clinicians will investigate the effects of the
system on workflow. Data from site 1 will be used to develop improvements in the eMM and
implemented at site 2, where the SWCRCT design repeated (stage 2).
Ethics and Dissemination The research has been approved by the Human Research Ethics
Committee of the Sydney Children’s Hospitals Network and Macquarie University. Results will be
reported through academic journals and seminar and conference presentations.
Page 3 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 3
Strengths
• The Stepped-Wedge Cluster Randomised Controlled Trial (SWCRCT) study design is the
closest to the gold-standard of an RCT, and has rarely been applied to study the effects of
information technology in healthcare.
• Few previous studies have investigated the impact of electronic systems on medication
administration error rates. We present a novel approach to conduct direct observation of this
process using the Precise Observation System for Safe Use of Medicines (POSSUM).
POSSUM allows observers to quickly and accurately record drug information e.g. name,
strength, compliance with procedures f as well as the number and length of interruptions and
multi-tasking.
• We will assess potential harm from medication errors identified and importantly also
measure actual harm to children.
Limitations
• Direct, close observation lends itself to the “Hawthorne effect” whereby participants may
seek to ‘improve’ their performance. This may result in an underestimatione of the ‘true’
medication administration error rate.
Page 4 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 4
INTRODUCTION
Prescribing, administering, and monitoring medicines in children is highly complex. Compared
with adults, medication errors in children are three times more likely to result in harm.(1) Paediatric
patients are at particular risk of certain errors, such as tenfold dosing errors, facilitated by mistakes
in dose calculation, poor documentation of decimal points and confusion with the use of zeros.
Some errors have led to death.(2-4) The complex medication decision process in children often
requires information about age, weight, dosing ranges and off-label use of medicines.(2) Systematic
reviews show errors occur in 5-27% of all medication orders for children.(5-7) Prescribing errors
account for 3-37% of errors, dispensing 5-58%, administration 72-75%, and documentation 17-
21%, although accuracy of estimates is affected by the great variability in definitions and
measurement methods used.(6) There is much less research about the harms associated with
medicine use. One Australian study found 19.2% of paediatric inpatients experienced adverse drug
events (ADEs) and estimated 12.3% were potential ADEs, 7.0% were actual ADEs, and 3.6% were
preventable.(8)
The frequency and severity of medication administration errors (MAEs) in Australian adult
hospitals is a cause for concern.(9, 10) In 4,271 medications administered to 720 patients, 74.4%
were found to have at least one procedural failure (e.g. failure to check a patient’s identification).(9,
10) One in four had a clinical error (e.g. wrong dose). Of intravenous drug administrations, 70%
had one clinical error, of which 25.5% were judged to be serious and likely to cause permanent
harm.(9) MAEs among children are rarely studied.(6, 11) A major barrier to MAE research is the
methodological challenges. MAEs cannot be accurately detected from retrospective record reviews;
they require direct observation of nurses administering medications to patients.
Can information technology reduce medication errors?
Electronic medication management (eMM) systems are expected to reduce medication errors and
ADEs significantly. However, rigorous evidence demonstrating these effects is limited.(7) A
systematic review identified eight studies of eMM effectiveness among paediatric patients. Meta-
analysis showed a significant reduction in prescribing error risk (RR 0.08) but not in ADEs or
mortality.(12) There are no Australian studies of eMM system use in a paediatric setting. Previous
studies have often relied upon incident reports to measure error rates, which are generally
unreliable.(13) Only one study(14) of paediatric inpatients has used a control group to assess eMM
effectiveness. Internationally there is currently insufficient evidence to demonstrate clinical benefit
from eMM in paediatric patients. This view was confirmed by a policy review for the American
Academy of Pediatrics(15) which called for the demonstrable enhancement of eMM systems to
Page 5 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 5
better meet the specific needs of paediatrics to ensure their safety and effectiveness. Further,
existing eMM systems used in paediatrics have been found lacking in functionalities required for
safe prescribing, dispensing and administration of medications.(16, 17)
In 2005, Han et al(18) reported a significant increase in the mortality rate among critical care
children at a US paediatric hospital following introduction of a commercial eMM system. The rapid
implementation process and limited attention to the significant workflow re-design required were
considered major factors in this outcome. Subsequent studies(19) have shown no increase in
mortality rates. However, the Han et al(18) study caused considerable alarm and served to
demonstrate the substantial dangers of poor implementation and the importance of monitoring
outcomes following system implementation and responding to problems identified.(20)
eMM use in adult hospitals, while highly effective at reducing medication errors,(21) also
introduced new ‘system-related’ errors. An investigation of 1,164 prescribing errors post eMM in
two adult Australian hospitals found 42.4% were facilitated by the system (78 per 100
admissions).(22) The most frequent mechanism was incorrect selection from a drop-down menu.
Results from that study were used to instigate changes to eMM software and to inform changes to
user training programs. The study was one of the first to quantify the rate of ‘system-related’ errors
and produced an innovative dual classification for categorising both the manifestations and
mechanisms of these system-related errors.(22) The nature and magnitude of such potential new
risks have rarely been studied in paediatrics.
Why evaluate eHealth systems
eHealth systems are having an increasing impact on the delivery of health care services, yet, despite
their widespread effects and vast cost, they are rarely subject to rigorous research.(23) This limited
evidence-base significantly hinders improvements and innovation in the design, implementation and
use of health information technology (IT) systems. Internationally the need for IT evaluation studies
to employ more robust designs and sophisticated analyses is well recognised.(24) Excessive
reliance has been placed upon uncontrolled before and after, and qualitative studies. Evidence of
eHealth system effectiveness and safety is crucial to facilitate policy-makers’ and health care
organisations’ informed decisions about investments and prioritisation of health IT systems.
This project presents an innovative and comprehensive program to assess eMM system
effectiveness in reducing medication errors, ADEs and length of stay (LOS) and rapidly deploying
new knowledge into practice for subsequent implementation.
Page 6 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 6
The project aims are:
Aim 1 To quantify the safety and effectiveness of an eMM system to reduce medication
errors, ADEs and average LOS among paediatric patients using a stepped-wedge
cluster randomised controlled trial (SWCRCT) design;
Aim 2 To assess the effects of the eMM on clinicians’ workflow and efficiency; and
Aim 3 To assess the extent to which feedback of study results and subsequent
modifications to the eMM design and associated work practices can improve eMM
effectiveness in reducing medication errors at a subsequent implementation tested
via a second SWCRCT.
METHODS AND ANALYSIS
Research Plan
Aim 1: Quantify the safety and effectiveness of an eMM system to reduce medication errors
(potential and actual ADEs), and average LOS among paediatric patients.
Study Design and Setting: We will conduct a SWCRCT to measure changes in prescribing and
medication administration errors (MAEs) which result in potential and actual ADEs, along with
changes in average ward LOS, pre and post eMM. Cluster randomised controlled trials (CRCT) are
ideally suited to test interventions where individual patient randomisation is not possible. CRCTs
commonly use a parallel group design, in which the clusters are randomised to either the
intervention or control arm of the study. It is often regarded as unethical to withhold an intervention
from a proportion of participants if it is believed that the intervention will do more good than harm.
The SWCRCT design, where the intervention is delivered sequentially to all trial clusters over a
number of time periods, is an alternative to the traditional parallel groups design. The order in
which the clusters (wards) receive the intervention is randomised, and by the end of the study all
clusters will have adopted the intervention.(25) The steps represent the predetermined periods
when data relating to each of the clusters will be collected. This design is the closest to the gold-
standard of an RCT, when such a design is not possible. The stepped-wedge design offers particular
strengths in allowing the modelling of the effect of time on the effectiveness of the intervention. In
stage 1 a SWCRCT will be conducted at site 1, and in stage 2 a SWCRCT will be conducted at site
2.
The study setting is the Sydney Children’s Hospitals Network which incorporates the two tertiary
paediatric hospitals in Sydney, The Children’s Hospital at Westmead (CHW; site 1) and Sydney
Page 7 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 7
Children’s Hospital, Randwick (SCH; site 2). The network provides care for 51,000 inpatient
admissions, 92,000 emergency department presentations and over 1,000,000 outpatient services
events each year. Both hospitals provide a complex and comprehensive range of services caring for
seriously ill and injured children and adolescents across the state of New South Wales and beyond.
Randomisation: In stage 1, 8 wards within site 1 will be randomised to receive the eMM system.
Randomisation will be conducted by a person blinded to ward identity. At baseline all 8 wards are
in the control period (Figure 1). At step 1, the eMM system will be implemented in the first ward.
The eMM system will then be implemented to a new ward in sequence weekly. By the end of step
8, all wards will have the eMM system at site 1.
<Insert Figure 1 here>
Medication error and ADE definitions: Medication errors are defined as any error in the
prescribing, supply, preparation, administration, or monitoring of a medication, regardless of
whether such errors lead to adverse consequences. In this study we will not measure dispensing
errors. ADEs are defined as harm or injury as a consequence of the use or non-use of medicines.(8)
Medication errors may result in actual ADEs or potential ADEs. For example, a medication error
may occur but is intercepted prior to administration thus preventing harm to the patient.
In this study we will be seeking to identify medication errors, and to determine those that resulted in
harm (actual ADEs) or potential harm (potential ADEs). Figure 2 illustrates the medication error
and ADE classification processes for this study.
<Insert Figure 2 here>
Data Collection: Medication error data collection occurs at baseline and each stage, (i.e. in every
subsequent week as eMM implementation occurs and for two additional weeks after full
implementation). For the primary study objective of determining eMM effectiveness to reduce
medication errors and ADEs we will collect data at 11 points on all wards (baseline and at each step
including two weeks after full implementation, Figure 1). This will allow us to measure changes pre
and post eMM system introduction in: 1) Prescribing error rates per order and per admission by type
and severity (potential and actual ADEs); 2) MAEs per order and per admission by type and
severity (potential and actual ADEs). For the secondary outcome of changes in LOS we will obtain
data for a further 21 steps in the follow-up period to provide greater statistical power. As these are
routine administrative data, no additional data collection is required.
Page 8 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 8
Prescribing error and ADE detection: A review of medication charts at baseline and each step will
be conducted complying with a standard error protocol.(8-10, 21) This protocol will be extended to
develop methods for determining the rate at which errors are detected and intercepted by staff,
actions taken, and any harm experienced.
Medication administration error (MAE) and ADE detection: For the MAE study, data will be
collected using direct observation. Nurses will be observed preparing and administering
medications. In our previous studies using this approach in adult hospitals,(10) over 80% of nurses
consented to participate and we expect similar rates for this study. Direct observations will be
supported by an innovative data collection tool, the Precise Observation System for Safe Use of
Medicines (POSSUM, Figure 3).(10, 26) The POSSUM tool allows observers to quickly and
accurately record drug information e.g. name, strength, and dose. The POSSUM tool also allows
collection of the number and length of interruptions experienced and multi-tasking (e.g. answering a
question while also selecting medicines). Nurses’ compliance with core procedures, such as
checking a patient’s identification, will also be recorded. Comparing observational data with
patients’ medical records (via retrospective audit) will enable identification of the number, types
and severity of MAEs.
<Insert Figure 3 here>
Observers will have a pre-allocated observation period to ensure coverage across the day and the
week.(9, 10) Observers will follow a “serious error” protocol i.e. they must intervene if they witness
an administration that is potentially dangerous to the patient. Observers will not have access to
patients’ medication charts and will record only what they observe. Thus, most MAEs will not be
identifiable until chart review. Past inter-rater reliability tests showed kappa scores from 0.94 to
0.96 following training in the use of POSSUM.(9, 10)
Direct, close observation lends itself to the “Hawthorne effect” whereby participants may seek to
‘improve’ their performance. If nurses change their practices, and are more careful when observed,
this will lead to an underestimation of the ‘true’ MAE rate. This bias would be present both pre and
post eMM. Our prior research suggests the likelihood of sustained change on busy wards is low.(27,
28)
Prescribing error, MAE and ADE classifications: Prescribing errors and MAEs will be classified
into: (i) procedural errors (for prescribing errors these include illegible order, illegal order,
Page 9 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 9
incomplete order; for MAEs these include failure to follow correct patient identification process,
compliance with double-checking requirements, etc.); and (ii) clinical errors (e.g. wrong dose,
wrong drug, wrong route, wrong strength) using previously applied classifications.(10, 21) .
Once an error has been identified, a rating of the potential severity of that error will be made.
Subsequently, records will be reviewed for evidence of error detection and interception, and for any
actual harm to the patient. Thus, medication errors which occur will receive both a ‘potential’ harm
rating and an ‘actual harm rating’ (Figure 2). As most previous medication error studies do not
assess actual harm, this double classification process will allow us to compare our findings with
previous studies, as well as allow an assessment of the accuracy of such approaches compared to
estimating the actual harm from medication errors.
Evidence of harm as a consequence of a medication error will be identified through a
comprehensive review of patients’ medical records. This clinical review process will be assisted by
the provision of specific harm identification guides for reviewers which will identify, for specific
drugs and error types, the types of evidence which would suggest harm had occurred following the
medication error. Figure 4 presents an example of one of the harm identification guides to be used.
<Insert Figure 4 here>
Experienced clinicians will abstract data from medical records using a structured data collection
form and the harm identification guides. A multi-disciplinary clinical review panel will re-assess a
minimum 5% sample of the records and will also review any records which reviewers identify as
particularly complex. Panel members will be blind to the location, and whether data were generated
pre or post eMM. Panel members will not know the ward order of rollout and specific dates when
each ward became an intervention ward with the stepped wedge design and therefore blinding of
pre and post data will be possible. Actual and potential severity will be assigned using the National
Coordinating Council for Medication Error Reporting and Prevention (NCC-MERP) scale for
adverse event outcomes (29) and the 5-point Severity Assessment Code (SAC) Scale,(30) as used in
our past research.(9, 10, 21) This will allow comparison with a greater number of previous studies.
System-related errors: We will apply our two dimensional classification, modified to incorporate
recent recommendations in this area,(20) to assess whether medication errors post eMM were
facilitated by eMM design, i.e. are ‘system-related’. This process identifies the manifestation (e.g.
wrong dose) and mechanisms (e.g. incorrect menu selection). These results will be used to provide
Page 10 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 10
recommendations about IT design and user training.(22) Any changes to the eMM design features,
training or work processes during the study will be documented.
Sample sizes and analyses: Sample size calculations have taken into account the estimated
between-cluster variance, i.e. between wards variance, and the design effect associated with the
stepped-wedge design.(31) Calculations were based on our previous studies in adult hospitals and
hospital data from the paediatric sites. Each ward has on average 14 admissions per week with an
average LOS 3.78 days (SD=7.39) with seven medications per admission.
Prescribing errors: Based on our previous studies(21) the expected reduction in overall prescribing
error rate is 60%, from 4.06 errors per admission (SD=5.27) to 1.62 (SD=2.87) with an estimated
intraclass correlation coefficient (ICC) of 0.06 (Table 1). The number of wards required to detect a
60% change for two-sided tests (80% power; �<5%) is one, with 10 data collection steps after
baseline. For ADEs, the required number of wards is seven to detect a 60% reduction (Table 1). To
be conservative and provide greater power, we will collect data on all eight wards allowing
detection of a minimum change of 20% for overall errors and 42% for ADEs. At each step records
for 112 patient admissions will be reviewed, totalling 1,232 across the study.
MAEs: Based on our previous studies(9, 10) we expect the overall MAE rate per administration to
fall by 27%, from 0.37 (SD=0.65) to 0.27 (SD=0.52) with an estimated ICC of 0.03 (Table 1). The
required number of wards (two-sided test; 80% power; �<5%) is seven, with 10 steps after baseline.
For ADEs, the required number of wards is six. We will collect data using all eight wards which
will allow detection of a minimum 20% change overall and 48% for ADEs. At each step we will
observe at least 240 medication administrations, totalling 2,640 across the study.
LOS: There are very limited data on the impact of eMM systems on LOS at ward level. A study in
an ICU showed a 23% reduction in LOS post eMM system.(32) To detect a 23% reduction in LOS,
i.e. from 3.78 (SD=7.39) to 2.92 days, with eight wards, will require additional (routinely collected
LOS) data in a total of 31 steps for a two-sided test with 80% power �<5%.
Table 1 Prescribing error and MAE power calculations
ER Pre-
(SD)
ER Post
(SD)
% changes
from past
eMM
study
ICC
(7)
Avg. # of
Adm or
Admins/
study step
# of
steps
(k)
Min.
#
wards
req.
# of
wards
Min. %
change
detectable
Max.
power
Presc.
Errorsβ
4.1 (5.3) 1.6 (2.9) 60% 0.06 14 10 1 8 20% 100%
Page 11 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 11
Data analyses: Medication error rates per order, stratified by error type, study step, and ward will
be calculated. For each outcome of interest, data collected across all measurement periods and all
study steps will be used in the analyses comparing intervention status (pre versus post eMM).
Analyses will apply the intention-to-treat principle. Patient data will be analysed according to the
status of the wards (i.e. pre or post eMM) where patients were admitted. Outcomes will be assessed
at the patient-level using mixed models, taking into account correlation of patient admissions within
wards (clusters) and multiple admissions for the same patient, with adjustment for potential
confounding factors. For the MAE analyses we will adjust for contextual factors including
interruptions, multi-tasking, nurse age, gender, and adherence to policies. For LOS analysis we will
adjust for patient characteristics, such as major diagnoses, comorbidity, age, and gender. The mixed
models will incorporate fixed terms for ward intervention status, measurement time steps (including
baseline) and other confounders. The analyses will include multiple time points pre and post eMM
implementation. The study design will allow us to determine temporal changes in system
effectiveness, e.g. to determine if error rates continue to decline over time. We will apply the
‘system-related’ error classification (22) to identify system-related error rate and associated
mechanisms.
Aim 2: To assess the effects of the eMM on workflow and efficiency
Study design and sample: Observations and interviews will be held with medical, nursing, and
pharmacy staff at baseline to allow mapping of core work processes associated with medication
provision. At each step in the stepped-wedge design, a small number of interviews will be
conducted with nursing and medical staff to gain insights into clinical staff perceptions of the
impact of the system on workflow, efficiency and care delivery. Purposive sampling will continue
until theme saturation. Trustworthiness of the qualitative data will be achieved through triangulation
of data and investigators, engagement with the field with a documented audit trail and member
checking.(33)
The eMM is anticipated to have a significant impact on the work of hospital pharmacists. We will
conduct a direct observational study of approximately eight pharmacists at site 1. We will observe
them for 200 hours between 7:30 – 18:00 pre and post eMM system implementation to examine
ADEsβ 0.3 (0.7) 0.1 (0.4) 44% 0.005 14 10 7 8 42% 83%
MAEs¥ 0.4 (0.6) 0.3 (0.5) 27% 0.03 30 10 4 8 20% 97%
ADEs~ 4.2% 1.8% 57% 0.003 30 10 6 8 48% 93%
β Per admission; ER = error rate;
~ % all medication administrations;
# =number; Adm =
admissions; Admins=administrations; req= required; Min=minimum,¥
per admin.
Page 12 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 12
changes in i) task time distributions ii) location of work and iii) communication patterns. Using the
validated WOMBAT approach(26, 34) multiple dimensions of work will be captured (e.g. tasks
performed, with whom, with what, location, interruptions, and multi-tasking). On data entry, tasks
are automatically time stamped when entered in the WOMBAT data collection tool. Figure 5 is an
example of data collection within WOMBAT. An additional sample of 140 hours of observation
will be conducted to capture the work of oncology pharmacists whose work involves supporting the
delivery of complex drug regimens to children with cancer.
<Insert Figure 5 here>
Data generated will allow changes in task time distributions and sequencing of work to be
determined. These data will be examined in relation to changes in outcome indicators generated
(from Aim 1, e.g. medication error rates, LOS) on the same wards, along with additional quality
process indicators (routinely collected) pre and post the eMM system. These will include time to
antibiotic treatment for children presenting with fever and neutropenia, a demonstrated indicator of
patient outcome,(35) and data on protocol compliance obtained from audits conducted by the
hospital. The research team have extensive experience in analysis of such work measurement
data.(28)
Aim 3: Assess the extent to which feedback (from Aims 1 and 2) and subsequent modifications to
an eMM system design can improve eMM system effectiveness in reducing medication errors
Evaluations of health IT serve multiple purposes, ranging from providing an objective assessment
of the success of the new technology in delivering anticipated benefits, to identification of deficits
in the system, their source, and the ways they can be addressed. This is critical to improving system
effectiveness, relevance, and responsiveness. For Aim 3, the findings of the SWCRCT at CHW (i.e.
stage 1) will be reported to the Project Evaluation Committee (PEC) made up of members of the
research team, and the Hospitals’ eMM system Project Steering Committee. The PEC will meet
every month to consider the implications of study findings across a number of domains including
the system’s technical features (e.g. compatibility with other hospital systems), effectiveness (e.g.
error reduction and system-related errors); professional attitudes (e.g. satisfaction) and
organisational features (e.g. work processes), as a means of formulating changes to eMM system
design features and user training.
This will form the key component of an action-oriented approach aimed at optimising system
Page 13 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 13
performance leading to an enhanced eMM system which will then be implemented across the
second site, SCH (stage 2). The SWCRCT design will be repeated at site 2 using the same methods
as above. Power calculations for stage 2 will be based on results from stage 1. We will conduct
separate analyses for all outcomes specified. Results for the two sites will be compared, using
multilevel and longitudinal analysis approaches to determine changes in error rates (taking baseline
data into account).
Expected outcomes and significance of the research project
This project will generate the first Australian data, in a paediatric setting, on the effectiveness of
eMM systems to reduce medication errors and ADEs, and provide an assessment of how systems
impact on the work of clinicians and the consequences for the delivery of care to children.
Importantly, the findings will be directly applied to enhance the eMM system design, and work
processes and then tested further through evaluation of the enhanced eMM system at a second
paediatric hospital. These results will be particularly valuable for other paediatric hospitals yet to
commence implementations. Exploiting the SWCRCT design within an action-research model is
highly innovative, and will deliver high quality data on system effectiveness. Such a model of
formally integrating health IT assessment results as a basis for active engagement with IT vendors
and clinicians to bring about system change has both national and international significance. The
study advances explicit methods for the systematic identification of harm associated with
medication errors. The data generated will also provide the basis for a robust cost-effectiveness
analysis, which will be the subject of a separate protocol.
ETHICS AND DISSEMINATION
The research has been approved by the Human Research Ethics Committee of the Sydney
Children’s Hospitals Network (HREC/15/SCHN/370). In the first instance, results from site 1 will
be reported to the PEC so that they can be used to inform eMM system and work process design
prior to implementation at site 2. Results will also be reported through academic journals and
conference presentations. The project is funded through a National Health and Medical Research
Council Partnership Grant. As such, the project team includes academic researchers, hospital
clinicians and experts involved in the implementation of the eMM system at the two hospital sites,
along with senior policy makers from agencies within the State Health Department involved in
eHealth system strategy and policy. This provides the project with access to a range of other
conduits through which to disseminate results to, for example, policy makers and system
implementers.
Page 14 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 14
Page 15 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 15
Authors Contributions: JW, LL, MB, AGe and CM initiated the project and led the development
of the NHMRC grant proposal. JW, LL, MB, AGe, RD, JK, LDP, CTC, GA, and TOB are chief
investigators on the project and all made contributions to the protocol in their specific areas of
expertise. CM, LW, NB, AGa, CUL, MG, PK, MK, AB and DB are associate investigators on the
NHMRC grant and provided input to the protocol, particularly in the areas of paediatric clinical
practice and broader eHealth strategy in relation to eMM systems. MZR, MP, VM, TK, RW, RL are
members of the project team and have made significant contributions to the protocol in terms of the
design of details regarding the collection and classification of medication errors and harm. JW
prepared the first draft of this manuscript based upon the grant proposal and all authors have
reviewed and provided input.
Funding Statement: The project is supported by a National Health and Medical Research Council
Partnership Grant (APP1094878) in partnership with: Sydney Children’s Hospitals Network;
eHealth New South Wales; Office of Kids and Families, New South Wales.
Competing Interests Statement: None to declare.
Page 16 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 16
References:
1. Kaushal R, Barker KN, Bates DW. How can information technology improve patient safety
and reduce medication errors in children's health care? Archives of Pediatrics & Adolescent
Medicine. 2001;155(9):1002-7.
2. Doherty C, Mc Donnell C. Tenfold Medication Errors: 5 Years’ Experience at a University-
Affiliated Pediatric Hospital. Pediatrics. 2012;129(5):916-24.
3. Gazarian M, Drew A, Bennett A. Medical mishap. Intravenous paracetamol in paediatrics:
cause for caution Australian Prescriber. 2014;37(1):24-5.
4. Kim GR, Chen AR, Arceci RJ, Mitchell SH, Kokoszka KM, Daniel D, et al. Error reduction
in pediatric chemotherapy: computerized order entry and failure modes and effects analysis.
Archives of pediatrics & adolescent medicine. 2006;160(5):495.
5. Kaushal R, Bates DW, Landrigan C, McKenna KJ, Clapp MD, Federico F, et al. Medication
errors and adverse drug events in pediatric inpatients. J Amer Med Assoc. 2001;285(16):2114-20.
6. Miller M, Robinson K, Lubomski L, Rinke M, Pronovost P. Medication errors in paediatric
care: a systematic review of epidemiology and an evaluation of evidence supporting reduction
strategy recommendations. Quality and Safety in Health Care. 2007;16(2):116-26.
7. Rinke ML, Bundy DG, Velasquez CA, Rao S, Zerhouni Y, Lobner K, et al. Interventions to
Reduce Pediatric Medication Errors: A Systematic Review. Pediatrics. 2014.
8. Gazarian M, Graudins LV. Long-term Reduction in Adverse Drug Events: An Evidence-
Based Improvement Model. Pediatrics. 2012;129(5):e1334-e42.
9. Westbrook J, Rob M, Woods A, Parry D. Errors in the administration of intravenous
medications in hospital and the role of correct procedures and nurse experience. BMJ Quality and
Safety. 2011:doi: 10.1136/bmjqs-2011-000089.
10. Westbrook JI, Woods A, Rob MI, Dunsmuir WTM, Day RO. Association of interruptions
with an increased risk and severity of medication administration errors. Archives of Internal
Medicine. 2010;170(8):683-90.
11. Manias E, Kinney S, Cranswick N, Williams A, Borrott N. Interventions to Reduce
Medication Errors in Pediatric Intensive Care. Annals of Pharmacotherapy. 2014.
12. van Rosse F, Maat B, Rademaker CMA, van Vught AJ, Egberts ACG, Bollen CW. The
Effect of Computerized Physician Order Entry on Medication Prescription Errors and Clinical
Outcome in Pediatric and Intensive Care: A Systematic Review. Pediatrics. 2009;123(4):1184-90.
13. Westbrook J, Li L, Lehnbom E, M B, Braithwaite J, Burke R, et al. What are incident
reports telling us? A comparative study at two Australian hospitals of medication errors identified at
audit, detected by staff and reported to an incident system. International Journal of Quality in
Health Care. 2015;27(1):1-9.
14. King WJ, Paice N, Rangrej J, Forestell GJ, Swartz R. The effect of computerized physician
order entry on medication errors and adverse drug events in pediatric inpatients. Pediatrics.
2003;112(3 Pt 1):506-9.
15. Lehmann CU, Johnson KB, Council on Clinical Information Technology American
Academy of Pediatrics. Electronic Prescribing in Pediatrics: Toward Safer and More Effective
Medication Management. Pediatrics. 2013;131(4):e1350-e6.
16. Dufendach KR. Core functionality in pediatric electronic health records.
17. Slight SP, Berner ES, Galanter W, Huff S, Lambert BL, Lannon C, et al. Meaningful Use of
Electronic Health Records: Experiences From the Field and Future Opportunities. JMIR medical
informatics. 2015;3(3):e30.
18. Han Y, Carcillo J, Venkataraman S, Clark R, Watson R, Nguyen T, et al. Unexpected
increased mortality after implementation of a commercially sold computerized physician order
entry system. Pediatrics. 2005;116:1506 - 12.
19. Longhurst CA, Parast L, Sandborg CI, Widen E, Sullivan J, Hahn JS, et al. Decrease in
Hospital-wide Mortality Rate After Implementation of a Commercially Sold Computerized
Physician Order Entry System. Pediatrics. 2010;126(1):14-21.
Page 17 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 17
20. Brigham and Women's Hospital HMS, Partners HealthCare. . Computerized Prescriber
Order Entry Medication Safety (CPOEMS): Uncovering and learning from issues and errors Silver
Spring, MD: US Food and Drug Administration 2015.
21. Westbrook J, Reckmann M, Li L, Runciman W, Burke R, Lo C, et al. Effects of two
commercial electronic prescribing systems on prescribing error rates in hospital inpatients: a before
and after study. PLoS Medicine. 2012;9(1):e1001164. doi:10.1371/journal.pmed.
22. Westbrook JI, Baysari MT, Li L, Burke R, Richardson KL, Day RO. The safety of
electronic prescribing: manifestations, mechanisms, and rates of system-related errors associated
with two commercial systems in hospitals. Journal of the American Medical Informatics
Association. 2013;20(6):1159-67.
23. Koppel R, Lehmann CU. Implications of an emerging EHR monoculture for hospitals and
healthcare systems. Journal of the American Medical Informatics Association. 2015;22(2):465-71.
24. McKibbon K, Lokker C, Handler S, Dolovich LR, Holbrook A, O'Reilly D, et al. Enabling
medication management through health information technology. Rockville MD: Agency for
Healthcare Research and Quality, 2011.
25. Brown C, Lilford R. The stepped wedge trial design: a systematic review. BMC Medical
Research Methodology. 2006;6(1):54.
26. Westbrook J, Woods A. Development and testing of an observational method for detecting
medication administration errors using information technology. Studies in Health Technology &
Informatics. 146. Amsterdam: IOS Press; 2009. p. 429-33.
27. Dean B, Barber N. Validity and reliability of observational methods for studying medication
administration errors. American journal of health-system pharmacy : AJHP : official journal of the
American Society of Health-System Pharmacists. 2001;58(1):54-9.
28. Westbrook JI, Li L, Georgiou A, Paoloni R, Cullen J. Impact of an electronic medication
management system on hospital doctors’ and nurses’ work: a controlled pre–post, time and motion
study. Journal of the American Medical Informatics Association. 2013;20(6):1150-8.
29. Prevention NCCfMERa. NCC MERP Index for Categorizing Medication Errors 2001 [cited
2015]. Available from: http://www.nccmerp.org/types-medication-errors.
30. New South Wales Health Department. Severity Assessment Code (SAC) Matrix. Sydney:
NSW Health, 2005.
31. Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials.
Contemporary Clinical Trials. 2007;28(2):182-91.
32. Lellouche F, Mancebo J, Jolliet P, Roeseler J, Schortgen F, Dojat M, et al. A Multicenter
Randomized Trial of Computer-driven Protocolized Weaning from Mechanical Ventilation.
American Journal of Respiratory and Critical Care Medicine. 2006;174(8):894-900.
33. Callen J, Paoloni R, Li J, Stewart M, Gibson K, Georgiou A, et al. Perceptions of the Effect
of Information and Communication Technology on the Quality of Care Delivered in Emergency
Departments: A Cross-Site Qualitative Study. Annals of Emergency Medicine. 2013;61(2):131-44.
34. Ballerman M, Shaw N, Mayes D, Gibney R, Westbrook J. Validation of the Work
Observational Method By Activity Timing (WOMBAT) method of conducting time-motion
observations in critical care settings: an observational study BMC Med Inform Decis.
2011;11:doi:10.1186/472-6947-11-32.
35. Gafter-Gvili A, Fraser A, Paul M, Leibovici L. Meta-Analysis: Antibiotic Prophylaxis
Reduces Mortality in Neutropenic Patients. Annals of Internal Medicine. 2005;142(12_Part_1):979-
95.
Page 18 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 18
FIGURES
Figure 1 Schematic of stepped-wedge cluster randomised controlled trial study design
Figure 2 Medication error, ADE and harm identification and classification process
Medication errors
Identified
Prescribing errors
identified via record
review
Administration errors via
direct observation &
record review
Classify procedural
&
Clinical error types
Error
intercepted
Error NOT
intercepted
Harm detected =
ADE
No harm detected
= Potential ADE
Classify types of harm
& resource use
Confirmed potential
ADE
Classify severity of
Actual ADE
Confirmed potential
ADENo harm occurred
=Potential ADE
Classify potential
error severity
Page 19 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 19
Figure 3 POSSUM tool for data collection during the direct observational study of medication
administration
Page 20 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 20
Figure 4 Example of harm assessment guide for paediatric opioid errors, to be used during medical
record review
Errors resulting in Identify harm indicators
Higher than
recommended dose or
concentration of the
drug
Symptoms: drowsiness, nausea
Signs: pinpoint pupils, lower level of consciousness, respiratory
depression Medications: naloxone, abrupt cessation of opioid
Tests: blood gasses – high CO2, O2 saturation <95%
Actions: any code or arrest
Care record: increased monitoring, increased level of care (e.g.
transfer to ICU), family notified, incident report filed, discharge
delayed Lower than
recommended dose or
concentration of the
drug
Symptoms: complaints of pain, agitation/restlessness
Signs: high pulse rate, high BP, pain scores noted and increasing
Medications: additional doses of analgesic/s
Page 21 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 21
Figure 5 Work Observation Method By Activity Timing (WOMBAT) for conducting observational
studies of health professionals work pre and post eMM system implementation
Page 22 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
The SPIRIT Checklist
Item 1: Descriptive title identifying the study design, population, interventions, and, if
applicable, trial acronym.
“A stepped-wedge cluster randomised controlled trial to assess the effectiveness of an
electronic medication management system to reduce medication errors, adverse drug
events, and average length of stay at two paediatric hospitals.”
Can be found on title page, and page two of submitted protocol document.
Item 2a: Trial identifier and registry name. If not yet registered, name of intended registry.
Australian New Zealand Clinical Trials Registry (ANZCTR)
Item 2b: All items from the World Health Organization Trial Registration Data Set.
Nil.
Item 3: Date and version identifier.
7 March 2016 – Protocol Manuscript - Version One
Item 4: Sources and types of financial, material, and other support.
This project is funded by a National Health and Medical Research Council Partnership Grant
(APP1094878) in partnership with the Sydney Children’s Hospitals Network, eHeath New
South Wales, and the Office of Children and Families, New South Wales. [Page 16]
Item 5a: Names, affiliations, and roles of protocol contributors.
As listed on page 2 and 16
Item 5b: Name and contact information for the trial sponsor.
The principle investigator and sponsor of this study is Professor Johanna Westbrook, of the
Centre for Health Systems and Safety Research, Australian Institute of Health Innovation,
Macquarie University, Sydney, NSW.
E-mail: johanna.westbrook@mq.edu.au
Phone: +61 02 9850 2402
Item 5c: Role of study sponsor and funders, if any, in study design; collection,
management, analysis, and interpretation of data; writing of the report; and the decision
to submit the report for publication, including whether they will have ultimate authority
over any of these activities.
The trial has been principally designed by Professor Westbrook, and the research team at
the Macquarie University. The research team have full independence from the funding
bodies in terms of decisions regarding the interpretation and publication of findings.
Item 5d: Composition, roles, and responsibilities of the coordinating centre, steering
committee, endpoint adjudication committee, data management team, and other
individuals or groups overseeing the trial, if applicable (see Item 21a for Data Monitoring
Committee).
Page 23 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
The research team at CHSSR, headed by Professor Westbrook will have responsibility for
designing and managing the study, and coordinating the collection, management, analysis
and interpretation of data.
The steering committee will be convened at least every three months, although more
frequently at pertinent times during the study, and will be engaged to provide their
expertise and advice on data collection and study progress. The research at CHSSR will have
responsibility for data management and analysis.
Item 6a: Description of research question and justification for undertaking the trial,
including summary of relevant studies (published and unpublished) examining benefits
and harms for each intervention.
Introduction: Prescribing, administering, and monitoring medicines in children is highly
complex. Compared with adults, medication errors in children are three times more likely to
result in harm. Electronic medication management (eMM) systems are expected to reduce
medication errors and ADEs significantly. However, rigorous evidence demonstrating these
effects is limited. [Page 5]
Existing Knowledge: A systematic review identified eight studies of eMM effectiveness
among paediatric patients. Meta-analysis showed a significant reduction in prescribing error
risk (RR 0.08) but not in ADEs or mortality. There are no Australian studies of eMM system
use in a paediatric setting. Previous studies have often relied upon incident reports to
measure error rates, which are generally unreliable. Only one study of paediatric inpatients
has used a control group to assess eMM effectiveness. Internationally there is currently
insufficient evidence to demonstrate clinical benefit from eMM in paediatric patients.
In 2005, Han et al. reported a significant increase in the mortality rate among critical care
children at a US paediatric hospital following introduction of a commercial eMM system.
The rapid implementation process and limited attention to the significant workflow re-
design required were considered major factors in this outcome. Subsequent studies have
shown no increase in mortality rates. However, the Han et al. study caused considerable
alarm and served to demonstrate the substantial dangers of poor implementation and the
importance of monitoring outcomes following system implementation and responding to
problems identified.
eMM use in adult hospitals, while highly effective at reducing medication errors, also
introduced new ‘system-related’ errors. An investigation of 1,164 prescribing errors post
eMM in two adult Australian hospitals found 42.4% were facilitated by the system (78 per
100 admissions). [Pages 5-6]
See further justification pages 5-6
Item 6b: Explanation for choice of comparators.
This is an pre-, post-, evaluation study, assessing changes between medication
administration errors, adverse drug events (ADEs) and effects on patient length of stay, pre
and post eMM roll-out. See pages 7-8 for further details
Item 7: Specific objectives or hypotheses.
Aim 1 - To quantify the safety and effectiveness of an eMM system to reduce medication
errors, ADEs and average LOS among paediatric patients using a stepped-wedge cluster
randomised controlled trial (SWCRCT) design;
Page 24 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Aim 2 - To assess the effects of the eMM on clinicians’ workflow and efficiency; and
Aim 3 - To assess the extent to which feedback of study results and subsequent
modifications to the eMM design and associated work practices can improve eMM
effectiveness in reducing medication errors at a subsequent implementation tested
via a second SWCRCT. [Page 7]
Item 8: Description of trial design including type of trial (e.g., parallel group, crossover,
factorial, single group), allocation ratio, and framework (e.g., superiority, equivalence,
non-inferiority, exploratory).
The study is a stepped wedge cluster randomised controlled trial (SWCRCT) evaluating
differences pre- and post- eMM implementation.
Item 9: Description of study settings (e.g., community clinic, academic hospital) and list of
countries where data will be collected. Reference to where list of study sites can be
obtained.
The study will be conducted in Sydney, New South Wales and will occur at two large tertiary
pediatric hospitals from within the Sydney Children’s Hospitals Network. [Pages 7-8]
Item 10: Inclusion and exclusion criteria for participants. If applicable, eligibility criteria for
study centres and individuals who will perform the interventions (e.g., surgeons,
psychotherapists).
All patients on the study wards will be participants. [Pages 9-10]
Item 11a: Interventions for each group with sufficient detail to allow replication, including
how and when they will be administered.
Detailed in methods section.
Item 11b: Criteria for discontinuing or modifying allocated interventions for a given trial
participant (e.g., drug dose change in response to harms, participant request, or
improving/worsening disease).
See methods section
Item 11c: Strategies to improve adherence to intervention protocols, and any procedures
for monitoring adherence (e.g., drug tablet return; laboratory tests).
The intervention – the eMM will be mandatory for use in the study hospitals.
Item 11d: Relevant concomitant care and interventions that are permitted or prohibited
during the trial.
N/A
Item 12: Primary, secondary, and other outcomes, including the specific measurement
variable (e.g., systolic blood pressure), analysis metric (e.g., change from baseline, final
value, time to event), method of aggregation (e.g., median, proportion), and time point
for each outcome. Explanation of the clinical relevance of chosen efficacy and harm
outcomes is strongly recommended.
Outcome: Medication administration errors (MAEs), ADEs and LOS
See methods section
Page 25 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Item 13: Time schedule of enrolment, interventions (including any run-ins and washouts),
assessments, and visits for participants. A schematic diagram is highly recommended (see
Figure)
For Aim 1, the data collection at CHW will commence on 25th April, 2016, one week before
the eMM rollout will commence. Direct observations of medication administration will
continue until 10th
July, two weeks after the roll out is complete, while chart reviews will
continue into 2017, until they are completed.
Figure 1: Timeline for the SWCRCT, with the baseline commencing at 25th April
For Aim 2, data will be collected during the same schedule as the implementation.
Item 14: Estimated number of participants needed to achieve study objectives and how it
was determined, including clinical and statistical assumptions supporting any sample size
calculations
See methods section
Item 15: Strategies for achieving adequate participant enrolment to reach target sample
size
See methods section
Item 16a: Method of generating the allocation sequence (eg, computer-generated random
numbers), and list of any factors for stratification. To reduce predictability of a random
sequence, details of any planned restriction (eg, blocking) should be provided in a
separate document that is unavailable to those who enrol participants or assign
interventions
Randomisation of the 8 wards will be conducted by a person blinded to the ward identity.
[Page 8]
Item 16b: Mechanism of implementing the allocation sequence (eg, central telephone;
sequentially numbered, opaque, sealed envelopes), describing any steps to conceal the
sequence until interventions are assigned
Page 26 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Randomisation will not be concealed. The study is a stepped wedge cluster randomised
controlled trial.
Item 16c: Who will generate the allocation sequence, who will enrol participants, and who
will assign participants to interventions?
The research team will determine the allocation sequence and communicate with the
hospital team who are responsible for the implementation of the eMM.
Item 17a: Who will be blinded after assignment to interventions (eg, trial participants,
care providers, outcome assessors, data analysts), and how
No one will be blinded to the assignment of the intervention.
Item 17b: If blinded, circumstances under which unblinding is permissible, and procedure
for revealing a participant’s allocated intervention during the trial
Methods: Data collection, management, and analysis
Item 18a: Plans for assessment and collection of outcome, baseline, and other trial data,
including any related processes to promote data quality (eg, duplicate measurements,
training of assessors) and a description of study instruments (eg, questionnaires,
laboratory tests) along with their reliability and validity, if known. Reference to where
data collection forms can be found, if not in the protocol
See methods
Item 18b: Plans to promote participant retention and complete follow-up, including list of
any outcome data to be collected for participants who discontinue or deviate from
intervention protocols
See methods. Retrospective review hospital medical records is the main source of data.
Item 19: Plans for data entry, coding, security, and storage, including any related
processes to promote data quality (eg, double data entry; range checks for data values).
Reference to where details of data management procedures can be found, if not in the
protocol
The methods section outlines the main data management processes. Information in the
individual ethics applications specified further details.
Item 20a: Statistical methods for analysing primary and secondary outcomes. Reference to
where other details of the statistical analysis plan can be found, if not in the protocol
Data analyses are specified in the methods section/
Page 27 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Item 20b: Methods for any additional analyses (eg, subgroup and adjusted analyses)
See methods section
Item 20c: Definition of analysis population relating to protocol non-adherence (eg, as
randomised analysis), and any statistical methods to handle missing data (eg, multiple
imputation)
See methods section. Analyses will be performed as per intention to treat.
Methods: Monitoring
Item 21a: Composition of data monitoring committee (DMC); summary of its role and
reporting structure; statement of whether it is independent from the sponsor and
competing interests; and reference to where further details about its charter can be
found, if not in the protocol. Alternatively, an explanation of why a DMC is not needed
A formal DMC has not been convened, however members of the research team will have
roles in monitoring the collection and quality of the data. Funding bodies will not have
access to the raw study data.
Item 21b: Description of any interim analyses and stopping guidelines, including who will
have access to these interim results and make the final decision to terminate the trial.
N/A
Item 22: Plans for collecting, assessing, reporting, and managing solicited and
spontaneously reported adverse events and other unintended effects of trial
interventions or trial conduct
The project manager (MZR) and the principle investigator (JIW), as well as the site
coordinators (CM and TOB) will monitor any adverse events which occur and wil report to
the Steering Committee.
Item 23: Frequency and procedures for auditing trial conduct, if any, and whether the
process will be independent from investigators and the sponsor
The project will be undertaken in line with specific human ethics approval requirements.
Item 24: Plans for seeking research ethics committee/institutional review board (REC/IRB)
approval
This study has been approved by the SCHN HREC (EC00130), reference code
HREC/15/SCHN/370.
Item 25: Plans for communicating important protocol modifications (eg, changes to
eligibility criteria, outcomes, analyses) to relevant parties (eg, investigators, REC/IRBs,
trial participants, trial registries, journals, regulators)
Any proposed modifications to the study will be discussed with the project steering
committee, and where required modification from the SCHN HREC will be sought.
Page 28 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Item 26a: Who will obtain informed consent or assent from potential trial participants or
authorised surrogates, and how (see Item 32)
See methods
Item 26b: Additional consent provisions for collection and use of participant data and
biological specimens in ancillary studies, if applicable
Not applicable.
Item 27: How personal information about potential and enrolled participants will be
collected, shared, and maintained in order to protect confidentiality before, during, and
after the trial
See methods
Item 28: Financial and other competing interests for principal investigators for the overall
trial and each study site
The research team declare no competing interests.
Items 29: Statement of who will have access to the final trial dataset, and disclosure of
contractual agreements that limit such access for investigators
Members of the research team involved in the data analysis and the writing up of the study
results will have access to the data set. Access will also be granted to selected members of
the steering committee and the site coordinators, where appropriate. The main funding
bodies will not have access to the raw data.
Item 30: Provisions, if any, for ancillary and post-trial care, and for compensation to those
who suffer harm from trial participation
The intervention implementation is the sole responsibility of the hospital sites and was
intended to proceed regardless of the evaluation study outlined here.
Item 31a: Plans for investigators and sponsor to communicate trial results to participants,
healthcare professionals, the public, and other relevant groups (eg, via publication,
reporting in results databases, or other data sharing arrangements), including any
publication restrictions
Study results will be communicated to the staff of the two study hospitals in meetings and
briefing sessions, to be arranged in coordination with senior staff. The results will be further
communicated with healthcare professionals and the public, through publication of journal
articles, conference presentations and other appropriate means.
Item 31b: Authorship eligibility guidelines and any intended use of professional writers
Page 29 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Standard academic and ethical guidelines for authorship and publication of results will
apply.
Item 31c: Plans, if any, for granting public access to the full protocol, participant-level
dataset, and statistical code
The study protocol will be published and thus available to the public. There are no plans to
provide public access to the full dataset or statistical code at this time.
Page 30 of 29
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
A stepped-wedge cluster randomised controlled trial to assess the effectiveness of an electronic medication
management system to reduce medication errors, adverse drug events, and average length of stay at two paediatric
hospitals: a study protocol
Journal: BMJ Open
Manuscript ID bmjopen-2016-011811.R1
Article Type: Protocol
Date Submitted by the Author: 18-Jul-2016
Complete List of Authors: Westbrook, Johanna; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Li, Ling; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Raban, Magdalena; Australian Institute of Healht Innovation, Faculty of
Medicine and Health Sciences, Macquarie University Baysari, Melissa; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Mumford, Virginia; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Prgomet, Mirela; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Georgiou, Andrew; Macquarie University, Australian Institute of Health Innovation Kim, Tara; Australian Institute of Health Innovation, Faculty of Medicine and Health Sciences, Macquarie University Lake, Rebecca; Australian Institute of Health Innovation, Faculty of
Medicine and Health Sciences, Macquarie University McCullagh, Cheryl; The Sydney Childrens Hospitals Network Dalla Pozza, Luciano; The Sydney Childrens Hospitals Network Karnon, Jon; The University of Adelaide O'Brien, Tracey; The Sydney Childrens Hospitals Network Ambler, Geoff; The Children’s Hospital at Westmead, Institute of Endocrinology and Diabetes Day, Richard; St Vincents Hospital Sydney and UNSW, Clinical Pharmacology; University of New South Wales, Pharmacology Cowell, Christopher; University of Sydney, Sydney Medical School; Sydney Children's Hospitals Network, Institute of Endocrinology and Diabetes, Children’s Hospital at Westmead
Gazarian, Madlen; University of New South Wales, School of Women's and Children's Health; Sydney Children's Hospital, Department of Immunology and Infectious Diseases Worthington, Rachael; The Sydney Childrens Hospitals Network Lehmann, Christoph; Vanderbilt University White, Les; NSW Health , Office of Kids and Families Barbaric, Draga; The Sydney Childrens Hospitals Network Gardo, Alan; The Sydney Childrens Hospitals Network
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open on D
ecember 26, 2020 by guest. P
rotected by copyright.http://bm
jopen.bmj.com
/B
MJ O
pen: first published as 10.1136/bmjopen-2016-011811 on 21 O
ctober 2016. Dow
nloaded from
For peer review only
Kelly, Margaret; NSW Health , Office of Kids and Families Kennedy, Peter; New South Wales Ministry of Health, eHealth
<b>Primary Subject Heading</b>:
Health informatics
Secondary Subject Heading: Paediatrics
Keywords: PAEDIATRICS, Medication errors, Adverse drug events, Electronic Prescribing, medical order entry systems, hospital medication systems
Page 1 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 1
A stepped-wedge cluster randomised controlled trial to assess the effectiveness
of an electronic medication management system to reduce medication errors,
adverse drug events, and average length of stay at two paediatric hospitals: a
study protocol
Westbrook JI1, Li L
1, Raban MZ
1, Baysari MT
1, Mumford V
1, Prgomet M
1, Georgiou A
1, Kim T
1,
Lake R1, McCullagh C
2, Dalla-Pozza L
2, Karnon J
4, O’Brien TA
2, Ambler G
3, Day R
5, Cowell CT
2,
Gazarian M5 ,Worthington R
2, Lehmann CU
6, White L
7, Barbaric D
2, Gardo A
2, Kelly M
7, Kennedy
P 8
1Centre for Health Systems and Safety Research, Australian Institute of Health Innovation,
Macquarie University, Sydney, Australia, 2The Sydney Children’s Hospitals Network,
3The Sydney
Children’s Hospitals Network and The University of Sydney, 4University of Adelaide,
5School of
Medical Sciences, Faculty of Medicine, University of New South Wales, 6Vanderbilt University,
USA, 7Office of Kids and Families NSW Health,
8eHealth NSW Health Ministry
Corresponding Author: Professor Johanna Westbrook, Centre for Health Systems and Safety
Research, Australian Institute of Health Innovation, Macquarie University, level 6, 75 Talavera Rd,
Macquarie Park, 2109 johanna.westbrook@mq.edu.au (+61 2) 9850 2402
Keywords: Paediatrics; medication errors; adverse drug events (ADEs); electronic prescribing,
medical order entry systems, hospital medication systems, electronic medical records
Word count: 4515
Page 2 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 2
ABSTRACT
Introduction Medication errors are the most frequent cause of preventable harm in hospitals.
Medication management in paediatric patients is particularly complex and consequently potential
for harms are greater than in adults. Electronic medication management (eMM) systems are
heralded as a highly effective intervention to reduce adverse drug events (ADEs), yet internationally
evidence of their effectiveness in paediatric populations is limited. This study will assess the
effectiveness of an eMM system to reduce medication errors, ADEs, and length of stay (LOS). The
study will also investigate system impact on clinical work processes.
Methods and Analysis A stepped-wedge cluster randomised controlled trial (SWCRCT) will
measure changes pre and post eMM system implementation in prescribing and medication
administration error (MAE) rates, potential and actual ADEs, and average LOS. In stage 1, 8 wards
within the first paediatric hospital will be randomised to receive the eMM system one week apart.
In stage 2, the second paediatric hospital will randomise implementation of a modified eMM and
outcomes will be assessed.
Prescribing errors will be identified through record reviews, and MAEs through direct observation
of nurses and record reviews. Actual and potential severity will be assigned. Outcomes will be
assessed at the patient-level using mixed models, taking into account correlation of admissions
within wards and multiple admissions for the same patient, with adjustment for potential
confounders. Interviews and direct observation of clinicians will investigate the effects of the
system on workflow. Data from site 1 will be used to develop improvements in the eMM and
implemented at site 2, where the SWCRCT design will be repeated (stage 2).
Ethics and Dissemination The research has been approved by the Human Research Ethics
Committee of the Sydney Children’s Hospitals Network and Macquarie University. Results will be
reported through academic journals and seminar and conference presentations.
Page 3 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 3
Strengths
• The Stepped-Wedge Cluster Randomised Controlled Trial (SWCRCT) study design is the
closest to the gold-standard of an RCT, and has rarely been applied to study the effects of
information technology in healthcare.
• Few previous studies have investigated the impact of electronic systems on medication
administration error rates. We present a novel approach to conduct direct observation of this
process using the Precise Observation System for Safe Use of Medicines (POSSUM).
POSSUM allows observers to quickly and accurately record drug information e.g. name,
strength, compliance with procedures, as well as the number and length of interruptions and
multi-tasking.
• We will assess potential harm from medication errors identified and importantly also
measure actual harm to children.
Limitations
• Direct, close observation lends itself to the “Hawthorne effect” whereby participants may
seek to ‘improve’ their performance. This may result in an underestimation of the ‘true’
medication administration error rate.
• This study will not evaluate adverse drug events occurring post discharge.
Page 4 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 4
INTRODUCTION
Prescribing, administering, and monitoring medicines in children is highly complex. Compared
with adults, medication errors in children are three times more likely to result in harm.(1) Paediatric
patients are at particular risk of certain errors, such as tenfold dosing errors, facilitated by mistakes
in dose calculation, poor documentation of decimal points and confusion with the use of zeros.
Some errors have led to death.(2-4) The complex medication decision process in children often
requires information about age, weight, dosing ranges and off-label use of medicines.(2) Systematic
reviews show errors occur in 5-27% of all medication orders for children.(5-7) Prescribing errors
account for 3-37% of errors, dispensing 5-58%, administration 72-75%, and documentation 17-
21%, although accuracy of estimates is affected by the great variability in definitions and
measurement methods used.(6) There is much less research about the harms associated with
medicine use. One Australian study found 19.2% of paediatric inpatients experienced adverse drug
events (ADEs) and estimated 12.3% were potential ADEs, 7.0% were actual ADEs, and 3.6% were
preventable.(8)
The frequency and severity of medication administration errors (MAEs) in Australian adult
hospitals is a cause for concern.(9, 10) In 4,271 medications administered to 720 patients, 74.4%
were found to have at least one procedural failure (e.g. failure to check a patient’s identification).(9,
10) One in four had a clinical error (e.g. wrong dose). Of intravenous drug administrations, 70%
had one clinical error, of which 25.5% were judged to be serious and likely to cause permanent
harm.(9) MAEs among children are rarely studied.(6, 11) A major barrier to MAE research is the
methodological challenges. MAEs cannot be accurately detected from retrospective record reviews;
they require direct observation of nurses administering medications to patients.
Can information technology reduce medication errors?
Electronic medication management (eMM) systems incorporate software which provides users with
the ability to prescribe, monitor and administer medications to patients. These systems also provide
the capacity to incorporate decision support tools such as alerts for drug-drug interactions(12).
eMM are usually integrated into a hospital’s clinical information system (Computerised provider
order entry system). These systems are expected to reduce medication errors and adverse drug
events significantly as a result of improved legibility of medication orders, complete and legally
compliant documentation, and through both the active and passive decision support tools embedded
in them. However, rigorous evidence demonstrating these effects are limited.(7) A systematic
review identified eight studies of eMM effectiveness among paediatric patients. Meta-analysis
showed a significant reduction in prescribing error risk (RR 0.08; 95%CI 0.01-0.77) across the three
Page 5 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 5
included studies, but not in ADEs or mortality.(13) There are no Australian studies of eMM system
use in a paediatric setting. Previous studies have often relied upon incident reports to measure error
rates, which are generally unreliable.(14) Only one study(15) of paediatric inpatients has used a
control group to assess eMM effectiveness. Internationally there is currently insufficient evidence to
demonstrate clinical benefit from eMM in paediatric patients. This view was confirmed by a policy
review for the American Academy of Pediatrics(16) which called for the demonstrable
enhancement of eMM systems to better meet the specific needs of paediatrics to ensure their safety
and effectiveness. Further, existing eMM systems used in paediatrics have been found lacking in
functionalities required for safe prescribing, dispensing and administration of medications.(17, 18)
In 2005, Han et al(19) reported a significant increase in the mortality rate among critical care
children at a US paediatric hospital following introduction of a commercial eMM system. The rapid
implementation process and limited attention to the significant workflow re-design required were
considered major factors in this outcome. Subsequent studies(20) have shown no increase in
mortality rates. However, the Han et al(19) study caused considerable alarm and served to
demonstrate the substantial dangers of poor implementation and the importance of monitoring
outcomes following system implementation and responding to problems identified.(21)
eMM use in adult hospitals, while highly effective at reducing medication errors,(22) also
introduced new ‘system-related’ errors. An investigation of 1,164 prescribing errors post eMM in
two adult Australian hospitals found 42.4% were facilitated by the system (78 per 100
admissions).(23) The most frequent mechanism was incorrect selection from a drop-down menu.
Results from that study were used to instigate changes to eMM software and to inform changes to
user training programs. The study was one of the first to quantify the rate of ‘system-related’ errors
and produced an innovative dual classification for categorising both the manifestations and
mechanisms of these system-related errors.(23) The nature and magnitude of such potential new
risks have rarely been studied in paediatrics.
Why evaluate eHealth systems
eHealth systems are having an increasing impact on the delivery of health care services, yet, despite
their widespread effects and vast cost, they are rarely subject to rigorous research.(24) This limited
evidence-base significantly hinders improvements and innovation in the design, implementation and
use of health information technology (IT) systems. Internationally the need for IT evaluation studies
to employ more robust designs and sophisticated analyses is well recognised.(25) Excessive
reliance has been placed upon uncontrolled before and after, and qualitative studies. Evidence of
Page 6 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 6
eHealth system effectiveness and safety is crucial to facilitate policy-makers’ and health care
organisations’ informed decisions about investments and prioritisation of health IT systems.
This project presents an innovative and comprehensive program to assess eMM system
effectiveness in reducing medication errors, ADEs and length of stay (LOS) and rapidly deploying
new knowledge into practice for subsequent implementation.
The project aims are:
Aim 1 To quantify the safety and effectiveness of an eMM system to reduce medication
errors, ADEs and average LOS among paediatric patients using a stepped-wedge
cluster randomised controlled trial (SWCRCT) design;
Aim 2 To assess the effects of the eMM on clinicians’ workflow and efficiency; and
Aim 3 To assess the extent to which feedback of study results and subsequent
modifications to the eMM design and associated work practices can improve eMM
effectiveness in reducing medication errors at a subsequent implementation tested
via a second SWCRCT.
METHODS AND ANALYSIS
Research Plan
Aim 1: Quantify the safety and effectiveness of an eMM system to reduce medication errors
(potential and actual ADEs), and average LOS among paediatric patients.
Study Design and Setting: We will conduct a SWCRCT to measure changes in prescribing and
medication administration errors (MAEs) which result in potential and actual ADEs, along with
changes in average LOS, pre and post eMM. Cluster randomised controlled trials (CRCT) are
ideally suited to test interventions where individual patient randomisation is not possible. CRCTs
commonly use a parallel group design, in which the clusters are randomised to either the
intervention or control arm of the study. It is often regarded as unethical to withhold an intervention
from a proportion of participants if it is believed that the intervention will do more good than harm.
The SWCRCT design, where the intervention is delivered sequentially to all trial clusters over a
number of time periods, is an alternative to the traditional parallel groups design. The order in
which the clusters (wards) receive the intervention is randomised, and by the end of the study all
clusters will have adopted the intervention.(26) The steps represent the predetermined periods
when data relating to each of the clusters will be collected. This design is the closest to the gold-
standard of an RCT, when such a design is not possible. The stepped-wedge design offers particular
Page 7 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 7
strengths in allowing the modelling of the effect of time on the effectiveness of the intervention. In
stage 1 a SWCRCT will be conducted at site 1, and in stage 2 a SWCRCT will be conducted at site
2.
The study setting is the Sydney Children’s Hospitals Network which incorporates the two major
urban tertiary referral paediatric hospitals in Sydney, Australia, The Children’s Hospital at
Westmead (CHW; site 1) and Sydney Children’s Hospital, Randwick (SCH; site 2). The network
provides care for 51,000 inpatient admissions, 92,000 emergency department presentations and over
1,000,000 outpatient services events each year. Both sites are acute paediatric tertiary hospitals with
emergency departments, out-patient and home services. During the study period the eMM will not
be available in the ICUs, theatres or outpatients. Site 1 accommodates 310 beds and site 2 has 180
beds. Both hospitals provide a complex and comprehensive range of services caring for seriously ill
and injured children and adolescents across the state of New South Wales and beyond.
The eMM Intervention: At baseline medication orders are written on paper medication charts and
details of medications administered are written on the same charts. The eMM clinical module will
be an addition to the hospitals’ existing commercial electronic clinical information system (Cerner
Corporation). Both hospital sites use the same commercial clinical system, however the software
can be customised to meet each hospital’s individual requirements. Based on results from our first
site and applying our action research methodology, we will provide advice to the second
implementation site on optimal customisation of the eMM (and associated processes) for their site.
The eMM allows electronic prescribing, recording of drug dispensing, drug administration and
medication reconciliation and monitoring processes. (See supplementary file for screenshots from
the eMM). The system allows for the ordering and administration of all oral, and IV medications
and fluids, but excludes anaesthesia medications. The eMM contains both passive and active
decision support in the form of links to guidelines, policies, protocols, order-sets, order sentences,
safety alerts (e.g. drug-drug interactions, dose range checks), and dosage calculators. During the
course of the study the eMM system will be accessible via any computer in the hospital allocated
for inpatient clinical care, but will not be available for patients in the intensive care units, theatres or
outpatients. The system will be predominantly accessed on hospital wards and in the hospital
pharmacy. Both fixed and mobile computing devices are available to staff using the system.
Medication reconciliation on admission and at discharge will be performed using the eMM system
when implemented. On admission, medication histories are taken and converted to inpatient orders.
While the patient is in hospital any new medication orders will be created within the eMM system.
Page 8 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 8
On discharge a discharge medication reconciliation occurs and orders are converted to paper
prescriptions for the patient. Patients then have their prescriptions filled at community pharmacies.
Randomisation: In stage 1, 8 wards within site 1 will be randomised to receive the eMM system.
Randomisation will be conducted by a person blinded to ward identity. At baseline all 8 wards are
in the control period (Figure 1). At step 1, the eMM system will be implemented in the first ward.
The eMM system will then be implemented to a new ward in sequence weekly. By the end of step
8, all wards will have the eMM system at site 1.
<Insert Figure 1 here>
Medication error and ADE definitions: Medication errors are defined as any error in the
prescribing, supply, preparation, administration, or monitoring of a medication, regardless of
whether such errors lead to adverse consequences. In this study we will not measure dispensing
errors. ADEs are defined as harm or injury as a consequence of the use or non-use of medicines.(8)
Medication errors may result in actual ADEs or potential ADEs. For example, a medication error
may occur but is intercepted prior to administration thus preventing harm to the patient.
In this study we will be seeking to identify medication errors, and to determine those that resulted in
harm (actual ADEs) or potential harm (potential ADEs). Figure 2 illustrates the medication error
and ADE classification processes for this study.
<Insert Figure 2 here>
Data Collection: Medication error data collection occurs at baseline (a one week period) and each
stage, (i.e. in every subsequent week as eMM implementation occurs and for two additional weeks
after full implementation). For the primary study objective of determining eMM effectiveness to
reduce medication errors and ADEs we will collect data at 11 points on all wards (baseline and at
each step including two weeks after full implementation, Figure 1). This will allow us to measure
changes pre and post eMM system introduction in: 1) Prescribing error rates per order and per
admission by type and severity (potential and actual ADEs); 2) MAEs per order and per admission
by type and severity (potential and actual ADEs). For the secondary outcome of changes in LOS we
will obtain data for a further 21 steps in the follow-up period to provide greater statistical power. As
these are routine administrative data, no additional data collection is required. Data on adverse drug
events occurring post discharge is not within the scope of this study.
Prescribing error and ADE detection: A review of medication charts at baseline and each step will
Page 9 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 9
be conducted complying with a standard error protocol.(8-10, 22) This protocol will be extended to
develop methods for determining the rate at which errors are detected and intercepted by staff,
actions taken, and any harm experienced.
Medication administration error (MAE) and ADE detection: For the MAE study, data will be
collected using direct observation. Nurses will be observed preparing and administering
medications. In our previous studies using this approach in adult hospitals,(10) over 80% of nurses
consented to participate and we expect similar rates for this study. Direct observations will be
supported by an innovative data collection tool, the Precise Observation System for Safe Use of
Medicines (POSSUM, Figure 3).(10, 27) The POSSUM tool allows observers to quickly and
accurately record drug information e.g. name, strength, and dose. The POSSUM tool also allows
collection of the number and length of interruptions experienced and multi-tasking (e.g. answering a
question while also selecting medicines). Nurses’ compliance with core procedures, such as
checking a patient’s identification, will also be recorded. Comparing observational data with
patients’ medical records (via retrospective audit) will enable identification of the number, types
and severity of MAEs.
<Insert Figure 3 here>
Observers will have a pre-allocated observation period to ensure coverage across the day and the
week.(9, 10) Observers will follow a “serious error” protocol i.e. they must intervene if they witness
an administration that is potentially dangerous to the patient. Observers will not have access to
patients’ medication charts and will record only what they observe. Thus, most MAEs will not be
identifiable until chart review. Past inter-rater reliability tests showed kappa scores from 0.94 to
0.96 following training in the use of POSSUM.(9, 10)
Direct, close observation lends itself to the “Hawthorne effect” whereby participants may seek to
‘improve’ their performance. If nurses change their practices, and are more careful when observed,
this will lead to an underestimation of the ‘true’ MAE rate. This bias would be present both pre and
post eMM. Our prior research suggests the likelihood of sustained change on busy wards is low.(28,
29)
Prescribing error, MAE and ADE classifications: Prescribing errors and MAEs will be classified
into: (i) procedural errors and (ii) clinical errors using previously applied classifications.(10, 22)
Procedural errors include, for prescribing orders, illegible orders, illegal orders (in which an aspect
Page 10 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 10
of the prescription does not comply with hospital policy, with law, or with the State Department of
Health policies) and incomplete orders. Medication administration procedural errors include, for
example, failure to follow the correct patient identification process prior to drug administration, and
correct conduct of medication double-checking requirements (many drugs within paediatric
hospitals require two nurses to independently check aspects of the drug preparation and
administration process). Clinical errors include wrong dose, wrong drug, wrong route, and wrong
strength errors.
Once an error has been identified, a rating of the potential severity of that error will be made, based
upon the NCC-MERP rating scale for adverse event outcomes.(30) Subsequently, records will be
reviewed for evidence of error detection and interception, and for any actual harm to the patient.
Thus, medication errors which occur will receive both a ‘potential’ harm rating and an ‘actual harm
rating’ (Figure 2). As most previous medication error studies do not assess actual harm, this double
classification process will allow us to compare our findings with previous studies, as well as allow
an assessment of the accuracy of such approaches compared to estimating the actual harm from
medication errors.
Evidence of harm as a consequence of a medication error will be identified through a
comprehensive review of patients’ medical records. This clinical review process will be assisted by
the provision of specific harm identification guides for reviewers which will identify, for specific
drugs and error types, the types of evidence which would suggest harm had occurred following the
medication error. Figure 4 presents an example of one of the harm identification guides to be used.
<Insert Figure 4 here>
Page 11 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 11
Experienced clinicians will abstract data from medical records using a structured data collection
form and the harm identification guides. A multi-disciplinary clinical review panel will re-assess a
minimum 5% sample of the records and will also review any records which reviewers identify as
particularly complex. Panel members will be blind to the location, and whether data were generated
pre or post eMM. Panel members will not know the ward order of rollout and specific dates when
each ward became an intervention ward with the stepped wedge design and therefore blinding of
pre and post data will be possible. Actual and potential severity will be assigned using the National
Coordinating Council for Medication Error Reporting and Prevention (NCC-MERP) scale for
adverse event outcomes (30) and the 5-point Severity Assessment Code (SAC) Scale,(31) as used in
our past research.(9, 10, 22) This will allow comparison with a greater number of previous studies.
System-related errors: We will apply our two dimensional classification, modified to incorporate
recent recommendations in this area,(21) to assess whether medication errors post eMM were
facilitated by eMM design, i.e. are ‘system-related’. This process identifies the manifestation (e.g.
wrong dose) and mechanisms (e.g. incorrect menu selection). These results will be used to provide
recommendations about IT design and user training.(23) Any changes to the eMM design features,
training or work processes during the study will be documented.
Sample sizes and analyses: Sample size calculations have taken into account the estimated
between-cluster variance, i.e. between wards variance, and the design effect associated with the
stepped-wedge design.(32) Calculations were based on our previous studies in adult hospitals and
hospital data from the paediatric sites. Each ward has on average 14 admissions per week with an
average LOS 3.78 days (SD=7.39) with seven medications per admission.
Prescribing errors: Based on our previous studies(22) the expected reduction in overall prescribing
error rate is 60%, from 4.06 errors per admission (SD=5.27) to 1.62 (SD=2.87) with an estimated
intraclass correlation coefficient (ICC) of 0.06 (Table 1). The number of wards required to detect a
60% change for two-sided tests (80% power; �<5%) is one, with 10 data collection steps after
baseline. For ADEs, the required number of wards is seven to detect a 60% reduction (Table 1). To
be conservative and provide greater power, we will collect data on all eight wards allowing
detection of a minimum change of 20% for overall errors and 42% for ADEs. At each step records
for 112 patient admissions will be reviewed, totalling 1,232 across the study.
MAEs: Based on our previous studies(9, 10) we expect the overall MAE rate per administration to
fall by 27%, from 0.37 (SD=0.65) to 0.27 (SD=0.52) with an estimated ICC of 0.03 (Table 1). The
required number of wards (two-sided test; 80% power; �<5%) is seven, with 10 steps after baseline.
Page 12 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 12
For ADEs, the required number of wards is six. We will collect data using all eight wards which
will allow detection of a minimum 20% change overall and 48% for ADEs. At each step we will
observe at least 240 medication administrations, totalling 2,640 across the study.
LOS: There are very limited data on the impact of eMM systems on LOS at ward level. A study in
an ICU showed a 23% reduction in LOS post eMM system.(33) To detect a 23% reduction in LOS,
i.e. from 3.78 (SD=7.39) to 2.92 days, with eight wards, will require additional (routinely collected
LOS) data in a total of 31 steps for a two-sided test with 80% power �<5%.
Data analyses: Medication error rates per order, stratified by error type, study step, and ward will
be calculated. For each outcome of interest, data collected across all measurement periods and all
study steps will be used in the analyses comparing intervention status (pre versus post eMM).
Analyses will apply the intention-to-treat principle. Patient data will be analysed according to the
status of the wards (i.e. pre or post eMM) where patients were admitted. Outcomes will be assessed
at the patient-level using mixed models, taking into account correlation of patient admissions within
wards (clusters) and multiple admissions for the same patient, with adjustment for potential
confounding factors. For the MAE analyses we will adjust for contextual factors including
interruptions, multi-tasking, nurse age, gender, and adherence to policies. For LOS analysis we will
adjust for patient characteristics, such as major diagnoses, comorbidity, age, and gender. The mixed
models will incorporate fixed terms for ward intervention status, measurement time steps (including
baseline) and other confounders. The analyses will include multiple time points pre and post eMM
implementation. The study design will allow us to determine temporal changes in system
effectiveness, e.g. to determine if error rates continue to decline over time. We will apply the
‘system-related’ error classification(23) to identify system-related error rate and associated
mechanisms.
Page 13 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 13
No. = Number
Table 1 Prescribing error and medication administration error power calculations
Error Rate
Pre-
(SD)
Error Rate
Post-
(SD)
% changes
from past
eMM study
ICC (7) Mean No. of
admissions or
administrations/
study step
No. of
steps (k)
Min. No.
wards
required
No. of
wards
Min. % change
detectable
Max.
power
Prescribing
errors
(per admission)
4.1 (5.3) 1.6 (2.9) 60% 0.06 14 10 1 8 20% 100%
ADEs
(per admission)
0.3 (0.7) 0.1 (0.4) 44% 0.005 14 10 7 8 42% 83%
MAEs
(per
administration)
0.4 (0.6) 0.3 (0.5) 27% 0.03 30 10 4 8 20% 97%
ADEs (% all
medication
administrations)
4.2% 1.8% 57% 0.003 30 10 6 8 48% 93%
Page 14 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on December 26, 2020 by guest. Protected by copyright. http://bmjopen.bmj.com/ BMJ Open: first published as 10.1136/bmjopen-2016-011811 on 21 October 2016. Downloaded from
For peer review only
Page 14
Aim 2: To assess the effects of the eMM on workflow and efficiency
Study design and sample: Observations and interviews will be held with medical, nursing, and
pharmacy staff at baseline to allow mapping of core work processes associated with medication
provision. At each step in the stepped-wedge design, a small number of interviews (approximately
four on each ward) will be conducted with nursing and medical staff to gain insights into clinical
staff perceptions of the impact of the system on workflow, efficiency and care delivery. These
interviews will be held one week, three weeks, six weeks and six months post eMM implementation
on each study ward. Members of the research team will directly approach hospital staff and invite
them to participate in interviews which will take approximately 10 minutes each. We anticipate
approximately 80-90 interviews will be conducted across the course of the study. Trustworthiness
of the qualitative data will be achieved through triangulation of data and investigators, engagement
with the field with a documented audit trail and member checking.(34)
The eMM is anticipated to have a significant impact on the work of hospital pharmacists. We will
conduct a direct observational study of approximately eight pharmacists at site 1. We will observe
them for 200 hours between 7:30 – 18:00 pre and post eMM system implementation to examine
changes in i) task time distributions ii) location of work and iii) communication patterns. Using the
validated WOMBAT approach(35, 36) multiple dimensions of work will be captured (e.g. tasks
performed, with whom, with what, location, interruptions, and multi-tasking). On data entry, tasks
are automatically time stamped when entered in the WOMBAT data collection tool. Figure 5 is an
example of data collection within WOMBAT. An additional sample of 140 hours of observation
will be conducted to capture the work of oncology pharmacists whose work involves supporting the
delivery of complex drug regimens to children with cancer.
<Insert Figure 5 here>
Data generated will allow changes in task time distributions and sequencing of work to be
determined. These data will be examined in relation to changes in outcome indicators generated
(from Aim 1, e.g. medication error rates, LOS) on the same wards.
Aim 3: Assess the extent to which feedback (from Aims 1 and 2) and subsequent modifications to
an eMM system design can improve eMM system effectiveness in reducing medication errors
Evaluations of health IT serve multiple purposes, ranging from providing an objective assessment
Page 15 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 15
of the success of the new technology in delivering anticipated benefits, to identification of deficits
in the system, their source, and the ways they can be addressed. This is critical to improving system
effectiveness, relevance, and responsiveness. For Aim 3, the findings of the SWCRCT at CHW (i.e.
stage 1) will be reported to the Project Evaluation Committee (PEC) made up of members of the
research team, and the Hospitals’ eMM system Project Steering Committee. The PEC will meet
every month to consider the implications of study findings across a number of domains including
the system’s technical features (e.g. compatibility with other hospital systems), effectiveness (e.g.
error reduction and system-related errors); professional attitudes (e.g. satisfaction) and
organisational features (e.g. work processes), as a means of formulating changes to eMM system
design features and user training.
This will form the key component of an action-oriented approach aimed at optimising system
performance leading to an enhanced eMM system which will then be implemented across the
second site, SCH (stage 2). The SWCRCT design will be repeated at site 2 using the same methods
as above. Power calculations for stage 2 will be based on results from stage 1. We will conduct
separate analyses for all outcomes specified. Results for the two sites will be compared, using
multilevel and longitudinal analysis approaches to determine changes in error rates (taking baseline
data into account).
Expected outcomes and significance of the research project
This project will generate the first Australian data, in a paediatric setting, on the effectiveness of
eMM systems to reduce medication errors and ADEs, and provide an assessment of how systems
impact on the work of clinicians and the consequences for the delivery of care to children.
Importantly, the findings will be directly applied to enhance the eMM system design, and work
processes and then tested further through evaluation of the enhanced eMM system at a second
paediatric hospital. These results will be particularly valuable for other paediatric hospitals yet to
commence implementations. Exploiting the SWCRCT design within an action-research model is
highly innovative, and will deliver high quality data on system effectiveness. Such a model of
formally integrating health IT assessment results as a basis for active engagement with IT vendors
and clinicians to bring about system change has both national and international significance. The
study advances explicit methods for the systematic identification of harm associated with
medication errors. The data generated will also provide the basis for a robust cost-effectiveness
analysis, which will be the subject of a separate protocol.
Page 16 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 16
ETHICS AND DISSEMINATION
The research has been approved by the Human Research Ethics Committee of the Sydney
Children’s Hospitals Network (HREC/15/SCHN/370). In the first instance, results from site 1 will
be reported to the PEC so that they can be used to inform eMM system and work process design
prior to implementation at site 2. Results will also be reported through academic journals and
conference presentations. The project is funded through a National Health and Medical Research
Council Partnership Grant. As such, the project team includes academic researchers, hospital
clinicians and experts involved in the implementation of the eMM system at the two hospital sites,
along with senior policy makers from agencies within the State Health Department involved in
eHealth system strategy and policy. This provides the project with access to a range of other
conduits through which to disseminate results to, for example, policy makers and system
implementers.
Page 17 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 17
Authors Contributions: JW, LL, MB, AGe and CM initiated the project and led the development
of the NHMRC grant proposal. JW, LL, MB, AGe, RD, JK, LDP, CTC, GA, and TOB are chief
investigators on the project and all made contributions to the protocol in their specific areas of
expertise. CM, LW, NB, AGa, CUL, MG, PK, MK, AB and DB are associate investigators on the
NHMRC grant and provided input to the protocol, particularly in the areas of paediatric clinical
practice and broader eHealth strategy in relation to eMM systems. MZR, MP, VM, TK, RW, RL are
members of the project team and have made significant contributions to the protocol in terms of the
design of details regarding the collection and classification of medication errors and harm. JW
prepared the first draft of this manuscript based upon the grant proposal and all authors have
reviewed and provided input.
Funding Statement: The project is supported by a National Health and Medical Research Council
Partnership Grant (APP1094878) in partnership with: Sydney Children’s Hospitals Network;
eHealth New South Wales; Office of Kids and Families, New South Wales.
Competing Interests Statement: None to declare.
Page 18 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 18
References:
1. Kaushal R, Barker KN, Bates DW. How can information technology improve patient safety
and reduce medication errors in children's health care? Arch Pediat Adol Med.
2001;155(9):1002-7.
2. Doherty C, McDonnell C. Tenfold medication errors: 5 years’ experience at a university-
affiliated pediatric hospital. Pediatrics. 2012;129(5):916-24.
3. Gazarian M, Drew A, Bennett A. Medicinal mishap. Intravenous paracetamol in
paediatrics: cause for caution. Aust Prescr. 2014;37(1):24-5.
4. Kim GR, Chen AR, Arceci RJ, Mitchell SH, Kokoszka KM, Daniel D, et al. Error reduction
in pediatric chemotherapy: computerized order entry and failure modes and effects analysis.
Arch Pediat Adol Med. 2006;160(5):495.
5. Kaushal R, Bates DW, Landrigan C, McKenna KJ, Clapp MD, Federico F, et al.
Medication errors and adverse drug events in pediatric inpatients. J Amer Med Assoc.
2001;285(16):2114-20.
6. Miller MR, Robinson KA, Lubomski LH, Rinke ML, Pronovost PJ. Medication errors in
paediatric care: a systematic review of epidemiology and an evaluation of evidence
supporting reduction strategy recommendations. Qual Saf Health Care. 2007;16(2):116-26.
7. Rinke ML, Bundy DG, Velasquez CA, Rao S, Zerhouni Y, Lobner K, et al. Interventions to
reduce pediatric medication errors: a systematic review. Pediatrics. 2014.
8. Gazarian M, Graudins LV. Long-term reduction in adverse drug events: an evidence-based
improvement model. Pediatrics. 2012;129(5):e1334-e42.
9. Westbrook JI, Rob MI, Woods A, Parry D. Errors in the administration of intravenous
medications in hospital and the role of correct procedures and nurse experience. BMJ Qual
Saf. 2011:doi: 10.1136/bmjqs-2011-000089.
10. Westbrook JI, Woods A, Rob MI, Dunsmuir WTM, Day RO. Association of interruptions
with an increased risk and severity of medication administration errors. Arch Intern Med.
2010;170(8):683-90.
11. Manius E, Kinney S, Cranswick N, Williams A, Borrot N. Interventions to reduce
medication errors in pediatric intensive care. Ann Pharmacother. 2014;48(10):1313-31.
12. Australian Commission on Safety and Quality in Health Care. Electronic medication
management (EMM): a guide for healthcare providers 2015.
13. van Rosse F, Maat B, Rademaker CMA, van Vught AJ, Egberts ACG, Bollen CW. The
effect of computerized physician order entry on medication prescription errors and clinical
outcome in pediatric and intensive care: a systematic review. Pediatrics. 2009;123(4):1184-
90.
14. Westbrook JI, Li L, Lehnbom EC, M Baysari, Braithwaite J, Burke R, et al. What are
incident reports telling us? A comparative study at two Australian hospitals of medication
errors identified at audit, detected by staff and reported to an incident system. Int J Qual
Health Care. 2015;27(1):1-9.
15. King WJ, Paice N, Rangrej J, Forestell GJ, Swartz R. The effect of computerized physician
order entry on medication errors and adverse drug events in pediatric inpatients. Pediatrics.
2003;112(3 Pt 1):506-9.
Page 19 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 19
16. Lehmann CU, Johnson KB, Technology Council on Clinical Information, American
Academy of Pediatrics. Electronic prescribing in pediatrics: toward safer and more effective
medication management. Pediatrics. 2013;131(4):e1350-e6.
17. Dufendach KR, Eichenberger JA, McPheeters ML, al. et. Core functionality in pediatric
electronic health records: Agency for Healthcare Research and Quality; 2015.
18. Slight SP, Berner ES, Galanter W, Huff S, Lambert BL, Lannon C, et al. Meaningful use of
electronic health records: experiences from the field and future opportunities. JMIR Med
Inform. 2015;3(3):e30.
19. Han YY, Carcillo JA, Venkataraman ST, Clark RS, Watson RS, Nguyen TC, et al.
Unexpected increased mortality after implementation of a commercially sold computerized
physician order entry system. Pediatrics. 2005;116:1506 - 12.
20. Longhurst CA, Parast L, Sandborg CI, Widen E, Sullivan J, Hahn JS, et al. Decrease in
hospital-wide mortality rate after implementation of a commercially sold computerized
physician order entry system. Pediatrics. 2010;126(1):14-21.
21. Brigham and Women's Hospital, Partners HealthCare, Harvard Medical School.
Computerized Prescriber Order Entry Medication Safety (CPOEMS): Uncovering and
learning from issues and errors. 2015.
22. Westbrook JI, Reckmann M, Li L, Runciman WB, Burke R, Lo C, et al. Effects of two
commercial electronic prescribing systems on prescribing error rates in hospital inpatients:
a before and after study. PLoS Med. 2012;9(1):e1001164. doi:10.1371/journal.pmed.
23. Westbrook JI, Baysari MT, Li L, Burke R, Richardson KL, Day RO. The safety of
electronic prescribing: manifestations, mechanisms, and rates of system-related errors
associated with two commercial systems in hospitals. J American Med Inform Assn.
2013;20(6):1159-67.
24. Koppel R, Lehmann CU. Implications of an emerging EHR monoculture for hospitals and
healthcare systems. J Amer Med Inform Assoc. 2015;22(2):465-71.
25. McKibbon KA, Lokker C, Handler SM, Dolovich LR, Holbrook AM, O'Reilly D, et al.
Enabling medication management through health information technology. Rockville MD:
Agency for Healthcare Research and Quality, 2011.
26. Brown C, Lilford R. The stepped wedge trial design: a systematic review. BMC Med Res
Methodol 2006;6(1):54.
27. Westbrook JI, Woods A. Development and testing of an observational method for detecting
medication administration errors using information technology. St Heal T. 2009;146:429-
33.
28. Dean B, Barber N. Validity and reliability of observational methods for studying
medication administration errors. Am J Health Syst Pharm. 2001;58(1):54-9.
29. Westbrook JI, Li L, Georgiou A, Paoloni R, Cullen J. Impact of an electronic medication
management system on hospital doctors’ and nurses’ work: a controlled pre–post, time and
motion study. J Am Med Inform Assn. 2013;20(6):1150-8.
30. National Coordinating Council for Medication Error Reporting and Prevention. NCC
MERP Index for categorizing medication errors 2001 [cited 2015]. Available from:
http://www.nccmerp.org/types-medication-errors.
31. Government NSW, Department Health. Incident Management Policy. 2014.
32. Hussey MA, Hughes JPs. Design and analysis of stepped wedge cluster randomized trials.
Contemp Clin Trials. 2007;28(2):182-91.
Page 20 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 20
33. Lellouche F, Mancebo J, Jolliet P, Roeseler J, Schortgen F, Dojat M, et al. A multicenter
randomized trial of computer-driven protocolized weaning from mechanical ventilation. Am
J Resp Crit Care. 2006;174(8):894-900.
34. Callen J, Paoloni R, Li J, Stewart M, Gibson K, Georgiou A, et al. Perceptions of the effect
of information and communication technology on the quality of care delivered in
emergency departments: a cross-site qualitative study. Ann Emerg Med. 2013;61(2):131-44.
35. Ballerman MA, Shaw NT, Mayes DC, Gibney RTN, Westbrook JI. Validation of the Work
Observational Method By Activity Timing (WOMBAT) method of conducting time-motion
observations in critical care settings: an observational study BMC Med Inform Decis.
2011;11:doi:10.1186/472-6947-11-32.
36. Westbrook JI, Ampt A. Design, application and testing of the Work Observation Method
by Activity Timing (WOMBAT) to measure clinicians’ patterns of work and
communication. Int J Med Inform. 2009;78S:S25-S33.
Page 21 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Page 21
FIGURES
Figure 1 Schematic of stepped-wedge cluster randomised controlled trial study design
Figure 2 Medication error, ADE and harm identification and classification process
Figure 3 POSSUM tool for data collection during the direct observational study of medication
administration
Figure 4 Example of harm assessment guide for paediatric opioid errors, to be used during medical
record review following identification of an opioid prescribing error
Figure 5 Work Observation Method By Activity Timing (WOMBAT) for conducting observational
studies of health professionals work pre and post eMM system implementation
Page 22 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Figure 1: Schematic of stepped-wedge cluster randomised controlled trial study design
124x30mm (300 x 300 DPI)
Page 23 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Figure 2: Medication error ADE and harm identification and classification process
163x97mm (300 x 300 DPI)
Page 24 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Figure 3: POSSUM tool for data collection during the direct observational study of medication administration
270x433mm (300 x 300 DPI)
Page 25 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Figure 4: Example of harm assessment guide
184x173mm (300 x 300 DPI)
Page 26 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Figure 5: WOMBAT tool for conducting observational studies of health professionals work pre and post eMM system implementation
318x527mm (300 x 300 DPI)
Page 27 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Example of a medication administration screen
Page 28 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Example of a medication order screen in the eMM system
Page 29 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
The SPIRIT Checklist
Item 1: Descriptive title identifying the study design, population, interventions, and, if
applicable, trial acronym.
“A stepped-wedge cluster randomised controlled trial to assess the effectiveness of an
electronic medication management system to reduce medication errors, adverse drug
events, and average length of stay at two paediatric hospitals.”
Can be found on title page, and page two of submitted protocol document.
Item 2a: Trial identifier and registry name. If not yet registered, name of intended registry.
Australian New Zealand Clinical Trials Registry (ANZCTR)
Item 2b: All items from the World Health Organization Trial Registration Data Set.
Nil.
Item 3: Date and version identifier.
7 March 2016 – Protocol Manuscript - Version One
Item 4: Sources and types of financial, material, and other support.
This project is funded by a National Health and Medical Research Council Partnership Grant
(APP1094878) in partnership with the Sydney Children’s Hospitals Network, eHeath New
South Wales, and the Office of Children and Families, New South Wales. [Page 16]
Item 5a: Names, affiliations, and roles of protocol contributors.
As listed on page 2 and 16
Item 5b: Name and contact information for the trial sponsor.
The principle investigator and sponsor of this study is Professor Johanna Westbrook, of the
Centre for Health Systems and Safety Research, Australian Institute of Health Innovation,
Macquarie University, Sydney, NSW.
E-mail: johanna.westbrook@mq.edu.au
Phone: +61 02 9850 2402
Item 5c: Role of study sponsor and funders, if any, in study design; collection,
management, analysis, and interpretation of data; writing of the report; and the decision
to submit the report for publication, including whether they will have ultimate authority
over any of these activities.
The trial has been principally designed by Professor Westbrook, and the research team at
the Macquarie University. The research team have full independence from the funding
bodies in terms of decisions regarding the interpretation and publication of findings.
Item 5d: Composition, roles, and responsibilities of the coordinating centre, steering
committee, endpoint adjudication committee, data management team, and other
individuals or groups overseeing the trial, if applicable (see Item 21a for Data Monitoring
Committee).
Page 30 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
The research team at CHSSR, headed by Professor Westbrook will have responsibility for
designing and managing the study, and coordinating the collection, management, analysis
and interpretation of data.
The steering committee will be convened at least every three months, although more
frequently at pertinent times during the study, and will be engaged to provide their
expertise and advice on data collection and study progress. The research at CHSSR will have
responsibility for data management and analysis.
Item 6a: Description of research question and justification for undertaking the trial,
including summary of relevant studies (published and unpublished) examining benefits
and harms for each intervention.
Introduction: Prescribing, administering, and monitoring medicines in children is highly
complex. Compared with adults, medication errors in children are three times more likely to
result in harm. Electronic medication management (eMM) systems are expected to reduce
medication errors and ADEs significantly. However, rigorous evidence demonstrating these
effects is limited. [Page 5]
Existing Knowledge: A systematic review identified eight studies of eMM effectiveness
among paediatric patients. Meta-analysis showed a significant reduction in prescribing error
risk (RR 0.08) but not in ADEs or mortality. There are no Australian studies of eMM system
use in a paediatric setting. Previous studies have often relied upon incident reports to
measure error rates, which are generally unreliable. Only one study of paediatric inpatients
has used a control group to assess eMM effectiveness. Internationally there is currently
insufficient evidence to demonstrate clinical benefit from eMM in paediatric patients.
In 2005, Han et al. reported a significant increase in the mortality rate among critical care
children at a US paediatric hospital following introduction of a commercial eMM system.
The rapid implementation process and limited attention to the significant workflow re-
design required were considered major factors in this outcome. Subsequent studies have
shown no increase in mortality rates. However, the Han et al. study caused considerable
alarm and served to demonstrate the substantial dangers of poor implementation and the
importance of monitoring outcomes following system implementation and responding to
problems identified.
eMM use in adult hospitals, while highly effective at reducing medication errors, also
introduced new ‘system-related’ errors. An investigation of 1,164 prescribing errors post
eMM in two adult Australian hospitals found 42.4% were facilitated by the system (78 per
100 admissions). [Pages 5-6]
See further justification pages 5-6
Item 6b: Explanation for choice of comparators.
This is an pre-, post-, evaluation study, assessing changes between medication
administration errors, adverse drug events (ADEs) and effects on patient length of stay, pre
and post eMM roll-out. See pages 7-8 for further details
Item 7: Specific objectives or hypotheses.
Aim 1 - To quantify the safety and effectiveness of an eMM system to reduce medication
errors, ADEs and average LOS among paediatric patients using a stepped-wedge cluster
randomised controlled trial (SWCRCT) design;
Page 31 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Aim 2 - To assess the effects of the eMM on clinicians’ workflow and efficiency; and
Aim 3 - To assess the extent to which feedback of study results and subsequent
modifications to the eMM design and associated work practices can improve eMM
effectiveness in reducing medication errors at a subsequent implementation tested
via a second SWCRCT. [Page 7]
Item 8: Description of trial design including type of trial (e.g., parallel group, crossover,
factorial, single group), allocation ratio, and framework (e.g., superiority, equivalence,
non-inferiority, exploratory).
The study is a stepped wedge cluster randomised controlled trial (SWCRCT) evaluating
differences pre- and post- eMM implementation.
Item 9: Description of study settings (e.g., community clinic, academic hospital) and list of
countries where data will be collected. Reference to where list of study sites can be
obtained.
The study will be conducted in Sydney, New South Wales and will occur at two large tertiary
pediatric hospitals from within the Sydney Children’s Hospitals Network. [Pages 7-8]
Item 10: Inclusion and exclusion criteria for participants. If applicable, eligibility criteria for
study centres and individuals who will perform the interventions (e.g., surgeons,
psychotherapists).
All patients on the study wards will be participants. [Pages 9-10]
Item 11a: Interventions for each group with sufficient detail to allow replication, including
how and when they will be administered.
Detailed in methods section.
Item 11b: Criteria for discontinuing or modifying allocated interventions for a given trial
participant (e.g., drug dose change in response to harms, participant request, or
improving/worsening disease).
See methods section
Item 11c: Strategies to improve adherence to intervention protocols, and any procedures
for monitoring adherence (e.g., drug tablet return; laboratory tests).
The intervention – the eMM will be mandatory for use in the study hospitals.
Item 11d: Relevant concomitant care and interventions that are permitted or prohibited
during the trial.
N/A
Item 12: Primary, secondary, and other outcomes, including the specific measurement
variable (e.g., systolic blood pressure), analysis metric (e.g., change from baseline, final
value, time to event), method of aggregation (e.g., median, proportion), and time point
for each outcome. Explanation of the clinical relevance of chosen efficacy and harm
outcomes is strongly recommended.
Outcome: Medication administration errors (MAEs), ADEs and LOS
See methods section
Page 32 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Item 13: Time schedule of enrolment, interventions (including any run-ins and washouts),
assessments, and visits for participants. A schematic diagram is highly recommended (see
Figure)
For Aim 1, the data collection at CHW will commence on 25th April, 2016, one week before
the eMM rollout will commence. Direct observations of medication administration will
continue until 10th
July, two weeks after the roll out is complete, while chart reviews will
continue into 2017, until they are completed.
Figure 1: Timeline for the SWCRCT, with the baseline commencing at 25th April
For Aim 2, data will be collected during the same schedule as the implementation.
Item 14: Estimated number of participants needed to achieve study objectives and how it
was determined, including clinical and statistical assumptions supporting any sample size
calculations
See methods section
Item 15: Strategies for achieving adequate participant enrolment to reach target sample
size
See methods section
Item 16a: Method of generating the allocation sequence (eg, computer-generated random
numbers), and list of any factors for stratification. To reduce predictability of a random
sequence, details of any planned restriction (eg, blocking) should be provided in a
separate document that is unavailable to those who enrol participants or assign
interventions
Randomisation of the 8 wards will be conducted by a person blinded to the ward identity.
[Page 8]
Item 16b: Mechanism of implementing the allocation sequence (eg, central telephone;
sequentially numbered, opaque, sealed envelopes), describing any steps to conceal the
sequence until interventions are assigned
Page 33 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Randomisation will not be concealed. The study is a stepped wedge cluster randomised
controlled trial.
Item 16c: Who will generate the allocation sequence, who will enrol participants, and who
will assign participants to interventions?
The research team will determine the allocation sequence and communicate with the
hospital team who are responsible for the implementation of the eMM.
Item 17a: Who will be blinded after assignment to interventions (eg, trial participants,
care providers, outcome assessors, data analysts), and how
No one will be blinded to the assignment of the intervention.
Item 17b: If blinded, circumstances under which unblinding is permissible, and procedure
for revealing a participant’s allocated intervention during the trial
Methods: Data collection, management, and analysis
Item 18a: Plans for assessment and collection of outcome, baseline, and other trial data,
including any related processes to promote data quality (eg, duplicate measurements,
training of assessors) and a description of study instruments (eg, questionnaires,
laboratory tests) along with their reliability and validity, if known. Reference to where
data collection forms can be found, if not in the protocol
See methods
Item 18b: Plans to promote participant retention and complete follow-up, including list of
any outcome data to be collected for participants who discontinue or deviate from
intervention protocols
See methods. Retrospective review hospital medical records is the main source of data.
Item 19: Plans for data entry, coding, security, and storage, including any related
processes to promote data quality (eg, double data entry; range checks for data values).
Reference to where details of data management procedures can be found, if not in the
protocol
The methods section outlines the main data management processes. Information in the
individual ethics applications specified further details.
Item 20a: Statistical methods for analysing primary and secondary outcomes. Reference to
where other details of the statistical analysis plan can be found, if not in the protocol
Data analyses are specified in the methods section/
Page 34 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Item 20b: Methods for any additional analyses (eg, subgroup and adjusted analyses)
See methods section
Item 20c: Definition of analysis population relating to protocol non-adherence (eg, as
randomised analysis), and any statistical methods to handle missing data (eg, multiple
imputation)
See methods section. Analyses will be performed as per intention to treat.
Methods: Monitoring
Item 21a: Composition of data monitoring committee (DMC); summary of its role and
reporting structure; statement of whether it is independent from the sponsor and
competing interests; and reference to where further details about its charter can be
found, if not in the protocol. Alternatively, an explanation of why a DMC is not needed
A formal DMC has not been convened, however members of the research team will have
roles in monitoring the collection and quality of the data. Funding bodies will not have
access to the raw study data.
Item 21b: Description of any interim analyses and stopping guidelines, including who will
have access to these interim results and make the final decision to terminate the trial.
N/A
Item 22: Plans for collecting, assessing, reporting, and managing solicited and
spontaneously reported adverse events and other unintended effects of trial
interventions or trial conduct
The project manager (MZR) and the principle investigator (JIW), as well as the site
coordinators (CM and TOB) will monitor any adverse events which occur and wil report to
the Steering Committee.
Item 23: Frequency and procedures for auditing trial conduct, if any, and whether the
process will be independent from investigators and the sponsor
The project will be undertaken in line with specific human ethics approval requirements.
Item 24: Plans for seeking research ethics committee/institutional review board (REC/IRB)
approval
This study has been approved by the SCHN HREC (EC00130), reference code
HREC/15/SCHN/370.
Item 25: Plans for communicating important protocol modifications (eg, changes to
eligibility criteria, outcomes, analyses) to relevant parties (eg, investigators, REC/IRBs,
trial participants, trial registries, journals, regulators)
Any proposed modifications to the study will be discussed with the project steering
committee, and where required modification from the SCHN HREC will be sought.
Page 35 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Item 26a: Who will obtain informed consent or assent from potential trial participants or
authorised surrogates, and how (see Item 32)
See methods
Item 26b: Additional consent provisions for collection and use of participant data and
biological specimens in ancillary studies, if applicable
Not applicable.
Item 27: How personal information about potential and enrolled participants will be
collected, shared, and maintained in order to protect confidentiality before, during, and
after the trial
See methods
Item 28: Financial and other competing interests for principal investigators for the overall
trial and each study site
The research team declare no competing interests.
Items 29: Statement of who will have access to the final trial dataset, and disclosure of
contractual agreements that limit such access for investigators
Members of the research team involved in the data analysis and the writing up of the study
results will have access to the data set. Access will also be granted to selected members of
the steering committee and the site coordinators, where appropriate. The main funding
bodies will not have access to the raw data.
Item 30: Provisions, if any, for ancillary and post-trial care, and for compensation to those
who suffer harm from trial participation
The intervention implementation is the sole responsibility of the hospital sites and was
intended to proceed regardless of the evaluation study outlined here.
Item 31a: Plans for investigators and sponsor to communicate trial results to participants,
healthcare professionals, the public, and other relevant groups (eg, via publication,
reporting in results databases, or other data sharing arrangements), including any
publication restrictions
Study results will be communicated to the staff of the two study hospitals in meetings and
briefing sessions, to be arranged in coordination with senior staff. The results will be further
communicated with healthcare professionals and the public, through publication of journal
articles, conference presentations and other appropriate means.
Item 31b: Authorship eligibility guidelines and any intended use of professional writers
Page 36 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
For peer review only
Standard academic and ethical guidelines for authorship and publication of results will
apply.
Item 31c: Plans, if any, for granting public access to the full protocol, participant-level
dataset, and statistical code
The study protocol will be published and thus available to the public. There are no plans to
provide public access to the full dataset or statistical code at this time.
Page 37 of 36
For peer review only - http://bmjopen.bmj.com/site/about/guidelines.xhtml
BMJ Open
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960
on Decem
ber 26, 2020 by guest. Protected by copyright.
http://bmjopen.bm
j.com/
BM
J Open: first published as 10.1136/bm
jopen-2016-011811 on 21 October 2016. D
ownloaded from
Recommended