t echnolo g y r ep ort HTA Technologies to Reduce Errors ... · medication dispensing devices, bar-coding verification for medication dispensing and administration, and electronic

Supporting Informed Decisions

Technologies to Reduce Errors in Dispensing and Administration of Medication in Hospitals: Clinical and Economic Analyses

t e c h n o l o g y r e p o r t

Canadian Agency forDrugs and Technologies

in Health

Agence canadienne des médicaments et des technologies de la santé

HTAIssue 121

August 2009

Until April 2006, the Canadian Agency for Drugs and Technologies in Health (CADTH) was known as the Canadian Coordinating Office for Health Technology Assessment (CCOHTA).

Cite as: Perras C, Jacobs P, Boucher M, Murphy G, Hope J, Lefebvre P, McGill S, Morrison A. Technologies to Reduce Errors in Dispensing and Administration of Medication in Hospitals: Clinical and Economic Analyses [Technology report number 121]. Ottawa: Canadian Agency for Drugs and Technologies in Health; 2009. Production of this report is made possible by financial contributions from Health Canada and the governments of Alberta, British Columbia, Manitoba, New Brunswick, Newfoundland and Labrador, Northwest Territories, Nova Scotia, Nunavut, Prince Edward Island, Saskatchewan, and Yukon. The Canadian Agency for Drugs and Technologies in Health takes sole responsibility for the final form and content of this report. The views expressed herein do not necessarily represent the views of Health Canada, or any provincial or territorial government. Reproduction of this document for non-commercial purposes is permitted, provided appropriate credit is given to CADTH. CADTH is funded by Canadian federal, provincial, and territorial governments. Legal Deposit – 2009 National Library of Canada ISBN: 978-1-926680-12-5 (print) ISBN: 978-1-926680-13-2 (online) H0472 – August 2009 PUBLICATIONS MAIL AGREEMENT NO. 40026386 RETURN UNDELIVERABLE CANADIAN ADDRESSES TO CANADIAN AGENCY FOR DRUGS AND TECHNOLOGIES IN HEALTH 600-865 CARLING AVENUE OTTAWA ON K1S 5S8

Publications can be requested from:

CADTH 600-865 Carling Avenue

Ottawa ON Canada K1S 5S8 Tel.: 613-226-2553 Fax: 613-226-5392

Email: [email protected]

or downloaded from CADTH’s website: http://www.cadth.ca

Canadian Agency for Drugs and Technologies in Health

Technologies to Reduce Errors in Dispensing and Administration of Medication in Hospitals:

Clinical and Economic Analyses

Christine Perras, BScPhm MPH1 Philip Jacobs, PhD2

Michel Boucher, BPharm MSc3 Gaetanne Murphy, BScPharm1

John Hope, BSc BScPharm4 Patricia Lefebvre, BPharm MSc FCSHP5

Sarah McGill, BSc MLIS3 Andra Morrison, BSc aCLIP3

August 2009

1 Canadian Agency for Drugs and Technologies in Health, Edmonton, Alberta 2 University of Alberta and Institute of Health Economics Edmonton, Alberta 3 Canadian Agency for Drugs and Technologies in Health, Ottawa, Ontario 4 BC Children's and Women's Health Centre, Vancouver, British Columbia 5 McGill University Health Centre, Montreal, Quebec


i

Reviewers

These individuals kindly provided comments on this report. External Reviewers

Carlo Marra PharmD PhD Associate Professor, Faculty of Pharmaceutical Sciences University of British Columbia Vancouver, British Columbia

Bonnie Salsman, BSc Pharm FCSHP Consultant Pharmacist BMS Consultants and Institute for Safe Medication Practices Canada Halifax, Nova Scotia

Bernard M. Dickens LLB LLM PhD LLD FRSC Professor Emeritus of Health Law and Policy Faculty of Law University of Toronto Toronto, Ontario

Ceri J. Phillips BSc(Econ) MSc(Econ) PhD Professor of Health Economics Head of Institute for Health Research Swansea University Swansea, Wales, United Kingdom

Denis Bois BSc Pharm, DPH Director, Pharmacy Department Centre Hospitalier de l’Université de Montréal (CHUM) Montreal, Quebec

CADTH Peer Review Group Reviewers

Greg S. Zaric, PhD Associate Professor Ivey School of Business, University of Western Ontario London, Ontario

Timothy Caulfield, LLM, FRSC Canada Research Chair in Health Law & Policy Senior Health Scholar, Alberta Heritage Foundation for Medical Research Professor, Faculty of Law and School of Public Health Research Director, Health Law Institute Edmonton, Alberta

Industry: The following manufacturers were provided with an opportunity to comment on an earlier version of this report: AmerisourceBergen Corporation, AutoMed Canada, Baxter International Inc. (Canada), Cardinal Health Canada Inc., Cerner Corporation (Canada), Eclipsys Corporation, GE Healthcare Canada, Healthmark Ltd., Lionville Systems Inc., Manrex Limited McKesson Canada, Omnicell, PointClickCare, Rubbermaid Medical Solutions, ScriptPro (Canada), Swisslog AG. All comments that were received were considered when preparing the final report. This report is a review of existing public literature, studies, materials, and other information and documentation (collectively the “source documentation”) which are available to CADTH. The accuracy of the contents of the source documentation on which this report is based is not warranted, assured, or represented in any way by CADTH and CADTH does not assume responsibility for the quality, propriety, inaccuracies, or reasonableness of any statements, information, or conclusions contained in the source documentation.


ii

CADTH takes sole responsibility for the final form and content of this report. The statements and conclusions in this report are those of CADTH and not of its Panel members or reviewers. Authorship

Christine Perras was the project lead during the second phase of the project. She contributed to the development of the protocol and participated in study selection, quality assessment, and data extraction and analysis. She was the principal author of the introduction, the clinical methods, the clinical results, the ethics, the discussion, and the conclusion sections. She also contributed to the revision of other sections of the project. Philip Jacobs was the lead for the economic section. He contributed to the development of the protocol and participated in study selection, quality assessment, and data extraction and analysis. He was the principal author of the economic methods, the economic review, the economic analysis, the budget impact, the discussion, and the conclusion sections. Michel Boucher was the project lead during the first phase of the project. He participated in the development of the protocol, selection of the clinical studies, quality assessment and extraction, and analysis of the clinical data. He contributed to the writing of the introduction, the clinical results, and the conclusion sections. He reviewed and commented on other sections of the report. He approved the final version of the report. Gaetanne Murphy participated in the development of the protocol, selection of studies, quality assessment, and extraction and analysis of the data. She reviewed and commented on the report. She approved the final version of the report. John Hope participated in the development of the protocol and in the writing of the introduction section. He provided clinical input and assistance in building the economic model. He reviewed drafts and approved the final version of the report. Patricia Lefebvre participated in the development of the protocol, in the data analysis and interpretation of the results, and in the writing of the introduction section. She reviewed drafts and approved the final version of the report. Sarah McGill was responsible for literature search updates, additional background literature searches, and referencing support. She wrote the search method sections and appendix, reviewed drafts, and approved the final version of the report. Andra Morrison participated in the development of the protocol, drafted and performed the search strategies, and provided bibliographic support. Acknowledgements

The authors are grateful to Karen Cimon, Research Assistant with CADTH, who participated in the development of the protocol, provided technical assistance, and provided feedback on the writing of the report. She assisted in obtaining vendor information.


iii

The authors thank Sarah Ndegwa, Research Officer with CADTH, for assisting in the data extraction and quality assessment. The authors are grateful to Hussein Noorani, Lead, HTA Impact with CADTH, who provided input during the drafting of the ethics section. Conflicts of Interest

Christine Perras, Michel Boucher, Gaetanne Murphy, Sarah McGill, and Andra Morrison have no conflicts of interest to declare. Patricia Lefebvre has declared that McKesson Canada is one of the Sponsors of “Prix d’excellence CUSM-McKesson” of the Pharmacy Department, given annually to pharmacists who distinguished themselves during the year. She is ineligible to receive one of the awards. All other authors have no conflicts of interest to declare.


iv

EXECUTIVE SUMMARY

The Issue

The technologies currently used to automate the dispensing and administration of medication may decrease medication errors, improve the quality of care, and reduce the cost that is associated with adverse events due to medication errors. These technologies include automated medication dispensing devices, bar-coding verification for medication dispensing and administration, and electronic medication administration records. Objectives

This report describes an assessment of the clinical and economic impact of adopting technologies that are designed to facilitate medication dispensing and administration in hospitals by addressing the following research questions: 1) What is the clinical effectiveness of using technologies that are intended to reduce

medication errors in hospitals in preventing medication errors, potential adverse drug events, adverse drug events, morbidity, and mortality?

2) What is the cost-effectiveness of using technologies that are intended to reduce medication errors in hospitals?

3) What is the budget impact of adopting these technologies in hospitals in terms of initial capital investment, training at implementation, training required for new employees, maintenance costs, and operational costs (for example, database update, software update, hardware, and human resources)?

Methods

A search for systematic reviews, health technology assessments, and clinical studies with comparison groups was conducted. A narrative synthesis of economic evaluations was performed. A primary economic analysis was also completed. Clinical Effectiveness

The systematic review that was identified during the literature search did not meet the criteria for quality. As a result, we conducted a new systematic review. The equipment that was used in two studies on pharmacy-based automatic dispensing devices is no longer available for purchase. These studies showed a decrease in dispensing errors. Five studies were conducted on devices available in Europe. The applicability of these results to Canadian hospital pharmacies is questionable. Based on the results of three studies, carousel systems (series of revolving shelves set on rails) reduced filling or dispensing errors. Three of four studies on profiled, ward-based automatic dispensing devices were conducted using an older model of device. These studies showed a decrease in dispensing or medication errors. One study showed an increase in medication errors in a cardiac intensive care unit. In a


v

more recent study that did not specify the model of the device that was used, medication-related events were decreased. Among studies on the replacement of paper medication administration records with bar-coding, one study did not detect a difference in medication errors, one showed an increase in medication administration errors, two studies showed a decrease in medication errors, and three studies showed a decrease in medication administration errors. In one of three studies that used bar-coding for the administration of blood products, one wrong transfusion was avoided among 50 units of blood that were transfused. The simultaneous use of several technologies reduced error rates. These findings are limited because of several factors. The definitions that were used to describe the outcomes were inconsistent among studies. The errors were counted using different methods. Compelling evidence was lacking. Observational study designs were used in all the studies. Most were uncontrolled before and after studies in which the participants were not blinded to the purpose of the study. Not all studies reported the use or results of statistical tests of significance. Factors other than automation may have led to changes in work practices. All of these factors could have affected the error rates, and the risk reduction may have been overestimated. Economic Analysis

Economic Review: A systematic review of available economic studies on the automation of medication dispensing and administration in hospitals was conducted. There is evidence that nursing time is saved with the use of automatic dispensing devices. Less storage space may be needed with the use of pharmacy-based dispensing devices. The financial analyses indicated that overall, there would be savings to hospitals. In studies from the United States, savings accrue to hospitals because the use of automated systems allows for more complete billings. These savings do not apply to Canada. Most studies had limitations. There was an absence of statistical tests of significance in the studies that were not conducted by modelling. Some of the studies on workload showed mixed results. Many costs were excluded from some of the studies. None of the studies looked at the clinical significance of medication errors or the downstream costs. Economic Evaluation: An economic model was designed to explain the difference in costs when a manual drug distribution system (with medication cassettes) is compared with ward-based automated dispensing devices (with or without patient medication profiles). When the analysis was conducted for unprofiled devices, there were savings of approximately $34,000 per patient care unit annually. Each intensive care unit had additional costs of $17,000, annually. After discounting and adjusting for inflation, there were net savings of $152,000 per patient care unit over a five-year period. Each intensive care unit costs an additional $75,000. Overall, a 400-bed hospital would achieve five-year savings of $2.7 million with the use of unprofiled equipment. The savings would be $2.2 million if profiled units were acquired.


vi

Sensitivity analyses showed that these results were robust for an unprofiled system. In several sensitivity analyses, a profiled automated system was more costly than a manual system. Budget Impact

The equipment costs for each patient care unit or intensive care unit are $123,000 for an unprofiled automatic dispensing device and $138,000 for a profiled device. The planning costs are $73,800 and $82,800. The up-front costs are $196,800 and $220,800 per patient or intensive care unit for unprofiled and profiled automatic dispensing devices, respectively. For a 400-bed hospital with approximately nineteen 20-bed patient care units and two eight-bed intensive care units, there would be up-front capital costs, as follows: For an unprofiled system, the cost of capital equipment would be $2.5 million, and planning

costs would be $1.5 million, for a total of approximately $4 million. For a profiled system, the cost of capital equipment would be $2.9 million, and planning

costs would be $1.7 million, for a total initial outlay of $4.6 million. There is some outstanding uncertainty regarding budget impact as these results are sensitive to underlying assumptions regarding equipment costs. Actual budget impact may change if more precise data are obtained. Conclusions

From a clinical perspective, based on studies of lower internal validity, the use of bar-coding for medication dispensing systems, bar-coding for medication administration systems, and the simultaneous use of technologies reduced the risk of dispensing or medication errors in hospitals. Studies of previous models of profiled, ward-based automatic dispensing devices also reported benefits. One study showed an increase in error rate in a cardiac intensive care unit. We cannot reliably estimate the magnitude of benefit from pharmacy-based automatic dispensing devices because the studies were conducted using equipment that is no longer available for purchase or the studies used devices available in Europe. We cannot reliably estimate how automation affects the rate of potential adverse drug events, adverse drug events, morbidity, and mortality because these outcomes were not measured in most studies. The implementation of a ward-based automatic dispensing device in a hospital can reduce costs while reducing error rates. This conclusion is only valid for medical-surgical patient care units. The implementation of ward-based automatic dispensing devices in the intensive care unit results in a net increase in costs. This is due to the large capital expenditures that are incurred for a small number of patients. There is also uncertainty about the clinical impact of this type of automation in intensive care. The results are more robust for unprofiled rather than profiled systems. We cannot reliably estimate the economic impact of other technologies because of gaps in knowledge.


vii


viii

TABLE OF CONTENTS

EXECUTIVE SUMMARY ............................................................................................................. iv ACRONYMS AND ABBREVIATIONS .........................................................................................x 1 INTRODUCTION...................................................................................................................1

1.1 Background and Setting in Canada...............................................................................1 1.1.1 Adverse events, medication errors, and adverse drug events......................1 1.1.2 Medication distribution cycle in hospital pharmacies....................................5

1.2 Overview of Technology................................................................................................7 1.2.1 Description of technologies ..........................................................................7 1.2.2 Regulatory status........................................................................................10 1.2.3 Unit cost......................................................................................................10 1.2.4 Utilization pattern........................................................................................10

2 ISSUE .................................................................................................................................10 3 OBJECTIVES .....................................................................................................................11 4 CLINICAL REVIEW ............................................................................................................11

4.1 Methods.......................................................................................................................11 4.1.1 Literature searches.....................................................................................11 4.1.2 Selection criteria .........................................................................................12 4.1.3 Selection method........................................................................................13 4.1.4 Data extraction strategy..............................................................................15 4.1.5 Strategy for validity assessment.................................................................15 4.1.6 Data analysis methods ...............................................................................15

4.2 Results ........................................................................................................................16 4.2.1 Quantity of research available ....................................................................16 4.2.2 Study characteristics ..................................................................................16 4.2.3 Data analyses and synthesis......................................................................17

5 ECONOMIC ANALYSIS .....................................................................................................23

5.1 Review of Economic Studies: Methods .......................................................................23 5.1.1 Literature searches.....................................................................................23 5.1.2 Selection criteria .........................................................................................24 5.1.3 Selection method........................................................................................24 5.1.4 Data extraction strategy..............................................................................24 5.1.5 Strategy for quality assessment .................................................................24 5.1.6 Data analysis methods ...............................................................................25

5.2 Review of Economic Studies: Results.........................................................................27 5.2.1 Studies identified ........................................................................................27 5.2.2 Quality of studies ........................................................................................27 5.2.3 Individual results.........................................................................................28

5.3 Primary Economic Evaluation: Methods......................................................................30 5.3.1 Types of economic evaluation ....................................................................30 5.3.2 Target population........................................................................................30 5.3.3 Comparators...............................................................................................30 5.3.4 Perspective.................................................................................................31


ix

5.3.5 Effectiveness ..............................................................................................31 5.3.6 Time horizon...............................................................................................31 5.3.7 Modelling ....................................................................................................31 5.3.8 Valuing outcomes .......................................................................................31 5.3.9 Resource use and costs .............................................................................31 5.3.10 Discount rate ..............................................................................................33 5.3.11 Sensitivity analysis .....................................................................................33

5.4 Primary Economic Evaluation: Results .......................................................................34 5.4.1 Analysis and results....................................................................................34 5.4.2 Results of the sensitivity analysis ...............................................................35

6 HEALTH SERVICES IMPACT............................................................................................35

6.1 Budget Impact .............................................................................................................35 6.1.1 Up-front costs .............................................................................................35 6.1.2 Annualized analysis....................................................................................36

6.2 Planning and Implementation Considerations.............................................................36 6.2.1 Pharmacy staff............................................................................................36 6.2.2 Nursing staff ...............................................................................................37

6.3 Ethical Considerations.................................................................................................38 6.3.1 Efficiency versus equity ..............................................................................38 6.3.2 Process or procedural issues .....................................................................39

6.4 Psychosocial Considerations From the Patient Perspective .......................................39 7 DISCUSSION......................................................................................................................40

7.1 Summary of Results ....................................................................................................40 7.1.1 Clinical review.............................................................................................40 7.1.2 Economic review ........................................................................................42 7.1.3 Economic evaluation ..................................................................................43

7.2 Strengths and Weaknesses of This Assessment ........................................................44 7.2.1 Clinical review.............................................................................................44 7.2.2 Economic review ........................................................................................46 7.2.3 Economic evaluation ..................................................................................46

7.3 Generalizability of Findings .........................................................................................47 7.3.1 Clinical review.............................................................................................47 7.3.2 Economic review ........................................................................................48 7.3.3 Economic model .........................................................................................48

7.4 Knowledge Gaps.........................................................................................................48 7.4.1 Clinical review.............................................................................................48 7.4.2 Economic review ........................................................................................48

8 CONCLUSIONS..................................................................................................................49 9 REFERENCES....................................................................................................................49 APPENDIX 1: Clinical search strategy 2003 – 2008 and Economic search strategy 1990 – 2008 APPENDIX 2: Vendors and Distributors Contacted APPENDIX 3: Clinical Data Extraction Form APPENDIX 4: Forms for Quality Assessment APPENDIX 5: Clinical Tables APPENDIX 6: Economic Tables


x

ACRONYMS AND ABBREVIATIONS

ADD automatic dispensing device

AE adverse event

ADE adverse drug event

ADR adverse drug reaction

BCMA bar code medication administration

BCMD bar code medication dispensing

CADTH Canadian Agency for Drugs and Technologies in Health

CDSS clinical decision support systems

CI confidence interval

CPOE computerized prescriber order entry

eMAR electronic medication administration record

HTA health technology assessment

ICU intensive care unit

IOM Institute of Medicine

MAR medication administration record

ME medication error

RR relative risk

RRI relative risk increase

RRR relative risk reduction


xi


1

1 INTRODUCTION

1.1 Background and Setting in Canada

Patient safety, which has been a component of quality health care in Canada, has become a key issue in this country. In 2002, two Canadian experts in patient safety submitted the report Patient Safety and Healthcare Error in the Canadian Healthcare System to the government.1 The report was based on a literature review and on a survey of Canadians working on patient safety. It recommended that health care organizations be encouraged to focus on errors, adverse events (AE), and near misses so as to support system change. The report also recommended making safety research a priority.1 In 2002, the National Steering Committee on Patient Safety issued Building a Safer System, which proposed a national integrated strategy for improving patient safety in the Canadian health care system.2 It recommended the establishment of the Canadian Patient Safety Institute, which was inaugurated in 2003. Health professionals, health care organizations, regulatory bodies, and governments work together, through the Institute, to foster a safer Canadian health care system.2 One of the Institute’s initiatives is a campaign called “Safer Healthcare Now!”, which was launched in 2005. Its goal is to improve patient safety by building a network of health care professionals who are implementing evidence-based interventions that reduce the risk of AEs. For example, one intervention that is targeted by the campaign is the implementation of medication reconciliation to prevent adverse drug events (ADE).2,3 Other patient safety strategies that have been reported in the literature and proven to prevent medical errors include reducing the work hours of medical residents and interns to avoid fatigue, including pharmacists on hospital rounds, and limiting the performance of high-risk medical procedures to hospitals that perform them frequently.4 Technologies that are intended to reduce medication dispensing and administration errors are one strategy in the continuum of patient safety strategies. These technologies have not been evaluated in a systematic review. As a result, the Canadian Agency for Drugs and Technologies in Health (CADTH) undertook a review of the benefits of, together with an economic analysis of, the technologies that are used in medication dispensing and administration. 1.1.1 Adverse events, medication errors, and adverse drug events

An AE is an unintended injury that is caused by medical care or management. An ADE is an AE that involves the use of medication.5-7 AEs and ADEs may range in severity from minor to fatal.8 A medication error (ME) is an error that occurs in the medication-use process5-7 and may result in an ADE.9,10 The definitions of ME, AE, ADE, near miss, and adverse drug reaction (ADR) from the Canadian Patient Safety Dictionary, the US Agency for Healthcare Research and Quality, and the Institute of Medicine (IOM) appear in Table 1.


2

Table 1: Definitions

The Canadian Patient Safety Dictionary6

Medication error: the failure to complete a planned action as it was intended, or when an incorrect plan is used, at any point in the process of providing medications to patients. Adverse event: i) an unexpected and undesired incident directly associated with the care or services provided to the patient; ii) an incident that occurs during the process of providing health care and results in patient injury or death; iii) an adverse outcome for a patient, including an injury or complication The US Agency for Healthcare Research and Quality Patient Safety Network Glossary7

Adverse event: an injury caused by medical care. Adverse drug event: an adverse event involving medication use. Potential adverse drug event: a medication error or other drug-related mishap that reached the patient but happened not to produce harm (e.g., the penicillin-allergic patient receives penicillin but happens not to have an adverse reaction). It can also refer to errors or other problems that, if not intercepted, would be expected to cause harm. Near miss: an event or situation that did not produce patient injury, but only because of chance. This good fortune might reflect the robustness of the patient (e.g., a patient with penicillin allergy receives penicillin, but has no reaction) or a fortuitous, timely intervention (e.g., a nurse happens to realize that a physician wrote an order in the wrong chart). This definition is identical to that for close call. Adverse drug reaction: an adverse effect produced by the use of a medication in the recommended manner. These effects range from “nuisance effects” (e.g., dry mouth with anticholinergic medications) to severe reactions, such as anaphylaxis to penicillin. The Institute of Medicine Key Definitions5

Error: the failure of a planned action to be completed as intended (error of execution) or the use of a wrong plan to achieve an aim (error in planning). An error may be an act of commission or an act of omission. Medication error: an error occurring in the medication-use process. Examples include wrong dosage prescribed, wrong dosage administered for a prescribed medication, or a failure to give (by the provider) or take (by the patient) a medication. Adverse drug event: any injury due to medication. Examples include a wrong dosage leading to injury (e.g., rash, confusion, or loss of function) or an allergic reaction occurring in a patient not known to be allergic to a given medication.


3

Complications may occur throughout every aspect of patient care, but a primary concern is MEs.4 The steps and persons involved in the provision of medications to hospitalized patients create opportunities for errors. These can occur at any stage of the medication distribution cycle. Leape et al. found that errors occurred most often in the physician ordering (39%) and nurse administration stages (38%). Errors also occurred at the pharmacy dispensing (11%) and the transcription of order (12%) stages.11 a) Epidemiology of adverse events, medication errors, and adverse drug events The epidemiology of medical injuries, AEs, MEs, and ADEs has been studied. The incidence of such events is difficult to quantify, because of the differences in study design, population, settings, and methods that are used to identify them.12 In 1991, the Harvard Medical Practice Study examined medical injuries in the US and found that AEs (defined as unintended injuries caused by medical management and resulting in measurable disability) occurred in 3.7% of hospital admissions.13 An estimated 13.6% of AEs resulted in death. Drug complications were the most common non-surgical cause at approximately a fifth (19%) of all AEs.14 Although this study focused on AEs and did not provide data on MEs, the same methods were used in subsequent studies.

In 1995, the Quality in Australian Health Care Study found that an AE (defined as an unintended injury or complication that resulted in disability, death, or prolonged hospital stay and that was caused by health care management) occurred in 16.6% of hospitalized patients.15 Investigators measured preventability, instead of determining negligence, the approach taken in the Harvard Medical Practice Study. The Australian study found that 51.2% of AEs were preventable and that 13.7% led to permanent disability. MEs were not reported. Yet, drugs were the fourth most common cause (10.8%) of AEs. Of these 10.8% of AEs, 17% led to permanent disability, and 8% resulted in death. Among ADEs, 43% were judged to be highly preventable (there was strong evidence that an error occurred because of a failure to follow accepted practice at an individual or system level).15 In 2000, a study that was conducted in Utah and Colorado used a design similar to that of the Harvard Medical Practice Study. In these states, AEs (defined as injuries caused by medical management and resulting in prolonged hospital stays or disabilities at discharge) occurred in 2.9% of hospitalizations.16 Of these AEs, 27% to 33% were related to errors, and 8.8% resulted in death. ADEs were the most common non-surgical AEs (19.3% of all AEs).16 In a similar study in New Zealand in 2001, researchers determined that 12.9% of hospital admissions were associated with AEs (defined as unintended injuries resulting in disabilities and caused by health care management). Most of the patients had minimal or moderate disabilities. However, 10.2% of AEs resulted in permanent disability, and 4.5% resulted in death.17 Of the AEs occurring in hospital, 34.9% were preventable.18 Drugs were the third most common source of preventable in-hospital AEs. These ADEs represented 7.5% of all events, and 9.3% led to permanent disability or death.18 In 2003, the results from the first large-scale Canadian study of AEs were released. The methods were based on those of the Harvard Medical Practice Study, as modified by the Quality in Australian Health Care Study. The Canadian study reported that 7.5% of patients who were


4

admitted to hospital during the fiscal year 2000 experienced one or more AEs (defined as unintended injuries or complications that resulted in disability at the time of discharge, death, or prolonged hospital stay and that was caused by health care management).19 More than a third (36.9%) of these AEs were preventable. Although more than half (55.7%) of the AEs resulted in minimal or no physical impairment, 5.2% resulted in permanent disability, and 15.9% resulted in death. Medications and injectable solutions were the second most common cause of AEs (23.6%) after surgical causes.19 A review of studies that were published between 1990 and 2005 assessed MEs and ADEs in hospitals.20 Reported MEs occurred in 5.7% of all episodes of drug administration (range 0.038% to 56.1% in 31 studies). ADEs were reported in 4.2% of hospitalized patients (range 0.17% to 65% in 46 studies). The authors reported that there was high variability in the frequencies of MEs among the studies, because of the drugs that were used (for example, there was a higher error rate with drugs that were administered parenterally with antibiotics, with cardiac drugs, and with cancer drugs), the hospital setting (for example, there was a higher rate in non-teaching hospitals), and the methods that were used to determine the rate of errors (more errors were detected using patient monitoring compared with spontaneous reporting or chart reviews). The high variability in ADEs occurred because of the methods that were used to determine the rate of events (which was higher with patient monitoring and chart reviews than with spontaneous reporting) and because of the patient care units (for example, the rate was higher on internal medicine, geriatric, and intensive care units than on general medical units). In 2000, the IOM released To Err is Human: Building a Safer Health System.21 This report described a comprehensive strategy that can be used by government, health care providers, industry, and consumers to reduce medical errors in hospitals. By extrapolating the results from the Harvard Medical Practice Study and the Utah and Colorado study to all hospital admissions in the US, the report estimated that medical errors are the eighth leading cause of death (44,000 to 98,000 deaths every year). Errors in medication in hospital and in the community accounted for more than 7,000 deaths.21 This report, which was published eight years after the Harvard Medical Practice Study, helped establish the international movement for patient safety. Despite the IOM’s recommendations, a study has shown that little has been done to improve patient safety in hospitals.22 Three other reports by the IOM are pertinent to this review.5,23,24 The purpose of Crossing the Quality Chasm: A New Health System for the 21st Century23 was to provide strategies to redesign the health care delivery system with a view to innovate and improve care. A chapter is devoted to evidence-based medicine, and another is on information technology.23 Patient Safety:Achieving a New Standard for Care24 provides a framework to develop a national health information infrastructure to support health care delivery, including data standards for the collection, coding, and classification of patient safety information. A section of this publication considers adverse events and near-miss detection and analysis.24 Preventing Medication Errors considers medication errors in all health care settings.5 It focuses on error prevention strategies that should be implemented in health care. It also focuses on the role of government, regulatory bodies, and industry in improving the quality of and safety in the use of medications, and provides a research agenda for government and industry.5


5

b) Epidemiology of dispensing errors Dispensing or filling errors are one type of ME that may also cause ADEs, if they are not caught before reaching the patient.12 Cina et al. reported a 3.6% filling error rate by pharmacy technicians in a direct observational study that was conducted in a 725-bed tertiary care centre.25 Although pharmacists caught most of the errors upon routine verification, 0.75% of doses filled would have been delivered to the patient care unit with errors undetected. Among the undetected errors, 23% were potential ADEs, of which 28% were serious and 0.8% were life-threatening. The most common potential ADEs were incorrect medications (36%), incorrect strength (35%), and incorrect dosage form (21%).25 c) Epidemiology of medication errors and adverse drug events in children Several studies have measured the incidence of MEs in children. The results of these studies were synthesized in a systematic review that included studies published from 2000 to 2005, in all care settings and for all types of medications.26 The authors reported that the definitions of MEs in the included studies were not uniform and not always stated. Overall ME data were reported in 14 studies: 5% to 27% of medication orders included an error. Two studies provided data on prescribing, dispensing, administration, and documentation errors for all medications. The distributional epidemiological estimates of the relative percentages of types of errors were 3% to 37% for prescribing, 5% to 58% for dispensing, 72% to 75% for administration, and 17% to 21% for documentation. The high variability may be due to how the errors were reported, how often, and by whom. d) Economic burden of adverse events and adverse drug events The economic burden of MEs is unknown. Two studies considered the cost of AEs, and one study looked at the economic burden of ADEs. Baker et al.’s Canadian study estimated that the 255 patients with an AE had to stay an additional 1,521 days in hospital. These patients extended their hospital stay by an average of six days, compared to patients without AEs.19 In the US, the national cost of preventable AEs (including lost income, lost household production, disability, and health care) is estimated to be between US$17 billion and US$29 billion. More than half of these costs are for health care.21 A study that was conducted in a large US teaching hospital estimated that patients who experienced preventable ADEs during hospitalization had a longer length of stay and used additional hospital resources, resulting in an additional cost of US$4,685 per admission.27 1.1.2 Medication distribution cycle in hospital pharmacies

The drug distribution system is defined as the method used to receive and process medication orders from practitioners, and the method used to dispense, deliver, and administer the medication.28 Hospitals may have pharmacy departments that are responsible for the provision of pharmaceutical services, including the dispensing of medications and pharmaceutical care to


6

patients. Some hospitals use a mixture of centralized and decentralized pharmacy services, with satellite pharmacies located in specialized patient care units such as oncology. Most members of the public are unaware of the existence or functions of the pharmacy department, or how medications are procured and distributed. There is a chain of events that involves pharmacy, medical, and nursing staff before the medication reaches the patients. The list of required medications is contained in the hospital’s drug formulary, which is typically prepared by a pharmacy and therapeutics committee and approved by a medical advisory committee or other council. Drugs are purchased from manufacturers and wholesalers through contracts and tenders, to ensure the best available price. When the stock is received in the pharmacy department, it is inventoried and stored until a prescription order or requisition is received. a) Medication ordering Prescriptions are patient-specific. Medications are ordered by practitioners who use hard-copy prescriptions or a computerized prescriber order entry (CPOE) system. The hard copy is delivered to the pharmacy by hand, facsimile, or electronically by scanner. When the order is received in the pharmacy department, it is processed through the information system. The indication, dose, route, and duration of treatment are reviewed for appropriateness. Clinical validation is performed for drug allergies, drug interactions, and dosage modification in cases of renal or hepatic impairment. b) Medication dispensing Limited amounts of medications that are urgently needed or those that are prescribed as part of a protocol, such as anti-nauseants and stool softeners, may be stored at the nursing station as ward stock. They may be used by any patient for whom an order has been written. Narcotic and controlled drugs, which are also dispensed as ward stock, are kept in a secured area to comply with federal legislation. There is a greater risk of errors when medications are not labelled for a specific patient and are used by several patients than when medications are patient-specific. Most medications are dispensed for a specific patient in a unit dose package for a limited period. A medication to be dispensed is manually picked from the inventory, re-packaged into a smaller quantity, labelled appropriately, and sent to the patient care unit. c) Medication administration When the medication is received at the patient care unit, it is stored in a medication cabinet in the patient’s room, in patient-specific drawers in medication carts, or in the designated ward stock area. When a medication is to be administered, the nurse selects it from the appropriate patient drawer or from the ward stock supply. The nurse giving the medication is responsible for noting all administered doses in the patient’s medication administration record (MAR). The MAR is a written or pharmacy computer-generated transcription of the prescriber’s medication orders. For prescriptions written outside usual pharmacy hours, a nurse coordinator may access a pharmacy night cupboard, where a limited amount of urgently needed medications is stored, or a pharmacist may be on call after hours for consultation and for providing medications. According to a survey of 162 Canadian hospitals, the dispensing and the administration of medications is achieved through centralized unit dose (64%), decentralized automated unit doses


7

(27%), total ward stock (12%), carded doses (25%), or a more traditional drug distribution system (46%; for example, a patient-specific medication dispensed in limited quantity).29

1.2 Overview of Technology

1.2.1 Description of technologies

Because of the frequency of MEs and ADEs, we should develop and implement systems that aim to reduce errors and improve medication safety. Technologies have been developed to improve the productivity of pharmacy services and to minimize the potential for error by avoiding a reliance on human memory. These technologies automate the stages of the medication distribution cycle: CPOE for the ordering stage (which is not discussed in this report); automated medication dispensing devices and bar code verification of medications for the dispensing stage; and automated medication cabinets, bar-coding for administration, and electronic medication administration records (eMARs) for the administering stage. a) Technologies supporting dispensing activities Automated dispensing devices Automated dispensing devices (ADD) are pharmacy-based (centralized) or ward-based (decentralized). Pharmacy-based automated dispensing devices Canister-type ADDs: Canister-type ADDs repack solid oral dosage forms of medication (for example, capsules and tablets) into unit dose packages with or without patient-specific information. The medications are stored in canisters, which are calibrated by the vendor for one specific drug, strength, and manufacturer. The vendor charges a fee for every calibration. These machines may hold approximately 500 different medications. The canisters are assigned a numbered location. A tablet or capsule is dispensed from its home canister when an electronic order is received. The medication is ejected into a unit-dose packing device, where it is labelled and sealed.30,31 Examples of these devices include the Baxter ATC 212™ (which is no longer available for purchase) and products such as PacMed® by McKesson and the AutoMed FastPack™ EXP by AmerisourceBergen.32,33 Robotic-type ADDs: Centralized robotic drug distribution systems automate the storage, dispensing, and return of medication, including oral solid and other dosage forms. They can dispense bar-coded unit dose medication to medication carts, envelopes, or medication rings. The system is contained in a glass room with a central robotic arm that has a suction device to pluck medication packets hanging from metal rods. To restock the machine, a technician loads the packets using an inside door. The machine reads the packet’s bar code, then places it on the appropriate rod. This equipment requires the repackaging of every medication that is intended to be dispensed through this technology. Different pieces of equipment, such as the ROBOT-Rx® by McKesson32 or an integrated system such as the PillPick® system by Swisslog, are used.34 The ROBOT-Rx®, which is a storage and dispensing system, may be used with the ROBOT-Ready™ PACMED™


8

packaging unit. The PillPick system comprises a packaging unit (the PillPicker), a storage unit (the DrugNest), and a dispensing unit (the PickRing). Robotics for intravenous preparations: Robotic intravenous automation (RIVA), which was developed by Intelligent Hospital Systems Inc.,35 is distributed in Canada by Manrex Ltd.36 It is used by hospital pharmacies to automate the preparation of intravenous syringes and bags, including chemotherapy and pediatric preparations.35 Canister-type ADDs, robotic-type ADDs, and robotics for intravenous preparations may be used with bar-coding to facilitate the dispensing and the filling of carts. Ward-based automated dispensing devices The Pyxis® MedStation® by Cardinal Health, the AcuDose-Rx® by McKesson, and the MedSelect® by AmerisourceBergen are examples of automated dispensing units that are used on patient care units. They are also called automated dispensing cabinets,32,33,37 controlled-access medication cabinets, medication distribution systems, or automated decentralized pharmacy dispensing systems.38 The cabinets are locked, and access is gained through passwords or biometric identification (fingerprints). Pharmacy technicians access the pharmacy information system to determine which units need reloading. Bar-coding technology may be used for replenishing the ADD.30,39 Profiled or unprofiled methods are used. In the profiled mode, medication orders are entered into the pharmacy information system manually or electronically (for example, CPOE). In some hospitals, the prescription order must be verified by a pharmacist before the medication information is brought across a computer interface to a medication profile that is displayed on the ADD. From the computer screen, the nurse selects a patient, and the medication to be administered, from the patient’s profile. The drawer where the medication is stored opens when the selection is made. Some drawers are used to store many items (for example, matrix drawer), while others are single-item drawers. The medication is then retrieved and administered. Administration may be done with bar-coding for medication administration (BCMA). Access to certain medications before the medication order is verified by a pharmacist may be allowed through an override function; for example, in urgent situations or when the pharmacy department is closed.40 In the unprofiled mode, the ADD becomes a controlled-access ward-stock system.30,39 Nurses may access any medication in the cabinet. The Institute for Safe Medication Practices has issued Guidance on the Interdisciplinary Safe Use of Automated Dispensing Cabinets.41 Bar-coding for medication dispensing Carousels Carousels (for example, MedCarousel® by McKesson32) were first developed for the storage of medication and for managing inventories. Because stocking of and dispensing from the carousel are bar code driven, more pharmacy departments are using carousels for dispensing first and


9

urgent doses. A carousel is a series of revolving shelves set on rails. It is designed to improve space efficiency in the pharmacy and to maximize productivity. When dispensing, the technician scans the demand label, and the shelves rotate to the proper bin. The machine indicates the amount of medication to retrieve from the bar-coded bin. The medication bar code is scanned to ensure that the proper medication is being dispensed.39,42 IntelliShelf-Rx™ IntelliShelf-Rx™ by McKesson32 has shelves and bins that include bar code verification. The software can prioritize the orders to be dispensed. It indicates what drug is to be retrieved from the bin, the amount to be retrieved, and the location where the bin is stored. The medication to be picked is highlighted by a red light on a radiofrequency identification tag attached to the pharmacy bin.32 It requires no pharmacy redesign. b) Technologies supporting medication administration Bar-coding for medication administration BCMA may be used with or without CPOE. CPOE will reduce the chance of transcription errors. To use BCMA with CPOE, the prescriber enters a prescription order electronically. The hospital may have a policy that a pharmacist is to verify the orders before a nurse can administer the medication. Without CPOE, the pharmacy technician enters all the orders into the computer and creates the eMAR. After the order is verified by a pharmacist, the new order appears on the patient medication profile. The patient wears an identification wristband that is bar-coded. When the bar code is scanned, the patient’s personal information is displayed. The medications to be administered must also be bar-coded by the manufacturer or by the pharmacy staff. Hand-held scanners or computer consoles that are attached to a mobile station (with or without medication drawers) are used at the patient’s bedside for medication administration. The nurse scans the patient’s hospital wristband, the medication, and his or her own badge, to ensure that the correct medication is given to the correct patient at the correct time. The nurse, by scanning his or her identification badge, signs off electronically that the administration has occurred. If the wrong medication or patient is selected, an alarm will alert the nurse to an error.43 These devices may include other applications such as clinical decision support systems (CDSS) and charting. CareFusion® by Cardinal Health and Horizon Admin-Rx™ by McKesson are examples of hand-held scanners.32,37 SafetyMed™ by Omnicell is another type of BCMA that is used with the OmniRx® ward-based ADD.44 The mobile Smart Cart™ by MDG Medical is a computerized medication cabinet which includes a touch screen, bar-code technology, and 24-hour patient-specific drawers. The unit travels with the nurse for medication administration at the bedside.45 c) Technologies supporting transcription and administration eMAR An eMAR is generated from a CPOE system or from a pharmacy information system. It replaces the traditional method of manually transcribing prescriber orders to a hard copy MAR or using a pharmacy computer-generated MAR. Electronic MARs may be used with BCMA.


10

1.2.2 Regulatory status

The technologies under review are not regulated as medical devices by Health Canada (Nancy Shadeed, Medical Devices Bureau, Ottawa: personal communication, 25 March 2008). 1.2.3 Unit cost

The unit cost of the technologies under review could not be obtained from sources such as vendor price lists or the vendors. 1.2.4 Utilization pattern

A survey that was performed in 2003 and funded by CADTH was addressed to the pharmacy directors of Canada’s 100 largest acute care hospitals. These hospitals either had more than 200 acute care beds or 50% of the total number of beds were acute care beds. The response rate was 78%. Of the responses, 56% used automated dispensing (defined as drug storage devices or cabinets), and 9% and 1% used bar-coding for medication dispensing (BCMD) and BCMA, respectively.46 Data for the fiscal year 2007 to 2008 were collected by the Hospital Pharmacy in Canada Survey Editorial Board.29 Of 164 Canadian hospitals, 60 (37%) used bar-coding in the following areas: drug selection before dispensing from the pharmacy (31%), return of doses to the pharmacy inventory (38%), verification of unit dose stocking (22%), and verification of the stock in automated dispensing cabinets (24%). Two hospitals (3%) used BCMA for patient identification, and one (2%) used it for drug selection. Of 102 respondents, 77 (75%) used automation in their centralized unit dose systems. Of these 77 hospitals, 72 (94%) used a canister type system, and nine (12%) used a robotic system. Decentralized automated unit dose systems were used by 59 of 162 respondents (36%).29

2 ISSUE

The steps and persons involved in the provision of medications to hospitalized patients create opportunities for errors. Most medication errors are minor, but some may result in an adverse drug event. A Canadian study reported that 7.5% of patients who were admitted to hospital during the fiscal year 2000 experienced one or more adverse events. Medications and injectable solutions were the second most common causes of adverse events. Technologies that are used to automate the dispensing and administration of medications may decrease medication errors, improve quality of care, and reduce the cost that is associated with adverse events due to medication errors. These technologies include automated medication dispensing devices, bar-coding verification for medication dispensing and administration, and electronic medication administration records. Informed decision-making about the use of these technologies requires an assessment of the clinical and economic consequences of adoption in a Canadian setting.


11

3 OBJECTIVES

This report describes an assessment of the clinical and economic impact of adopting technologies that are designed to facilitate medication dispensing and administration in hospitals. This health technology assessment (HTA) is intended to encourage the efficient allocation of scarce health care resources as hospitals are investing in computerized technologies. These objectives will be met by addressing the following research questions: 1) What is the clinical-effectiveness of technologies that are intended to reduce medication

errors in hospitals ― including automated dispensing devices, bar-coding for medication dispensing and administration, and eMAR ― in preventing medication errors, potential adverse drug events, adverse drug events, morbidity, and mortality?

2) What is the cost-effectiveness of technologies that are intended to reduce medication errors in hospitals, including automated dispensing devices, bar-coding for medication dispensing and administration, and eMAR?

3) What is the budget impact of adopting these technologies in hospitals on initial capital investment, training at implementation, training required for new employees, maintenance costs, and operational costs (for example, database update, software update, hardware, and human resources)?

4 CLINICAL REVIEW

4.1 Methods

A protocol was written a priori and followed throughout the review process. 4.1.1 Literature searches

Literature searches were conducted for the clinical review and for the economic evaluation. The results from both literature searches were combined. The literature search for the clinical review was conducted in two parts. The original search included other technologies, such as CPOE, CDSS, interventions (for example, clinical pharmacist participation on rounds), and systems (for example, continuous quality improvement). A decision was made to narrow the scope of research to technologies that are used in the automation of the dispensing and administration of medication. The methods for the original search (1992 to 2002) are available upon request (MB, unpublished observations, 2004). The second search was conducted for the years 2003 to 2008, with search updates until January 2009. All search strategies were developed by an Information Specialist, with input from the project team, and were peer-reviewed by another Information Specialist.


12

For the clinical component of the report, the following bibliographic databases were searched through the Ovid interface: MEDLINE, MEDLINE In-Process & Other Non-Indexed Citations, EMBASE, BIOSIS Previews, CINAHL, ACP Journal Club, The Cochrane Library, and the Centre for Reviews and Dissemination. The search strategy comprised controlled vocabulary, such as the National Library of Medicine’s MeSH (Medical Subject Headings) and keywords. The main search concepts included specific medication ordering devices (BCMD, BCMA, ADD, eMAR) and their impact on the quality of care and medical errors in hospitals. Search filters were applied to limit retrieval to randomized controlled trials, controlled clinical trials, observational studies, and systematic reviews. Appendix 1 shows the detailed search strategies. Papers that were published in all languages were considered. Ovid AutoAlerts were set up to send monthly updates with new literature. A final update was performed on the Centre for Reviews and Dissemination and The Cochrane Library databases. Grey literature was identified by searching the websites of HTA and related agencies, professional associations, and other specialized databases. Google and other Internet search engines were used to search for additional information. These searches were supplemented by hand searching the bibliographies and abstracts of key papers and conference proceedings, and through contacts with appropriate experts and agencies. A final grey literature update was performed during the writing of the report. Google and other Internet search engines were used to search for vendors and devices. The search included terms specific to medication ordering devices (BCMD, BCMA, ADD, eMAR). Additional vendors and devices were found in the grey literature search, which included ECRI’s Gold and device regulatory websites (including Health Canada and the FDA). The vendors were contacted to obtain unpublished material (Appendix 2). 4.1.2 Selection criteria

a) Study design Systematic reviews, HTAs, and clinical studies with comparison groups — including randomized controlled trials, controlled clinical trials, observational studies (cohort and case-control studies), before and after studies, and time series analyses — were considered. Case series, case reports, implementation studies, chart reviews, guidelines, surveys, focus groups and interviews, letters (unless they contained original research data), news articles, opinions, and editorials were excluded. b) Population groups The population groups were inpatients of a hospital, including acute care (adult, children, and psychiatric), critical care, rehabilitation and long-term care, and emergency rooms. Ambulatory care (outpatient clinics and community-based practice), nursing homes, and retirement homes with assisted living programs were excluded. c) Interventions The interventions to be included were based on the results of a survey of Canadian hospital pharmacy directors. In the survey, which was conducted in the fall of 2007 (CP, unpublished observations, 2008), pharmacy directors were asked to rate the relevance of conducting an HTA


13

that would inform their purchasing decisions about health technologies that may contribute to the development of more effective and safer medication systems in Canadian hospitals. It was determined that the technologies related to medication dispensing (BCMD, ADD) and medication administration (BCMA, eMAR) would be most relevant. These include technologies that are used in hospitals, that are commercially available, that are customized, or that are developed in-house. Based on the survey findings, our review excluded automated devices that are designed to support medication ordering in hospitals (for example, CPOE and CDSS). Other excluded technologies were radiofrequency identification, infusion pumps, health care provider interventions (for example, automatic stop orders, education and training programs, clinical pharmacist participation on medical and para-medical rounds, and therapeutic drug monitoring services) and systems, processes, or policies (for example, automatic stop orders and continuous quality improvement). d) Comparators The comparators were another technology or standard practice. e) Outcomes The outcomes were MEs, including change in rate of MEs, potential ADEs, ADEs, morbidity, and mortality. Other outcomes such as human resources requirements, staff training, and pharmacy space allocation were addressed in the economic section. f) Publication characteristics For the clinical review, papers that were published from 1992 to the present were considered. 4.1.3 Selection method

Reviewers (MB, CP, GM) examined the results of the literature search and independently selected potentially relevant articles. First, the reviewers selected titles or abstracts that met all of the inclusion criteria. If all the criteria were met, or if there was uncertainty or disagreement, the paper was obtained in full text. Second, the reviewers independently selected those papers that met all of the inclusion criteria.


14

3,502 citations excluded

18 citations identified from other sources

122 potentially relevant reports retrieved for scrutiny (full text, if

available)

40 potentially relevant reports retrieved from other sources

162 potentially relevant reports

140 reports excluded: no primary data (61); other (9) no intervention or control group of interest (23) no outcome of interest (11) study design inappropriate (36)

22 reports (21 studies+1 systematic review)

3,606 citations identified from electronic search and screened

Figure 1: Clinical Studies From 2003 to Present


15

Disagreements were resolved through consensus. The number of studies that were included or excluded is shown in Figure 1. 4.1.4 Data extraction strategy

Data were extracted using a data extraction form (Appendix 3) that was tested on two articles to ensure that both reviewers were interpreting the form similarly and extracting the information consistently. No changes were made to the form. The study data were extracted by two reviewers (CP, MB), and other reviewers (CP, GM, and SN) verified the extraction. Disagreements were resolved through consensus. 4.1.5 Strategy for validity assessment

Reviewers (MB, CP, SN) independently assessed the validity of the systematic reviews and of the included studies (Appendix 4). Disagreements were resolved through consensus. Systematic reviews were assessed using the Oxman and Guyatt Scale.47 A review was considered to be of high quality if it obtained a score of five or higher. The internal validity of the randomized controlled trials was evaluated using the Jadad Scale,48 which assesses the appropriateness of randomization and double-blinding, and how withdrawals and dropouts were counted. The adequacy of concealment of allocation to treatment was also considered for randomized controlled trials.49 The validityof cohort and case-controlled studies was assessed using the Newcastle-Ottawa Scale.50 The internal and external validity of epidemiological studies were also assessed. They were further evaluated for potential bias, including selection, performance measurement, and detection biases. Relevant definitions, such as those for the error-ascertainment methods that were used in the studies, were extracted. The data analysis and interpretation of the results were made, taking into account the findings from the quality assessment. Allan and Barker51 developed a list of criteria for analyzing the validity of medication error studies. Their list was not used in this assessment because it could not be determined whether or not the list had been validated and was reliable. 4.1.6 Data analysis methods

The question of whether or not to accept or update a systematic review or conduct a new systematic review was answered based on whether or not: the published systematic review met the selection criteria the systematic review was of higher methodological quality (a score of 5 or greater on the

Oxman and Guyatt Scale) the authors conducted a thorough search strategy consistent with current CADTH standards the systematic review was current (no new studies that would likely change the results were

published after the systematic review’s last search date). If the data could be used for a meta-analysis, then a meta-analysis was to be done. Otherwise, a descriptive review of findings would be considered. For the included studies, risk estimates such as relative risk (RR), relative risk reduction (RRR), relative risk increase (RRI), and 95% confidence intervals (CI) were calculated using the software Confidence Interval Analysis.52


16

4.2 Results

4.2.1 Quantity of research available

Of the 162 potentially relevant reports that were retrieved for a full text review, 140 did not meet the selection criteria, leaving 22 reports (one systematic review and 21 studies) that met our inclusion criteria (Figure 1). Ten reports describing nine studies53-61 were retrieved from the original search (MB, unpublished observations, 2004) and included in our review (total of 30 studies). Fifteen vendors were contacted to obtain unpublished clinical studies (Appendix 2). Five responses were received, but none provided additional studies that met the inclusion criteria. 4.2.2 Study characteristics

a) Systematic review One systematic review on ADDs and bar-coding met the selection criteria.62 Its quality was assessed using the Oxman and Guyatt Scale.47 The systematic review was not considered to be acceptable for an update because it was deemed to be of lower quality (Appendix 5, Table 1). b) Studies All the selected studies were available in the public domain. One study was available only as an abstract.63 A before and after design was used in most of the studies. Fifteen studies were prospective,53-

56,63-73 three were retrospective,57,58,74 and six did not state whether the study was done prospectively or retrospectively.59,60,75-78 One study was a controlled before and after study,79 three studies were cohort studies,61,80,81 one was a time series analysis,82 and one used a before and after study design with a time series analysis.83 No randomized controlled trials were found. The studies included interventions such as pharmacy-based ADDs,53,61,64-66,75,82 ward-based ADDs,54-56,67 BCMD,68,69,76 BCMA,57-60,63,70,71 and bar-coding for blood or blood product administration.72,74,80 Six studies were conducted using several technologies simultaneously. One study was on ward-based ADD with BCMA,83 four were on BCMA with eMAR,77-79,81 and one was on ward-based ADD with BCMA and eMAR.73 Different methods were used in the studies to ascertain errors before and after the implementation of the interventions, including doing voluntary or solicited medication error reports, conducting chart reviews and audits, using direct observation techniques, and using automated reports (Appendix 5, Table 2). The quality of the three cohort studies was evaluated using the Newcastle-Ottawa Scale.50 All scored 4 out of a possible 8. The remainder of the studies were assessed for internal and external validities. They were evaluated for potential bias, including selection, performance measurement, and detection biases. The study limitations appear in Appendix 5, Tables 3 to 6. The results are discussed in light of these limitations.


17

4.2.3 Data analyses and synthesis

The study characteristics, results (risk estimates that we calculated or that were reported by the investigators), and the study limitations appear in Appendix 5, Tables 3 to 6. The results were not meta-analyzed, because the study designs did not lend themselves to pooling. Given the heterogeneity of the study characteristics ― such as study design, hospital setting, study duration, interventions, comparators, error-ascertainment methods, and outcomes ― a descriptive review was done. A summary of the findings appears in Table 2.

Table 2: Findings From the Included Clinical Studies Technology Outcome Measured (Number of Studies) RRR or RRI

Dispensing errors (1) ↓28.7%* Total MEs (1) ↓38.4%* MEs in surgical unit (1) ↓33.8% MEs in ICU (1) ↑70.0%

Profiled, ward-based (decentralized) ADD

Medication-related events (1) ↓36.6%* Dispensing errors using ATC-212™ (1) ↓22.3%† Cart-filling errors using ATC-212™ (1) ↓99.7%*

Pharmacy-based (centralized) ADD

Dispensing errors using original-pack dispensing systems (5)

↓16.0% to ↓61.3%

Filling errors for first dose or missing dose (1) ↓15.2%† Filling errors for automated dispensing cabinet fill (1) ↓74.7%* Dispensing errors for first dose or missing dose (1) ↑9.0%† Dispensing errors for automated dispensing cabinet fill (1)

↓28.9%†

Dispensing errors (2) ↓36% and ↓96%

BCMD (carousels)

Potential ADE (1) ↓63% MEs (1) ↓86.2% Medication administration errors (4) ↑18.0%†

↓77.9% to ↓86.8%

BCMA used for drug administration

Total MEs (2) 0% and ↓70.6% Blood transfused to wrong patient (1) 0 pre and post BCMA used for blood and

blood products administration

Near miss per 50 units of blood (1) 1 near miss

Dispensing errors (1) ↓99.0% Ward-based ADD and BCMA MEs (1) ↓9.8%

MEs (3) ↑14.7%* ↓44.0 % and ↓80%

Medication administration errors in cardiac telemetry (1)

↓24.1%†

Medication administration errors in medical-surgical unit (1)

↓35.9%*

BCMA and eMAR

Preventable ADEs (1) ↓47%* Ward-based ADD, BCMA, and eMAR

Medication administration errors (1) ↓47.5%*

ADD=automatic dispensing device; ADE=adverse drug event; BCMA=bar code medication administration; BCMD=bar code medication dispensing; eMAR=electronic medication administration record; ICU=intensive care unit; ME=medication error; RRI=relative risk increase; RRR=relative risk reduction *met an investigator-defined threshold of statistical significance †did not meet an investigator-defined threshold for statistical significance


18

Among the weaknesses of the studies, one limitation was that few studies adjusted for confounders. Factors other than automation may have led to changes in work practices. These factors include physical reconfiguration of the pharmacy department or patient care units, or staff training (pharmacy and nursing staff receiving additional training before the intervention began, and the technologies having been generally evaluated when they were first implemented) and sensitization to patient safety. These factors could have had an impact on error rates, and the risk reduction may have been overestimated. Furthermore, investigators did not take into account the baseline risk of participants in the studies or the context in which machines were used. For example, the rates and severity of MEs that are observed in an ICU are not comparable to what may be observed on a general medicine patient care unit, given the differences in the type and intensity of care. Thus, if error rates are greater in certain types of patient care units, with certain types of drugs, or in certain types of patients, it would be difficult to compare risk reduction between settings and populations. The error detection capacity varies between error-ascertainment methods. In a review of medication error research, Allan and Baker51 noted that the error rates across studies should be compared with caution when considering the variations in error-ascertainment definitions and methods.51 In our included studies, different techniques for measuring errors were used. In other instances, error-ascertainment methods were not reported or described. Flynn et al.84 compared three error detection methods (direct observation, incident report, and chart review) that were used to detect the same medication error and the incurred cost. Data collectors included pharmacy technicians, registered nurses, and licensed practical nurses. Of 2,556 comparison doses, a pharmacist confirmed that there were 457 errors (a true error rate of 17.9%). With the direct observation method, 300 errors were caught (11.7% error rate). Seventeen errors (0.7% error rate) were detected using chart reviews, and one error (0.04% error rate) was detected using incident reports. Of the 457 errors, 35 (7.6%) were considered to be potentially clinically significant by a three-physician panel. Of these 35 errors, 25 (71.4%) were caught by direct observation, three (8.6%) by chart review, and none by incident report review. The mean cost per error detection was US$4.82 for direct observation and US$0.62 for a chart review.84 a) Pharmacy-based automated dispensing devices All seven studies that met the inclusion criteria (Appendix 5, Table 3)53,61,64-66,75,82 measured dispensing errors. Two US studies compared the use of the now retired ATC-212™ to the manual filling of unit-dose medication carts by pharmacy technicians. All medication drawers were verified by a pharmacist. Klein et al. reported a 22.3% reduction in the rate of medication dispensing errors. This was not statistically significant (RR:0.78 [95% CI: 0.46 to 1.30]).53 Kratz and Thygesen compared the accuracy of manual versus automated cart filling by a pharmacy technician (not verified by a pharmacist).61 They found that the ATC-212™ was 99.98% accurate, and manual filling was 92.62% accurate. The overall accuracy (ATC-212™ doses plus manual doses) was 98.77%. Of 12,660 doses, three errors occurred during the use of the ATC-212™ compared to 183 filling errors in 2,493 doses filled manually (RR:0.003 [95%


19

CI: 0.001 to 0.01]). The most common error was due to missing drugs from the unit dose cart, followed by excess drugs in the cart. The authors reported that three of the seven pharmacy technicians were new and had limited experience. Two of them contributed to more than 50% of the errors.61 The other five studies were conducted in the UK.64-66,75,82 German-made original-pack dispensing systems are used in Europe.85 These devices automate the storage and dispensing of drug products in the original package supplied by the manufacturer. The ROWA Speedcase by ARX Ltd. and the Consis by Baxter are two examples that were used in the included studies. Overall, the RRR of dispensing errors ranged from 16.0% to 61.3%. One study reported a statistically significant result, but the study duration was only a few weeks.64 The wide range of the results may be due to the limitations. For example, in Franklin et al.,64 there may have been an under- reporting of errors, because reporting was done voluntarily. Fitzpatrick et al.65 did not discriminate between the items dispensed using Consis and those dispensed manually post-implementation. In Whittlesea’s75 study, the data collection periods were short. The pharmacy staff involved in the data collection had no research experience. There was a lack of resources for doing second independent checks. The errors were undefined.75 b) Ward-based automated dispensing devices Four prospective before and after studies met our inclusion criteria (Appendix 5, Table 3).54-56,67 All four studies used an ADD with patient medication profiles. Three US studies that were published in 1995 compared an earlier version of the MedStation® Rx to a centralized unit dose system.54-56 Conversion to a ward-based ADD led to a reduction in the rates of dispensing or medication errors. Ray et al.55 showed that there was a statistically significant decrease in dispensing errors by technicians (RRR: 28.7% [95% CI: 3.6 to 53.8]). One possible explanation for this finding is that pharmacy technicians had fewer medications to select from in filling the ADD units compared to filling the unit-dose medication carts in the central pharmacy. Borel and Rascati54 showed that there was a statistically significant decrease in total MEs (RR: 0.62 [95% CI: 0.49 to 0.78], RRR: 38.4%). Most errors were due to wrong-time administration of medication, omission, and delays in delivery from the pharmacy before and after the implementation of the ADD. From incident reports, Schwarz and Brodowy56 observed fewer MEs per month on a surgical unit (RRR: 33.8%), and more MEs in the cardiac ICU (RRI: 70%). This could have been due to the fact that nurses were allowed to access more than one medication in a matrix drawer. Nurses could also override the pharmacy profile to obtain certain medications. Although the overall number of events was small, the findings may have been affected by the introduction of a new ME reporting form during the deployment of the ADD.56 A more recent study, which was conducted in Saudi Arabia, compared unit dose and open floor stock systems to an un-named ADD.67 Pharmacists and nurses were blinded about the purpose of the study. The study lasted six months, with no pause between pre- and post-implementation. ADEs were measured. The results were in favour of the ADD (RR: 0.63 [95% CI: 0.46 to 0.88],


20

RRR: 36.6%). The errors, however, were not defined clearly, because the authors combined all medication-related events. They may have under-reported the number of errors by using incident reports to ascertain errors. c) Bar-coding for medication dispensing Three before and after studies were retrieved (Appendix 5, Table 4).68,69,76 Two prospective before and after studies measured the impact of using a carousel system on filling or dispensing errors.68,69 Oswald and Caldwell looked at three filling and dispensing processes that were affected by the introduction of the carousel: first dose for new patient-specific medication orders or a missing dose of a medication, automated dispensing cabinet refills, or interdepartmental requests.68 The only statistically significant result that was seen with the introduction of the carousel was a reduction in errors when filling the automated dispensing cabinet (RR: 0.25 [95%CI: 0.11 to 0.58], RRR = 74.7%). All other results were non-statistically significant: for first dose or missing medication, filling errors were reduced by 15.2%, and dispensing errors were increased by 9.0%. There was an increase in dispensing errors when filling interdepartmental requests (no errors pre-implementation out of 123 orders compared with one error out of 85 orders post-implementation). A 20-month study that was sponsored by the Agency for Healthcare Research and Quality, considered dispensing errors and ADEs while observing 115,164 doses pre-implementation and 253,984 doses post-implementation of the carousel.69 The calculated error rates were controlled for the proportion of doses that were dispensed using each process (pre- and post-). Relative risk reductions of 36% and 63% were reported for all dispensing errors and for all potential ADEs due to dispensing errors respectively. A 2.8-fold increase in the number of life-threatening ADEs was reported (two life-threatening ADEs pre-implementation compared with 13 ADEs post-implementation). The statistical significance was not reported for any of the results, because the investigators were concerned about the configuration of the technologies being different during each observation period. This may have led to confounding. The authors also stated that pharmacy staff knew they were being observed. This may have resulted in better performance compared with usual practice (the Hawthorne effect). One before and after study considered packaging systems for oral solid doses, liquids, and odd- shaped dosage forms with the carousel to determine dispensing errors.76 It reported a RRR of 96% in dispensing errors, which the authors describe as “pharmacy errors in selecting products from storage.” No details were provided about study design and duration, and there was no explanation about how the errors were measured.76 d) Bar-coding for medication administration Eleven reports describing ten studies met our inclusion criteria (Appendix 5, Table 5).57-60,63,70-

72,74,80,86 The studies were divided into those that were specific to drugs and those specific to blood or blood products. Drugs Seven US before and after studies were published from 1995 to 2006.57-60,63,70,71 Two were retrospective,57,58 three were prospective,63,70,71 and two were not described.59,60 They compared standard practice and paper MARs to BCMA products. Some hospitals used systems such as


21

CliniCare,59,60 Tremont,58 or CareFusion,71 and others did not state the brand that was used.63,70 One system was developed in-house.57 The study settings included small community hospitals and larger acute care hospitals. Two studies reported statistical significance.58,70 Studies measuring medication administration errors One study reported a non-statistically significant increase in the medication administration error of 18% (P = 0.62).58 The increase may be attributed to the fact that nurses voluntarily reported incidents before the implementation of the BCMA, which tends to lead to under-reporting. Post-implementation, the process of collecting errors was automated. In addition, staff were unfamiliar with using the new software during the first month of BCMA implementation. This may have influenced the results post-implementation.58 Three other studies reporting administration errors had RRRs of 77.9%,70 82.3%,71 and 86.8%.63 One study reported near- misses in 3.2% of doses post-implementation.63 Studies measuring total medication errors In 1995, Brown et al. showed no difference in medication errors per 1,000 doses dispensed, possibly because a reduction in the number of staff occurred post-implementation.59 Also in 1995, Puckett showed a decrease in the rate of MEs from 0.17% to 0.005% (RRR 70.6%).60 Johnson et al.57 reported a decrease in MEs (RRR 86.2%), but they did not indicate if total errors or administration errors only were measured. The magnitude of differences between study results may be explained by how MEs were measured (some studies used incident reports in the pre- and post-phases; others used an automated process post-implementation) and calculated (some studies considered the number of errors over the total number of doses dispensed; others reported means). In addition, some studies included wrong or delayed administration time, which accounted for a great proportion of errors. Blood and blood products A Chinese retrospective before and after study compared two methods of verification in the blood administration process: a conventional system of using two persons to do a visual check compared to bar-coding (PathFinder® Ultra®).74 Four years of data pre-implementation and three years of data post-implementation were reviewed. Before bar-coding, 13 of 41,000 blood sampling procedures had wrong labels (on the blood sample or on the request forms). This was decreased to zero after bar-coding was implemented. No errors in blood transfusions were reported in either period. The study was conducted in a regional hospital and excluded the emergency room, where blood is often administered. A study that was sponsored by the UK National Blood Service compared standard transfusion procedures to bar-coding (Symbol) in a cardiac recovery unit and a cardiothoracic unit.72 Of 50 first-unit red blood cell transfusions in the cardiac recovery unit, the transfusion of one incorrect blood component was prevented by using bar-coding.


22

A US historical cohort study by Porcella and Walker reported results in favour of bar-coding for the collection, dispensing, and administration of blood products, compared with a manual process.80 A pilot study that included adult and pediatric inpatient units, the ICU, and the adult transplant unit reported a RR of 9.98 (95% CI: 1.28 to 78.0) when comparing 2003 and 2004 data. When bar-coding was implemented hospital-wide, the RR increased to 30.6 (95% CI: 9.5 to 98.4). A high RR means that it was more likely that errors were caught using the bar code system compared with the manual system. e) Multiple technologies Six studies evaluated the simultaneous use of several technologies (Appendix 5, Table 6).73,77-

79,81,83 Ward-based automated dispensing device and bar-coding for medication administration A before and after study with interrupted time series compared a centralized patient-specific medication with cart system and visual wristband checks to ward-based ADD and BCMA (POC system with RX).83 All medications were included, except emergency medications. Converting from a centralized system to a ward-based system resulted in a 99.0% decrease in the dispensing error rate. The use of BCMA prevented medications from being administered to the wrong patients (RRR: 90.0%, P = 0.003). A 0.2% error rate for giving a wrong medication did not change with the implementation of BCMA. The hospital-wide mean ME per 1,000 patient-days was reduced by 9.8%, based on the voluntary reporting of incidents. Bar-coding for medication administration and eMAR In 2004, Anderson and Wittwer77 described their use of BCMA and eMAR. A one-month pilot study of a 35-bed unit reported 80 near-misses post-implementation. A 44% decrease in MEs was obtained over a six-month period for all units. The authors did not state what types of machines were used in the hospital, nor did they describe the comparators. The error-ascertainment methods that were used pre- and post-intervention were not provided. A controlled before and after study compared a manual five-day MAR to BCMA and eMAR (brand and supplier not provided).79 The control group consisted of a 20-bed cardiac telemetry unit, and the comparator groups were a 20-bed cardiac telemetry unit and a 36-bed medical surgical unit. The control group, which did not receive the intervention, had an increase in medication administration errors. When comparing the medication administration errors pre- and post-implementation, the two groups that received the intervention had a reduction in errors of 24.1% (P = 0.065) and 35.9% (P = 0.035), respectively. Paper-based MARs were compared to a BCMA and eMAR product by Cerner Corporation in two studies. One before and after, four-week pilot study reported an 80% reduction in MEs.78 The ascertainment methods were not reported. The second study was a prospective cohort study that was conducted in a 36-bed neonatal ICU over 50 weeks.81 An unadjusted 15% (P < 0.001) increase in MEs was reported after implementation. This increase was mostly due to wrong-time errors. Unadjusted 71% (P < 0.001) and 50% (P = 0.008) reductions in potential ADEs and preventable ADEs were reported, respectively. When controlling for doses per participant, per day, and other covariates, there was a RRR of 47% (RR: 0.53 [95% CI: 0.29 to 0.98]) for preventable ADEs. All findings were statistically significant. The control group, which did not get equipped with BCMA and eMAR, had no changes in all measured outcomes in the before and after study period.81


23

Ward-based automated dispensing device, bar-coding, and eMAR A UK before and after study that lasted two weeks compared a traditional pharmacy system to an all-inclusive system of electronic prescribing, ward-based ADD, BCMA, and eMAR73 on a surgical unit, where fewer medications are administered than on a medical unit. The outcomes that were measured included prescribing errors, medication administration errors, and severity of MEs. The prescribing errors are not described in this review, because CPOE was not included as an intervention. Medication administration errors were statistically less frequent, with a RRR of 47.5% (RR: 0.53 [95% CI: 0.39 to 0.71]). With this integrated system, patient identification was more likely to be verified before administration (RR: 0.23 [95% CI: 0.20 to 0.26], RRR: 77.1%). The severity of MEs, as assessed by four “judges,” was reduced by 7.4% (P = 0.39). The authors did not describe what was considered to be a severe error, so it is difficult to interpret this finding.

5 ECONOMIC ANALYSIS

5.1 Review of Economic Studies: Methods

The protocol for the review was written a priori and was followed throughout the review process. 5.1.1 Literature searches

A literature search was conducted to look for prior economic evaluations. The search strategies were developed by an Information Specialist, with input from the project team, and were peer reviewed by another Information Specialist. A parallel search was conducted in the Health Economic Evaluations Database (HEED), in addition to the bibliographic databases that were searched for the clinical review. The search strategy comprised controlled vocabulary, such as the National Library of Medicine’s MeSH, and keywords. The main search concepts included specific devices supporting the medication cycle (BCMD, BCMA, ADD, eMAR) and their impact on the quality of care and medical errors in hospitals. Search filters were applied to limit retrieval to economic studies (Appendix 1). The search included papers that were published in all languages from 1990 onwards. Ovid AutoAlerts were set up to send monthly updates with new literature. A final update was performed on the Health Economic Evaluations Database, on the Centre for Reviews and Dissemination database, and on The Cochrane Library database. Google and other Internet search engines were used to search for vendors and devices. The search included terms specific to medication ordering devices (BCMD, BCMA, ADD, eMAR). Additional vendors and devices were found in the grey literature search, which included ECRI’s Gold and device regulatory websites (including Health Canada and the FDA). The vendors were contacted to obtain unpublished material (Appendix 2).


24

5.1.2 Selection criteria

To be included, a study had to meet the following criteria: the topic was an automation of the hospital drug distribution or delivery system; there was a study and comparison intervention; and an economic component was included (the study had to examine costs or resources, such as personnel workload, for the interventions). 5.1.3 Selection method

Two reviewers (PJ, GM) reviewed titles and abstracts and, based on pre-determined criteria, selected titles and abstracts that met the inclusion criteria. The results were coded and compared. Any disagreements were discussed, and any further disagreements were resolved by a third party (CP). 5.1.4 Data extraction strategy

The review had three components: interventions and context, methods used by the authors, and results of the studies and quality of the analysis. An economic study can compare resources or costs (cost comparison), or it can include costs and outcomes. The latter can be measured physically (cost-effectiveness or cost utility) or in monetary terms (cost benefit). The components of a study are described in CADTH’s economic guidelines.87 They include the timelines (all downstream effects to be included), perspective (resources provided by government, hospital, or both), costs (physical resources, unit costs, and cost by intervention), outcomes (measures of health outcomes are preferred to those of health care services), and method of determining effectiveness. Each element comes with preferred methods (Appendix 6, Table 7). The data extraction forms were developed by two reviewers (PJ, GM). Each reviewer extracted and compared the information. Disagreements were discussed, and any further disagreements were resolved by a third party (CP). 5.1.5 Strategy for quality assessment

Two aspects of quality were assessed for each study. First, the study design was assessed using the categories that were developed by Hailey et al.88 according to whether or not there was randomization and whether or not the data-gathering was prospective or retrospective. A prospective study (in which the study is designed before data collection) has a higher level of evidence than a retrospective study. In addition, a prospective study with an independent control was preferred to one without an independent control. Modelling studies were excluded in this component of quality assessment. The assessment of the economic components was based on a checklist that was developed by Drummond and Jefferson.89 Our assessment had the following eight criteria: whether actual or potential health outcomes (such as errors or AEs) were included whether all relevant resources (costs) were included whether the timelines of the study included relevant downstream events whether the analysis was incremental


25

whether future period items were discounted whether there were statistical tests of significance or tests of sensitivity whether a sensitivity analysis was conducted whether the methods and results were transparent.

For this dimension of quality, a designation of “met,” “not met,” or “not applicable” was assigned for each item. A value of 1 was given for each criterion that was met. The maximum value was 8. All articles, including those with a model, were assessed. Two reviewers assessed quality (PJ, GM). Any disagreements were resolved by a third party (CP). 5.1.6 Data analysis methods

The study outcomes included resource items and health outcomes. Resource items included any of automation system costs, nursing and pharmacy personnel costs, drug costs, inventory costs, and space storage costs. The costs of equipment can be evaluated as a capital amount or annual rental, but should include annual maintenance and the original cost. It should also include set-up and planning costs, because resources are involved. The use of automated techniques can have an impact on nursing and pharmacy personnel. This impact can be reported in physical or dollar terms. It can also have an impact on drug costs, through the amount of drugs that are used and through its impact on average inventories held, stock shortages, and stock outdates. If inventory levels are low, excess costs that are associated with shortages may occur. If inventories are kept too long, they can become outdated. If levels are kept too high, holding costs are incurred. Finally, automation can lead to less space being used to store drugs. In other instances, some technologies, such as dispensing robots, take up a lot of space and are noisy.


26

1,527 citations excluded

14 citations identified from other sources

62 potentially relevant reports retrieved for scrutiny (full text, if available)

17 potentially relevant reports retrieved from

other sources

79 potentially relevant reports

64 reports excluded: no primary or economic data (30) no interventions or outcomes of interest (13) study design inappropriate (14) other (7)

15 reports describing 15 studies

1,575 citations identified from electronic search and screened

Figure 2: Economic Studies


27

Benefits can be reported in physical terms (for example, hours or full-time equivalents) or dollar values. In the American studies, with direct patient or third-party billing, charges that are captured are a benefit to the hospital. Where these are included, they do not refer to all hospitals, but only to those in the US. The outcomes have been measured in terms of dispensing errors, medication administration errors, total MEs, AEs related to these errors, or waiting times. Adverse events have been reported in physical units or in dollar values. Captured charges have been identified as an outcome in the US studies. The results are expressed in descriptive terms and, where possible, in relative terms (cost per unit of outcomes, cost-benefit ratio).

5.2 Review of Economic Studies: Results

5.2.1 Studies identified

In the electronic search, 1,575 references were identified. After a review of the abstracts and hand searching, 79 articles were selected for consideration. Fifteen vendors were contacted to obtain unpublished economic analyses and cost information. One economic paper (Baker J, Draves M, Ramudhin A. Analysis of the Medication Management System in Seven Hospitals. Cardinal Health: unpublished data, 2008) was obtained from the five responses that were received. This paper was excluded from the review because it did not have a comparator without automation, but it did provide some of the workload data for the economic model. Fifteen articles were retained for the economic review (Figure 2). The results are presented by resource component. Nine of the 15 reviewed articles (Appendix 6, Table 8) studied the implementation of ward-based ADDs.56,67,90-96 Of these studies, three were using ward-based ADDs with patient medication profiles.56,67,94 Three studies focused on the implementation of pharmacy-based ADDs,53,65,66 two on BCMD,97,98 and one on BCMA.99 Six studies were conducted in the 1990s; the remaining nine were conducted after 2001. A cost-benefit analysis was done in six studies,56,92,95-98 and the remainder reported on physical resources (for example, workload) or costs only. Of the cost-benefit analyses, four were done from the viewpoint of the hospital, and two from a societal viewpoint that included patient outcomes related to AEs. 5.2.2 Quality of studies

Two dimensions of study quality were included in the review. The first was study design. Twelve of the 15 studies were before and after studies, and 11 were prospective.53,56,65-67,90-93,96,99 Three studies were models,94,97,98 which are not judged with the usual classification of observational studies.100 The second dimension of quality was a numerical indicator that was based on the Drummond checklist.89 The checklist item that was most often met by the studies was incremental analysis. All observational studies met this criterion. Seven studies included outcomes of MEs or AEs. Four observational studies reported on statistical significance, and four included all relevant resources in the analysis.


28

5.2.3 Individual results

The results are summarized (Appendix 6, Table 9) as reported outcomes, reported findings related to the use of resources that are measured in physical (non-dollar) units, and measures of costs. a) Ward-based automated dispensing devices Nurse time allocation The results of three workload studies56,95,96 favoured ADDs. The results of two studies could not be used.90,91 The most frequently used indicator was the amount of nursing time that was spent on medication administration. Because of differences in the reporting of results, it was not possible to obtain a summary measure of the effect. There were varying results in the few studies that had a standard measure (for example, the change in medication-related time per eight-hour shift). Kheniene et al.96 showed a reduction from 4.4 hours to 2.5 hours per workday (43% reduction). Schwarz and Brodowy56 reported a 45% reduction. Poveda Andrés et al.95 reported an 18% reduction. Pharmacy time allocation The indicators for pharmacy workload that were most widely reported were the amount of time that pharmacists and pharmacy technicians spent in preparing to dispense and distribute drugs. There was a variation in this indicator: Wise et al.92 reported that pharmacists’ time fell from 45 minutes to five minutes per eight-hour shift. Kheniene et al.96 reported a reduction in “faculty staff” from 0.8 hours to 0.4 hours with automation. It was unclear, however, whether the staff worked on patient care units. On the other hand, Buchanan reported an increase of 42% with automation.94 Poveda Andrés et al.95 reported a 10% reduction in pharmacy technician time with automation, and Buchanan reported a 7% reduction. Two studies indicated results that occurred in the opposite direction, with increases of 7% to 8%.56,92,96 Schwarz and Brodowy56 reported an increase in pharmacy staff workload in the ward and a reduction in the ICU. It was unclear whether or not staff included pharmacists and pharmacy technicians. The results of one study could not be used.90 Inventory management The costs that were related to inventory management were identified in three studies. Poveda Andrés et al.95 estimated inventory holding cost savings of €16,000, which was the full amount of the reduction in inventories. It is questionable whether this is an appropriate valuation approach. Kheniene et al.96 has an alternative method of measurement. Kheniene et al.96 estimated that the annual cost of an inventory that was held in the manual (ward stock) system was €14,000 greater than that in the automated system. Kheniene et al. assumed that the annual holding cost of excess inventory was 9.5% of the total inventories held, resulting in an excess cost of €1,400. Kheniene et al. also reported a one-time reduction in outdated drugs of €9,086 for the automated system. Dib et al.67 identified that more drugs were dispensed from the pharmacy when the manual system was used, but the cost to the hospital was unclear. Schwarz and Brodowy56 identified a savings of US$108,000 over five years due to reductions in theft and less onerous preparation of narcotic tracking forms using an automated system. Allocation of space No studies assessed the impact of ward-based ADDs on nursing unit space.


29

Captured charges The financial benefits accrued to the hospital instead of the patient. These were measured as captured charges for dispensed drugs. Wise et al.92 and Lee et al.90 reported gains in captured charges. Economic efficiency and financial analysis Four studies reported aggregate financial analyses. Lee et al.,90 Wise et al.,92 Schwarz and Brodowy,56 and Poveda Andrés et al.95 conducted the analyses from the perspective of the hospital. All studies reported financial savings. Lee et al.90 estimated a gain in hospital revenue based on an increase in captured charges from a 28-bed patient care unit. The authors then deducted annual equipment rental and service fees, and inventory stocking costs from the revenue gains, and showed a net gain to the hospital of US$35,000. The study did not include installation, nursing, and other pharmacy costs. In a study of a 26-bed patient care unit, Wise et al.92 accounted for savings in pharmacy time and deducted automation system costs. The result was an annual net gain to the hospital of US$14,480 per patient care unit. The net savings increased to US$80,910 per ward if the gains from recovered charges were included. This may be an overestimate, because not all the captured charges would have been collected. Schwarz and Brodowy56 conducted a five- year study that incorporated annual leasing and service costs, reductions in nursing personnel, and reductions in narcotic pilferage costs. The net savings were US$1 million over the five years. Poveda Andrés et al.,95 in a Spanish study, conducted a cost-benefit analysis over a five-year span. The study included the cost of the equipment (€316,000) and savings from personnel, including nurses and pharmacy assistants (€234,000); inventory reductions (€16,000); and the reduction in the use of medications (€302,318). The net benefit to the hospital was €300,525. a) Pharmacy-based automated dispensing devices Pharmacy time allocation All three workload studies53,65,66 reported that the use of ADD was favourable to pharmacy staff workload. Slee et al.66 reported a 30% reduction in pharmacy technician time. Allocation of space Two studies reported the impact of using pharmacy-based ADDs on drug inventory storage space. Slee et al.66 reported a reduction in floor space by half and dispensary shelving space of 70%. Fitzpatrick et al.65 reported a reduction in the dispensary floor space from 14.3 m2 to 10.2 m2 and of total storage space from 9 m3 to 7 m3. a) Bar-coding for medication dispensing Two studies were conducted on BCMD.97,98 In one of the modelling studies, Maviglia et al.97 recorded a dispensing error rate of 0.19% before automation and 0.07% after implementation. This study only included errors that had the potential to harm patients. Both modelling studies reported cost-benefit analyses from a societal perspective. The benefits were defined in monetary terms as patients’ health benefits resulting from reductions in AEs. Karnon et al.’s98 study was largely based on data from Maviglia et al.’s study, in which the benefits and costs of the BCMD over a five-year span in a 735-bed tertiary hospital were


30

estimated. There were equipment costs, set-up costs, planning costs (all one-time), and recurring costs (operations, labour, lease, and repackaging). The benefits consisted of the costed differences in AEs between the pre- and post-situations. The net benefit over five years, discounted for time value, was US$3.49 million. This is an underestimate, because the pre-implementation costs were omitted from the analysis. Karnon’s results agreed with those of Maviglia — the benefits due to the reduction in AEs outweighed the costs of the intervention. b) Bar-coding for medication administration One observational study reported the nursing time that was associated with BCMA.99 This study showed no change in direct nursing time spent on drug administration and a statistically significant reduction in indirect nursing time (for example, documentation).

5.3 Primary Economic Evaluation: Methods

The economic evidence from the literature was scattered. No author developed a comprehensive picture of the economic impact of automation. As a result, a model was developed, incorporating evidence from various sources to obtain a more complete picture. The basis for this model was written a priori and was followed throughout the process. CADTH’s Guidelines for the Economic Evaluation of Health Technologies87 was followed. 5.3.1 Types of economic evaluation

A cost-consequence analysis involves a measurement of the costs of the alternatives and an enumeration of outcomes and their measurement. If there is more than one type of outcome, these are measured individually (they are not aggregated into an index). A model of the economic impact of increasing the number of unprofiled and profiled, ward-based ADDs in a Canadian hospital was developed. The results from the clinical section were used to obtain the impact on outcomes. The economic papers with clinical data were included in the clinical section if they met the clinical inclusion criteria. The clinical outcomes were measured as differences in error rates. The costs and outcomes were then compared. 5.3.2 Target population

The approach that was taken was to estimate the costs of replacing a manual cassette system with an automated system in the patient care units of a “representative” hospital. The model had a hospital unit component and a hospital component. 5.3.3 Comparators

The two comparators were a manual cassette exchange system and the ward-based ADDs for a surgical or medical patient care unit or the ICU. In the manual cassette system, a 24-hour supply of patient-specific medications is filled by pharmacy staff and placed into medication cassettes or carts. Restocked cassettes are exchanged daily for the empty cassettes on the patient care units. A supply of any new medications ordered after the cassette is filled is sent through a different process until the medication can be added to the patient’s cassette at the next scheduled refill.


31

In the automated scenario, medications are stored in ward-based ADDs. Nurses access the cabinets through a secure log-in and withdraw medications as they are due to be administered. Pharmacy staff restock the cabinet as medications become depleted and add medications as needed. 5.3.4 Perspective

The perspective was that of a hospital. 5.3.5 Effectiveness

The effectiveness was based on the findings of the clinical section of this report. 5.3.6 Time horizon

A five-year time horizon was used in this analysis. 5.3.7 Modelling

The economic model was presented in two stages. First, the costs were estimated. Then, the outcomes from the clinical review were added. The cost-computational model identified the costs for a set of services in a patient care unit and an ICU. The resources were estimated separately for the two scenarios. Information was unavailable on the use of ADDs in other units, such as the operating and emergency rooms, which were excluded from the model. Equipment, software, and initial planning costs were assumed to have a five-year life, which is the time horizon of the model. The manual distribution system did not use equipment, software, and maintenance, and did not require additional planning. All other costs – nursing, pharmacist, and technician time, and pharmaceutical inventories – were evaluated annually for each of the five years. The costs were discounted and summed over the five years. Separate costs were estimated for each of the two interventions. 5.3.8 Valuing outcomes

The outcomes were not in the cost-estimation model, but were obtained from the clinical part of this study. They were presented in physical terms and were used to estimate the consequences. 5.3.9 Resource use and costs

The resources used in distributing drugs to patient care units and to patients in the ICU were equipment and software for the automated system; planning; annual maintenance of the system; nursing medication administration and documentation; drug distribution by pharmacy technicians and pharmacists; and drug inventories. The annual use of these resources was estimated separately for a patient care unit and an ICU. Equipment, software, and initial planning costs were assumed to have a five-year life, so they were amortized over a five-year period. The manual distribution system did not involve the use of equipment, software, and maintenance, and did not require additional planning.


32

a) Data for model In the first component of the model, the hospital unit was a patient care unit (medical or surgical) or an ICU. It was assumed that the patient care unit had 20 beds and the ICU had eight beds. The occupancy rates were assumed to be 90% in each (at any time, the patient care unit had 18 patients and the ICU had seven). In the second component of the model, the costs were analyzed for a hospital of approximately 400 beds and two ICUs. It was assumed that the hospital had 19 patient care units and a total of 16 ICU beds. Appendix 6, Table 10 presents the data and sources for the base case that was used to estimate the costs for automated and manual units. Because of concerns about vendor confidentiality, an estimate of the capital costs of new equipment from purchasers of the equipment was obtained. The types of cabinets that were selected for the model were those used by Capital Health in Edmonton, Alberta: a station with six main and six mini or cubie drawers, with a seven drawer auxiliary unit, with and without patient medication profiles (profiled and unprofiled). Because the brands and models could not be identified, a sensitivity analysis was applied to account for uncertainty in the pricing. The cost of the equipment was annualized over five years, using a 5% discount. The only estimate that was available for planning costs was from Maviglia et al.97 (in a BCMD study); they estimated that planning costs were 60% of capital equipment costs. That analysis, however, was for a hospital-wide drug distribution system, which might be more far-reaching than estimating the planning costs for one patient care unit. Therefore, a lower percentage (30%) was used in the sensitivity analysis. An estimate of the nursing time that was spent on drug administration and documentation per patient was based on data from Baker et al. (Baker et al. Cardinal Health: unpublished data, 2008). In that study of seven hospitals, all had adopted ADDs in at least some patient care units. Alberta wages for a registered nurse101 were applied to these times to derive the drug administration costs per patient day. To estimate the nursing times in non-automated hospitals, the increased nursing time that occurred when automated and non-automated patient care units were compared was applied (Appendix 6, Table 11), averaged over all economic studies in this systematic review. Pharmacist and pharmacy technician times per patient day, in the automated scenario, were based on the average times that were reported in Baker et al.’s study (Baker et al. Cardinal Health: unpublished data, 2008). The results from the economic literature review were used to estimate the difference in times between the automated and manual systems (Appendix 6, Table 11). Alberta wages for pharmacists and technicians were used to derive a cost per patient day.102 One study in the review96 appropriately addressed inventory differences between systems. In that study, however, the comparator was a ward-stock system, where differences in inventories are likely to be observed. In our comparator, a cassette exchange system, drug expiry is unlikely to differ from the automated system.


33

b) Assumptions Given the paucity of information from the economic literature review, it was necessary to make assumptions: All costs — such as those for nursing time, pharmacy staff time, inventory, and maintenance — were the same for a profiled and an unprofiled ADD. The only difference occurred in the acquisition of the equipment and in planning costs. The data that were obtained from studies conducted in the 1990s were applicable today. In our scenarios, 100% of the doses were dispensed using ADDs.

The costs of the pharmacy software (for example, Pyxis® Profile™) and interfaces that were associated with profiled ADDs were excluded, because an estimate of these costs could not be obtained. Other patient care units, such as operating rooms and emergency rooms, were excluded from the model and from the budget impact analysis. Furthermore, the equipment that was included in the model was one station with six main and six mini or cubie drawers, with a seven drawer auxiliary unit. In reality, busy patient care units may require several auxiliary units and towers, which would increase the equipment costs. 5.3.10 Discount rate

A discount of 5% was used for all future costs. This is the rate that is recommended in the CADTH guidelines for economic evaluations. 5.3.11 Sensitivity analysis

Because there were several sources of uncertainty in the model, one-way sensitivity analyses were conducted as appropriate. Given the high degree of uncertainty, it was unnecessary to conduct two-way or multi-way analyses, which are used when one-way confidence intervals are narrow. There may be underlying correlations between independent variables. First, the estimate of capital equipment costs were increased by 10%. We presented the net change in the cost differential if nursing costs were reduced by 45% and 18%, rather than by 38%, as in the base case. Because three studies recorded reductions in nursing time due to automation of approximately 45% (Appendix 6, Table 11), this figure was used in a sensitivity analysis. The smallest reduction in nursing time was 18%, so this figure was also used. Sensitivity analyses were conducted for pharmacist time and pharmacy technician time, which were based on high and low estimates from the literature. Sensitivity analyses were conducted for inventory costs and planning costs for the installation of capital equipment. In the case of inventories, an analysis was done using a value of inventory savings equal to 10% of nursing costs (one-time and annual savings). An estimate of costs was conducted, assuming that capital planning costs were 30% of the value of the equipment, which was half of the amount presented in the Maviglia et al. study (and the base case).97


34

5.4 Primary Economic Evaluation: Results

5.4.1 Analysis and results

The estimated costs for the base case patient care unit analysis, in Canadian dollars, appear in Table 3.

Table 3: Base-Case Analysis (Five-Year Drug Distribution Costs)

Patient care unit Manual $968,000 Unprofiled ADD $816,000 Profiled ADD $840,000 Intensive care unit Manual $353,000 Unprofiled ADD $429,000 Profiled ADD $453,000 400-bed hospital Unprofiled ADD Not applicable Profiled ADD Not applicable

ADD=automatic dispensing device

Unprofiled ADD Using the base case assumptions, the five-year drug distribution costs for the 20-bed patient care unit were $968,000 and $816,000 for the manual and automated units, respectively. The difference was $152,000 in favour of the automated unit. In the ICU, the manual system was less costly. The drug distribution in a non-automated ICU was $76,000 less than in an automated ICU. Profiled ADD Using the base case assumptions, the five-year drug distribution costs for the patient care unit were $968,000 and $840,000 for the manual and automated units, respectively. The difference was $128,000 in favour of the automated unit. In the ICU, the manual system was less costly. The drug distribution in a non-automated ICU was $100,000 less than in an automated ICU. The breakdown of costs for each of the five years appears in Appendix 6, Tables 12 to 15. With a manual system, nursing costs were 68% of the total cost, and pharmacy technician costs were 26% of the total cost. Under automation, equipment costs were approximately 14% of the total, and planning costs were approximately 6%. Nursing costs fell to 50%, and technician costs fell to 13% of the total.


35

a) Costs and Outcomes Profiled ADD With a reduction in dispensing and MEs reported (in the clinical section) for ADDs with profiling, automation was associated with annual savings in a 20-bed patient care unit. The combination of better outcomes with lower costs indicates that automation is a dominant strategy. Unprofiled ADD No clinical studies were retrieved on unprofiled ADDs. The error rates would be expected to be higher for the ADD without profiling than for the profiled ADD. In the ICU, there was a net increase in costs due to automation, because of the large capital expenditures that were incurred for a smaller number of patients. In addition, Schwarz and Brodowy56 showed an increase in reported medication errors in a cardiac ICU. No cost data were available for a cost-effectiveness or a cost-consequences analysis. 5.4.2 Results of the sensitivity analysis

The results of the sensitivity analyses for a patient care unit appear in Appendix 6, Table 16. Unprofiled ADD on patient care units In one instance, the manual system was less costly by $19,000 when nursing time was reduced by 18%. In all other instances, sensitivity analyses showed that an unprofiled ADD was the less costly alternative. Profiled ADD on patient care units The sensitivity analysis for the profiled system indicates that the results were favourable to the automated system, but they were not as robust. For nurse time and pharmacy technician time, the costs were the same or higher for automation than for the manual system.

6 HEALTH SERVICES IMPACT

6.1 Budget Impact

The budget impact analysis for a 400-bed hospital is based on the primary economic analysis. 6.1.1 Up-front costs

The equipment cost for each patient care unit or ICU was $123,000 for an unprofiled ADD and $138,000 for a profiled ADD. The planning costs, at 60% of equipment costs, were $73,800 and $82,800. The total up-front costs were $196,800 and $220,800 per patient care unit or ICU for unprofiled and profiled ADD, respectively.


36

For a 400-bed hospital with approximately nineteen 20-bed patient care units and two eight-bed ICUs, the up-front capital costs would be: for an unprofiled system, $2.5 million for capital equipment costs, and $1.5 million for

planning costs, for a total of approximately $4 million for the hospital. for a profiled system, $2.9 million for capital equipment costs, and $1.7 million for planning

costs, for a total initial outlay of $4.6 million. 6.1.2 Annualized analysis

When the annualized analysis was conducted for an unprofiled ADD, there were savings of approximately $34,000 per patient care unit. Each ICU had additional costs of $17,000 annually. After discounting and adjusting for inflation, there were net savings of $152,000 per patient care unit over a five-year period. Each intensive care unit costs an additional $75,000. For a 400-bed hospital, the savings would be $2.8 million for the medical or surgical patient care units minus increases of approximately $150,000 for the two ICUs. The hospital would achieve five-year savings of $2.7 million for unprofiled devices. The savings are less for ADDs with medication profiles (five-year savings of $2.2 million). The sensitivity analysis has shown that, with our inability to obtain more precise data on equipment costs, there is uncertainty about the results.

6.2 Planning and Implementation Considerations

The automation of parts or all of the services that are related to medication dispensing and administration requires the support of the hospital administrators, nursing staff, pharmacy staff, medical staff, finance staff, informatics staff, human resources staff, and union representatives.103 Each party has its own interests and set of values that will influence decisions; for example, about which technology to implement first. U and Jelincic list the critical factors for success in planning and implementing new technologies: leadership in patient safety; staff buy-in; integration of new technologies with related information systems; critical evaluation before buying technologies; and adequate training and support.104 From a hospital resource use perspective, automation of the dispensing and the administration of medications will have the largest impact on pharmacy and nursing staff. The implementation of these technologies will affect the workflow, requiring standardization of processes beyond the technology (for example, charting forms, medication administration times). Some staff may be reluctant to learn new processes or accept standardization, or may feel that using automation is too onerous and intimidating, while others may be concerned about job security. 6.2.1 Pharmacy staff

In the last 20 years, the responsibilities of the pharmacist have evolved from product-related activities to patient-focused activities. The automation of technical tasks may reduce the time spent by pharmacists on the dispensing and technical activities that they are required to perform. Pharmacists can focus on clinical work, and technicians can take charge of the dispensary.


37

One qualitative study looked at the restructuring of roles and workplace after implementation of dispensing automation at two-tertiary care hospitals in England.105 After implementation of a robot, the dispensary became less dependent on the presence of pharmacists, who had more time to spend in a more clinical role. The pharmacy technicians enjoyed their new visibility, because they interacted with vendors, and the robot enabled them to modernize their roles. Pharmacy assistants had less autonomy in their work and were more dependent on the pharmacy technicians, but, overall, they recognized that the robot was increasing their efficiency.105 One director of pharmacy in England reported that the implementation of automation in his department106 led to a less stressful environment. Staff engagement at the outset was key. It was emphasized that the new process would improve their working life and should not be seen as a threat to their jobs. It was, instead, an opportunity to expand their roles and improve their services. A Canadian paper described case studies that were conducted at three hospitals with ADDs.107 The paper’s purpose was to examine the changing role of pharmacists with the implementation of an ADD. It showed that pharmacists were reluctant to let go of dispensing activities in favour of clinical work. This paper was published in 2000, when clinical pharmacy was gaining momentum. Today, the movement toward the delegation of pharmacist distributive responsibilities to pharmacy technicians is stronger. It is unknown if the same level of reluctance would be encountered. 6.2.2 Nursing staff

Ward-based ADDs, BCMA, and eMARs will have an impact on the medication administration process. Nurses may be reluctant to use the new technologies if they feel that more time is needed to perform their duties. Compliance may be an issue, and workarounds may develop.103 In a study evaluating workarounds to BCMA, Koppel et al.108 identified fifteen types of workarounds, including not scanning the patient identification or the medication bar code. Thirty-one causes of workarounds were reported, including malfunctioning scanners, unreadable medication bar codes or patient identification bracelets, and emergencies. Error alerts were overridden in 4.2% of patients and in 10.3% of medication administrations. An evaluation of nursing compliance with BCMA in a Dutch hospital showed that bar codes were verified in half of the medications that were administered (55.3%).109 The reasons for not using BCMA included a shortage of time and difficulty in scanning the medication bar code. The route of administration, the age of the nurse, and the type of patient care unit influenced the frequency of bar code verification. The number of patients and and the types of medications that were administered did not affect the use of BCMA.109 A survey by Rough, Ludwig, and Wilson showed that, overall, nursing satisfaction improved by 42% after implementation of BCMA.63 Nurse resistance was due to the additional time that was required to scan the patient, the medication, and the nurse. This was not a lasting issue when nurses saw how the system prevented errors. The potential for a higher frequency of errors exists with the use of a new technology. Schwarz and Brodowy56 reported a 70% increase in error rate after deployment of an ADD in an ICU. In


38

this study, nurses could access more than one medication in a matrix drawer and could override the pharmacy profile to obtain certain medications. These issues were identified as possible explanations for the increase in MEs. The selection and implementation of technologies are multidisciplinary and require planning time and costs. One of the economic studies that considered planning costs determined that these costs represented 60% of the capital expenses.97 Furthermore, a change in process may be difficult for staff, and ongoing support is needed. The evaluation of the technology and processes and staff feedback are required during and after implementation, especially because workarounds develop when the new technology hinders workflow. Finally, manufacturers’ awareness of the potential workarounds should result in improved designs that will maximize the error reduction and provide an efficient interface with users.

6.3 Ethical Considerations

The ethics related to automation was not an a priori research question. We conducted a literature search that was designed to retrieve articles on the ethical issues about patient safety in medication and automation. MEDLINE and MEDLINE In-Process & Other Non-Indexed Citations were searched through the Ovid interface, and a parallel search run in ISI Web of Science. The search strategy was developed by an Information Specialist and comprised subject headings and keywords for medication ordering devices (BCMD, BCMA, AMDD, eMAR) and medical errors in hospitals, combined with ethical and legal issues. The search was restricted to English language documents published in the last 10 years. The complete search strategy is available upon request. Few publications were found that met the criteria. This review identified some key ethical issues, including equity and procedural issues. It is possible that there are other important ethical issues, not identified in this review that will require further consideration. 6.3.1 Efficiency versus equity

Patient safety comes at a cost. The benefits and cost of the devices used in the automation of medication dispensing and administration must be weighed against the greater good for society. Buchanan considered the ethics of automated compounding and dispensing.110 In Buchanan’s view, the ethical dilemma faced by a pharmacist when selecting new technologies includes the choice between quality of care and cost of care. When facing such a dilemma, the pharmacist must consider the Acts and Regulations governing his or her profession, the professional code of ethics, and the principles of beneficence (preventing harm by minimizing the risks and maximizing the benefits), non-maleficence (doing no harm), self-determination (respecting the patient), utility (usefulness to patient and society), and justice (treating each person fairly and equitably).4,110 If automating the processes of medication dispensing and administration reduces the chance of errors, then these principles will be met. What constitutes the minimum acceptable risk of errors? The evidence reviewed in this report suggests that, even with automation, total error elimination may never be attained. In some


39

instances, error rates may increase despite automation, such as during system downtime, and in instances where the system is inflexible (for example, non-standard medication orders).56,111 Then, automation alone may be insufficient, and built-in verification procedures may be needed to ensure that errors are minimized. This may have an impact on implementation costs.112 Furthermore, the policy-maker, the clinician, and the patient will each have a different perception of what is an acceptable risk at an acceptable cost. The policy-maker will decide whether or not to provide funds to acquire and use these technologies. The policy-maker will need to weigh the benefits of automation, the optimal allocation of limited resources, and the rights of patients to be treated in an environment where the risk of errors is minimized. Competing priorities are reviewed when making decisions that will have an impact on the greater good of society. The clinician’s primary responsibility is to the patient. Thus, the clinician will be guided by what is best for the patient and what is accepted practice, and not necessarily by what is best from a system’s point of view or by cost. This may place a clinician in conflict with policy-makers and hospital administrators. When a dispensing or medication error occurs with subsequent injury to the patient, the clinician faces the possibility of malpractice litigation. Automation will be considered to be justified, regardless of cost, if the errors result in costly legal defence and damage settlements. 6.3.2 Process or procedural issues

Hospital staff are bound by the ethical duties of confidentiality. Patient confidentiality and privacy may be improved with automation, because a limited number of staff will have access to password-controlled, ward-based ADDs and eMARs. A traditional system has few safeguards to limit access to the medication cupboard or to the paper MAR. The availability of eMARs and their shared use, however, has raised ethical questions. For example, who is authorized to access the MAR, for what purpose, and in which conditions?113 Hospitals may face legal issues that are related to data sharing.114 To limit access may mean that staff may be denied patient information. This may impair the quality of care that is provided to patients. Errors may also occur because of limited access to patient information. Patients may need to be educated about how care could be improved if clinical information was shared with health care providers.114

6.4 Psychosocial Considerations From the Patient Perspective

Whether the dispensing process is manual or automated is not an issue that will evoke the concerns of the patient, unless an error occurs. The patient expects that the correct medication will be given at the correct time, but patients may not be concerned about how the medications reach them. A BCMA will be the one technology that will be obvious to the patient. It may reassure the patient that he or she is being given the correct medication. On the other hand, if the scanner emits a sound signifying an error, the patient or family may realize that an error is about to occur,


40

and this may jeopardize the patient-provider relationship. The patient or family may lose confidence in the nurse and wonder if another error could occur.115 In all cases of errors, transparency is important. Health care providers should inform patients of all errors in medication, including those indirectly affecting health or treatment, those remedied by appropriate action, and near misses.116

7 DISCUSSION

7.1 Summary of Results

7.1.1 Clinical review

The goal of the clinical review was to examine the impact of using automated technologies in the dispensing and administration of medications on the outcomes of errors, ADEs, morbidity, and mortality. In the search for systematic reviews, no reports that met the eligibility criteria were retrieved. Literature searches for the previous 16 years revealed a total of 30 studies that were included in the analysis. The reported outcomes were MEs, medication administration errors, dispensing errors, ADE errors, near misses or potential ADEs, and preventable ADEs. The definitions that were used to describe these events were inconsistent among studies. Errors were counted using different methods, including direct observation, incident reports, audits, and automation. The use of some methods may lead to an underestimation of the error rate. Study duration ranged from one month to several years in the studies that reported these data. Most studies were conducted among adult patients in tertiary care teaching or general hospitals. The comparators that were described included manual or traditional drug distribution systems. High-quality evidence is lacking. All studies that were retrieved in the literature search used an observational study design. Not all studies reported statistical significance. a) Pharmacy-based automated dispensing devices All seven studies on pharmacy-based ADDs reported a reduction in filling or dispensing errors. Two studies were published in the early 1990s. The intervention that was studied, the ATC-212™, is no longer available for purchase. The other five studies were conducted in the UK using German-made, original-pack dispensing systems that are only available in Europe. b) Ward-based automated dispensing devices Two US studies on ward-based ADDs with profiling reported 34% and 38% reductions in MEs, and a 70% increase in errors in the ICU. One study reported a decrease of 29% in dispensing errors. An earlier version of the MedStation™ Rx was used in all studies. The error rate may have been higher or errors not detected if an unprofiled ADD had been evaluated. A study that was conducted in Saudi Arabia did not describe the intervention and comparator. It showed a decrease of 37% in medication AEs.


41

c) Bar-coding for medication dispensing Three US before and after studies examined the effect of using carousels on filling errors, dispensing errors, or ADEs. Relative risk reductions ranged from 15% to 96%. There was a 9% increase in dispensing errors for first or missing medication doses in one study, and a 2.8-fold increase in life-threatening ADEs due to dispensing errors in another study. d) Bar-coding for medication administration Medication and blood and blood product studies were separated in this review to ensure appropriate comparisons. Drugs Seven before and after studies compared standard practice and paper MARs to various BCMA products. All were conducted in the US. Four studies reported the outcomes of medication administration errors. One study reported an 18% increase in medication administration errors, and the other three showed a decrease in administration errors, with RRR ranging from 78% to 87%. Near misses are easy to measure with the use of BCMA. Yet, only one study included this outcome. It showed that near misses occurred in 3.2% of doses after the implementation of BCMA. Two studies showed that total MEs may be reduced by 70% to 86%. Another study showed no difference. Blood and blood products Three studies examined the impact of BCMA on the administration of blood and blood products. When compared to standard procedures, BCMA prevented one near miss among 50 transfusions in one study. A historical cohort study reported that BCMA was more likely to prevent errors than the manual system. The third study noted a decrease in labelling errors. e) Multiple technologies Six studies took a system’s approach and evaluated several technologies that were implemented simultaneously. Ward-based automated dispensing devices and bar-coding for medication administration One study showed a 99% decrease in dispensing errors when a ward-based ADD and BCMA replaced a manual system. Although using the BCMA did not prevent the wrong medications from being administered, it did prevent the administration to the wrong patients by 90%. Overall, the ME rate was reduced by 10%. Bar-coding for medication administration and eMARs Four US studies, one of which was conducted in neonates, examined BCMA and eMARs. Two studies reported reductions in MEs of 44% and 80%. The study on neonates reported a statistically significant increase of 15%, mostly due to wrong-time errors and a statistically significant reduction in preventable ADEs of 47%. One study reported 80 near misses in one month post-implementation.


42

One study showed a non-statistically significant decrease of 24% in medication administration errors in a cardiac telemetry unit, and a statistically significant decrease of 36% in medication administration errors in a medical-surgical unit. Ward-based ADDs, bar-coding for medication administration, and eMARs An inclusive system of ADDs, BCMA, and eMARs was evaluated in a two-week prospective before and after study conducted in the UK. Medication administration errors were statistically significantly reduced by 48%. The errors were also judged to be less severe post-implementation of the integrated system. 7.1.2 Economic review

A systematic review of available studies on the economics of automation of medication dispensing and administration in hospitals was conducted. Of 15 papers that were reviewed, nine were on ward-based ADDs (three on profiled ADDs), three on pharmacy-based ADDs, two on BCMD, and one on BCMA. There is evidence that with ADDs, nursing time is saved. There is less evidence about pharmacy time. Less pharmacy space is needed when using pharmacy-based ADDs. There is less evidence on inventory costs. Because there was conflicting evidence on pharmaceutical workload, a financial analysis would provide more information to assess the monetary implications of offsetting factors. All financial analyses indicated that overall, there would be savings to the hospital. Most studies had methodological limitations. In the studies that were not models, there was an absence of statistical tests of significance. This made it difficult to draw conclusions. Some of the studies on workload, especially pharmacy workload, showed mixed results (reductions and increases in times spent on different tasks). A costing analysis would have helped to resolve the issue of overall savings. On the other hand, there were prospective and observational studies in which data collection was more detailed. Finally, many costs were excluded from the studies (for example, system implementation costs), and there were few studies that were comprehensive in costing. None of the studies looked at the clinical significance of MEs or the downstream costs.

Table 4: Findings From the Included Economic Studies Technology Resources Measured

(Number of Studies) Range of Findings

Nurse time allocation (3) ↓18% to ↓45% Pharmacy time allocation (5) ↓88% to ↑42% Inventory management (3) Costs greater with manual system Captured charges (2) Gains in captured charges

Profiled, ward-based ADD

Economic efficiency and financial analysis (4)

Benefits exceed costs

Pharmacy time allocation (3) Reduction in staff time Pharmacy-based ADD Allocation of space (2) Reduction in space

BCMD (carousels) Cost-benefit analyses (2) Net benefit over 5 years US$3.49 million BCMA Nursing time (1) No change in time spent on drug

administration

ADD=automatic dispensing device; BCMA=bar code medication administration; BCMD=bar code medication dispensing


43

7.1.3 Economic evaluation

An economic model was designed to explain the difference in total patient care units and hospital costs between a scenario where there was a manual distribution system (with medication cassettes) compared with an unprofiled or profiled ward-based ADD. The differences were based on the studies in the literature review. They showed that automated distribution was less costly. A sensitivity analysis showed, however, that these results were sensitive to the savings in nurses’ time. Even in the case where there were net increases in costs due to automation, a reduction in errors produces a better outcome, at a higher cost. The magnitudes of the change in costs and the change in outcomes become important in reaching a policy conclusion. A high cost per unit of improved outcome would be a factor that would be considered in policy-making. The use of the outcome “error reduction,” however, may be inadequate to obtain a policy conclusion. The implications of the errors need to be identified, and none of the studies on ward-based ADDs provided such information.

Table 5: Economic Model and Budget Impact Base-Case Analysis

Budget Impact Analysis

Five-Year Drug

Distribution Costs

Up-Front Equipment

Costs

Planning Costs

Total Up-Front Costs

Annualized Savings

Net Savings Over

Five Years

Patient care unit Manual $968,000 0 0 0 0 0 Unprofiled ADD

$816,000 $123,000 $73,800 $196,800 −$34,000 −$152,000

Profiled ADD

$840,000 $138,000 $82,800 $220,800 −$34,000 −$128,000

Intensive care unit Manual $353,000 0 0 0 0 0 Unprofiled ADD

$429,000 $123,000 $73,800 $196,800 $17,000 $75,000

Profiled ADD

$453,000 $138,000 $82,800 $220,800 $17,000 100,000

400-bed hospital Unprofiled ADD

Not applicable

$2.5 million $1.5 million $4.0 million Not applicable

$2.7 million

Profiled ADD

Not applicable

$2.9 million $1.7 million $4.6 million Not applicable

$2.2 million

ADD=automatic dispensing device


44

7.2 Strengths and Weaknesses of This Assessment


This clinical review is a comprehensive examination of studies on the automation of the medication dispensing and administration processes. The studies met the inclusion criteria. The methods were robust and met CADTH’s standards for systematic reviews. The clinical review did not address the benefits of technologies that were used in the ordering of medication (for example, CPOE and CDSS). Most preventable ADEs occur at this stage.117 A survey that was conducted to define the scope of this project showed that stakeholders were most interested in technologies that were used to automate the dispensing and administration processes (CP, unpublished observations, 2008). Moreover, there are several systematic reviews on CPOE and CDSS.118-123 The strength of the conclusions of systematic reviews depends on the quality of the primary literature. All the included studies were observational and susceptible to selection bias because of non-randomized designs. The measurement of the benefits of automation in hospitals may require an experimental design other than randomized controlled trials or observational studies. Few studies adjusted for confounders. The reduction in errors may, at least, be partially explained by factors other than automation; for example, physical reconfiguration of the pharmacy department or patient care units, or staff training and sensitization to patient safety, leading to changes in work practices. Pharmacy and nursing staff received additional training before the intervention began, and the technologies were generally evaluated when they were first implemented. It is unknown if the positive impact of the technologies would be sustained. The study characteristics were too different to allow meta-analysis. None of the studies were randomized controlled trials. There was a lack of uniformity in the definitions of dispensing errors, medication administration errors, total MEs, and ADEs. The error rates were calculated differently. The studies were conducted in various hospital settings and patient care units. The interventions and comparators were not always described. The data were collected over different periods, ranging from a few weeks to a few years. The technologies carry a potential for inappropriate and inefficient use.62 Different types of medication errors may be introduced. For example, an overreliance on a machine may lead staff to complete their tasks with minimal reflection and less checking.123,124 Detection and performance biases may be issues. Different methods for measuring errors were used in the studies. These methods may capture only a small proportion of errors. The error detection capacity varies between the methods: Voluntary incident reporting is an insensitive method of collecting data on MEs,125 because it

may lead to under-reporting. Practitioners will not always report their own errors or their peers’ errors, or they may not recognize the errors.126 The use of incident reports leads to under-reporting by a factor of 50 to 1,000, compared to direct observation.51


45

Direct observation is a more sensitive method of collecting data on MEs than the other types of reporting.84,125,127 The use of this method, however, may lead to the Hawthorne effect. Someone who knows that he or she is being evaluated may change his or her practice. This can influence the outcome that is associated with the intervention being evaluated.128 Direct observation is prone to observer bias and will give inconsistent results. It is labour- and time-intensive.84,125

Daily chart reviews, incident reporting using logs, and nurse-solicited information were studied by Bates, Leape, and Petrycki, who compared these three methods of identifying incidents.129 Fifty-nine per cent of incidents were identified using solicited reports; 37% were found using logs; 22% were found using verbal reports; and 67% were found during chart reviews. Chart reviews were thought to be a better method of identifying incidents, but this technique is expensive and labour-intensive, and requires training. Despite training, there can be a variation in the reviewers’ ability to extract data. Also, inadequate documentation may be a problem.127

Automation, through computerized monitoring, is a low-cost alternative to error detection. In theory, it should lead to a better detection of errors; but an increase in errors post-implementation was not observed, as would be expected.

Some have argued that the use of triggers to identify errors may improve data capture.125 These triggers are signals that are identified through chart reviews or electronically. An example of a trigger is the prescription of naloxone (an antidote for narcotics), which would indicate that a narcotic overdose has occurred.125

The decrease or increase in error rate was expressed as relative risk reduction or relative risk increase. These were chosen as a common measure of effects, to facilitate the comparison between studies. The RRR and RRI do not discriminate between large and small absolute differences (absolute numbers are reported in the tables). Some of the studies reported few errors or events, and others had small sample sizes. Therefore, statistical significance was difficult to achieve and, hence, not reported. Furthermore, the studies did not take into account the baseline risk of participants or the context in which machines were deployed. For example, the rates and severity of errors that were observed in an ICU are not comparable to what may be observed in a general medicine patient care unit, given the differences in the type and intensity of care. A review of studies20 showed that the error rates varied by type of medication. For example, higher error rates occurred among patients receiving medication administered parenterally, and patients receiving antibiotics, cardiac drugs, and cancer drugs. Similarly, the rates of ADEs were higher in internal medicine, and geriatric patient care units and ICUs, than in general medical patient care units. Most studies reported dispensing error, administration error, or ME rates. A few studies (that were typically underpowered) reported preventable ADEs and ADEs. Hayward and Hofer showed that clinicians had difficulty in agreeing whether or not an error had occurred and whether it was preventable.130 These authors presented a case study to show that the link between medical errors and AEs is complex.131 Hence, MEs may be an inappropriate proxy for adverse clinical outcomes. The funding sources were reported in seven studies.


46

7.2.2 Economic review

This economic review is a comprehensive examination of studies on the automation of the medication dispensing and administration processes. The studies met the inclusion criteria. The methods were robust and met CADTH’s standards for systematic reviews. There are limitations. First, the quality of the studies is an issue. It is difficult to set up an ideally controlled study when assessing organizational change. As a result, simple before and after comparisons were conducted and, in general, the quality was poor. The definitions and resources varied from study to study. Seldom could results be compared between studies. The nursing costs, for example, were measured differently in the studies, and key elements, such as the time to deliver drugs, were often missing. When nursing time was directly measured, it was not reported the same way. For example, one study reported the percentage of tasks, but did not report actual time. It is not possible to conduct an economic analysis using such data. Data on planning costs were determined to be critical by Maviglia et al.97 Yet they were missing from all other studies. The inventory differences were not examined. The outcomes that were obtained from the clinical and economic reviews were expressed as error reductions. There was no information available to extrapolate the outcomes to those that are more commonly used in economic studies, such as reductions in specific conditions or quality- adjusted life-years. With such data, a better estimate of the cost-effectiveness of automation could be provided. The statistical significance was almost never measured. It then becomes difficult to interpret the scientific and decision-making implications of the observed differences. Because of these differences across the economic studies, it was difficult to draw conclusions about the impact of automation. The reduction in errors has received attention from policy- makers. Automation in drug distribution is a technology that can help achieve such reductions. Yet the quality of studies did not mirror the importance of the topic. 7.2.3 Economic evaluation

The economic analysis focused on ADDs for patient care units and the ICU, because supporting data were only available for ward-based ADD in medical-surgical and intensive care units. This economic analysis is the only one that has been completed for Canadian hospitals. The technology is relevant in Canada. Yet the analysis was limited by several factors. There was no consistent method that was followed by the various studies reviewed. In each study, the cost items that were considered were unique, the method of measuring economic variables varied, and the measurement of outcomes was often missing. The model was limited by the poor quality of resources and cost data available in the literature. The resources that would be affected by automation have been listed. These include equipment costs, planning costs, inventories, nurse time, and pharmacy staff time. No prior study included


47

all of these. For example, although planning costs were deemed to be important by Maviglia et al.97 in their analysis of hospital automated unit-dosing systems, no study of automated distribution measured pre-implementation planning costs. This persisted with almost every variable. As a result, the model only estimates a portion of all costs. As another example, no study included inventories. It may be that for the alternatives that we considered ― manual medication cassette and automation ― no differences would have been observed. There was no direct evidence of this. The equipment configuration was simple: one main station with an auxiliary unit. Busy medical patient care units and ICUs may need additional modular stations to allow for a greater number of items to be stored. These different equipment configurations would have different costs and would have an impact on the cost estimates. The measurement of economic variables differed between studies. An economic analysis should include all downstream costs, yet in almost no case were such costs available. As a result, they had to be estimated. Some studies only reported differences in personnel, without identifying the personnel in the base case. Most studies did not report nurse and pharmacy technician time, and unit costs, which are used for comparing differences. When differences exist between health care strategies, an economic evaluation should have data on outcomes and costs to be useful for health care decision-making. Few studies reported both. Information from the clinical studies was used in the model, yet the circumstances in which the clinical studies were done differed from those in the studies providing the costs. Thus, there is an incomplete estimate of the costs and outcomes of automation.

7.3 Generalizability of Findings


The setting, year, and country where the studies were conducted may affect generalizability. The setting is a consideration when determining whether or not the results are transferable to a pharmacy department or patient care unit. Studies were conducted in different types of hospitals (from community general hospitals to university-affiliated tertiary care hospitals) and patient care units (from medical and surgical patient care units to critical care), and results will only be applicable to similar settings. Twenty-three studies were published after the year 2000. Seven other studies were published between 1992 and 1995 (two for pharmacy-based ADDs, three for ward-based ADDs, and two for BCMA). Furthermore, some of the technologies are no longer available (for example, ATC-212™), another model is available (for example, MedStation™ Rx), and in other instances, the investigators did not provide the name and supplier of the technologies or assessed customized systems. The clinical conclusions of this report are made in the Canadian context, although 21 studies were conducted in the US.


48

7.3.2 Economic review

The location of the studies has an impact on generalizability. The results of most of the target variables in the analyses — physical resources, costs, health outcomes — are transferrable from country to country. Some financial variables are not. In the US, hospital revenues are collected through billings (charges) to individual payers. The use of better inventory systems enables hospitals to recover charges that would otherwise be lost. Some of the US studies identified recovered charges as a benefit of automation. These benefits would not be generalizable to other countries, including Canada and most of Europe, where revenues are not tied to individual drug billings. 7.3.3 Economic model

The conclusions of the model and budget impact analysis are generalizable to hospitals wanting to implement profiled and unprofiled ward-based ADDs on patient care units and in the ICU. There was insufficient information to model ADDs in the operating and emergency rooms. As well, models for the other technologies (pharmacy-based ADDs, BCMD, and BCMA) could not be designed. The economic model of ward-based ADDs is based on nine studies, five of which were published in the 1990s. Although the pertinence of using these data in the model could be questioned, experts (PL, JH) indicated that the system changes to ward-based ADDs have been minimal (for example, added features such as log-in management for biometric identification, interface designs that simplify replenishment tasks, options for drawer sizes, and additional safety features). The changes likely have had limited impact on nursing or pharmacy resources (PL, JH).

7.4 Knowledge Gaps


The studies focused on dispensing, administration, or ME rates. Few studies reported on the outcomes of ADEs and potential ADEs (near misses). More research is needed to better evaluate the effect of technologies and the association between their use and ADE reduction. Furthermore, researchers should strive to perform studies with better internal validity, and statistical reporting. Finally, assessing how the error ascertainment methods could be used conjointly is necessary to develop one overall measure of errors.132 7.4.2 Economic review

Randomized controlled trials, which are the gold standard in clinical investigation, are poorly suited for the investigation of organizational changes. The tools of organizational observation and evaluation, which are used in sociology and business, are better suited to study automation. Yet in almost no study were these methods used. Most studies seem to have been designed by clinical specialists or non-methodologists.


49

8 CONCLUSIONS

From a clinical perspective, based on studies of lower internal validity, the use of bar-coding for medication dispensing systems, bar-coding for medication administration systems, and the simultaneous use of technologies reduced the risk of dispensing or medication errors in hospitals. Studies of previous models of profiled, ward-based automatic dispensing devices also reported benefits. One study showed an increase in error rate in a cardiac intensive care unit. We cannot reliably estimate the magnitude of benefit from pharmacy-based automatic dispensing devices because the studies were conducted using equipment that is no longer available for purchase or the studies used devices available in Europe. We cannot reliably estimate how automation affects the rate of potential adverse drug events, adverse drug events, morbidity, and mortality because these outcomes were not measured in most studies. The implementation of a ward-based automatic dispensing device in a hospital can reduce costs while reducing error rates. This conclusion is only valid for medical-surgical patient care units. The implementation of ward-based automatic dispensing devices in the intensive care unit results in a net increase in costs. This is due to the large capital expenditures that are incurred for a small number of patients. There is also uncertainty about the clinical impact of this type of automation in intensive care. The results are more robust for unprofiled rather than profiled systems. We cannot reliably estimate the economic impact of other technologies because of gaps in knowledge. Up-front capital costs between $4 million and $4.6 million would be expected during the implementation of a ward-based ADD in a 400-bed acute care general hospital, depending on whether or not an unprofiled or profiled system is purchased. Over five years of operation, however, the hospital would be expected to achieve savings in the order of $2.2 million (if purchasing an ADD with medication profiles) or $2.7 million (for an unprofiled system). This projection is sensitive to assumptions about equipment costs.

9 REFERENCES 1. Baker GR, Norton P. Patient Safety and Healthcare Error in the Canadian Healthcare System: A Systematic

Review and Analysis of Leading Practices in Canada with Reference to Key Initiatives Elsewhere. Ottawa: Health Canada; 2008. Available: http://www.hc-sc.gc.ca/hcs-sss/alt_formats/hpb-dgps/pdf/pubs/2001-patient-securit-rev-exam/2001-patient-securit-rev-exam-eng.pdf (accessed 2008 Nov 27).

2. Canadian Patient Safety Institute. Edmonton: Canadian Patient Safety Institute; 2008. Available: http://www.patientsafetyinstitute.ca/ (accessed 2008 Dec 9).

3. Safer Healthcare Now! Toronto: Safer Healthcare Now; 2008. Available: http://www.saferhealthcarenow.ca/ (accessed 2008 Dec 9).

4. Clark PA. Medication errors in family practice, in hospitals and after discharge from the hospital: an ethical analysis. J Law Med Ethics 2004;32(2):349-57.

5. Aspden P, Wolcott J, Bootman JL, Cronenwett JL, editors, Committee on Identifying and Preventing Medication Errors. Preventing Medication Errors [Quality Chasm Series]. Washington: National Academies Press; 2007. Available: http://www.nap.edu/catalog.php?record_id=11623 (accessed 2009 Mar 3).


50

6. The Canadian patient safety dictionary. Edmonton (AB): Canadian Patient Safety Institute; 2003. Available: http://rcpsc.medical.org/publications/PatientSafetyDictionary_e.pdf (accessed 2008 Jun 4).

7. AHRQ patient safety network glossary. Rockville (MD): Agency for Healthcare Research and Quality (AHRQ); 2008. Available: http://www.psnet.ahrq.gov/glossary.aspx (accessed 2008 Dec 1).

8. Leape LL. Preventing adverse drug events. Am J Health Syst Pharm 1995;52(4):379-82.

9. Bates DW, Teich JM, Lee J, Seger D, Kuperman GJ, Ma'Luf N, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999;6(4):313-21.

10. Ackroyd-Stolarz S, Hartnell N, Mackinnon NJ. Demystifying medication safety: making sense of the terminology. Res Social Adm Pharm 2006;2(2):280-9.

11. Leape LL, Bates DW, Cullen DJ, Cooper J, Demonaco HJ, Gallivan T, et al. Systems analysis of adverse drug events. JAMA 1995;274(1):35-43.

12. American Society of Hospital Pharmacists. ASHP guidelines on preventing medicaton errors in hospitals. Am J Hosp Pharm 1993;50:305-14. Available: http://www.ashp.org/DocLibrary/BestPractices/ASHPGuidelinesPreventingMedicationErrorsinHospitals.aspx (accessed 2009 Jan 19).

13. Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, Lawthers AG, et al. Incidence of adverse events and negligence in hospitalized patients. Results of the Harvard Medical Practice Study I. N Engl J Med 1991;324(6):370-6.

14. Leape LL, Brennan TA, Laird N, Lawthers AG, Localio AR, Barnes BA, et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 1991;324(6):377-84.

15. Wilson RM, Runciman WB, Gibberd RW, Harrison BT, Newby L, Hamilton JD. The Quality in Australian Health Care Study. Med J Aust 1995;163(9):458-71.

16. Thomas EJ, Studdert DM, Burstin HR, Orav EJ, Zeena T, Williams EJ, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care 2000;38(3):261-71.

17. Davis P, Lay-Yee R, Briant R, Ali W, Scott A, Schug S. Adverse events in New Zealand public hospitals I: occurrence and impact. N Z Med J 2002;115(1167):U271. Available: http://www.nzma.org.nz/journal/115-1167/271/content.pdf (accessed 2004 Jun 9).

18. Davis P, Lay-Yee R, Briant R, Ali W, Scott A, Schug S. Adverse events in New Zealand public hospitals II: preventability and clinical context. N Z Med J 2003;116(1183):U624. Available: http://www.nzma.org.nz/journal/116-1183/624/content.pdf (accessed 2004 Jun 9).

19. Baker GR, Norton PG, Flintoft V, Blais R, Brown A, Cox J, et al. The Canadian Adverse Events Study: the incidence of adverse events among hospital patients in Canada. CMAJ 2004;170(11):1678-86. Available: http://www.cmaj.ca/cgi/reprint/170/11/1678 (accessed 2007 Feb 22).

20. Krähenbühl-Melcher A, Schlienger R, Lampert M, Haschke M, Drewe J, Krähenbuhl S. Drug-related problems in hospitals: a review of the recent literature. Drug Saf 2007;30(5):379-407.

21. Kohn LT, Corrigan JM, Donaldson MS, editors, Committee on Quality of Health Care in America. To err is human: building a safer health system [Quality Chasm Series]. Washington: National Academies Press; 2000 Jan. Available: http://www.nap.edu/catalog.php?record_id=9728 (accessed 2009 Mar 3).


51

22. Longo DR, Hewett JE, Ge B, Schubert S. The long road to patient safety: a status report on patient safety systems. JAMA 2005;294(22):2858-65. Available: http://jama.ama-assn.org/cgi/reprint/294/22/2858 (accessed 2008 Jan 2).

23. Committee on Quality of Health Care in America. Crossing the Quality Chasm: a New Health System for the 21st Century [Quality Chasm Series]. Washington: National Academies Press; 2001. Available: http://www.nap.edu/catalog.php?record_id=10027 (accessed 2009 Mar 3).

24. Aspden P, Corrigan JM, Wolcott J, Erickson SM, editors, Committee on Data Standards for Patient Safety. Patient Safety: Achieving a New Standard for Care [Quality Chasm Series]. Washington: National Academies Press; 2004. Available: http://www.nap.edu/catalog.php?record_id=10863 (accessed 2009 Mar 3).

25. Cina JL, Gandhi TK, Churchill W, Fanikos J, McCrea M, Mitton P, et al. How many hospital pharmacy medication dispensing errors go undetected? Jt Comm J Qual Patient Saf 2006;32(2):73-80.

26. Miller MR, Robinson KA, Lubomski LH, Rinke ML, Pronovost PJ. Medication errors in paediatric care: a systematic review of epidemiology and an evaluation of evidence supporting reduction strategy recommendations. Qual Saf Health Care 2007;16(2):116-26. Available: http://qshc.bmj.com/cgi/reprint/16/2/116 (accessed 2008 Nov 7).

27. Bates DW, Spell N, Cullen DJ, Burdick E, Laird N, Petersen LA, et al. The costs of adverse drug events in hospitalized patients. Adverse Drug Events Prevention Study Group. JAMA 1997;277(4):307-11.

28. Canadian Society of Hospital Pharmacists. Medication Safety and Drug Use Management Enhanced by Drug Distribution: Background Paper. Ottawa: CSHP; 2008 Jun. Available: http://www.cshp.ca/members/dmsgetfile.asp?file=%7BD2119190-7224-43F1-9345-CE5156451DB4%7D (accessed 2008 Dec 1).

29. Hall K, Wilgosh C, editors. Hospital Pharmacy in Canada: 2007 / 2008 Report. Toronto: Eli Lilly Canada; 2008. Available: http://www.lillyhospitalsurvey.ca/hpc2/content/rep_2008_toc.asp (accessed 2009 Mar 17).

30. Murray MD. Automated Medication Dispensing Devices. In: Shojanja KG, Duncan BW, McDonald KM, Wachter RM, editors. Making Health Care Safer: a Critical Analysis of Patient Safety Practices [Evidence Report/Technology Assessment no. 43]. Rockville (MD): Agency for Healthcare Research and Quality; 2001. Prepared by the University of California at San Francisco (UCSF)-Stanford University Evidence-based Practice Center, for the Agency for Healthcare Research and Quality, Contract No. 290-97-0013. AHRQ Publication 01-E058. Available: http://www.ahrq.gov/Clinic/ptsafety/chap11.htm (accessed 2008 Sep 1).

31. Posey LM. Drug Distribution: Options for Enhancing Efficiency. Consult Pharm 1996;11(11). Available: http://www.ascp.com/publications/tcp/1996/nov/drugdistrib.html (accessed 2008 Sep 24).

32. McKesson Corporation. San Francisco: McKesson; 2008. Available: http://www.mckesson.com (accessed 2008 Oct 22).

33. AutoMed: Efficiency Pharmacy Automation Solutions. Buffalo Grove (IL): AmerisourceBergen Technology Group; 2008. Available: http://www.automedrx.com/pages/products.html (accessed 2008 Oct 22).

34. Swisslog. Buchs (Switzerland): Swisslog; 2008. Available: http://www.swisslog.com (accessed 2008 Oct 22).

35. RIVA: Intelligent Hospital Systems (IH Systems). Winnipeg: Intelligent Hospital Systems; 2009. Available: http://www.rivasystem.com/ (accessed 2009 Mar 31).

36. Robotic IV Automation (RIVA). Winnipeg: Manrex; 2009. Available: http://www.manrex.com/pc_automation2.html (accessed 2009 Mar 31).


52

37. Cardinal Health. Dublin (OH): Cardinal Health; 2008. Available: http://www.cardinal.com/ (accessed 2008 Oct 22).

38. Purchasing a decentralized medication management system. Health Devices 2006;35(3):96-101.

39. Kuiper SA, McCreadie SR, Mitchell JF, Stevenson JG. Medication errors in inpatient pharmacy operations and technologies for improvement. Am J Health Syst Pharm 2007;64(9):955-9.

40. The Institute for Safe Medication Practices Canada. Automated dispensing cabinets in the Canadian enviroment. ISMP Canada Safety Bulletin 2007;7(3):1-3. Available: http://www.ismp-canada.org/download/ISMPCSB2007-03ADCs.pdf.

41. Institute for Safe Medication Practices (ISMP) guidance on the interdisciplinary safe use of automated dispensing cabinets. Institute for Safe Medication Practices (ISMP); 2008. Available: http://www.ismp.org/Tools/guidelines/ADC_Guidelines_Final.pdf.

42. Colen HBB, Neuenschwander M, Neef C, Krabbendam KJ. Using automated dispensing machines to improve medication safety. EJHP Pract 2008;12(5):71-3. Available: http://www.eahp.eu/EJHP/EJHP-Practice/Issue-5-2006/Trends-in-Drug-Distribution/Using-automated-dispensing-machines-to-improve-medication-safety (accessed 2008 Dec 1).

43. Coyle GA, Heinen M. Scan your way to a comprehensive electronic medical record. Augment medication administration accuracy and increase documentation efficiency with bar coding technology. Nurs Manage 2002;33(12):56, 58-9.

44. Omnicell. Mountain View (CA): Omnicell; 2008. Available: http://www.omnicell.com/ (accessed 2008 Oct 22).

45. SmartCart™. Beachwood (OH): MDG Medical; 2008. Available: http://www.mdgmedical.com/our_solutions/smart_cart?pagename=systems (accessed 2008 Oct 22).

46. Saginur M, Graham ID, Forster AJ, Boucher M, Wells GA. The uptake of technologies designed to influence medication safety in Canadian hospitals. J Eval Clin Pract 2008;14(1):27-35.

47. Shea B, Dubé C, Moher D. Assessing the quality of reports of systematic reviews: the QUOROM statement compared to other tools. In: Egger M, Smith GD, Altman DG, editors. Systematic reviews in health care: meta-analysis in context. 2nd ed. London: BMJ Publishing Group; 2001. p.122-39.

48. Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, et al. Assessing the quality of reports of randomized clinical trials: is blinding necessary? Control Clin Trials 1996;17(1):1-12.

49. Schulz KF, Grimes DA. Allocation concealment in randomised trials: defending against deciphering. Lancet 2002;359(9306):614-8.

50. Wells GA, Shea B, O'Connell D, Peterson J, Welch V, Losos M, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Ottawa: Ottawa Health Research Institute; 1999. Available: http://www.ohri.ca/programs/clinical_epidemiology/oxford.htm (accessed 2004 Feb 9).

51. Allan EL, Barker KN. Fundamentals of medication error research. Am J Hosp Pharm 1990;47(3):555-71.

52. Confidence interval analysis [computer program]. Version 2.1.0 for Windows. Southampton (UK): University of Southampton; 2000.

53. Klein EG, Santora JA, Pascale PM, Kitrenos JG. Medication cart-filling time, accuracy, and cost with an automated dispensing system. Am J Hosp Pharm 1994;51(9):1193-6.


53

54. Borel JM, Rascati KL. Effect of an automated, nursing unit-based drug-dispensing device on medication errors. Am J Health Syst Pharm 1995;52(17):1875-9.

55. Ray MD, Aldrich LT, Lew PJ. Experience with an automated point-of-use unit-dose drug distribution system. Hosp Pharm 1995;30(1):18, 20-3, 27-30.

56. Schwarz HO, Brodowy BA. Implementation and evaluation of an automated dispensing system. Am J Health Syst Pharm 1995;52(8):823-8.

57. Johnson CL, Carlson RA, Tucker CL, Willette C. Using BCMA software to improve patient safety in Veterans Administration Medical Centers. J Healthc Inf Manag 2002;16(1):46-51. Available: http://www.himss.org/content/files/ambulatorydocs/BCMASoftwareToImprovePatientSafety.pdf (accessed 2008 Jul 9).

58. Low DK, Belcher J, V. Reporting medication errors through computerized medication administration. Comput Inform Nurs 2002;20(5):178-83.

59. Brown SJ, Cioffi MA, Schinella P, Shaw A. Evaluation of the impact of a bedside terminal system in a rapidly changing community hospital. Comput Nurs 1995;13(6):280-4.

60. Puckett F. Medication-management component of a point-of-care information system. Am J Health Syst Pharm 1995;52(12):1305-9.

61. Kratz K, Thygesen C. A comparison of the accuracy of unit dose cart fill with the Baxter ATC-212 computerized system and manual filling. Hosp Pharm 1992;27(1):19-20,22.

62. Oren E, Shaffer ER, Guglielmo BJ. Impact of emerging technologies on medication errors and adverse drug events. Am J Health Syst Pharm 2003;60(14):1447-58.

63. Rough S, Ludwig B, Wilson E. ASHP best practices award in health system pharmacy: award paper abstracts. Improving the medication administration process: the impact of point of care bar code medication scanning technology. Bethesda (MD): American Society of Health-System Pharmacists; 2003. Available: http://www.ashpadvantage.com/bestpractices/2003_papers/rough.htm (accessed 2008 Aug 19).

64. Franklin BD, O'Grady K, Voncina L, Popoola J, Jacklin A. An evaluation of two automated dispensing machines in UK hospital pharmacy. Int J Pharm Pract 2008;16:47-53. Available: http://ijpp.publisher.ingentaconnect.com/content/rpsgb/ijpp/2008/00000016/00000001/art00009 (accessed 2009 Jan 19).

65. Fitzpatrick R, Cooke P, Southall C, Kauldhar K, Waters P. Evaluation of an automated dispensing system in a hospital pharmacy dispensary. Pharm J 2005;274(7354):763-5.

66. Slee A, Farrar K, Hughes D. Implementing an automated dispensing system. Pharm J 2002;268(7191):437-8. Available: http://www.pharmj.com/pdf/articles/pj_20020330_automated.pdf (accessed 2009 Jan 19).

67. Dib JG, Abdulmohsin SA, Farooki MU, Mohammed K, Iqbal M. Effects of an automated drug dispensing system on medication adverse event occurrences and cost containment at SAMSO. Hosp Pharm 2006;41(12):1180-4.

68. Oswald S, Caldwell R. Dispensing error rate after implementation of an automated pharmacy carousel system. Am J Health Syst Pharm 2007;64(13):1427-31.

69. Poon EG, Cina JL, Churchill W, Patel N, Featherstone E, Rothschild JM, et al. Medication dispensing errors and potential adverse drug events before and after implementing bar code technology in the pharmacy. Ann Intern Med 2006;145(6):426-34.


54

70. Reifsteck M, Swanson T, Dallas M. Driving out errors through tight integration between software and automation. J Healthc Inf Manag 2006;20(4):35-9.

71. Work M. Improving medication safety with a wireless, mobile barcode system in a community hospital. Patient Safety & Quality Healthcare 2005;(May/June). Available: http://www.psqh.com/mayjun05/casestudy.html (accessed 2008 Jun 24).

72. Davies A, Staves J, Kay J, Casbard A, Murphy MF. End-to-end electronic control of the hospital transfusion process to increase the safety of blood transfusion: Strengths and weaknesses. Transfusion 2006;46(3):352-64.

73. Franklin BD, O'Grady K, Donyai P, Jacklin A, Barber N. The impact of a closed-loop electronic prescribing and administration system on prescribing errors, administration errors and staff time: a before-and-after study. Qual Saf Health Care 2007;16(4):279-84.

74. Chan JC, Chu RW, Young BW, Chan F, Chow CC, Pang WC, et al. Use of an electronic barcode system for patient identification during blood transfusion: 3-year experience in a regional hospital. Hong Kong Med J 2004;10(3):166-71.

75. Whittlesea C. What did the research reveal about the effects of introductin automation? Hospital Pharmacist 2004;11:453. Available: http://www.pharmj.com/pdf/hp/200412/hp_200412_machinepower.pdf (accessed 2009 Jan 19).

76. Ragan R, Bond J, Major K, Kingsford T, Eidem L, Garrelts JC. Improved control of medication use with an integrated bar-code-packaging and distribution system. Am J Health Syst Pharm 2005;62(10):1075-9.

77. Anderson S, Wittwer W. Using bar-code point-of-care technology for patient safety. J Healthc Qual 2004;26(6):5-11.

78. Foote SO, Coleman JR. Medication administration: the implementation process of bar-coding for medication administration to enhance medication safety. Nurs Econ 2008;26(3):207-10.

79. Paoletti RD, Suess TM, Lesko MG, Feroli AA, Kennel JA, Mahler JM, et al. Using bar-code technology and medication observation methodology for safer medication administration. Am J Health Syst Pharm 2007;64(5):536-43.

80. Porcella A, Walker K. Patient safety with blood products administration using wireless and bar-code technology. AMIA Annu Symp Proc 2005;614-18. Available: http://www.pubmedcentral.nih.gov/picrender.fcgi?artid=1560432&blobtype=pdf (accessed 2008 Jun 19).

81. Morriss FH, Jr., Abramowitz PW, Nelson SP, Milavetz G, Michael SL, Gordon SN, et al. Effectiveness of a Barcode Medication Administration System in Reducing Preventable Adverse Drug Events in a Neonatal Intensive Care Unit: A Prospective Cohort Study. J Pediatr 2009;154(3):363-8. Epub 2008 Sept 29.

82. Slee A. Processes and benefits of automation in hospitals. Hospital Pharmacy Europe 2004;Jul/Aug:21-2.

83. Skibinski KA, White BA, Lin LI, Dong Y, Wu W. Effects of technological interventions on the safety of a medication-use system. Am J Health Syst Pharm 2007;64(1):90-6.

84. Flynn EA, Barker KN, Pepper GA, Bates DW, Mikeal RL. Comparison of methods for detecting medication errors in 36 hospitals and skilled-nursing facilities. Am J Health Syst Pharm 2002;59(5):436-46.

85. Swanson D. Automated dispensing - An overview of the types of systems available. Hospital Pharmacist 2004;11(2):66-8+77. Available: http://www.pharmj.com/Hospital/Editorial/200402/articles/automateddispensing.html (accessed 2008 Dec 8).


55

86. Malcom B, Carlson RA, Tucker CL, Willette C. Veterans affairs: eliminating medication errors through point-of-care devices. Technical paper for 2000 Annual HIMSS Conference submitted November 30, 1999. Falls Church (VA): Office of the Assistant Secretary of Defense (Health Affairs) and the TRICARE Management Activity; 1999. Available: http://www.tricare.osd.mil/hcr/downloads/00050a.doc (accessed 2004 Feb 9).

87. Canadian Agency for Drugs and Technologies in Health. Guidelines for the economic evaluation of health technologies: Canada. 3rd ed. Ottawa: CADTH; 2006. Available: http://www.cadth.ca/media/pdf/186_EconomicGuidelines_e.pdf (accessed 2008 Jan 31).

88. Hailey D, Ohinmaa A, Roine R. Study quality and evidence of benefit in recent assessments of telemedicine. J Telemed Telecare 2004;10(6):318-24.

89. Drummond MF, Jefferson TO. Guidelines for authors and peer reviewers of economic submissions to the BMJ. The BMJ Economic Evaluation Working Party. BMJ 1996;313(7052):275-83.

90. Lee LW, Wellman GS, Birdwell SW, Sherrin TP. Use of an automated medication storage and distribution system. Am J Hosp Pharm 1992;49(4):851-5.

91. Frick T, Rein A, Parks V. Benefits of an automated narcotic retrieval system. Nurs Manage 1993;24(7):57-9.

92. Wise LC, Bostrom J, Crosier JA, White S, Caldwell R. Cost-benefit analysis of an automated medication system. Nurs Econ 1996;14(4):224-31.

93. Guerrero RM, Nickman NA, Jorgenson JA. Work activities before and after implementation of an automated dispensing system. Am J Health Syst Pharm 1996;53(5):548-54.

94. Buchanan EC. Computer simulation as a basis for pharmacy reengineering. Nurs Adm Q 2003;27(1):33-40.

95. Poveda Andrés JL, Garcia Gómez C, Hernández Sansalvador M, Valladolid Walsh A. Análisis coste-beneficio de la implantácion de los sistemas automáticos de dispensación de medicamentos en las Unidades de Críticos y Urgencias [Cost-benefit analysis of the implementation of automated drug-dispensing systems in Critical Care and Emergency Units]. Farm Hosp 2003;27(1):4-11. Spanish. Available: http://www.sefh.es/fh/2003/n1/2.pdf (accessed 2008 Jul 3).

96. Kheniene F, Bedouch P, Durand M, Marie F, Brudieu E, Tourlonnias MM, et al. Impact économique de la mise en place d'un automate de distribution des médicaments dans un service de réanimation. Ann Fr Anesth Reanim 2008;27(3):208-15.

97. Maviglia SM, Yoo JY, Franz C, Featherstone E, Churchill W, Bates DW, et al. Cost-benefit analysis of a hospital pharmacy bar code solution. Arch Intern Med 2007;167(8):788-94. Available: http://archinte.ama-assn.org/cgi/reprint/167/8/788 (accessed 2008 Jul 3).

98. Karnon J, McIntosh A, Dean J, Bath P, Hutchinson A, Oakley J, et al. Modelling the expected net benefits of interventions to reduce the burden of medication errors. J Health Serv Res Policy 2008;13(2):85-91.

99. Poon EG, Keohane C, Featherstone E, Hays B, Dervan A, Woolf S, et al. Impact of barcode medication administration technology on how nurses spend their time on clinical care. AMIA Annu Symp Proc 2006;1065. Available: http://www.pubmedcentral.nih.gov/picrender.fcgi?artid=1839328&blobtype=pdf (accessed 2008 Aug 12).

100. Weinstein MC, O'Brien B, Hornberger J, Jackson J, Johannesson M, McCabe C, et al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR Task Force on Good Research Practices--Modeling Studies. Value Health 2003;6(1):9-17.


56

101. Nursing Salaries: 2007-2010 Provincial Collective Agreement, Salary Appendix. Edmonton: United Nurses of Alberta; 2008. Available: http://www.una.ab.ca/collectiveagreements/nursingsalaries/ (accessed 2008 Dec 9).

102. Alberta Health & Wellness. Capital Health. Career Opportunities. Edmonton: Capital Health; 2008. Available: https://www.capitalhealth.ca/Careers/CareerOpportunities/default.htm (accessed 2008 Dec 9).

103. Englebright JD, Franklin M. Managing a new medication administration process. J Nurs Adm 2005;35(9):410-3.

104. U D, Jelincic V. Medication safety alerts. Can J Hosp Pharm 2003;56(5):280-2.

105. Oborn E, Barrett M. The adoption of dispensing technology: An occasion for restructuring hospital pharmacy work. Int J Pharm Pract 2008;16(2):109-14.

106. Fitzpatrick R. Implementing automation in a hospital pharmacy. Hospital Pharmacy Europe 2004;Mar/Apr:34-7.

107. Novek J. Hospital pharmacy automation: collective mobility or collective control? Soc Sci Med 2000;51(4):491-503.

108. Koppel R, Wetterneck T, Telles JL, Karsh BT. Workarounds to barcode medication administration systems: their occurrences, causes, and threats to patient safety. J Am Med Inform Assoc 2008;15(4):408-23. Available: http://www.jamia.org/cgi/reprint/15/4/408 (accessed 2008 Aug 12).

109. van Onzenoort HA, van de PA, Kessels AG, Veldhorst-Janssen NM, van der Kuy PH, Neef C. Factors influencing bar-code verification by nurses during medication administration in a Dutch hospital. Am J Health Syst Pharm 2008;65(7):644-8.

110. Buchanan EC. Ethics of automated compounding and dispensing. Pharm Pract Manag Q 1995;15(3):66-71.

111. Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, et al. Role of computerized physician order entry systems in facilitating medication errors. JAMA 2005;293(10):1197-203.

112. Stead WW. Rethinking electronic health records to better achieve quality and safety goals. Annu Rev Med 2007;58:35-47.

113. Mahmoud S, Cherni N. Les nouvelles technologies de l'information en médecine: a-t-on pensé au malade? Tunis Med 2003;81(2):80-5.

114. Burton LC, Anderson GF, Kues IW. Using electronic health records to help coordinate care. Milbank Q 2004;82(3):457-81, table.

115. Erlen JA. Medication errors: ethical implications. Orthop Nurs 2001;20(4):82-5.

116. Waite M. To tell the truth: the ethical and legal implications of disclosure of medical error. Health Law J 2005;13:1-33.

117. Bates DW, Cullen DJ, Laird N, Petersen LA, Small SD, Servi D, et al. Incidence of adverse drug events and potential adverse drug events. Implications for preventiong. JAMA 1995;274(1):29-34.

118. Wolfstadt JI, Gurwitz JH, Field TS, Lee M, Kalkar S, Wu W, et al. The effect of computerized physician order entry with clinical decision support on the rates of adverse drug events: a systematic review. J Gen Intern Med 2008;23(4):451-8.

119. Conroy S, Sweis D, Planner C, Yeung V, Collier J, Haines L, et al. Interventions to reduce dosing errors in children : a systematic review of the literature. Drug Saf 2007;30(12):1111-25.


57

120. Eslami S, de Keizer NF, Abu-Hanna A. The impact of computerized physician medication order entry in hospitalized patients; a systematic review [Epub ahead of print]. Int J Med Inf 2007.

121. Garg AX, Adhikari NK, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of computerized clinical decision support systems on practitioner performance and patient outcomes: a systematic review. JAMA 2005;293(10):1223-38.

122. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ 2005;330(7494):765.

123. Kaushal R, Shojania KG, Bates DW. Effects of computerized physician order entry and clinical decision support systems on medication safety: a systematic review. Arch Intern Med 2003;163(12):1409-16.

124. Barker KN. Ensuring safety in the use of automated medication dispensing systems. Am J Health Syst Pharm 1995;52(21):2445-7.

125. Stockwell DC, Slonim AD. Detecting medication errors: a job for Six Sigma. Pediatr Crit Care Med 2007;8(2):190-2.

126. Weingart SN, Wilson RM, Gibberd RW, Harrison B. Epidemiology of medical error. BMJ 2000;320(7237):774-7.

127. Gandhi TK, Seger DL, Bates DW. Identifying drug safety issues: from research to practice. Int J Qual Health Care 2000;12(1):69-76. Available: http://intqhc.oxfordjournals.org/cgi/reprint/12/1/69 (accessed 2009 Jan 19).

128. Kramer MS. Clinical trials: the Hawthorne effect. In: Clinical epidemiology and biostatistics: a primer for clinical investigators and decision-makers. Berlin: Springer-Verlag; 1998. p.87-8.

129. Bates DW, Leape LL, Petrycki S. Incidence and preventability of adverse drug events in hospitalized adults. J Gen Intern Med 1993;8(6):289-94.

130. Hayward RA, Hofer TP. Estimating hospital deaths due to medical errors: preventability is in the eye of the reviewer. JAMA 2001;286(4):415-20.

131. Hofer TP, Hayward RA. Are bad outcomes from questionable clinical decisions preventable medical errors? A case of cascade iatrogenesis. Ann Intern Med 2002;137(5 Part 1):327-33.

132. Lilford RJ, Mohammed MA, Braunholtz D, Hofer TP. The measurement of active errors: methodological issues. Qual Saf Health Care 2003;12 Suppl 2:ii8-12. Available: http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pubmed&pubmedid=14645889 (accessed 2009 Apr 20).

A-1

APPENDIX 1: Clinical search strategy 2003 – 2008 Economic search strategy 1990 – 2008

OVERVIEW

Interface: Ovid

Databases: BIOSIS Previews <1989 to 2008> CINAHL Cumulative Index to Nursing & Allied Health Literature <1982 to 2008> EMBASE <1980 to 2008> Medline <1950 to 2008> Medline In-Process & Other Non-Indexed Citations < 2008> Note: Subject headings have been customized for each database. Duplicates between databases were removed in Ovid.

Date of Search: June 20, 2008

Alerts: Monthly search updates ran until January 2009.

Study Types: Systematic reviews; meta-analyses; technology assessments; randomized controlled trials; controlled clinical trials; multicenter studies; cohort studies; cross-over studies; case control studies; comparative studies; also costs and cost analysis studies, quality of life studies, and economic literature.

Limits: Publication years 2003 – 2008 Human studies

SYNTAX GUIDE

/ At the end of a phrase, searches the phrase as a subject heading

.sh At the end of a phrase, searches the phrase as a subject heading

MeSH Medical Subject Heading

fs Floating subheading

exp Explode a subject heading

$ Truncation symbol, or wildcard: retrieves plural or variations of a word

ADJ# Requires words are adjacent to each other within # number of words (in any order)

.ti Title

.ab Abstract

.hw Heading Word; usually includes subject headings and controlled vocabulary

.pt Publication type

.mp Title fields, abstract, name of substance field, and subject headings

.rn CAS registry number

use b9o89 Limit search line to the Biosis Previews database

use emez " EMBASE

use nursing " CINAHL

use mesz " MEDLINE

use prem " MEDLINE In-Process & Other Non-Indexed Citations

A-2

CLINICAL SEARCH STRATEGY

Line Strategy 1. eMAR$.ti,ab. 2. AMDD$.ti,ab. 3. BCMA$.ti,ab. 4. BCMD$.ti,ab. 5. BPOC$.ti,ab. 6. Bar-cod$.ti,ab. 7. Barcod$.ti,ab. 8. Robot$.ti,ab. 9. (Electronic$ adj3 med$).ti,ab. 10. (Automat$ adj3 med$).ti,ab. 11. (Automat$ adj3 dispens$).ti,ab. 12. Point of care.ti,ab. 13. Automation.sh. use mesz 14. Automation.sh. use prem 15. Patient Identification Systems.sh. use mesz 16. Patient Identification Systems.sh. use prem 17. Patient Identification.sh. use nursing 18. Bar-coding.sh. use nursing 19. or/1-18 20. (Safety or safe).ti,ab. 21. Error$.ti,ab. 22. (adverse adj3 event$).ti,ab. 23. (adverse adj3 effect$).ti,ab. 24. mistake$.ti,ab. 25. complication$.ti,ab. 26. (risk$ adj5 manag$).ti,ab. 27. (risk$ adj5 assess$).ti,ab. 28. harm$.ti,ab. 29. exp medical errors/ use mesz 30. exp medical errors/ use prem 31. Safety management.sh. use mesz 32. Safety management.sh. use prem 33. Patient safety.sh. use emez 34. Patient safety.sh. use nursing 35. Medical error.sh. use emez 36. Medication errors.sh. use nursing 37. medication error.sh. use emez 38. risk management.sh. use mesz 39. risk management.sh. use prem 40. risk management.sh. use emez 41. risk management.sh. use nursing 42. risk assessment.sh. use mesz 43. risk assessment.sh. use prem 44. risk assessment.sh. use emez

A-3

45. risk assessment.sh. use nursing 46. adverse drug reaction reporting systems.sh. use mesz 47. adverse drug reaction reporting systems.sh. use prem 48. safety.sh. use emez 49. or/20-48 50. 19 and 49 51. (Randomized Controlled Trial or Controlled Clinical Trial or Clinical Trial).pt. 52. Randomized Controlled Trials as Topic/ use prem 53. Randomized Controlled Trials as Topic/ use mesz 54. Controlled Clinical Trials as Topic/ use prem 55. Controlled Clinical Trials as Topic/ use mesz 56. Clinical Trials as Topic/ use prem 57. Clinical Trials as Topic/ use mesz 58. Randomized Controlled Trial/ use emez 59. Randomization/ use emez 60. Controlled Clinical Trial/ use emez 61. Double-Blind Method.sh. use prem 62. Double-Blind Method.sh. use mesz 63. Double Blind Procedure.sh. use emez 64. Double-blind Studies.sh. use nursing 65. Single-blind Method.sh. use prem 66. Single-blind Method.sh. use mesz 67. Single Blind Procedure.sh. use emez 68. Single-blind Studies.sh. use nursing 69. Placebos/ use mesz 70. placebos/ use prem 71. Placebo$/ use emez 72. Placebo$/ use nursing 73. Random Allocation.sh. use prem 74. Random Allocation.sh. use mesz 75. Random Assignment.sh. use nursing 76. Clinical Trials.sh. use nursing 77. (random$ or sham$ or placebo$ or (singl$ adj (blind$ or dumm$ or mask$)) or (doubl$ adj (blind$

or dumm$ or mask$))).ti,ab,hw. 78. ((tripl$ adj (blind$ or dumm$ or mask$)) or (trebl$ adj (blind$ or dumm$ or mask$))).ti,ab,hw. 79. trial.ti. 80. or/51-79 81. 50 and 80 82. (MEDLINE or systematic review).tw. or meta-analysis.pt. or meta-analysis/ 83. exp Technology Assessment, Biomedical/ or (health technology assessment$ or HTA or HTAs or

biomedical technology assessment$ or bio-medical technology assessment$).ti,ab. 84. (Meta analysis or systematic review).ti,ab. use prem or review.ti. use prem 85. or/82-84 86. (Meta Analysis or Systematic Review or Biomedical Technology Assessment).mp. 87. (meta analy$ or metaanaly$ or met analy$ or metanaly$ or health technology assessment$ or HTA

or HTAs or biomedical technology assessment$ or bio-medical technology assessment$).ti,ab. 88. (meta regression$ or metaregression$ or mega regression$).ti,ab.

A-4

89. ((systematic$ adj (literature review$ or review$ or overview$)) or (methodologic$ adj (literature review$ or review$ or overview$))).ti,ab.

90. ((quantitative adj (review$ or overview$ or synthes$)) or (research adj (integration$ or overview$))).ti,ab.

91. ((integrative adj2 (review$ or overview$)) or (collaborative adj (review$ or overview$)) or (pool$ adj analy$)).ti,ab.

92. (data synthes$ or data extraction$ or data abstraction$).ti,ab. 93. (handsearch$ or hand search$).ti,ab. 94. (mantel haenszel or peto or der simonian or dersimonian or fixed effect$ or latin square$).ti,ab. 95. or/86-94 96. (Meta Analysis or Systematic Review).sh. 97. (meta analy$ or metaanaly$ or met analy$ or metanaly$ or health technology assessment$ or HTA

or HTAs or biomedical technology assessment$ or bio-medical technology assessment$).ti,ab. 98. medline.tw. 99. (meta regression$ or metaregression$ or mega regression$).ti,ab. 100. ((systematic$ adj3 (literature review$ or review$ or overview$)) or (methodologic$ adj3

(literature review$ or review$ or overview$))).ti,ab. 101. ((quantitative adj3 (review$ or overview$ or synthes$)) or (research adj3 (integration$ or

overview$))).ti,ab. 102. ((integrative adj3 (review$ or overview$)) or (collaborative adj3 (review$ or overview$)) or

(pool$ adj3 analy$)).ti,ab. 103. (data synthes$ or data extraction$ or data abstraction$).ti,ab. 104. (handsearch$ or hand search$).ti,ab. 105. (mantel haenszel or peto or der simonian or dersimonian or fixed effect$ or latin square$).ti,ab. 106. or/96-105 107. or/85,95,106 108. 50 and 107 109. exp cohort studies/ 110. cohort$.tw. 111. controlled clinical trial.pt. 112. epidemiologic methods/ 113. limit 112 to yr=1966-1989 114. exp case-control studies/ 115. (case$ and control$).tw. 116. (case$ adj2 series).tw. 117. (case$ adj2 stud$).tw. 118. exp cohort analysis/ 119. exp longitudinal study/ 120. exp prospective study/ 121. exp follow up/ 122. exp case control study/ 123. or/109-122 124. 123 and 50 125. 81 or 108 or 124 126. 125 use prem 127. limit 125 to human

128.126 or 127

A-5

ECONOMIC SEARCH STRATEGY

Line Strategy 1. economics/ 2. exp "Costs and Cost Analysis"/ 3. exp "economics hospital"/ 4. economics medical/ 5. economics nursing/ 6. economics pharmaceutical/ 7. or/1-6 8. (econom$ or cost or costs or costly or costing or price or prices or pricing or

pharmacoeconomic$).ti,ab. 9. (expenditure$ not energy).ti,ab. 10. (value adj1 money).ti,ab. 11. budget$.ti,ab. 12. or/8-11 13. 7 or 12 14. letter.pt. 15. editorial.pt. 16. historical-article.pt. 17. or/14-16 18. 13 not 17 19. ANIMALS.mp. 20. HUMAN.mp. 21. 19 not (19 and 20) 22. 18 not 21 23. (metabolic adj cost).ti,ab. 24. ((energy or oxygen) adj cost).ti,ab. 25. 22 not (23 or 24) 26. health-economics/ 27. exp economic-evaluation/ 28. exp health-care-cost/ 29. exp pharmacoeconomics/ 30. or/26-29 31. (econom$ or cost or costs or costly or costing or price or prices or pricing or

pharmacoeconomic$).ti,ab. 32. (expenditure$ not energy).ti,ab. 33. (value adj2 money).ti,ab. 34. budget$.ti,ab. 35. or/31-34 36. 30 or 35 37. letter.pt. 38. editorial.pt. 39. note.pt. 40. or/37-39 41. 36 not 40 42. (metabolic adj cost).ti,ab.

A-6

43. ((energy or oxygen) adj cost).ti,ab. 44. ((energy or oxygen) adj expenditure).ti,ab. 45. or/42-44 46. 41 not 45 47. exp animal/ 48. exp animal-experiment/ 49. nonhuman/ 50. (rat or rats or mouse or mice or hamster or hamsters or animal or animals or dog or dogs or cat or

cats or bovine or sheep).ti,ab,sh. 51. or/47-50 52. exp human/ 53. exp human-experiment/ 54. 52 or 53 55. 51 not (51 and 54) 56. 46 not 55 57. 25 or 56 58. eMAR$.ti,ab. 59. AMDD$.ti,ab. 60. BCMA$.ti,ab. 61. BCMD$.ti,ab. 62. Bar-cod$.ti,ab. 63. Barcod$.ti,ab. 64. Robot$.ti,ab. 65. Electronic$.ti,ab. 66. Automat$.ti,ab. 67. Point of care.ti,ab. 68. Automation.sh. use mesz 69. Automation.sh. use prem 70. Patient Identification Systems.sh. use mesz 71. Patient Identification Systems.sh. use prem 72. Patient Identification.sh. use nursing 73. Bar-coding.sh. use nursing 74. or/58-73 75. Medica$.ti,ab. 76. pharmacotherap$.ti,ab. 77. Drug$.ti,ab. 78. Pharmaceutical$.ti,ab. 79. Administrat$.ti,ab. 80. Prescription$.ti,ab. 81. Dispens$.ti,ab. 82. Pharmaceutical Preparations.sh. use mesz 83. Pharmaceutical Preparations.sh. use prem 84. Prescriptions, drug.sh. use mesz 85. Prescriptions, drug.sh. use prem 86. Prescriptions, drug.sh. use nursing 87. Pharmacy Service.sh. use nursing 88. Drugs, Prescription/ad use nursing

A-7

89. Medication Systems, Hospital.sh. use mesz 90. Medication Systems, Hospital.sh. use prem 91. Medication Systems.sh. use nursing 92. Drug labeling.sh. use mesz 93. Drug labeling.sh. use prem 94. Drug labeling.sh. use emez 95. Drug labeling.sh. use nursing 96. Drug administration.sh. use nursing 97. Drug administration.sh. use emez 98. Prescription.sh. use emez 99. Hospital pharmacy.sh. use emez 100. or/75-99 101. (Safety or safe).ti,ab. 102. Error$.ti,ab. 103. mistake$.ti,ab. 104. complication$.ti,ab. 105. risk$.ti,ab. 106. harm$.ti,ab. 107. Medical Errors.sh. use mesz 108. Medical Errors.sh. use prem 109. Safety management.sh. use mesz 110. Safety management.sh. use prem 111. Patient safety.sh. use emez 112. Patient safety.sh. use nursing 113. Medical error.sh. use emez 114. Medication errors.sh. use nursing 115. medication error.sh. use emez 116. risk management.sh. use mesz 117. risk management.sh. use prem 118. risk management.sh. use emez 119. risk management.sh. use nursing 120. risk assessment.sh. use mesz 121. risk assessment.sh. use prem 122. risk assessment.sh. use emez 123. risk assessment.sh. use nursing 124. adverse drug reaction reporting systems.sh. use mesz 125. adverse drug reaction reporting systems.sh. use prem 126. safety.sh. use emez 127. or/101-126 128. Efficiency workpath.ti,ab. 129. bcm barcode.ti,ab. 130. pyxis.ti,ab. 131. centricity.ti,ab. 132. RxTFC.ti,ab. 133. winpak.ti,ab. 134. robot-rx.ti,ab. 135. medlocker.ti,ab.

A-8

136. safetypak.ti,ab. 137. rxscan.ti,ab. 138. rxstation.ti,ab. 139. pillpick.ti,ab. 140. medstation.ti,ab. 141. sp central workflow.ti,ab. 142. pacemed.ti,ab. 143. accuMed.ti,ab. 144. autopack.ti,ab. 145. autolabel.ti,ab. 146. parata RDS.ti,ab. 147. robocheck.ti,ab. 148. avalo IMC medication cart.ti,ab. 149. MedSelect.ti,ab. 150. iPoint Med Cart.ti,ab. 151. AcuDose-Rx.ti,ab. 152. Connect-RN.ti,ab. 153. z series medication cart.ti,ab. 154. ultra rx medication.ti,ab. 155. OmniRx.ti,ab. 156. SinglePointe.ti,ab. 157. CareFusion.ti,ab. 158. Admin-Rx.ti,ab. 159. CAREt Medication.ti,ab. 160. VeriScan.ti,ab. 161. MetroLogic Orbit Scanner.ti,ab. 162. Accu-Care.ti,ab. 163. CODE CR2.ti,ab. 164. E-mar.ti,ab. 165. MediMAR.ti,ab. 166. RIVA.ti,ab. 167. TX10 medserver.ti,ab. 168. Clintec Automix.ti,ab. 169. OpenVista.ti,ab. 170. Safetymed.ti,ab. 171. Medcheck.ti,ab. 172. wCaremed.ti,ab. 173. pentapak.ti,ab. 174. Rx software.ti,ab. 175. or/128-174 176. 74 and 100 and 127 177. 175 and 127 178. 176 or 177 179. 178 and 57 180. workload$.ti,ab. 181. work load$.ti,ab. 182. 180 or 181

A-9

183. 182 and 178 184. 184. limit 183 to yr="1993 - 2008"

OTHER DATABASES

Cochrane Library Databases Issue 2 2008 Ovid EBM Reviews

Same MeSH, keywords, limits, and study types used as per Medline search, with appropriate syntax used.

ACP Journal Club, 2008 Ovid EBM Reviews

Same keywords, limits, and study types used as per Medline search, with appropriate syntax used.

Centre for Reviews and Dissemination Databases (CRD) University of York 2008

Same keywords and date limits used as per Medline search, excluding study types and Human restrictions. Syntax adjusted for HEED database.

Grey Literature and Hand Searches

Date of search: June – July 2008, main sites checked in December 2008

Keywords: Included terms BCMD, BCMA, AMDD, eMAR

Limits: Publication years 2003-present

NOTE: This section lists the main agencies, organizations, and web sites searched; it is not a complete list. For a complete list of sources searched, contact CADTH (http://www.cadth.ca). Health Technology Assessment Agencies Alberta Heritage Foundation for Medical Research (AHFMR) http://www.ahfmr.ab.ca Agence d’Evaluation des Technologies et des Modes d’Intervention en Santé (AETMIS). Québec http://www.aetmis.gouv.qc.ca Canadian Agency for Drugs and Technologies in Health (CADTH) http://www.cadth.ca Centre for Evaluation of Medicines. Father Sean O'Sullivan Research Centre, St.Joseph's Healthcare,Hamilton, and McMaster University, Faculty of Health Sciences. Hamilton, Ontario http://www.thecem.net/ Centre for Health Services and Policy Research, University of British Columbia http://www.chspr.ubc.ca/cgi-bin/pub Health Quality Council of Alberta (HQCA) http://www.hqca.ca Health Quality Council. Saskatchewan. http://www.hqc.sk.ca/ Institute for Clinical Evaluative Sciences (ICES). Ontario http://www.ices.on.ca/

A-10

Institute of Health Economics (IHE). Alberta http://www.ihe.ca/ Manitoba Centre for Health Policy (MCHP) http://www.umanitoba.ca/centres/mchp/ Ontario Ministry of Health and Long Term Care. Health Technology Analyses and Recommendations http://www.health.gov.on.ca/english/providers/program/mas/tech/ohtas_mn.html The Technology Assessment Unit of the McGill University Health Centre http://www.mcgill.ca/tau/ Therapeutics Initiative. Evidence-Based Drug Therapy. University of British Columbia http://www.ti.ubc.ca Health Technology Assessment International (HTAi) http://www.htai.org International Network for Agencies for Health Technology Assessment (INAHTA) http://www.inahta.org WHO Health Evidence Network http://www.euro.who.int/HEN Australian Safety and Efficacy Register of New Interventional Procedures – Surgical (ASERNIP-S) http://www.surgeons.org/Content/NavigationMenu/Research/ASERNIPS/default.htm Centre for Clinical Effectiveness, Monash University http://www.med.monash.edu.au/healthservices/cce/ Medicare Services Advisory Committee, Department of Health and Aging http://www.msac.gov.au/ NPS RADAR (National Prescribing Service Ltd.) http://www.npsradar.org.au/site.php?page=1&content=/npsradar%2Fcontent%2Farchive_alpha.html Institute of Technology Assessment (ITA) http://www.oeaw.ac.at/ita/index.htm Federal Kenniscentrum voor de Gezendheidszorg http://www.kenniscentrum.fgov.be Danish Centre for Evaluation and Health Technology Assessment (DCEHTA). National Board of Health http://www.dihta.dk/ Finnish Office for Health Care Technology and Assessment (FinOHTA). National Research and Development Centre for Welfare and Health http://finohta.stakes.fi/EN/index.htm L’Agence Nationale d’Accréditation et d’Evaluation en Santé (ANAES). Ministere de la Santé, de la Famille, et des Personnes handicappés http://www.anaes.fr/anaes/anaesparametrage.nsf/HomePage?ReadForm Committee for Evaluation and Diffusion of Innovative Technologies (CEDIT) http://cedit.aphp.fr/english/index_present.html

A-11

German Institute for Medical Documentation and Information (DIMDI). Federal Ministry of Health http://www.dimdi.de/static/de/hta/db/index.htm College voor Zorgverzekeringen/Health Care Insurance Board (CVZ) http://www.cvz.nl Health Council of the Netherlands http://www.gr.nl New Zealand Health Technology Assessment Clearing House for Health Outcomes and Health Technology Assessment (NZHTA) http://nzhta.chmeds.ac.nz/ Norwegian Centre for Health Technology Assessment (SMM) http://www.kunnskapssenteret.no/ Agencia de Evaluación de Tecnologias Sanitarias (AETS), Instituto de Salud “Carlos III”/ Health Technology Assessment Agency http://www.isciii.es/htdocs/investigacion/Agencia_quees.jsp Basque Office for Health Technology Assessment (OSTEBA). Departemento de Sanidad http://www.osasun.ejgv.euskadi.net/r52-2536/es/ Catalan Agency for Health Technology Assessment and Research (CAHTA) http://www.gencat.net/salut/depsan/units/aatrm/html/en/Du8/index.html CMT - Centre for Medical Technology Assessment http://www.cmt.liu.se/pub/jsp/polopoly.jsp?d=6199&l=en Swedish Council on Technology Assessment in Health Care (SBU) http://www.sbu.se/ Swiss Network for Health Technology Assessment http://www.snhta.ch/about/index.php European Information Network on New and Changing Health Technologies (EUROSCAN). University of Birmingham. National Horizon Scanning Centre http://www.euroscan.bham.ac.uk National Horizon Scanning Centre (NHSC) http://www.pcpoh.bham.ac.uk/publichealth/horizon NIHR Health Technology Assessment programme, Coordinating Centre for Health Technology Assessment (NCCHTA) http://www.hta.ac.uk/ NHS National Institute for Clinical Excellence (NICE) http://www.nice.org.uk NHS Quality Improvement Scotland http://www.nhshealthquality.org University of York NHS Centre for Reviews and Dissemination (NHS CRD) http://www.york.ac.uk/inst/crd

A-12

The Wessex Institute for Health Research and Development. Succinct and Timely Evaluated Evidence Review (STEER) http://www.wihrd.soton.ac.uk/ West Midlands Health Technology Assessment Collaboration (WMHTAC) http://www.wmhtac.bham.ac.uk/ Agency for Healthcare Research and Quality (AHRQ) http://www.ahrq.gov/ Dept. of Veterans Affairs Research & Development, general publications http://www1.va.gov/resdev/prt/pubs_individual.cfm?webpage=pubs_ta_reports.htm VA Technology Assessment Program (VATAP) http://www.va.gov/vatap/ ECRI http://www.ecri.org/ Institute for Clinical Systems Improvement http://www.icsi.org/index.asp Blue Cross and Blue Shield Association's Technology Evaluation Center (TEC) http://www.bcbs.com/blueresources/tec/ University HealthSystem Consortium (UHC) http://www.uhc.edu/ Health Economics Bases Codecs. CODECS (COnnaissances et Décision en EConomie de la Santé) Collège des Economistes de la Santé/INSERM http://infodoc.inserm.fr/codecs/codecs.nsf Centre for Health Economics and Policy Analysis (CHEPA). Dept. of Clinical Epidemiology and Biostatistics. Faculty of Health Sciences. McMaster University, Canada http://www.chepa.org Health Economics Research Group (HERG). Brunel University, U.K. http://www.brunel.ac.uk/about/acad/herg Health Economics Research Unit (HERU). University of Aberdeen http://www.abdn.ac.uk/heru/ Health Economic Evaluations Database (HEED) http://heed.wiley.com The Hospital for Sick Children (Toronto). PEDE Database http://pede.bioinfo.sickkids.on.ca/pede/index.jsp University of Connecticut. Department of Economics. RePEc database http://ideas.repec.org

A-13

Search Engines Google http://www.google.ca/ Yahoo! http://www.yahoo.com

A-14

APPENDIX 2: Vendors and distributors contacted

Vendors and distributors of automated technologies Contact information Examples of devices

AmerisourceBergen 1400 Busch Parkway Buffalo Grove, IL USA 60089 Phone: 1-888-537-3102 Fax: 1-888-808-3322 Email: [email protected]

AutoMed Canada* 2015 Fisher Drive Peterborough, ON K9J 6X6 Phone: 1-866-770-7702 Fax: (705) 746-3572 Web site: www.automed.com

AutoMed FastPack™ EXP (packaging) MedSelect® (dispensing; administration)

Baxter Canada 4 Robert Speck Parkway, Suite 700 Mississauga, ON L4Z 3Y4 Phone: 1-800-387-8399 Fax: (905) 281-6560 Email: NA – link through web site Web site: www.baxter.ca

ENLIGHTENED Bar-coding (BCMA)

Cardinal Health* 60 International Blvd Toronto, ON M9W 6J2 Phone: (416) 213-5150 Fax: (416) 213-5199 Email: [email protected] Web site: www.cardinal.com/ca/en/

Pyxis® MedStation™ (dispensing) Pyxis PARx® system (BCMD) CareFusion® (BCMA)

Cerner Canada* Phase 1, Tower 2, Level 3 800 Commissioners Road East London, ON N6A 4G5 Phone: (519) 685-8499 Fax: NA Web site: www.cerner.com

RxStation™ (dispensing) eMAR

Eclipsys 13511 Commerce Pkwy, Suite 100 Richmond, BC V6V 2J8 Phone: (604) 273-4900 Fax: (604) 273-2764 Web site: www.eclipsys.com

eMAR Knowledge-Based Medication Administration™ (BCMA)

GE Healthcare 2300 Meadowvale Blvd Mississauga, ON L5N 2P9 Phone: 1-800-367-2773 Fax: NA Email: NA – link through web site Web site: www.gehealthcare.com/caen

Centricity® Pharmacy (BCMA)

A-15


Healthmark 8815 Henri-Bourassa West Montreal, Quebec H4S 1P7 Phone: 1-800-665-5492 Fax: (514) 336-7111 Email: [email protected] Web site: www.healthmark.ca

Winpak ™ Bar Code Labelling Software medDISPENSE (dispensing)

Manrex Limited* 300 Cree Crescent Winnipeg, MN R3J 3W9 Phone: 1-800-665-7652 Fax: (204) 453-6350 Email: [email protected] Web site: www.manrex.com

RIVA (packaging) Medication Device Machine (verification of unit dose packaging)

McKesson Canada 4800 Levy Street Saint-Laurent, Quebec H4R 2P1 Phone: (514) 832-8333 Toll Free: 1-800-361-3757 Fax: (514) 832-8049 Email: [email protected] [email protected] Web site: www.mckesson.ca

PakPlus-Rx (packaging) AcuDose-Rx (dispensing) MedCarousel® Robot-Rx™ (dispensing) PACMED® (packaging and dispensing) Admin-Rx (BCMA)

Omnicell 1201 Charleston Road Mountain View, CA 94043-1337 Phone: 1-800-850-6664 Fax: 1-650-251-6266 Email: [email protected] Gary Robinson, International sales [email protected] 1-800-474-2355 ext 5111

SafetyStock™ (BCMD) Workflow Rx™ (carousel) SinglePointe™ (dispensing) OmniRx® (dispensing) SafetyMed™ (BCMA)

PointClickCare 6790 Century Avenue, Suite 100 Mississauga, ON L5N 2V8 Phone: 1-800-277-5889 Fax: (905) 858-2248 Email: NA – link through web site Web site: www.pointclickcare.com

eMAR

Rubbermaid Medical Solutions 16905 Northcross Drive Huntersville, NC USA 28078 Phone: 1-888-859-8294 Fax: 1-888-859-8297 Email: [email protected] Web site: www.rubbermaidmedical.com

Medication Cart (administration) eMAR

A-16


ScriptPro Canada 555 Burrard Street, Suite 900 Vancouver, BC V7X 1M8 Phone: (604) 443-5067 Fax: (604) 443-5001 Email: NA – link through web site Web site: www.scriptpro.com or www.scriptpro.ca

SP Automation Centre 200® (dispensing)

Swisslog AG* Healthcare Solutions Canada #7-1200 Aerwood Drive Mississauga, ON L4W 2S7 Phone: 1-877-294-2831 Fax: 1-905-629-2799 Email: [email protected] Web site: www.swisslog.com

PillPick® System: PillPicker (packaging), DrugNest (storage), PickRing (dispensing) BoxPicker (storage and dispensing – alternative to a carousel)

*companies that responded to our request for additional information.

Google and other Internet search engines were used to search for vendors and devices. The search included terms specific to medication ordering devices (BCMD, BCMA, ADD, eMAR). Additional vendors and devices were found in the grey literature search which included ECRI Gold and device regulatory websites including Health Canada and the FDA. They were contacted to obtain unpublished material.

A-17

APPENDIX 3: Clinical data extraction form Ref ID number

First Author

Year of publication

Country

Industry funding Yes No No information

Population and Setting Number of centres involved in study:

Teaching Non-teaching

General Paediatric Adult Specialty (specify)

Acute care Critical care Rehabilitation

Long-term care Emergency room Other (specify wards)

Number of beds: Interventions (device name or manufacturer)

AMDD BCMD BCMA eMAR

Comparator(s)

Nothing or usual care Another technology or system (specify):

Study Design

RCT Cohort Case-control

Before and after CCT Time-series

Prospective Retrospective

ITT

Yes No

Unclear Not applicable

A-18

Notes on study population Outcome measures (how defined in text; state page number) Medication errors Preventable adverse drug events Potential adverse drug events Error-ascertainment method How were the errors ascertained or how were the incidents reported? Period 1

incident report study-solicited voluntary report chart-review, single, retrospective chart-review, repeated daily direct observation automated undefined other (specify)

Period 2 incident report study-solicited voluntary report chart-review, single, retrospective chart-review, repeated daily direct observation automated undefined other (specify)

Study limitations – Comments Study Findings Baseline Group 1 Group 2 Group 3 Observation period Outcome Results Please report all results including variance. Please define observation periods in the outcome column.

A-19

APPENDIX 4: Forms for quality assessment Quality Assessment Form for Systematic Reviews Oxman and Guyatt Scale Reference ID number Author: 1. Were the search methods used to find evidence (original research) on the primary question(s)

stated? yes partially no

2. Was the search for evidence reasonably comprehensive?

yes can’t tell no

3. Were the criteria used for deciding which studies to include in the overview reported?

yes partially no

4. Was bias in the selection of studies avoided?

yes can’t tell no

5. Were the criteria used for assessing the validity of the included studies reported?

yes partially no

6. Was the validity of all studies referred to in the text assessed using appropriate criteria (either

in selecting studies for inclusion or in analysing the studies that are cited)? yes can’t tell no

7. Were the methods used to combine the findings of the relevant studies (to reach a

conclusion) reported? yes partially no

A-20

8. Were the findings of the relevant studies combined appropriately relative to the primary question the overview addresses?

yes can’t tell no

For question 8, if no attempt was made to combine findings, and no statement is made regarding the inappropriateness of combining findings, check “no.” If a summary (general) estimate is given in the abstract, the discussion, or the summary of the paper, and it is not reported how the estimate was derived, mark “no” even if there is a statement regarding the limitations of combining the findings of the studies reviewed. If in doubt mark “can’t tell.” 9. Were the conclusions made by the author(s) supported by the data or analysis reported in the

overview? yes partially no

For an overview to be scored as “yes” on question 9, data (not just citations) must be reported that support the main conclusions regarding the primary question(s) that the overview addresses. 10. How would you rate the scientific quality of the overview?

Extensive Major Minor Minimal Flaws Flaws Flaws Flaws

1 2 3 4 5 6 7

The score for question 10, the overall scientific quality, should be based on your answers to the first nine questions. The following guidelines can be used to derive a summary score. If the “can’t tell” option is used one or more times on the preceding questions, a review is likely to have minor flaws at best, and it is difficult to rule out major flaws (a score of 4 or lower). If the “no” option is used on question 2, 4, 6, or 8, the review is likely to have major flaws (a score of 3 or less, depending on the number and degree of the flaws).

A-21

Quality Assessment Form for Randomized Controlled Trials Jadad Scale and Schulz Scale Reference ID number Author:

Randomization Was the study described as randomized (including words such as randomly, random, randomization)? A trial reporting that it is “randomized” receives one point.

Yes =1 No =0 Trials describing an appropriate method of randomization (table of random numbers, computer generated) receive an additional point.

Appropriate =1 Not appropriate or not reported =0 If the report describes the trial as randomized and uses an inappropriate method of randomization (for example, date of birth, hospital numbers), a point is deducted.

Inappropriate = −1 Double-blinding Was the study described as double-blind? A trial reporting that it is “double-blind” receives one point.

Yes =1 No =0 Trials describing an appropriate method of double-blinding (identical placebo: colour, shape, taste) receive an additional point.

Yes =1 No or not reported =0 If the report describes a trial as double-blind and uses an inappropriate method (for example, comparison of tablets versus injection with no dummy), a point is deducted.

Inappropriate = −1 Withdrawals and dropouts Was there a description of withdrawals and dropouts? A trial reporting the number and reasons for withdrawals or dropouts receives one point. If there is no description, no point is given.

Yes =1 No =0

Total score (for above 3 categories) Adequacy of allocation concealment Central randomization; numbered or coded bottles or containers; drugs prepared by a pharmacy, serially numbered, opaque, sealed envelopes → Adequate Alternation; reference to case record number or date of birth → Inadequate Allocation concealment is not reported or fits neither category → Unclear

A-22

Quality Assessment Form for Cohort Studies Newcastle – Ottawa Scale A study can be awarded a maximum of one star* for each numbered item in the Selection and Outcome categories. A maximum of two stars** can be given for Comparability. Reference ID number Author: Selection 1. Representativeness of the exposed cohort

Truly representative of the average (describe) in the community* Somewhat representative of the average in the community* Selected group of users (for example, nurses, volunteers) No description of the derivation of the cohort

2. Selection of the non exposed cohort

Drawn from the same community as the exposed cohort* Drawn from a different source No description of the derivation of the non exposed cohort

3. Ascertainment of exposure

Secure record (for example, surgical records)* Structured interview* Written self report No description

4. Demonstration that outcome of interest was not present at start of study

Yes* No

Comparability 1. Comparability of cohorts on the basis of the design or analysis

Study controls for (select the most important factor)* Study controls for any additional factor* (These criteria could be modified to indicate specific control for a

second important factor.) Outcome 1. Assessment of outcome

Independent blind assessment* Record linkage* Self report No description

2. Was follow-up long enough for outcomes to occur?

Yes (select an adequate follow up period for outcome of interest)* No

3. Adequacy of follow up of cohorts

Complete follow-up – all subjects accounted for* Subjects lost to follow-up unlikely to introduce bias – small number lost > % (select an adequate %)

follow up or description provided of those lost* Follow up < % (select an adequate %) and no description of those lost No statement

A-23

Quality Assessment Form for Case-control Studies Newcastle – Ottawa Scale A study can be awarded a maximum of one star* for each numbered item in the Selection and Exposure categories. A maximum of two stars** can be given for Comparability. Reference ID number Author:

Selection 1. Is the case definition adequate?

Yes, with independent validation* Yes (for example, record linkage or based on self reports) No description

2. Representativeness of the cases

Consecutive or obviously representative series of cases* Potential for selection biases or not stated

3. Selection of controls

Community controls* Hospital controls No description

4. Definition of controls

No history of disease (endpoint)* No description of source

Comparability 1. Comparability of cases and controls on the basis of the design or analysis

Study controls for (Select the most important factor.)* Study controls for any additional factor* (These criteria could be modified to indicate specific control

for a second important factor.) Exposure 1. Ascertainment of exposure

Secure record (for example, surgical records)* Structured interview where blind to case/control status* Interview not blinded to case/control status Written self report or medical record only No description

2. Same method of ascertainment for cases and controls

Yes* No

3. Non-response rate

Same rate for both groups* Non respondents described Rate different and no designation

A-24

Quality Assessment Form for Other Types of Studies Reference ID number Author:

Internal validity (Are the groups comparable? Are the technologies comparable? Was the length of follow-up appropriate? Were confounders investigated and adjustments made? Are all patients accounted for?) External validity Applicability of results to other population and settings, for example: Are the settings comparable (country, type of hospitals?) How old are the technologies?

A-25

APPENDIX 5: Clinical Tables

Table 1: Oren et al.’s Systematic Review62 Country US Funding source not reported Objective -identify all published studies evaluating patient outcomes associated with use of 4

technologies; primary end-points were medication errors and adverse drug events; secondary end-points were costs, work efficiencies, and other measures

Inclusion and exclusion criteria Definitions -medication error: any preventable event that may cause or lead to inappropriate

medication use or patient harm while medication is in control of health care professional, patient, or consumer -adverse drug event: any response to drug which is noxious, unintended, and occurs at doses normally used in humans for prophylaxis, diagnosis, or therapy of disease

Population and setting not specified Interventions automated dispensing devices (ADD) and bar-coding Comparators not specified Study design controlled studies Search strategy -bar-coding: PubMed 1982 to March 2002

-ADD: PubMed 1966 to March 2002 -specific MeSH terms described -hand-searching of references -studies published in US, in peer-reviewed journals that could be obtained full-text

Methods Study selection not reported Data extraction not reported Method of synthesis -studies not combined but grouped according to interventions

-narrative synthesis provided -differences between studies not reported

Quality assessment and validity

not reported

Results Number of included studies

-ADD 7 -bar-coding 7

Interventions -ADD: Baxter ATC-212™, Pyxis® Medstation™ Rx, McLaughlin Dispensing System and Sure-Med Automated Dispensing System -bar-coding: not reported

Comparators -ADD: comparators described in 3 studies: manual filling by technicians; traditional unit dose cassette; decentralized unit dose system -bar-coding: 3 studies reported that interventions were compared to manual systems

Quality assessment and validity of included studies

not reported

Results: ADD centralized ADD -retrospective before and after study: dispensing error rate decreased compared to doses manually filled by technicians (0.65% versus 0.84%) decentralized ADD -prospective before and after study: medication administration error rate decreased (16.9% versus 10.4%, p<0.001); most were wrong time errors

A-26

Table 1: Oren et al.’s Systematic Review62 -prospective before and after study: lower filling error rate of ADD compared with filling of traditional unit dose cassettes (0.61% versus 0.89%, p=0.04) -prospective before and after study: medication administration error rate decreased (0.0075 error per patient-day to 0.0058, p>0.05) on cardiovascular surgical unit but not on cardiovascular ICU (0.0051 to 0.0090, p>0.05) -prospective controlled study: mean medication error rate decreased compared with decentralized unit dose system (10.6 % versus 15.9%, p<0.05); most were wrong time errors -prospective before and after study: medication-related nursing activities decreased in 1 unit (20.7% versus 18.4%) and increased in another (10.8% versus 11.0%); time spent on clinical activities increased for pharmacists on 2 units (36.5% to 49.1% and 27.9% to 35.1%) -prospective before and after study: increase in doses administered as scheduled with automation (18%, p=0.0235)

Results: bar-coding -prospective before and after study: using bar code stock ordering system, error rate in ambulatory care pharmacy decreased (1.0% versus 0.2%); time saving of 104 technician hours -prospective before and after study: using bar code inventory system for issuing medical supplies to nursing units; time needed to take order increased (4.48 versus 4.14 minutes, p<0.01); time needed to enter order decreased (1.36 versus 7.10 minutes, p<0.01); accuracy of inventory improved (p<0.001) -prospective controlled study: mean data entry time not statistically significantly faster (p>0.05); mean entry error lower with bar code (0.79% versus 1.53%, p=0.0167); comparator: manual keyboard entry -prospective cross-over controlled study: data entry error for documenting pharmacists’ clinical interventions lower (1.7% versus 5.8%); bar code associated with increased cost ($35.85/pharmacist/year); time per intervention using bar codes shorter (p<0.01); comparator: manual system -prospective controlled study: mean ± sd, total number of documentation errors per record decreased (2.63±0.24 versus 4.48±0.30, p<0.0001); lower mean number of omissions (p=0.0001) and inaccuracies (p=0.0038) per trauma record -prospective before and after study: time saving of 1.52 seconds per dose with BCMD -prospective before and after study: improved (19%) patient accountability for charges for large volume iv solutions in 2 nursing units

Conclusions

ADD: “Five studies observed a decrease in medication errors associated with ADMs… Furthermore, while the observed differences in medication errors between unit dose and automated systems are statistically significant, they often are of low magnitude.” bar-coding: “While bar-coding experimentally improves both the speed and accuracy of data entry, few other ‘real-life’ data are available to clarify its role. In particular, studies evaluating point-of-care systems that verify patient and drug information were not found.”

Quality assessment rating = 3 (major flaws) Studies limited to those published in peer-reviewed journals as full-text, in the US. One database searched. No attempt made to include studies from grey literature. This may have resulted in the omission of relevant studies. Validity or methodological limitations of each study were not assessed. For most of the included studies, no information provided on population and setting.

ADD=automated dispensing device; BCMD=bar-coding for medication dispensing; iv=intravenous; sd=standard deviation

A-27

Table 2: Errors and ascertainment methods Error-ascertainment methods Study Errors control intervention

Anderson77 medication error and near-miss: unconfirmed order discontinued, future order, wrong IV solution, wrong dose, order discontinued, wrong route, wrong drug, early dose

not reported not reported

Borel54 medication error: omission, wrong dose, unauthorized drug, wrong dosage form, wrong time, wrong route, wrong rate, wrong preparation of dose, extra dose, other

direct observation direct observation

Brown59 medication error: variation from standard practice

incident report incident report

Chan74 blood transfused to wrong patient, wrong labelling of blood samples and request forms

direct observation (verification and documentation by 1 staff with second staff as checker)

automated

Davies72 near-miss detailed auditing detailed auditing Dib67 medication adverse event: wrong drug,

wrong dose, wrong patient, medication administration, documentation error, policy and procedure, order entry, missed doses, adverse drug events, and others (injection-site complications, narcotic and controlled drug loss, infiltration, extravasations)

occurrence report occurrence report

Fitzpatrick65 dispensing errors: wrong drug, wrong strength, wrong formulation, wrong quantity, wrong label information, wrong label instructions, omissions, expired or deteriorated drug

logs kept of all dispensing errors picked up by technician

error log picked up errors from Consis-dispensed items and those dispensed manually

Foote78 medication error not reported not reported Franklin73 medication administration error: any dose

of medication that deviated from patient’s current medication orders and included wrong drug, dose, patient, route, form, or time, extra dose, expired drug, omission due to unavailability, other omission, wrong diluents, fast IV bolus; timing and documentation errors excluded; severity of errors assessed by 4 judges


Franklin64 dispensing errors at final-check stage which included content, labelling and documentation

study-solicited voluntary report

study-solicited voluntary report

Johnson57

wrong medication, wrong dose, wrong patient, wrong time and omission (cannot tell whether total medication errors or medication administration errors only)


Klein53 dispensing errors: incorrect strength of correct drug, incorrect number of doses of

direct observation and audit (carts

direct observation (carts randomly

A-28


correct drug, incorrect drug, expired medication, doses that are not intact (for example, crushed tablet or capsule, open package)

randomly selected and rechecked for errors by 2 observers)

selected and rechecked for errors by 2 observers)

Kratz61 cart filling errors: wrong drug, wrong strength, drug missing, excess drug and miscellaneous

quality assurance data collection form

quality assurance data collection form

Low58 medication administration errors regardless of severity

incident report forms automated by BCMA log generated

Morriss81 -medication error: error in ordering, transcribing, dispensing, administering, or monitoring medication -targeted, preventable ADE: harm to patient as result of medication error that is expected to be prevented by BCMA system -potential ADE: medication error that could have harmed patient but did not because it was intercepted or participant was lucky

structured daily audit of each participant’s paper and eMAR by nurses who were not blinded to intervention (included use of triggers to enhance identification of ADEs) and voluntary reports

structured daily audit of each participant’s paper and eMAR by nurses who were not blinded to intervention (included use of triggers to enhance identification of ADEs) and voluntary reports

Oswald68 -filling error: error caught by pharmacist during verification step -dispensing error: error caught by pharmacist observer after verification by pharmacist -errors detected by pharmacists may include technicians picking wrong medication, strength, dose or dosage form

observational audit observational audit

Paoletti79 medication administration error: wrong time (late doses), omissions, wrong technique, wrong dose, extra dose, wrong medication, wrong route, wrong formulation

voluntary report and direct observation

voluntary report and direct observation

A-29


Poon69 -dispensing error: discrepancy between dispensed medications and physician orders or replenishment requests or deviation from standard pharmacy policies Primary outcome: -target dispensing error: dispensing error that bar code technology was designed to address, including those in which wrong medication, wrong strength or dose, wrong formulation, or expired medication was dispensed. -potential ADE: dispensing error that can harm patients if not intercepted before medication administration. Primary outcome: -target potential ADE: target dispensing error that can harm patients if not intercepted before medication administration. 2 internists independently judged severity of errors.


Porcella80 identification errors including blood sample collection, sample arrival in blood bank, product dispensing, and administration

voluntarily reported incident reports

scanner-detected prevented errors

Puckett60 medication error: deviation from original physician’s order and deviation from hospital’s standards (for example, wrong patient, wrong medication, wrong dose, wrong route, or wrong time)

incident form incident form

Ragan76 dispensing errors: pharmacy errors in selecting products from storage


Ray55 dispensing errors (technician filling error rate)

pharmacist verification of unit-dose cassette and record of number of errors found

pharmacist verification of content of Medstation™ Rx units and record of number of errors found

Reifsteck70 -medication administration error: deviation from physician's medication order as written in patient’s chart -further categorized into 7 types: unauthorized drug, improper dose, wrong dosage form, wrong route, wrong time, wrong administration technique, omission -wrong time: medication administration >60 minutes before or after scheduled time -medication error rate: combined number of errors divided by observed doses plus omissions


A-30


Rough63 medication administration error and near-miss

direct observation technique (trained observer blinded to patient’s medical history) with data collection form; and audit of patient’s medical record

series of vendor created reports providing list of errors recorded in software; all potential and actual medication administration errors captured electronically at bedside; and audit of medical chart after each report

Schwarz56 medication errors missing dose form and incident report; new medication error report form introduced during study period

missing dose form and incident report; new medication error report form introduced during study period

Skibinski83 medication errors (included dispensing and administration errors): any preventable event that may cause or lead to inappropriate medication use or patient harm while medication is in control of health care professional, patient, or consumer; include failure to identify patient, omission, incorrect patient, incorrect medication, incorrect dose, incorrect dosage form, incorrect route, incorrect time of administration, medication administered without active order, and administration time not documented

unclear: states that comparison of documents to actual medication dispensed done; also states that voluntary error reports were completed; and direct observation method used to evaluate accuracy of medication administration

unclear: states that comparison of documents to actual medication dispensed done; also states that voluntary error reports completed; and direct observation method used to evaluate accuracy of medication administration

Slee66 dispensing errors not reported not reported Slee82 dispensing and picking errors not reported not reported Whittlesea75 dispensing errors not reported not reported Work71 medication administration errors: wrong

dosage, missed medication, missed drug reaction, or wrong IV bag hung

not reported hospital Medication Event Report

ADE=adverse drug event; BCMA=bar-coding for medication administration; eMAR=electronic medication administration record; IV=intravenous

A-31

Table 3: Results for automated dispensing devices Pharmacy- based automated dispensing devices

Study (funding)

Setting Control versus

intervention (study

duration)

Outcomes Occurrence Risk estimate (95% CI)

Manual dispensing in original packs versus robots at 2 main sites

site 1: Pack Picker 4.0 by Swisslog (2-week data collection period; pre in summer and fall 2003; post in spring and summer 2005)

dispensing errors identified (% of items dispensed)

pre: 35 (1.2) post: 26 (0.8)

RRc=0.67 (0.40, 1.10) RRRc=33.3%

Franklin64 prospective before and after study

UK large teaching trust with 950 beds in 3 hospitals; 2 main sites have 450 beds each

site 2: Rowa Speedcase 2.6.9.6 by ARX Ltd (2-week data collection period; pre in summer and fall 2003, and spring and summer 2005; post in spring 2006)

dispensing errors identified (% of items dispensed)

pre 1: 50 (1.5) pre 2: 41 (1.6) post: 22 (0.9)

RRc=0.60 (0.36, 0.99) RRRc=40.0% RRc=0.56 (0.34, 0.94) RRRc=43.8%

Fitzpatrick65 prospective before and after study

UK NHS Trust

Manual filling from open shelving versus Consis by Baxter (5 months pre and 4 months post)

dispensing error rate (errors/100,000 items dispensed)

change in error rate −104.9

RRRr=16.0%

Klein53 prospective before and after study

US 650-bed tertiary care teaching hospital

Manual filling of unit-dose carts versus ATC-212™ by Baxter (3 weeks pre and 3 weeks post)

dispensing error rate [total errors/total filled doses by technician (reached patient care area)]

pre: 34/4,029 (0.84%) post: 25/3,813 (0.65%)

RRc= 0.78 (0.46, 1.30) RRRc=22.3%

Kratz61 cohort study

US 173-bed general hospital (2 nursing wards,

manual filling of unit-dose carts versus ATC-212™ by Baxter (43 consecutive

accuracy of cart fill

manual filling: 92.62% ATC-212™: 99.98%

A-32


64 beds) days) cart-filling error rate [total errors/total filled doses by technician (before pharmacist verification)]

manual filling: 184/2,493 (7.38%) ATC-212™: 3/12,660 (0.02%)

RRc=0.003 (0.001, 0.01) RRRc=99.7%

Slee66 prospective before and after study

UK district general hospital

comparator not described versus Rowa Speedcase (pre undefined and 4 month post)

dispensing error rate

dropped by 50% for items dispensed in first 4 months; 38,000 original packs dispensed each month

RRRr=50.0%

Slee82 interrupted time series

UK 1 centre

patient-specific medications and ward stock supplied according to standard checklist versus Rowa system used for filling ward stock boxes and used in dispensary (study duration not reported but conducted from January 2002 to February 2004; automation introduced December 2003)

mean±sd dispensing errors* (errors/ 100,000 items)

too few data points to do time series analysis pre: 42.6±18.4 post: 16.5±14.8

trend towards decrease in errors RRRc=61.3%

Whittlesea75 before and after study

UK multi-centre

comparator and ADD not described (study duration not reported)

Average reduction in dispensing errors/10,000 items dispensed distribution incidents/1,000 items distributed

Ysbyty Glan Clywd: 2.6 to 2.9 Llandough Hospital: 2.0 West Wales General: 3.6 pre: 12.9 post: 7.2

RRRc=44.2%

A-33


Limitations Franklin:64 May have been under reporting of errors, because this was done voluntarily. Data collected over 2 weeks. Fitzpatrick:65 Post-implementation, study did not discriminate between items dispensed by Consis versus those dispensed manually. Kratz:61 Findings (99.7%) reflect cart-filling errors before pharmacist verification (near-misses) compared with Klein’s study, in which findings (22.6%) reflect cart-filling errors after pharmacist verification of unit-dose medication carts and which reached the ward (true medication dispensing errors). Slee:66 Too little information provided to assess validity of study. Slee:82 Data were estimates from graphs. Too little information provided to assess validity of study. Whittlesea:75 Data collection periods were short. Pharmacy staff involved in data collection had no previous research experience. Lack of resources for carrying out second independent checks. Dispensing errors not defined.

Ward-based automated dispensing devices Borel54 prospective before and after study

US 600-bed hospital (24-bed speciality unit with 10 beds added post; 86-bed general unit; total 120 beds)

unit-dose distribution system versus Medstation™ Rx with profiling by Pyxis (study duration not reported)

medication errors (dispensing or administration)

pre: 148/ 873 (16.9%) post: 97/929 (10.4%), p=0.001

RRc= 0.62 (0.49, 0.78)

RRRc=38.4%

Dib67 prospective before and after study; pharmacists, nurses, and data collectors blinded to purpose of study

Saudi Arabia 390-bed tertiary care facility (5 units: cardiac care, medical ICU, surgical and medical ICU step down, hemodialysis)

traditional unit dose cassette exchange and open-floor stock medication system versus ADD with profiling not described (3 months pre and 3 months post)

medication adverse events/ prescription orders

pre:82/15,894 post:60/18,352

RRc=0.63 (0.46, 0.88) RRRc=36.6%

Ray55 prospective before and after study (Pyxis Corporation)

US large teaching hospital (26-bed adult medicine ward)

traditional centralized unit-dose drug distribution system versus Medstation™ Rx with profiling by Pyxis (6 weeks pre and 6 weeks post)

dispensing error (technician filling error rate, as % of doses dispensed)

pre: 0.89% post: 0.61%, p=0.04

RRRr=28.7% (3.6, 53.8)

A-34


Schwarz56 prospective before and after study

US 560-bed tertiary care teaching hospital (Unit A: 36-bed surgical unit and Unit B: 8-bed cardiac ICU)

traditional unit-dose cassette system versus Medstation™ Rx with profiling by Pyxis (6 months pre and 3 months post)

medication errors/month (mean ± sd) medication errors/patient day

Unit A pre: 6.5 ±2.6 post: 4.3 ±2.3 Unit B pre: 1.0 ±1.3 post: 1.7 ±0.6 Unit A pre: 0.0075 post: 0.0058 Unit B pre: 0.0051 post: 0.0090

RRRc=33.8% RRIc=70.0%

Limitations Borel:54 Most errors due to wrong timing. Dib:67 Errors not clearly defined because authors combined all medication-related events. No rest periods provided between pre and post phases of study. Ray:55 Possible explanation for reduction in dispensing errors observed is that pharmacy technicians had fewer medications to select from in filling Medstation™ Rx units compared with filling unit-dose medication carts in central pharmacy. Schwarz:56 Fewer errors reported on Unit A (33.8%) and more errors reported on Unit B (70%). Overall number of events was small and potentially contaminated by introduction of new medication error reporting form during deployment of Medstation™ Rx.

* data obtained from graphs ADD=automatic dispensing device; CI=confidence interval; ICU= intensive care unit; RRc=relative risk calculated; RRRc=relative risk reduction calculated; RRIc=relative risk increase calculated; RRRr=relative risk reduction reported; sd=standard deviation

A-35

Table 4: Results of bar-coding for medication dispensing Study

(funding) Setting Control versus

intervention (study

duration)


Oswald68 prospective before and after study (1 co-author employed by Omnicell)

US 613-bed acute and tertiary care university hospital 3 dispensing processes affected by introduction of carousel evaluated: 1) first dose or missing medication fill 2) automated dispensing cabinet fill 3) interdepartmental request fill

manual fill of unit dose medication versus automated pharmacy carousel system with bar code (first dose or missing medication fill: 40 hours over 12 days pre, and 37 hours over 15 days post; automated dispensing cabinet fill: 21 days pre and 9 days post; interdepartmental request fill: 7 days pre and 7 days post)

orders with filling errors/ orders observed (% filling error rate) orders with dispensing errors/ orders observed (% dispensing error)

first dose or missing medication fill pre: 9/422 (2.1) post: 7/387 (1.8) automated dispensing cabinet fill pre: 18/1112 (1.6) post: 8/1954 (0.4) first dose or missing medication fill pre: 2/422 (0.5) post: 2/387 (0.5) automated dispensing cabinet fill pre: 4/1112 (0.4) post: 5/1954 (0.3) interdepartmental request fill pre: 0/123 (0) post: 1/85 (1)

RRc=0.85 (0.32, 2.26) RRRc=15.2% RRc=0.25 (0.11, 0.58) RRRc=74.7% RRc=1.09 (0.15, 7.70) RRIc=9.0% RRc=0.71 (0.19, 2.64) RRRc=28.9% RRc=ne

A-36



intervention (study

duration)


Poon69 prospective before and after study (AHRQ)

US 735-bed tertiary care academic medical centre

manual fill from shelves with visual verification versus carousel fill with bar code for compact and non-refrigerated medications; other medications dispensed manually (20-month study period)

doses observed target dispensing errors target potential ADE serious ADE life-threatening ADE all dispensing errors all potential ADEs

pre: 115,164 post: 253,984 % adjusted rates pre: 0.37 post: 0.06 pre: 0.17 post: 0.04 pre: 0.06 post: 0.03 pre: 0.001 (n=2) post: 0.003 (n=13) pre: 0.88 post: 0.57 pre: 0.19 post: 0.07

RRRr=85% RRRr=74% RRRr=54% increase by 2.8 fold RRRr=36% RRRr=63%

Ragan76 before and after study

US tertiary care community teaching hospital (640 beds and 102 bassinets)

drug packaging by central pharmacy versus BCMD Cadet Twin by Euclid (for solid oral dosages), Speedy Wet Cadet by Euclid (for liquids), Pace Setter by Accu-Chart Plus Health Care Systems (for odd shaped dosage forms) and Med Carousel® by McKesson for drug storage (study duration not reported)

dispensing errors

pre: 42 errors/ week post: 1.8 errors/ week

RRRr=96%

A-37



intervention (study

duration)


Limitations Oswald:68 Description of dispensing process pre-implementation of carousel unclear. Unclear how carousel fits in process. Poon:69 Authors stated that favourable results may be combined effect of bar-coding and redesign efforts; that Hawthorne effect may have been seen because staff knew they were being observed; and that study excluded dispensing narcotics or at night and weekends. Ragan:76 Insufficient information provided to assess validity of study. ADE=adverse event; AHRQ=Agency for Healthcare Research and Quality; BCMD=bar code for medication dispensing; CI= confidence interval; ne=not estimable; RRc=relative risk calculated; RRRc=relative risk reduction calculated; RRRr=relative risk reduction reported

Table 5: Results of bar-coding for medication administration Study (funding) Setting Control versus

intervention (study

duration)


Drug administration Brown59 before and after study

US 250-bed community hospital (53-bed surgical orthopedic unit)

standard practice versus Clinicare by Clinicom Inc., type of bar code technology that is part of BTS (8 months pre and 5 months post)

medication errors/1,000 doses dispensed

pre: 0.7 post 0.7

RRRc=0%

Johnson57,86 retrospective before and after study

US VA hospital

manual paper chart system versus BCMA developed by Veteran’s Health Administration (pre: 1993 to last year of manual system; post: up to 2001)

medication errors/patient doses dispensed (% reported errors)

pre: 409/ 1,885,651 (0.0217) post: 22/ 460,795 (0.003)

RRRr=86.2%

Low58 retrospective before and after study

US mid-west government hospital (2 medical- surgical units)

standard practice versus Tremont BCMA SC-2015 (12 months pre and 12 months post)

medication administration error (mean/ 1,000 doses of medication administered)

pre: 0.125 post: 0.145, p=0.616

RRIr=18%

A-38


intervention (study

duration)


Puckett60 before and after study

US 326-bed primary and tertiary care hospital

manual medication profile and administration record versus Clinicare by Clinicom Inc., type of bar code technology that is part of BTS (1 year pre and 2 years post)

medication error rate (reported errors / doses administered)

pre: 0.17% post: 0.05%

RRRc=70.6%

Reifsteck70 prospective before and after study

US 900-bed network of acute care hospitals (500 medication doses administered to 110 patients)

comparator and BCMA not described; pharmacy uses bar code-driven centralized robotic drug distribution system supplemented by unit-based medication cabinets on patient floors (study period not reported, study was conducted March 2000 to November 2003)

medication administration errors (%) errors excluding wrong time (%)

pre: 216 (23.5%) post: 32 (5.2%) pre: 55 (9.9%) post: 10 (1.9%)

RRc=0.22 (0.16, 0.32) RRRr=77.9% RRc=0.19 (0.10, 0.37) RRRr=80.8%

Rough63 abstract prospective before and after study

US teaching hospital (28-bed inpatient pilot unit)

paper-based Kardex system versus BCMA not described (450 observations pre-implementation; data collected from BCMA system over 17 days post-implementation with 7013 doses checked)

medication administration error rate near-miss

pre: 9.1% post: 1.2% post: 3.2% of doses

RRRc=86.8%

Work71 prospective before and after study

US 175-bed community

system pre-implementation not described versus Care

average monthly medication administration

pre: 2.37 post: 0.42

RRRr=82.3%

A-39


intervention (study

duration)


hospital (family care, critical care, oncology, surgery and telemetry)

Fusion Care Med 3.0; Symbol Technologies PPT 8800 handheld (range 4 to 15 months)

errors

Limitations Ascertainment methods used in some studies (how medication errors were identified). Brown:59 No impact could be measured with implementation of bar-coding for drug administration, part of a BTS, due to series of organizational events that occurred during study, particularly reduction in staffing. Johnson:57 BCMA developed in-house, and no description provided. Limited information on study design, the setting (1 centre included), and ascertainment methods used in pre-implementation phase. Low:58 Reporting of medication administration errors pre-BCMA implementation was responsibility of nurses, while reporting post-BCMA implementation automated. This suggests that greater percentage of errors potentially reported after BCMA implementation than before. Also lack of familiarity of staff with new software during first month of BCMA implementation, which may have influenced results post-implementation. Rough:63 Results reported in abstract, which contained little information on pre-implementation period.

Blood and blood product administration Chan74 retrospective before and after study

China regional hospital (all patient units except psychiatry and emergency room)

conventional second checker system versus PathFinder Ultra by Monarch UPN Alliance (4 years pre and 3 years post)

blood transfused to wrong patient wrong labelling of blood samples or request forms

pre: 0 post: 0/27,000 units of blood pre: 13 post: 0/41,000 blood sampling procedures

RRRc=0% RRRc cannot be calculated given the data

Davies72 prospective before and after study (National Blood Service)

UK 1 centre (cardiac recovery unit and cardiothoracic ward)

standard transfusion procedure versus handheld computer and laser scanner: Symbol; and portable printer: Zebra (study duration not reported but included first 50 RBC transfusions)

near-miss/first 50 RBC transfustions

post: 1/ 50

A-40


intervention (study

duration)


identification errors at any step of process (pilot)

post (2004): 10/ 8,824 instances of system activity (8 of which were in dispensing stage)

RRr 2002 versus 2004 =3.33 (0.92, 12.1) RRr 2003 versus 2004 =9.98 (1.28, 78.0)

Porcella80 historical cohort study (AHRQ)

US 772-bed tertiary care teaching hospital (pilot in adult in-patient, pediatric in-patient, ICU, and adult transplant units)

standard ID band with manual process versus bar-coded ID band and different mobile carts based on different work processes; bar-coding on collection, dispensing, and administration of blood products (pre: 2002 and 2003 versus post: April 21 to December 27, 2004)

identification errors at any step of process (house-wide go-live)

RRr 2003 versus 2004 =30.6 (9.5, 98.4) based on a total activity of 22,569

Limitations Chan:74 Blood samplings and units given pre-implementation not reported. Porcella:80 Patient care units chose to use different bar-coding technologies based on their preferences.

AHRQ= Agency for Healthcare Research and Quality; BCMA=bar code for medication administration; BTS=bedside terminal system; CI=confidence interval; ICU=intensive care unit; ID=identification; RBC=red blood cell; RRc=relative risk calculated; RRIr=relative risk increase reported; RRRc=relative risk reduction calculated; RRRr=relative risk reported; VA=Veteran’s Affairs

A-41

Table 6: Results for multiple technologies Ward-based ADD and bar-coding for medication administration

Study (funding)


intervention (study

duration)


Skibinski83 before and after study; interrupted time series (Baxter Healthcare Corporation)

US teaching centre (general medical and medical ICUs)

Traditional patient-specific distribution system (individual patient medication drawer) and visual check of wristband and second form of patient identification versus POC system (3 bar code checks of patient, nurse, and drug) with RX (automated cabinet) (3 months pre and 3 months post, with 6 months washout November 2002 to July 2005; implementation of system October 2003 to August 2004)

% errors for dispensing from central pharmacy % errors for BCMA for incorrect patient % errors for BCMA for incorrect medication medication errors/1,000 patient days mean±sd medication errors/1,000 patient days (entire hospital; from voluntary reports)*

pre: 2.1 post: 0.02 pre:0.7 post: 0.07 p=0.003 pre:0.2 post:0.2 p=0.677 too few data points to do time series analysis pre: 7.36±2.69 post: 6.64±2.42

RRRc=99.0% RRRc=90.0% RRRc=0% RRRc=9.8%

Bar-coding for medication administration and eMAR Anderson77 before and after study

US 440-bed acute care hospital (35-bed pilot unit)

comparator and BCMA/eMAR not described (1 month, for 35-bed pilot unit and 6 months for all units)

medication error rate (6 months) near-miss (1 month)

post: 80

RRRr=44.0%

A-42


Study (funding)


intervention (study

duration)


Foote78

before and after study

US 300-bed community hospital (33-bed medical- surgical ward)

comparator not described versus Cerner electronic documentation system and wireless PC containing MAR (4 week pilot)

medication error RRRr=80%

Morriss81 prospective cohort (American Society of Health-System Pharmacists Research and Education Foundation and University of Iowa Pharmaceutical Enterprise)

US teaching hospital (36-bed NICU)

manual MAR versus Cerner Bridge Medication Administration system v. 3.4 by Cerner Corporation (19 weeks pre; 4 weeks washout; half the beds received intervention for 9 weeks and 3 weeks with 4 week interval between; remainder of beds added for 19 weeks post, for total of 50 weeks)

unadjusted medication errors/1,000 doses unadjusted potential ADEs/1,000 doses unadjusted targeted, preventable ADEs/1,000 doses adjusted targeted, preventable ADEs (adjusted for covariates such as opportunities for error, birth weight, sex, race)

pre: 69.5 post: 79.7 pre: 15.1 post: 4.4 pre: 0.86 post: 0.43

RRIc=14.7% p<0.001 RRRc=70.9% p<0.001 RRRc=50.0% p=0.008 RRr=0.53 (0.29, 0.98) RRRr=47% p=0.044

A-43


Study (funding)


intervention (study

duration)


Paoletti79 controlled before and after study

US 521-bed general hospital (control group: 20-bed cardiac telemetry; intervention 1: 20-bed cardiac telemetry; intervention 2: 36-bed medical-surgical unit

manual 5-day MAR versus BCMA and eMAR not described (pre and post study period not reported; implementation began on August 2003)

medication administration error includes time and technique errors/doses observed (% errors)

pre control: 60/306 (19.6%) group 1: 78/308 (25.3%) group 2: 50/320 (15.6%) post control: 63/306 (20.6%) group 1: 61/ 318 (19.2%) group 2: 31/ 310 (10.0%)

control: RRIr=5.1% p=0.762 group 1: RRRr=24.1% p=0.065 group 2: RRRr=35.9% p=0.035

Ward-based ADD, bar-coding for medication administration, and eMAR Franklin73 prospective before and after study (MDG Medical and Department of Health’s Patient Safety Research Programme)

UK teaching hospital (28-bed general surgery ward)

medication prescribed on paper drug charts, and medication stored in 2 drug trolleys and stock cupboards versus closed-loop system incorporating electronic prescribing, ward-based automated dispensing, bar code patient identification and eMAR, ServeRx v.1:13 by MDG Medical (2 weeks)

medication administration errors (%) mean severity score patient identification not verified prior to administration

pre: 141/1644 opportunity for errors (8.6%) post: 53/1178 opportunity for errors (4.4%), difference of −4.2% (95% CI: −2.4%, −6.0%), p=0.0003 pre:2.7 post: 2.5, p=0.39 pre: 1110/ 1344 doses (82.6%) post: 244/1291 doses (18.9%), difference of 63.7% (95% CI: 60.8%, 66.6%) p<0.001

RRc=0.53 (0.39, 0.71) RRRc=47.5% RRRc=7.4% RRc=0.23 (0.20, 0.26) RRRc=77.1%

Limitations Skibinski:83 Setting described as Rehabilitation Centre, yet units chosen for study are general medical units and

A-44


Study (funding)


intervention (study

duration)


ICU; emergency medications excluded. Anderson:77 No information provided on brand and suppliers used. Study methods poorly described. Paoletti:79 Length of study period pre-implementation not reported, although number of doses observed pre and post were similar. Franklin:73 Observation period spanned 14 days. Results obtained from surgical ward where typically patients are not administered large number of medications. This may imply less chance of medication administration error occurring. Severity score used to judge potential harm of errors not explained.

* data obtained from graphs ADD=automatic dispensing device; BCMA=bar-coding for medication administration; CI=confidence interval; MAR=medication administration record; NICU=neonatal intensive care unit; POC=point of care; RRc=relative risk calculated; RRIr=relative risk increase reported; RRRc=relative risk reduction calculated; RRRr=relative risk reduction reported; sd=standard deviation

A-45

APPENDIX 6: Economic Tables

Table 7: Elements of economic evaluation and quality of study Element Criteria in CADTH guidelines Potential quality issues in

hospital pharmacy automation study

Interventions Study intervention and comparator should be identified

Study with fictitious intervention is ineligible.

Type of study Study can be cost comparison, cost- effectiveness (utility), or cost-benefit analysis

If cost-comparison study done, outcomes under 2 interventions should be same.

Perspective Government or societal perspective should be undertaken.

None

Timelines All future, downstream events related to interventions should be taken into account.

If automation lasts more than 1 year, then 1 year study may not capture all relevant events.

Outcomes Outcomes should be expressed in terms of patient health status.

Comparison of costs that ignores outcomes would be invalid.

Effectiveness Method should identify effectiveness. Quality of study can be ranked from highest to lowest, as randomized trial, observational study, literature review (where data are obtained), and professional opinion.

Costs Study should identify physical units, unit prices, and dollar costs. Future costs should be discounted.

Some resource items might be excluded. Future costs might not be discounted.

Sensitivity analysis Assumptions where there is uncertainty should be supplanted by sensitivity analyses.

None

Overall assessment Studies should be incremental and include costs and outcomes.

Some studies might include only costs.

A-46

Table 8: Characteristics of economic studies Study

(funding) Intervention Control Study

design Setting

Ward based automated dispensing devices Lee90

ADD Pyxis® Medstation™ by Pyxis

manual ward stock system

prospective before and after study

US; 1000-bed tertiary care centre. 1 general medical and 1 surgical unit. Items included controlled and non-controlled substances and IV solutions

Frick91

ADD not specified

narcotic ward stock in locked cabinet, manual distribution


US; 28-bed medical unit in large tertiary hospital. System delivered narcotics. Hospital not described

Schwarz56

ADD with profiling Medstation™ Rx by Pyxis

manual unit dose cart exchange

cost-benefit analysis, prospective before and after study

US; 36-bed surgical ward, 8- bed cardiac ICU ward in 560-bed teaching hospital.

Wise92 (Pyxis Corporation)

ADD Pyxis® Medstation™ by Pyxis

manual cart exchange plus narcotics in Pyxis® Medstation™

cost-benefit analysis, prospective, before and after study

US; 2 26-bed medical units, each with 2 depositories. Regular drugs and narcotics included. Hospital not described.

Guerrero93 (Baxter)

ADD Sure Med by Baxter and manual unit dose cart exchange

manual unit dose cart distribution


US; 400-bed teaching hospital. Surgical ICU and medicine unit. Drugs included all regular and first doses, missing doses, controlled and as-needed medications.

Buchanan94

centralized sterile product dispensing plus ADD with profiling (Pyxis® Profile™) or robotic unit dose dispensing

decentralized dispensing system, or combined centralized and decentralized dispensing system

modelling US; Large teaching hospital. Initially 4 pharmacy satellites providing decentralized dispensing services.

Poveda Andrés95

ADD by Pyxis


cost-benefit analysis, before and after study

Spain; 3 patient care units (medical ICU, surgical ICU, ER) in teaching hospital

Dib67

ADD with profiling

unit dose cassette exchange with open-floor-stock medication system


Saudi Arabia; 390-bed tertiary care hospital. 5 nursing patient care units. Drugs included unit dose, bulk medications, and narcotics

A-47

Table 8: Characteristics of economic studies Study

(funding) Intervention Control Study

design Setting

Kheniene96

ADD OmniRx® by Omnicell


cost-benefit analysis, prospective before after study

France; 12-bed ICU in teaching hospital

Pharmacy-based automated dispensing devices Klein53

automated packaging and dispensing system ATC 212™ by Baxter

manual unit dose cart exchange


US; Central pharmacy with 4 satellites in 650-bed tertiary teaching hospital. Excluded narcotics.

Slee66

centralized robotic dispensing Rowa Speedcase

not described prospective before and after study

UK; District general hospital.

Fitzpatrick65

ADD Consis by Baxter

manual dispensing system


UK; Centralized hospital dispensary. Hospital not described.

Bar-coding for medication dispensing Maviglia97 (AHRQ)

bar code assisted unit dose dispensing system with carousel storage system White system series 2400 by Omnicell

manually filled automated ward based cabinets plus patient specific unit dose dispensing

cost-benefit analysis, modelling

US; Centralized pharmacy in 735-bed tertiary care hospital

Kamon98

bar code assisted unit dose dispensing system with carousel storage system White system series 2400 by Omnicell

manually filled automated ward based cabinets plus patient specific unit dose dispensing

cost benefit analysis, modelling

UK; Hypothetical 400-bed acute care hospital

Bar-coding for medication administration Poon99 not specified not specified prospective

before and after study

US; Medical and surgical units and ICU in 735-bed tertiary care hospital

ADD=automated dispensing device; BCMA=bar code medication administration; CPOE=computerized prescriber order entry; eMAR=electronic medication administration record;

ER=emergency room; ICU=intensive care unit; IV=intravenous

A-48

Table 9: Results from economic studies Study Key findings

for outcomes Key findings for physical

resources Key findings for cost and

financial analysis

Quality assessment

Ward based automated dispensing devices Lee90 Not addressed Before ADD, nurses devoted

10.2% of time to medication provision; 5.6% after (p<0.05). * Before and after ADD pharmacy personnel spent 7.17 / 48.96 minutes per nursing unit on floor stock activity and 7.15 / 1.36 minutes on billing.*

Gain in net revenue resulted from increase in charge capture. Authors deducted annual equipment rental and service fees, and stocking costs from revenue gains, and showed net gain to hospital of US$35,000.

Prospective observational. Number of quality criteria met=3 or not met=3. N/A 2

Frick91 Not addressed For ward using 750 narcotics per month, savings in nurses’ time spent on narcotic administration 25 hours per month.

Not addressed Prospective observational. Number of quality criteria met=1 or not met=5. N/A 2

Schwarz56 On surgical ward, ADD associated with reduction from 0.53 to 0.13 missing doses per patient day. ADD associated with reduction in medication error incidents reported per patient day from 0.0075 to 0.0058. In ICU, missing doses were 0.54 / 0.19, and medication errors increased from 0.0051 to 0.0090.

Before and after ADD, nurse medication acquisition time 107 / 48 seconds per dose for both units. For pharmacy staff, processing time per unit dose increased from 2.7 minutes to 2.9 minutes per patient day on surgical ward, and fell from 13.1 to 5.8 minutes in ICU.

In projections to 10 acute care units and 4 ICUs, 5-year analysis incorporated annual leasing and service costs, reductions in nursing and pharmacy personnel, and reductions in narcotic pilferage costs. Net savings were US$1 million over 5-year period.

Prospective observational. Number of quality criteria met=5 or not met=3

A-49




financial analysis

Quality assessment

Wise92 Not addressed In 8 hour shift, nursing time to gather medications before and after ADD 33.8 / 36.1 minutes. Nurse medication administration time 29.4 / 22.7 minutes. Before and after ADD pharmacy technician time for related activities 65 / 70 minutes per 8 hour shift. Pharmacist time 45 / 5 minutes per shift.

Authors reported net savings of US$14,480 annually per ward after ADD costs subtracted from nursing and pharmacy cost savings. Inclusion of recovered charges led to savings of US$80,910 per nursing unit.


Guerrero93 Not addressed All nursing activities related to medications 20.7% before and 18.4% after ADD in medical ward, and 10.8% and 11.0% in ICU. Pharmacist activities that were clinical 27.9% before ADD and 35.1% after for SICU, and 36.5% and 49.1% for the medicine unit. No differences were statistically significant.


Buchanan94

Not addressed Simulation predicted pharmacist and technician FTEs to fall from 27 to 25 and 33 to 19 respectively with Pyxis™ Profile and to 27 and 21 with robotic dispensing. When robot dispensing implemented actual pharmacist and technician FTEs 22 and 30 respectively.

Not addressed Model. Number of quality criteria met=1 or not met=5. N/A 2

Poveda Andrés95

Not addressed Nurse time reduced by 18%, clinical assistants by 10%, administrative assistants by 22%.

5-year net benefits €300,525 including equipment costs, personnel costs, inventory reduction, and reduction in use of drugs.

Observational and model. Number of quality criteria met=5 or not met=3

A-50




financial analysis

Quality assessment

Dib67 Medication adverse events after ADD implemented fell by 27%.†

Not addressed Acquisition costs of drugs during 1-month fell from US$23,736 to US$13,804. Results not usable


Kheniene96

not addressed Nursing time on medication – related activities fell from 4.4 hours per workday before introduction of ADD to 2.5 hours after. Technician time 0.26 hours/workday before and 0.94 hours/day after ADD.

Net 5 year benefits €14,317 per year including equipment purchase and maintenance, nursing and pharmacy personnel, and inventory costs (cost of holding and stock outages).


Pharmacy based automated dispensing devices Klein53 Dispensing errors

0.84% of doses before automation, and 0.65% after.

Daily technician time to fill 35 carts 457 minutes without automation and 333 minutes with automation. Daily pharmacist checking time fell from 211 minutes to 186 minutes with automation.

Savings from automated system US$7,044 annually included drug acquisition costs and personnel time. Except unit dose packaging costs, automation costs excluded.

Prospective observational. Number of quality criteria met=2 or not met=2. N/A 1. Unclear 3

Slee66

Dispensing errors fell by 50% in first 4 months after ADD implementation.

Technician hours per day in dispensary fell by ~ 30%. Technician hours on patient care units increased by 50%. Shelving space fell by 70% after ADD, and floor space fell by 50% in dispensary.


A-51




financial analysis

Quality assessment

Fitzpatrick65

Dispensing errors fell by 104.9 per 100,000 items dispensed (decrease of 16%).

Pharmacy staff hours per week spent in labelling and dispensing, checking, and restocking fell from 458 to 371 (19%). Footprint in dispensary’s drug storage space fell after ADD from 14.3 m2 to 10.2 m2. Total pharmacy storage space fell from 9 m3 to 7m3.


Bar-coding for medication dispensing Maviglia97 Dispensing errors

(potential adverse drug events) fell from 0.19% to 0.07% of dispensed doses after bar code verification introduced.

Not reported Costs consisted of equipment, set up, planning (all one-time) and recurring costs (operations, labour, lease, repackaging). Benefits consisted of costed differences in adverse events between pre and post periods. Net benefits over 5 years US$3.49 million.

Cost benefit analysis, modeling. Number of quality criteria met=7 or not met=1. Costs of alternative intervention excluded.

Karnon98 Potential adverse drug events: baseline 432; automation 362 incidences per 162,000 prescriptions annually.

Not addressed Cost items were equipment purchase and operation, and treatment and human cost of adverse events. Net benefits in 40-bed general hospital £13.1 million over 5 years.

Model. Number of quality criteria met=4 or not met=2. N/A 1. Uncertain 1

A-52




financial analysis

Quality assessment

bar-coding for medication administration Poon99 Not addressed Nurses spent 26.5% and 24.5% of

time on drug administration before and after BCMA implementation (p=0.22). Nurses spent 38.3% of time on all medication related activities, including documentation, before BCMA and 33.4% after (p<0.001).

Not addressed Prospective observational Number of quality criteria met=2 or not met=3. N/A 3

Italicized text indicates results are in favour of automation. *Results were statistically significant. †Medication adverse events included wrong drug, dose, or patient, missed dose, policy, procedure, medication administration, order entry, or documentation error, and adverse drug reactions. ADD=automated dispensing device; BCMA=bar-coding for medication administration; ICU=intensive care unit; SICU=surgical intensive care unit

Table 10: Values and data sources for model

Item Value for manual option*

Value for automation

option*

Data source

Equipment, ward-based ADD, unprofiled

-- $28,400 Based on discussion with health care providers, we estimated cost of unprofiled ADD with 6 drawer main and 6 mini or cubie drawers ($61,000) and 7 drawer auxiliary ($62,000) which would be used in medical patient care unit. Total cost $123,000 allocated over 5 years and amortized with 5% time discount results in annual cost of $28,400.

Equipment, ward-based ADD, profiled

-- $31,900 Based on discussion with health care providers, we estimated cost of profiled ADD with 6 drawer main and 6 mini or cubie drawers ($76,000) and 7 drawer auxiliary ($62,000) which would be used in medical patient care unit. Total cost $138,000 allocated over 5 years and amortized with 5% time discount rate results in annual cost of $31,900.

Planning, unprofiled

-- $17,040 Maviglia97 estimated planning costs for central unit to be 60% of total capital costs. Annualized over 5 years at 5%, costs are $17,124.

Planning, profiled

-- $19,100 Maviglia97 estimated planning costs for central unit to be 60% of total capital costs. Annualized over 5 years at 5%, costs are $19,100.

Annual maintenance

$1,440 According to health care providers, monthly fee for equipment maintenance is $120; amounts to $1,440 per year.

A-53

Table 10: Values and data sources for model Item Value for

manual option*

Value for automation

option*

Data source

Nursing time spent on drug administration per patient day Patient care unit ICU Nurse costs per patient day Patient care unit ICU

0.585 hour 0.532 hour $23.40 $21.28

0.363 hour 0.33 hour $14.52 $13.20

For patient care unit in hospital with automated distribution, we used estimates from Baker (Cardinal Health: unpublished data, 2008) for number of medication orders per patient day (7.7). For nurse administration time, per order, under automation, we used average for all 7 hospitals in Baker’s study: 2.83 minutes (21.79 minutes per patient day for 7.7 orders). Using summary statistics (Appendix 6, Table 5), we estimated administration time in automated patient care unit to be 38% less than that in patient care unit with manual system. Nurse wages $40 per hour. For 20-bed patient care unit, 90% occupancy assumed. Same method used for ICU, except administrative time 2.61 minutes per order (Baker et al. Cardinal Health: unpublished data, 2008)

Pharmacist time per patient day

0.0423 hour 2 minutes and 31 seconds or 0.0419 hour

Time per patient with automated system based on Baker (Cardinal Health: unpublished data, 2008). Time per patient using manual system based on average from literature review.

Technician time per patient day

0.35 hour 13 minutes and 6 seconds or 0.21 hour

Time per patient with automated system based on Baker (Cardinal Health: unpublished data, 2008). Time per patient using manual system based on average from literature review.

Inventory 0 0 No difference in base model Hourly cost / nurse

$40 $40 For Alberta nurse with 9 years experience, in 2008101

Hourly cost / pharmacist

$40.24 $40.24 Mid range wage in 2007 for Capital Health, Edmonton, AB102

Hourly cost / technician

$25.60 $25.60 Mid range wage in 2007 for Capital Health, Edmonton, AB102

* All monetary values in Canadian 2008 dollars

Table 11: Estimates of changes in personnel time due to ward-based ADD

Study Registered nurses Pharmacists Pharmacy technicians Lee90 nu nr nu Frick91 nu nr nr Schwarz56 −45% nr 7% Wise92 nr −88% 8% Guerrero93 nr nu nr Buchanan94 nr 42% −7% Poveda Andrés95 −18% nr −10% Dib67 nr nr nr Kheniene96 −43% −50% nr

Average −38% −60% −1%

nr=differences were reported; nu=results were not usable

A-54

Table 12: Itemized costs in patient care unit model, unprofiled ADD Year one to five

No ADD Nurse $153,738 $153,738 $153,738 $153,738 $153,738 Pharmacist $11,183 $11,183 $11,183 $11,183 $11,183 Technician $58,867 $58,867 $58,867 $58,867 $58,867 Inventory $0 $0 $0 $0 $0 Total $223,788 $223,788 $223,788 $223,788 $223,788 Discount factor 0.95238095 0.907029 0.863838 0.822702 0.783526 Present value $213,132 $202,983 $193,317 $184,111 $175,344 $968,886ADD, unprofiled Equipment $28,400 $28,400 $28,400 $28,400 $28,400 Planning $17,040 $17,040 $17,040 $17,040 $17,040 Maintenance $1,440 $1,440 $1,440 $1,440 $1,440 Nurse $95,396 $95,396 $95,396 $95,396 $95,396 Pharmacist $11,077 $11,077 $11,077 $11,077 $11,077 Technician $35,320 $35,320 $35,320 $35,320 $35,320 Total $188,674 $188,674 $188,674 $188,674 $188,674 Discount factor 0.95238095 0.907029 0.863838 0.822702 0.783526 Total discounted $179,690 $171,133 $162,984 $155,223 $147,831 $816,860

difference of $152,026

ADD=automated dispensing device

Table 13: Itemized costs in patient care unit model, profiled ADD Year one to five

No ADD Nurse $153,738 $153,738 $153,738 $153,738 $153,738 Pharmacist $11,183 $11,183 $11,183 $11,183 $11,183 Technician $58,867 $58,867 $58,867 $58,867 $58,867 Inventory $0 $0 $0 $0 $0 Total $223,788 $223,788 $223,788 $223,788 $223,788 Discount factor 0.95238095 0.907029 0.863838 0.822702 0.783526 Present value $213,132 $202,983 $193,317 $184,111 $175,344 $968,886ADD, profiled Equipment $31,900 $31,900 $31,900 $31,900 $31,900 Planning $19,100 $19,100 $19,100 $19,100 $19,100 Maintenance $1,440 $1,440 $1,440 $1,440 $1,440 Nurse $95,396 $95,396 $95,396 $95,396 $95,396 Pharmacist $11,077 $11,077 $11,077 $11,077 $11,077 Technician $35,320 $35,320 $35,320 $35,320 $35,320 Total $194,233 $194,233 $194,233 $194,233 $194,233 Discount factor 0.952381 0.907029 0.863838 0.822702 0.783526 Total discounted $184,984 $176,175 $167,786 $159,796 $152,187 $840,927

difference of $127,959


A-55

Table 14: Itemized costs in ICU, unprofiled ADD Year one to five

No ADD Nurse $54,370 $54,370 $54,370 $54,370 $54,370 Pharmacist $4,323 $4,323 $4,323 $4,323 $4,323 Technician $22,893 $22,893 $22,893 $22,893 $22,893 Inventory $0 $0 $0 $0 $0 Total $81,586 $81,586 $81,586 $81,586 $81,586 Discount factor 0.95238095 0.907029 0.863838 0.822702 0.783526 Present value $77,701 $74,001 $70,477 $67,121 $63,925 $353,226ADD, unprofiled Equipment $28,400 $28,400 $28,400 $28,400 $28,400 Planning $17,040 $17,040 $17,040 $17,040 $17,040 Maintenance $1,440 $1,440 $1,440 $1,440 $1,440 Nurse $34,211 $34,211 $34,211 $34,211 $34,211 Pharmacist $4,282 $4,282 $4,282 $4,282 $4,282 Technician $13,736 $13,736 $13,736 $13,736 $13,736 Total $99,109 $99,109 $99,109 $99,109 $99,109 Discount factor 0.95238095 0.907029 0.863838 0.822702 0.783526 Total discounted $94,390 $89,895 $85,614 $81,537 $77,655 $429,091

difference of −$75,866


Table 15: Itemized costs in ICU, profiled ADD

Year one to five No ADD Nurse $54,370 $54,370 $54,370 $54,370 $54,370 Pharmacist $4,323 $4,323 $4,323 $4,323 $4,323 Technician $22,893 $22,893 $22,893 $22,893 $22,893 Inventory $0 $0 $0 $0 $0 Total $81,586 $81,586 $81,586 $81,586 $81,586 Discount factor 0.95238095 0.907029 0.863838 0.822702 0.783526 Present value $77,701 $74,001 $70,477 $67,121 $63,925 $353,226ADD, profiled Equipment $31,900 $31,900 $31,900 $31,900 $31,900 Planning $19,100 $19,100 $19,100 $19,100 $19,100 Maintenance $1,440 $1,440 $1,440 $1,440 $1,440 Nurse $34,211 $34,211 $34,211 $34,211 $34,211 Pharmacist $4,282 $4,282 $4,282 $4,282 $4,282 Technician $13,736 $13,736 $13,736 $13,736 $13,736 Total $104,669 $104,669 $104,669 $104,669 $104,669 Discount factor 0.95238095 0.907029 0.863838 0.822702 0.783526 Total discounted $99,685 $94,938 $90,417 $86,112 $82,011 $453,163

difference of −$99,938


A-56

Table 16: Costs of patient care units and ICU with or without automation Results Annual cost per

unit: manual delivery

Annual cost per unit: automation

Difference or savings*

Unprofiled ADD (base case 1) Patient care unit $968,000 $816,000 $152,000ICU $353,000 $429,000 −$76,000

Profiled ADD (base case 2) Patient care unit $968,000 $840,000 $128,000ICU $353,000 $453,000 −$100,000

Sensitivity analyses: patient care units, ward-based ADD, unprofiled

Base case except equipment cost increased by 10%

$968,000 $836,000 $132,000

Base case except under automation nurse time reduced by 45%

$968,000 $769,000 $199,000


$968,000 $949,000 -$19,000

Base care except pharmacist time increased by 42%

$968,000 $796,000 $172,000

Base care except pharmacist time reduced by 88%

$968,000 $764,000 $194,000

Base case except technician time reduced by 1%

$968,000 $815,000 $153,000

Base case except technician time increased by 8%

$968,000 $939,000 $29,000

Base case except inventory savings 10% of nursing costs

$1,035,000 $816,000 $219,000

Base case except planning costs 30% of capital

$968,000 $779,000 $189,000

Sensitivity analyses: patient care units, ward-based ADD, profiled Base case except equipment cost increased by 10%

$968,000 $864,000 $114,000


$968,000 $797,000 $171,000

A-57

Table 16: Costs of patient care units and ICU with or without automation Results Annual cost per

unit: manual delivery

Annual cost per unit: automation

Difference or savings*


$968,000 $977,000 -$9,000

Base care except pharmacist time increased by 42%

$968,000 $824,000 $144,000

Base care except pharmacist time reduced by 88%

$968,000 $792,000 $176,000

Base case except technician time reduced by 1%

$968,000 $844,000 $124,000

Base case except technician time increased by 8%

$968,000 $967,000 $1,000

Base case except inventory savings were 10% of nursing costs

$1,035,000 $872,000 $163,000

Base case except planning costs are 30% of capital

$968,000 $807,000 $161,000

*negative means higher cost for automation

Documents

t echnolo g y r ep ort HTA Technologies to Reduce Errors ... · medication dispensing devices, bar-coding verification for medication dispensing and administration, and electronic