Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Technological University Dublin Technological University Dublin
ARROW@TU Dublin ARROW@TU Dublin
Articles School of Marketing
2013
Simulation-based Framework to Improve Patient Experience in an Simulation-based Framework to Improve Patient Experience in an
Emergency Department Emergency Department
Waleed Abo-Hamad Technological University Dublin, [email protected]
Amr Arisha Technological University Dublin, [email protected]
Follow this and additional works at: https://arrow.tudublin.ie/buschmarart
Part of the Business Commons
Recommended Citation Recommended Citation Abo-Hamad, Waleed, Arisha, Amr: Simulation-based framework to improve patient experience in an emergency department, European Journal of Operational Research, Volume 224, Issue 1, 1 January 2013, Pages 154-166, ISSN 0377-2217. http://dx.doi.org/10.1016/j.ejor.2012.07.028.
This Article is brought to you for free and open access by the School of Marketing at ARROW@TU Dublin. It has been accepted for inclusion in Articles by an authorized administrator of ARROW@TU Dublin. For more information, please contact [email protected], [email protected].
This work is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 4.0 License
1
Simulation-based Framework to Improve Patient Experience in an
Emergency Department
Waleed Abo-Hamad, Amr Arisha
3S Group, College of Business
Dublin Institute of Technology
Dublin, Ireland
Abstract
The global economic crisis has a significant impact on healthcare resource provision
worldwide. The management of limited healthcare resources is further challenged by the high
level of uncertainty in demand, which can lead to unbalanced utilisation of the available
resources and a potential deterioration of patient satisfaction in terms of longer waiting times
and perceived reduced quality of services. Therefore, healthcare managers require timely and
accurate tools to optimise resource utility in a complex and ever-changing patient care
process. An interactive simulation-based decision support framework is presented in this
paper for healthcare process improvement. Complexity and different levels of variability
within the process are incorporated into the process modelling phase, followed by developing
a simulation model to examine the impact of potential alternatives. As a performance
management tool, balanced scorecard (BSC) is incorporated within the framework to support
continual and sustainable improvement by using strategic-linked performance measures and
actions. These actions are evaluated by the simulation model developed, whilst the trade-off
between objectives, though somewhat conflicting, is analysed by a preference model. The
preference model is designed in an interactive and iterative process considering decision
makers preferences regarding the selected key performance indicators (KPIs). A detailed
implementation of the framework is demonstrated on an emergency department (ED) of an
adult teaching hospital in north Dublin, Ireland. The results show that the unblocking of ED
outflows by in-patient bed management is more effective than increasing only the ED
physical capacity or the ED workforce.
Keywords: Simulation; Multiple Criteria Decision Analysis, Emergency Department;
Healthcare Management
2
1 Introduction
Healthcare managers are constantly under pressure to control rapidly escalating expenses
whilst simultaneously fulfilling the growing demand for healthcare services. As a result, they
are continuously studying the efficiency of existing healthcare systems and exploring
improvement opportunities. The evaluation of these proposed interventions is crucial prior to
their actual implementation, though challenged by intrinsic uncertainty of demands and
outcomes of healthcare systems; high level of human involvement at both patients level and
resource level (doctor, nurses, etc.); limited budget and resources; and large number of
variables (e.g., staff scheduling, number of beds, etc). Patients, on the other hand, in addition
to requiring a high service quality, are understandably no longer prepared to wait in queues
for essential health services. Accordingly, the healthcare service concept has shifted from
optimising resources utilisation to finding a balance between service for patients and
efficiency for providers (Brailsford & Vissers, 2011). Dealing with these inevitable
complexities within healthcare processes and services (e.g., hospitals, emergency
departments, and bed management) and addressing the challenges in the decision making
process is the focus of this paper.
Discrete-Event Simulation (DES) has proven to be an effective tool for process
modelling and improvement (Benneyan, 1997; Jun, Jacobson, & Swisher, 1999). Healthcare
managers can apply DES for assessing current performance, predicting the impact of
operational changes, and examining the tradeoffs between system variables (Wierzbicki,
2007). Furthermore, DES has been used to identify areas of improvement of service through
possible reorganisation of existing resources, for example; reorganisation of surgical and
anaesthesia care surrounding laparoscopic surgery (Stahl, et al., 2004); and planning for the
geographical locations of new healthcare services taking into account the demographics of
the population and the location of the patients who need the services (Harper, Shahani,
Gallagher, & Bowie, 2005). Furthermore, DES is well-suited to tackle problems in
emergency departments (EDs), where resources are scarce and patients arrive at irregular
times (Jun, et al., 1999), and effectively combine data mining (Ceglowski, Churilov, &
Wasserthiel, 2006) for better results. Though a substantial body of work has appeared in the
literature, most of these papers do not use strategy-linked performance measures and actions.
The development and application of a decision support system that can coordinate diverse
staff categories toward the strategy direction is becoming more pivotal for strategy
implementation (Voelker, Rakich, & French, 2001). Moreover, the decision-maker
3
preferences information is rarely considered in the process of alternative evaluation (Rosen,
et al. 2008).
The objective of this paper is to develop an interactive simulation-based decision
support framework to improve planning and efficiency of healthcare processes. A real-world
case study of an emergency department in one of the largest University Hospitals in Dublin is
investigated in order to enhance patients’ experience using the proposed framework.
2 Proposed Integrated Framework
2.1 Literature review
Efforts to develop DES models have been advancing since the late 1980s when Saunders et
al. (Saunders, Makens, & Leblanc, 1989) proposed a model to study the impact of key
resources on waiting times and throughput. Since that time, DES models have been used to
study the effect of a wide range of health interventions on healthcare processes’ performance,
for example; designing a new house staff work schedule (Dittus, Klein, DeBrota, Dame, &
Fitzgerald, 1996) and ambulance schedules (Ingolfsson, Erkut, & Budge, 2003); improving
capacity utilisation in intensive care units (S. C. Kim, Horowitz, Young, & Buckley, 1999;
Litvak, Van Rijsbergen, Boucherie, & Van Houdenhoven, 2008); planning healthcare
services (Oddoye, Jones, Tamiz, & Schmidt, 2009); and evaluating different health
interventions such as comparing the ‘individual surgeons’ strategy with the ‘pooled lists’
strategy for scheduling outpatient clinical appointments in surgical care (Vasilakis, Sobolev,
Kuramoto, & Levy, 2006).
Recently, calls for improved performance have grown significantly. Therefore,
applications for operational decision support are widespread and have become increasingly
significant (Eldabi, Paul, & Young, 2006). Most of these undertakings have focussed on
departmental operations; especially the more patient-intensive ones such as outpatient and
emergency departments. For example, Samaha et al. (Samaha, Armel, & Starks, 2003)
analysed the effect of the physical expansion of ED on patient stay times. However, physician
skills level was not considered in patients’ service time. The impact of different patient triage
methods on service times has been studied by (Connelly & Bair, 2004). Yet, variations in
patients’ arrival rate were not considered. The effect of staffing levels were investigated by
(Sinreich & Jabali, 2007) to reduce patient’s length of stay (LOS) and by (Ahmed &
Alkhamis, 2009) to determine the optimal number of required staff (e.g., doctors, lab
technicians and nurses) that maximise patient throughput and to reduce patient experience
4
time. In (Duguay & Chetouane, 2007), a number of alternatives based on adding resources
has been investigated with the objective to reduce patient waiting times and to improve
overall service delivery and system throughput. Additionally, (Thorwarth, Arisha, & Harper,
2009) examined the impact of staff scheduling on overall utilisation and burnout issues
related to over-utilised staff. The tradeoffs between different alternatives such as adding more
beds or altering the admission rate has been evaluated by (Khare, Powell, Reinhardt, &
Lucenti, 2009), where patient length of stay is considered as the key performance indicator
(KPI). However, aforementioned studies only consider a small number of KPIs (e.g., waiting
time and LOS), while other performance measures such as resource utilisation, productivity,
and layout efficiency are rarely considered. Moreover, linking these KPIs to the international
standard and national metrics is mostly neglected.
On the other hand, the balanced scorecard (BSC), pioneered by (Kaplan & Norton,
1992), is a systematic methodology that uses strategy-linked leading and lagging performance
measures and actions for planning and implementing an organization’s strategy (Kaplan &
Norton, 2001). With many successful implementations at different organizations, BSC is
considered as a popular model and effective means for performance management and strategy
execution. Furthermore, the BSC concept has been modified and successfully developed at
different types of healthcare organizations (Zelman, Pink, & Matthias, 2003). The BSC
usually has several perspectives (e.g., financial, internal operations, and patient) of the
healthcare facility performance, with each perspective composed of main objectives and sub-
objectives.
Yet, these objectives are interacting among themselves simultaneously, influencing
each other in a complex relationship network, often under conditions that involve
randomness, and requires the observation and evaluation of numerous decision criteria.
Therefore, a structured technique is needed for dealing with problems with multiple and
complex criteria influencing decision making (Dyer, Fishburn, Steuer, Wallenius, & Zionts,
1992; Liberatore & Nydick, 2008; Saaty, 1990).
2.2 An Interactive Simulation-based Decision Support Framework
An overview of the framework is given in Figure 1 where a detailed description of each
component is provided through the next sections. Further, the coordination between these
components is explained in details along with highlighting their points of integration.
5
2.2.1 Formulation and Understanding
Healthcare systems contain a high level of social interactions that are characterized by
complexity and in particular at decision points. Therefore, problems associated with
healthcare service delivery and managing patient flow are usually hard to define problems.
Gaining a better understanding of the healthcare process is essential for making correct
justifiable decisions and providing effective solutions. Therefore, it is necessary to formulate
the underlined problem from the point of view of the individuals who are directly involved in
the process of service delivery.
Data Collection
Data AnalysisBusiness Process Modelling
Simulation
Balanced Scorecard (BSC)
Conceptual Model
Multi-Criteria Decision Analysis (MCDA)
Aggregated Performance
Hospital Information System
Direct Observations Site VisitWork
Sampling
Processing Time
Problem Definition
Patient FlowResources TypesMain Processes
Tasks/SkillsShifts/IssuesLayout Analysis
Patients Allocation
Problem UnderstandingDecision Making Challenges
Interviews
Decision Maker (s)
Preferences of Decision Maker(s)
Scenarios/Plans
Populated BSC
Performance Value Tree
Performance Perspectives &
Measures
Form
ulat
ion
and
Und
erst
andi
ngC
once
ptua
lizat
ion
and
Anal
ysis
Impl
emen
tatio
n an
d D
ecis
ion
Mak
ing
Populated BSC
Patients Records
Data Preparation
Patients PatternsPatient Grouping
Inter-arrival TimeData Seasonality
Figure 1 An interactive simulation-based decision support framework
Accordingly, the data collection phase proceeds to gather relevant information of the
underlying processes. This phase focuses on the retrieval of the data and also on the
construction of a conceptual model (i.e., business process model). The quantitative data is
either stored in databases, written on documents, or recorded on any type of storage medium
where the qualitative data can be obtained via direct observation of the system and interviews
from the experts. In the healthcare context, experts are those who work in the hospital -
doctors, nurses, consultants, administrators and managers.
6
2.2.2 Conceptualizations and Analysis
The data collection phase combines data from observations and interviews with experts and
practitioners. This in return provided holistic insights for various system issues and aspects.
The underlined business processes are then mapped into a conceptual process model using
one of the well-developed modeling languages where sub-processes and activities are
identified. The control flow definition is created by identifying the entities that flow through
the system (e.g., patients, staff, and medical resources) and describing the connectors that link
the different parts of the process. Finally, the resources are identified and assigned to the
activities where necessary. The conceptual model is used in the simulation model for two
purposes: first it is guidance for the actual simulation model, which contains and considers a
higher degree of details, and second it is used as a communication platform in order to
validate the model with the experts working within the real system. Once the conceptual
model is completed, it is essential to validate it with the staff in the facility including senior
managers. This is an essential step for the credibility of the simulation model and hence its
output.
Interviews and observations have greatly contributed to a better understanding and an
accurate modeling of work flow in the healthcare facility. However, to incorporate the time
factor, patient’s records are collected from the hospital information system (HIS), including
information regarding patient care path, arrival time, mode of arrival, referral type, and time
of discharge or admission. The patient’s records are entered by different type of staff (e.g.
administrators, doctors, and nurses through the stages of patients care). Due to the high level
of pressures within healthcare processes, hospital records in many instances lack accuracy
and consistency. Therefore, prior to extracting any set of data from these records, data mining
procedures are required to extract the trustworthy set of records. This is followed by a further
analysis of the resulting records for extracting patient arrival patterns, patient groups,
distributions and seasonality features.
Planning, analysing, and evaluating the performance of any healthcare process are key
elements for setting a comprehensive framework to support continual improvement. To
transform objectives into a set of actions, managers need to set proper performance measures
and set a realistic and achievable target for each objective. Mostly, the targets are stemmed
from the national healthcare authorities (e.g., Health Service Executive (HSE) in Ireland and
National Health Service (NHS) in the UK). The performance perspectives and performance
measures collected through interviewing senior managers of the healthcare facility are used to
7
develop a BSC for the healthcare facility. This is essential to align the performance measures
of the healthcare facility (e.g., emergency department) with the strategic objectives of the
national health authorities.
These measures are used to represent the output of the simulation model.
Consequently, the simulation model will provide quantitative values of the provided
performance measures where qualitative measures such as patient satisfaction can be related
to measurable indicators such as average waiting time and LOS. Such integration allows the
evaluation of a wide range of actions and plans based on the recommendations of national
reports and surveys. These plans can then be evaluated in the form of what-if scenarios, with
the results are used to populate the design BSC.
2.2.3 Implementation and Decision Making
Once the conceptual model is validated, the model translation phase begins, which combines
the validated conceptual model and the results of the patients’ records analysis. The
simulation model can either be the programming of code, or modeling with the use of
simulation software package, which provides the modeler with tools that are typical and
essential for certain modeling. The procedure is often referred as model translation, because it
describes the transformation of the abstract conceptual model into a higher detailed complex
executable simulation model. Verification during the modeling phase ensures that the model
logic reflects the underlying business process. The difference between verification and
validation within the context of simulation modeling is that verification ensures that the
transformation of the conceptual model has been applied correctly, where validation
considers the representation of the model towards the system under investigation (Balci,
1997).
After the model verification and validation, the decision makers can use the model to
investigate the impact of decisions and alternatives (i.e., what-if scenarios) to foresee the
consequences of these decisions. The results can then be evaluated and interpreted by experts
and decision makers, which provide guidance on the implementation of suggested
alternatives and plans, as well as set benchmarks of the maximum performance that can be
achieved using the available resources and staffing levels. Hence, more practical solutions
and plans can be recommended and tested using the simulation model. Therefore, such
integration between simulation and BSC helps to focus on strategic visions to obtain desired
outcomes, assists in making better decisions, improves communication within the
organization, provides continual feedback on strategies, promotes adjustments to changes and
8
assists both individuals and organizations in achieving their goals and objectives. Moreover,
the capabilities of the simulation can provide interesting information about the causal-effect
relationships among performance.
However, the number of performance indicators (i.e., criteria) delays the evaluation
and analysis of the simulation results. This is due to the fact that some of these criteria are of
a conflicting nature and oppose each other. The trade-off between different objectives is
analysed using multi-criteria decision analysis (MCDA) in an interactive and iterative process
with senior managers. This is achieved by developing an aggregated preference model using
preference ratios in multi-attribute evaluation (PRIME) (Salo & Hamalainen, 2001). PRIME
is able to handle incomplete information (i.e, uncertainty) about decision maker preferences
through the use of interval-valued ratio statements (Lindstedt, Hämäläinen, & Mustajoki,
2001). MCDA can also effectively aggregate the marginal performance of the indicators
considering the preferences of the decision makers regarding the achievement of the defined
strategic objective.
2.3 Emergency Department – A Case Study
Overcrowding in emergency departments (EDs) tends to be a significant international crisis
that negatively affects patient safety, quality of care, and patient satisfaction (Graff, 1999).
ED overcrowding has been declared as a “National Emergency” in Ireland since 2006.
Several national reports have highlighted a growing demand for emergency care (1.2 million
patients attending EDs annually) and a simultaneous decrease in the number of operating
EDs. The result has increased crowding, high percentages of patients leaving EDs without
being seen and higher morbidity and mortality rates. Additionally, prolonged waiting times
has been reported with more than 500 patients on trolleys for hospital admission every day;
18% of patients are waiting more than 24 hours and 40% between 10-24 hours (HSE
Performance Monitoring Report, 2010). Although Ireland is not alone in experiencing these
kind of figures (Forero, et al., 2010; Rowe, et al., 2006; Schafermeyer & Asplin, 2003), it is
important not to underestimate the probable catastrophic consequences this situation has on
patients, staff and healthcare sector across the State. Therefore, analysing the patient flow in
emergency departments to minimize length of stay, improve efficiency, and reduce
overcrowding has become a crucial requirement.
The hospital studied is a an acute, public, voluntary, and adult teaching hospital that
holds a unique place in the delivery of healthcare not only to the community of North Dublin
but also to the rest of Ireland. This 570-bed hospital provides primary, specialized, and
9
tertiary healthcare services, with a 24hr “on-call” ED which services over 55,000 patients
annually.
According to the task force report in 2007, the overall ED physical space and
infrastructure is inadequate. Additionally, the hospital is operating at approximately 99%
occupancy with resultant difficulty in accommodating surges in numbers of ED admissions.
This is often aggravated by delays in patients transfer to critical care (ICU/HDC) beds.
Consequently, the hospital is not compliant with volume and wait time targets (6 hour patient
experience time target). The ED figures show a clear evidence of overcrowding with 17 % of
patients leaving the ED before being seen on average. Moreover, the average time from
registration to discharge is 9.16 hrs with 2.58 hrs standard deviation, i.e. 3.16 hrs over the
HSE metric (0-6 hrs). Besides, the average time from registration to acute admission is 21.3
hrs with a standard deviation of 17.2 hrs, which is 15.3 hrs above the national metric.
Obviously, patients to be admitted usually experience longer length of stay than discharged
patients due to the delays that can occur between admission referral by ED doctor, bed
allocation, and patient transferral from ED to the allocated bed.
The proposed framework is used to model the complexity of the ED to identify
performance bottlenecks and to explore potential solutions that can lead to better performance
and to meet the national metrics.
3 Model Development
3.1 Emergency Department Layout
The department has officially, 12 monitored trolley spaces; 3 of these trolley spaces
(resuscitation area) are reserved for major trauma and critical care patients. Besides, the ED
has a ambulatory car area with a capacity of six trolley spaces. Two isolation rooms, 1
psychiatric assessment room, two rapid assessment triage bays, and two triage rooms are also
provided by the ED. There are 5 distinct areas in the ED (Figure 2): walk-in patient waiting
room, X-Ray and CT scan area, ambulatory care area, ED resuscitation area, and ED major
assessment area. Patients that arrive by ambulance who are in critical conditions are routed
directly to the resuscitation area, while ill patients who require their conditions to be
monitored stay in the major assessment area. The ambulatory care area is for patients
suffering from abdominal pain, headache, limb problems, wounds, head injuries, and facial
problems (amongst all other ambulant patients).
10
Figure 2 ED physical layout and main areas
3.2 ED Staff
As a 24hr department, the ED has eleven nurses during the day and nine nurses at night
which collectively are divided into six types of nurses; Advanced Nurse Practitioner (ANP),
triage nurse, resuscitation nurse, respiratory nurse, majors/minors nurse, and healthcare
assistant. Physicians (excluding the 3 Consultants who provide shop floor cover between 9-5
or 8-8 with 24/7 on-call provision), referred to as non-consultant hospital doctors (NCHD),
are divided into three types: registrar/specialist registrar (i.e., receiving advanced training in a
specialist field of medicine in order eventually to become a consultant), Senior House Officer
(SHO) (i.e., a junior doctor undergoing training within a certain speciality), and intern that
are distributed as follows when the roster allows: three registrars per day with a 10hr shift
starting at 8am, 12pm, and 10pm; two interns with a one shift per day from 8am to 5pm
Monday to Friday; and overlapping shifts of SHOs during the day to make it possible to have
more than one SHO at specific time (i.e. from 2 to 6 SHOs during the day).
3.3 Key Performance Indicators Selection
Following repeated visits to the ED and interviewing the senior management team, the ED
manager has identified two main key performance areas: patient throughput and ED
efficiency. The performance measures for patient throughput are the average waiting time
and average length of stay (LOS), while for ED efficiency they are; ED productivity, resource
utilisation and layout efficiency. Figure 3 shows the breakdown of the key performance
indicators (KPIs) according to the ED senior managers.
11
ED Performance
Patient Throughput ED Efficiency
Length of StayWaiting Time ED Productivity Layout Efficiency
Discharged Patients
Admitted Patients
Patient to NCHD
Patient to Nurse
% Treated Patients
NCHD Distance
Nurse Distance
Resource Utilisation
Staff Capacity
NCHDs Nurses CPR Majors ACU
CPR: Cardio-Pulmonary Resuscitation Room ACU: Ambulatory Care Unit NCHD: Non-Consultant Hospital Doctor Figure 3 Key performance indicators for the ED
3.4 Patient Flow Analysis
Upon arrival at the ED and registration, walk-in patients (self-referral or GP referral) remain
in the waiting area to be triaged (Figure 4).
Patient Arrival
RegistrationTriage
CPR bed available?
Majors bed available?
ACU bed available?
X-Ray sub-waiting area
free?
CPR bed
Majors bed/trolley
ACU bed/trolley
X-Ray sub-waiting area
Waiting Room
Yes
Yes
Yes
Yes
Yes
Triage Category 1
Triage Category 2
Triage Category 3
Yes
Yes
Triage Category 4
Yes
Assessment by ED Doctor
ED doctor available ?
Yes
Diagnostics needed ?
Diagnostics
Yes
Patient waits results
Opinion needed ?
Request Opinion
No
Yes
Admit patient to hospital ?
No
Request Patient Admission
Hospital bed available ?
Yes
No
Speciality doctor available ?
Yes
Patient waits
Patient wait bed
Discharge Patient
Transfer patient to the Hospital bed “inpatient”Yes
Triage nurse available ?
Walk-in Patients
Yes
Patients arrive by ambulance
Waiting Room
No
No
No
NoNo
No
No
No
No
Patient waits doctor
NoNo
No
Figure 4 A detailed flowchart for the patient flow in the ED
12
When a patient’s name is called, depending on triage staff availability, the patient is assessed
by a triage nurse. Based on patient condition and triage assessment, each patient is assigned a
clinical priority (triage category) according to the Manchester Triage System (MTS) that is
widely used in UK, Europe, and Australia (Cronin, 2003). The MTS uses a five level scale
for classifying patients according to their care requirements; immediate, very urgent, urgent,
standard, and non-urgent. Once a triage category is assigned, the patient may be sent back to
the waiting room until a bed or trolley is available in an appropriate treatment area, based on
the type and intensity of their care requirements. The patient’s waiting time depends on the
triage category of patient and the availability of both medical staff (i.e. ED physician or
ANP) and empty trolleys, which are a prerequisite for a complete and accurate assessment.
Following the patient’s assessment by an ED clinician, a decision is made, either the patient
is to be discharged or admitted to the hospital. These are the primary care stages which are
relevant for all patients, whether they are discharged from or admitted to hospital. Secondary
patient stages are those steps involved in the care of some but not all patients such as
diagnostics (e.g. X-Ray and blood test), and second patient assessment by ED doctor.
Opinion may be requested by ED staff from a medical/surgical speciality doctor to confirm
that a patient should be admitted or to obtain advice on the best possible treatment for the
patient who is to be discharged.
3.5 ED Process Mapping
Based on the analysis of patient flow through the ED, a detailed flowchart is built which
highlights the common processes and decision points involved in the care of patients through
the ED. Each ED process is then broken down into smaller sub-functions with key resources
(e.g. staff and medical equipments) at each care stage are identified and detailed using IDEF0
(Colquhoun, Baines, & Cro, 1993). IDEF0 is a powerful tool for modelling complex systems
which allows users (e.g. ED managers, decision makers, system analysts) to comprehensively
understand the system through modelling decisions, actions, and processes in a hierarchical
form. Such organizational strategy allows the system to be easily refined into more detail
until the model is as descriptive as necessary for the decision maker (S. H. Kim & Jang,
2002). The top level of the developed IDEF0 model is shown in Figure 5. The main unit of an
IDEF0 model is an activity block that describes the main function of the process. ICOMs
(Input, Control, Output and Mechanism) are represented by horizontal and vertical arrows.
Process control (top arrow) can be patient information (e.g. arrival time, triage category, and
presenting complaint), safety regulations, or national/international standards whereas process
13
mechanisms are usually the agents/resources which facilitate the activity (e.g. ED physicians,
nurses, and physical beds/trolleys).
A1
Patient Arrival
A2
Registration
A3
Triage
A51
Seen by ED
Doctor
A53
Referred for
Admission
A52
Referred for
Opinion
A54
Awaiting Admission
A6
Patient Transfer
Patient TreatmentA5
A4
Patient Allocation
Patient Information
Mode of Arrival
Triage Category
Medical Staff
Cubicles
Bed/Trolleys/Seats
Medical Equipments
Administrative Staff
Ward/Hospital Capacity
Figure 5 Process mapping of main ED process using IDEF0
The utilization of IDEF for process modeling has not only improved the quality of
simulation models but also it enhanced the communication levels among decision makers and
the staff (e.g., doctors and nurses) through modeling the underlined work flow, decision
points, and processes in a hierarchical form. This hierarchical structure kept the model scope
within the boundaries represented by breaking down processes into smaller sub-functions.
Such organizational strategy allowed the system to be easily refined into more details until
the model is as descriptive as necessary for the decision maker.
3.6 Empirical Data Analysis
The analysis of empirical data is essential in developing a robust simulation model that
considers the time features of the intended system in terms of demand volume and patterns.
A thorough analysis of data enables the discovery of different type of patterns (i.e.,
clustering) that are essential to reduce the complexity of the simulated system in terms of
patient groupings and patient allocation and routing analysis. This valuable information is
needed to build a comprehensive and representative dynamic model for the underpinned
healthcare system. Historical patients records have been gathered for the ED information
system during a 16-month period provided by hospital managers with a total of 59,986
anonymous patient records. Each patient record is described by the following patient-level
variables: (1) triage category assigned to patient, (2) patient presenting medical complaint, (3)
14
mode of patient arrival, (4) patient attend date/time, (5) patient triage date/time, (6) date/time
patient seen by doctor, and (7) whether the patient left without seen, discharged, or admitted
to the hospital. We analysed patients records to extract qualitative information about patients
arrival pattern, patient grouping and allocation, and routing information. Patients were
grouped based on their triage category. Urgent patients (triage category 3) represent the
largest group of new attendees to the ED annually (58 % average) who are presented to the
ED with a wide range of medical complains and aging conditions. Patient Placement for each
patient group through ED treatment areas along with the arrival mode is summarized in
(Table 1).
Table 1 Input settings for different control parameters
Triage Category % of Patients Mode of Arrival
Walk-in Ambulance 1 1.1 % 5 % 95 % 2 16.5 % 40 % 60 % 3 58 % 61 % 39 % 4 23.9 % 81 % 19 % 5 0.5 % 72 % 28 %
For each patient group, an estimation of patient arrival distribution is used to
replicate the arrival pattern in the simulation model. From the simulation perspective, the
inter-arrival data is required, not the arrival time, which describe the time delay between two
consecutive patient arrivals. To do so, the difference between the arrival times of patients was
obtained for each group. These inter-arrival times were then grouped into time slots where
the relative frequency (i.e., percentage) of each time slot was accumulated and represented in
a histogram (see Figure 6a). This was followed by the determination of a fitted distribution
for each inter-arrival histogram. For example, Figure 6b shows different possible
distributions for the inter-arrival time for “Urgent” patient group.
Intervals of Patient Inter-arrival
a) Inter-arrival Time Density
Intervals of Patient Inter-arrival
Rel
ativ
e Fr
eque
ncy
b) Fitted Density for Inter-arrival
Rel
ativ
e Fr
eque
ncy
Intervals of Patient Inter-arrival
Figure 6 Relative frequency diagram – urgent patients
15
The best fit distribution was then determined for each patient group and validated by
using Kolmogorov Smirnov goodness of fit test with a 5 % significance level. Regarding
patient allocation data, Table 2 shows the analysis of the allocation of patients within the ED.
Based on that analysis, the ED staff failed to fully implement the recommendations of the
MTS concerning the disposition of patients, which is due to the overcrowding of the ED. For
example, 88% of immediate patients are seen in the resuscitation room and 9% in the majors’
cubicles, while only 40% percent of very urgent patient are seen in inappropriate assessment
areas. Moreover, due to the overcrowding status of the ED, the majority of standard and non-
urgent patient are assessed and treated in inappropriate areas (e.g., chairs) or wait in waiting
areas.
Table 2 Analysis of patient allocation within the emergency department
ED Areas Triage Category
IMM VURG URG STD NURG
Resuscitation Room 88% 25% 2% 0% 0% Majors Area 9% 15% 8% 1% 0%
Ambulatory Care Unit 0% 12% 10% 20% 11% Majors Chairs 0% 7% 6% 1% 1%
Rapid Assessment Triage 3% 12% 7% 2% 2% Waiting Room 0% 14% 56% 74% 85%
X-Ray Sub-Wait Area 0% 15% 12% 4% 1%
IMM: Immediate VURG: Very Urgent URG: Urgent STD: Standard NURG: Non-Urgent
4 Simulation Model Development and Validation
4.1 Model Construction
Based on the ED business process model, the designed BSC, and the empirical data analysis,
a comprehensive simulation model for the ED was constructed. Modules of the simulation
model were connected to resemble the ED business process model, where blocks are
connected similar to the conceptual flow chart, which eases the model construction phase.
Accordingly, the top-level of the simulation model defines the overall model structure, where
sub-level blocks containing additional modules with more details. Object-oriented
programming was used to customize pre-defined block for constructing the ED simulation
model. A database was used to save the measured KPIs after each simulation run (i.e.,
replicate), followed by exporting the populated BSC in a tabular form for future analysis and
validation.
16
4.2 Verification &Validation
To reduce the model development cycle time and to increase the confidence in the simulation
model results, the verification and validation were carried out all the way through the
development phases of the ED simulation mode. After each model development phase, the
model was verified and validated with respect to other previously completed phases. For the
verification process, the model logic is verified to ensure that patients follow the correct care
path as expected. This was achieved by visual tracking of patients using animation and by
checking intermediate output values such as queue lengths and waiting times between
processes. The conceptual model had been documented and validated by circulating the
document among ED senior managers and senior nursing staff. This is crucial to ensure that
the logic of the model and ED activities is correct. All distributions determined from the data
and used in the model were validated by using Kolmogorov Smirnov goodness of fit test with
a 5 % significance level (Massey, 1951). The final results of the simulation model have been
validated using three techniques; face validation, comparison testing, and hypothesis testing.
Face Validation is performed by interviewing ED senior managers and nursing staff in order
to validate the final results of the simulation model. The second approach is ‘Comparison
Testing’ which done by comparing the output of the simulation model with the real output of
the system under identical input conditions (Balci, 1997). Three main KPIs are used in this
approach; average waiting time (seen by doctor response), average LOS for discharged
patients, and average LOS for admitted patients. Besides the overall average for all patients,
each KPI has been also detailed for 3 patient categories; VURG, URG, and STD. Based on
the comparison testing approach, the deviation between actual and simulated results for the
selected KPIs range from 1 % to 9 % with an average of only 5 % deviation (Figure 7).
IMM: Immediate VURG: Very Urgent URG: Urgent STD: Standard NURG: Non-Urgent
Figure 7 Validation of simulation results against actual data
17
5 Stakeholders Engagement
The role that hospital managers and staff played was vital to the success of the project. Their
level of engagement (Figure 8) at the beginning of the project was exclusive where the
problem formulation and data collection phase was initiated by setting up interview sessions
with the ED senior managers (two ED consultants and two nursing managers) where deep
insights about the current challenges they face in managing their department were obtained.
The current awareness of business process management methods (e.g., simulation modeling,
multi-criteria decision analysis, and optimization) was also a key topic in the discussions and
interviews. A better understanding for healthcare processes, activities, challenges, and
variables was then acquired with valuable insights of the challenges in the decision making
process.
The interviews helped to develop significant inputs that critically supported the
development and validation phases of the proposed framework. This was followed by
constructing a focus group of ED doctors and nurses for discussing issues such as general
patient care paths, categories of patients and their complexities, and resources availability and
capacity issues. Meanwhile, a number of interviews with ED senior managers have been
taken place to discuss the performance measures. Incorporating these measures at that stage
was very useful for developing the balanced scorecard for the ED and setting the objectives
of the simulation model (i.e., the simulation output). Concurrently, a focus group for
historical data collection was formed to discuss issues related to electronic patients records,
existing information systems, and data entry procedures. That focus group included members
from the information system department in the partner hospital. The discussions with that
focus group was supported by a close observation of the data entry procedures through the
patient journey through the ED and by a series of short interviews with the ED staff (e.g.
registration staff, triage nurses, and physicians).
Figure 8 Engagement levels of the stakeholders during the development of the proposed
framework
18
Upon the completion of that stage, we started the conceptual model phase where the
main ED processes were mapped and detailed into a conceptual process model using IDEF
modeling languages. For the validation of the ED conceptual model, it was circulated among
the senior managers and the patient flow focus group, where it was then refined through
another cycle of interviews and observations. While developing the simulation model, the
level of staff engagement has been decreased while building the simulation model. Upon the
completion of this stage, a number of interviews with the ED senior managers were
established in order to validate the simulation model (i.e. simulation base-line). Subsequently,
senior managers suggested a number of scenarios to be evaluated using the simulation model.
These scenarios were suggested by the ED senior managers to evaluate the intended
new extension of the hospital which will include rebuilding of key parts of the hospital
including the ED. Expanding the capacity of the ED may eventually necessitate a
corresponding increase in the staffing levels. Therefore, the hospital managers and the
planners of the new ED express their interest to evaluate the effect of capacity expansion and
increasing the staffing levels against the effect of unblocking critical performance bottlenecks
such as the access block from the ED to the hospital. During the experimentation phase, the
preference of the ED manager regarding the KPIs was collected and used to build the
preference model and to evaluate the potential scenarios.
6 Experimentation and Scenario Analysis
6.1 Scenario Design
The simulation scenarios tested were the impact of variation in medical staffing, increasing
clinical assessment space and finally assessing the impact of incorporating a ‘zero-tolerance’
policy regarding exceeding the national 6-hour boarding time (i.e., length of stay). According
to the ED managers, the goal of the ‘zero-tolerance’ policy is to assess the performance of the
ED if the average LOS of patients complies with the HSE 6hrs target. The importance of this
assumption was emphasized by the senior hospital decision makers to identify the real factors
that contribute the unacceptable overcrowding status of the current ED; inappropriate
physical space, insufficient staffing levels, or operational difficulties beyond the direct
control of the ED. This scenario is implemented in the simulation model by dismissing
patients from the ED model who are waiting to be admitted to the hospital and their LOS
exceeds 6hrs. The rationale beyond this is that hospitals can provide a short stay unit, with an
appropriate capacity, for patients who are waiting to be admitted but there are no available
19
beds in the hospital. Therefore, distinct study scenario variables (Table 3) were added to the
simulation model and run for a 3 month continuous blocks. The three months was chosen by
the ED manager for the stability of ED staffing levels offer this period.
Table 3 Simulation variables for base scenario and scenario 1, 2, and 3
Control variables Admission
blockage Physical capacity
(number of trolleys) Additional
physician shift Base Scenario Yes 12 - Scenario 1 (no admission blockage / Zero-tolerance) No 12 -
Scenario 2 (increasing number of trolleys by 50%) Yes 18 -
Scenario 3 (adding one SHO shift over night) Yes 12 1 SHO
[9pm to 7am]
The principle variables introduced were increased clinical assessment capacity (extra
6 trolley cubicles), increased clinical assessors (1 Senior House Officer shift at night), and
absolute compliance with the national 6 –hour admission target for ED boarders (i.e., zero-
tolerance). These scenarios were selected based on the discussion in previous sections.
6.2 Result Analysis
Moving patients, who are waiting to be admitted, to a short stay unit or improving the
admission/discharge cycle within the hospital (i.e., scenario 1) has resulted in a significant
decrease in admitted patients LOS from 21.3hrs to 7.75hrs and 49% decrease in the average
distance travelled by nurses (Table 4). This is because ED resources that were utilised by
patients awaiting admission have been made available for other ED patients. For example,
physical beds and trolleys that were occupied by patients waiting admission are available for
new emergency patients, especially acute patients. Moreover, nursing staff that were
frequently monitoring patients in critical medical conditions in corridors or trolleys can be
reallocated to more pressurised areas in the ED. Consequently, upon their arrival, patients are
seen by an ED clinician or wait inside the ED. First, patients’ waiting time decreases,
however, the waiting time of patients waiting to be seen by clinician increases gradually as
more patients arrive. This is because the staffing level is the same and there are no more
servers (i.e. staff) for incoming patients. Scenario 1 resulted in shifting the bottle neck from
the waiting room to inside the ED. As a result, the average waiting time of patients,
particularly less acute patients, waiting to be seen by physician increased by 15 % and
consequently the average LOS of discharged patients has increased to 9.78 hrs.
20
Table 4 Simulation results of scenario 1, 2, and 3
Key Performance Indicators (KPIs) Base
Scenario Scenario 1 Scenario 2 Scenario 3
O/P O/P +/- O/P +/- O/P +/- Avg. WT
(min) First Clinical Contact 177.4 204.1 15% 185.9 05% 98.7 -44%
Avg. LOS (hrs)
Discharged Patients 08.9 09.8 09% 09.4 05% 07.5 -16%
Admitted Patients 21.3 07.6 64% 19.7 -08% 19.0 -11%
ED Productivity
Patient to Doctor (per shift) 07.3 07.5 02% 07.5 02% 07.1 -03%
Patient to Nurse (per shift) 09.9 10.2 03% 10.2 03% 10.2 02%
% Patients Treated 83% 83% - 85% 02% 94% 13%
% Patients Admitted 18% 18% - 18% - 20% 11%
% Patients Left Without Seen 17% 17% - 15% -12% 06% -65%
Resource Utilisation
Doctor 81% 80% - 79% -02% 63% -22%
General Nurse 82% 71% -13% 82% - 83% -
CPR 81% 67% -17% 80% - 79% -02%
Majors Trolleys 91% 85% -07% 87% -04% 88% -03%
Ambulatory Care Unit 90% 85% -06% 87% -03% 85% -06%
Avg. Distance (km/day)
Doctor 03.2 02.9 -12% 02.8 -13% 03.3 -
Nurse 06.5 03.3 -49% 06.4 -02% 06.7 04%
Number of Patients in WR 15.5 12.5 -19% 12.6 -19% 11.1 -28%
O/P: simulation output +/-: percentage of increase/decrease relative to the base scenario
Increasing the physical space by 50% (i.e. adding 6 trolleys/beds) will decrease the
number of patients in the waiting room, though the number of admitted patients will increase
by 3%. The effect will be cascaded back through the ED progressively with more patients
waiting on trolleys to be admitted to the hospital. As a result, there will be no space left to
meet the timely needs of the next patients who need emergency care; the average waiting
time increase by 5% and the average LOS run up to 9.38hrs. On the other hand, adding one
physician (i.e. SHO doctor) from 9pm to 7am will reduce the queue length in the waiting
room that keeps building up over the night time (especially weekends). Subsequently, the
average waiting time of patients will shorten by 44% (98.68mins) and the percentage of
treated patients will increase to 94%. Nevertheless, the average LOS of patients is still
beyond the national metric (6hrs compared to 7.53hrs for discharged patients and 19hrs for
admitted patients). Following the discussion of the results of these scenarios, it was clear for
the hospital senior decision makers that investing in improving the admission/discharge cycle
within the whole hospital is a more effective and practical strategy. However, they expressed
their interest to explore the potential benefits of all possible combinations of the three basic
scenarios (Table 5).
21
Table 5 Simulation settings for scenario 4, 5, 6, and 7
Control variables Admission
blockage Physical capacity
Additional physician shift
Scenario 4 (scenario 1 & scenario 2) No 18 -
Scenario 5 (scenario 1 & scenario 3) No 12 1 SHO
[9pm to 7am] Scenario 6 (scenario 2 & scenario 3) Yes 18 1 SHO
[9pm to 7am] Scenario 7 (scenario 1 & scenario 2 & scenario 3) No 18 1 SHO
[9pm to 7am]
The effect of scenario 1 on the ED performance is greater than that by adding more
ED beds or having an additional physician. Moreover, when this solution is combined with
one or both other solutions, the performance is improved significantly, as shown in Table 6.
Table 6 Simulation results of scenario 4, 5, 6, and 7
Key Performance Indicators (KPIs) Base
Scenario Scenario 4 Scenario 5 Scenario 6 Scenario 7
O/P O/P +/- O/P +/- O/P +/- O/P +/- Avg. WT
(min) First Clinical Contact 177.4 141.3 -20% 110.8 -38% 105.2 -41% 99.3 -44%
Avg. LOS (hrs)
Discharged Patients 08.9 08.5 -05% 07.5 -17% 07.5 -16% 07.4 -18% Admitted Patients 21.3 06.9 -68% 06.6 -69% 18.8 -12% 06.4 -70%
ED Productivity
Patient to Doctor (per shift) 07.3 07.7 05% 07.3 - 07.2 - 07.3 - Patient to Nurse (per shift) 09.9 10.2 03% 10.4 05% 10.3 04% 10.4 05% % Patients Treated 83% 90% 08% 94% 13% 93% 12% 95% 14% % Patients Admitted 18% 19% 06% 20% 11% 20% 11% 20% 11% % Patients Left Without Seen 17% 10% -41% 06% -65% 07% -59% 05% -71%
Resource Utilisation
Doctor 81% 71% -12% 61% -25% 63% -22% 60% -26% General Nurse 82% 67% -18% 69% -16% 83% - 68% -17% CPR 81% 64% -21% 62% -23% 79% -02% 62% -23% Majors Trolleys 91% 70% -23% 74% -19% 83% -09% 64% -30% Ambulatory Care Unit 90% 75% -17% 73% -19% 81% -10% 68% -24%
Avg. Distance (km/day)
Doctor 03.2 02.5 -23% 02.9 -11% 02.9 -11% 02.6 -19% Nurse 06.5 02.7 -59% 02.8 -56% 06.5 - 02.5 -62%
Number of Patients in Waiting Room 15.5 11.8 -24% 08.2 -47% 09.4 -39% 05.6 -64%
O/P: simulation output +/-: percentage of increase/decrease relative to the base scenario
As expected, scenario 7 dominates all other scenarios in terms of ED performance. As
shown in Table 6, the average LOS for admitted patients is approximately 6hrs, and
percentage of patients left without treatment is 5% which meet the national metric for both
22
LOS and percentage of untreated patients. A multi-criteria decision analysis approach has
been used to reflect the ED decision makers’ preferences and to account for the tradeoffs
between competing KPIs.
6.3 A Preference Model for ED
Based on the scenarios’ results in Table 4 and Table 6, a preference model has been
developed using preference ratios in multi-attribute evaluation (PRIME) (Salo &
Hamalainen, 2001). The suitability of PRIME approach to the analysis of these scenarios lies
into that 1) its ability to model a hierarchical structure of criteria (value tree); and 2) it is able
to handle incomplete information (uncertainty) about decision maker preferences through the
use of interval-valued ratio statements (Lindstedt, et al., 2001). Based on base line scenario,
and the results of the first three scenarios (Table 4), along with the simulation results of the
simulation output for their combinations (Table 6), PRIME (Appendix A) was used for
building the ED preference model. The imprecise preferences of the ED mangers were used
to construct weighted intervals of the ED KPIs as shown in Figure 9.
ED Performance
Patient Throughput
ED Efficiency
Length of Stay
Waiting Time
Resource Utilisation
ED Productivity
Layout EfficiencyScenario 1
Scenario 2
Scenario 3
Scenario 4
Scenario 5
Scenario 6
Scenario 7
[0.625 ... 0.69]
[031 ... 0.375]
[0.714 ... 0.8]
[0.2 ... 0.286]
[0.066 ... 0.172]
[0.588 ... 0.769]
[0.138 ... 0.29]
Figure 9 The value tree of the emergency department
The marginal performance of the KPIs was then aggregated using the PRIME value
tree and resulted in a performance value interval for each scenario. As shown in Figure 10,
the aggregate performance of ED is represented as an interval that represents the marginal
gain of each scenario. For example, an increase between 38% and 54% in the performance is
23
achieved for scenario 1 while the expansion of the physical ED capacity (scenario 2)
enhanced the ED performance only by 8% to 19%. The visual presentation of these value
intervals of the performance of each scenario (Figure 10) has provided the senior managers
with a powerful tool to interpret the simulation results and at the same time to account for
their preferences. For example, by the visual inspection of value intervals for the scenarios,
scenario 1 dominates both scenario 2 and 3. Consequently, any improvement strategy that
will not address the access blockage (i.e. scenario 1) will not lead to the desired performance
targets. Based on these insights, the hospital senior managers have prioritized improving the
admission/discharge planning activities across the hospital over the other expensive solution
alternatives (i.e. scenario 2 and 3).
Figure 10 Value intervals for all scenarios
7 Discussion
The proposed framework has been well-received by the ED managers and the hospital senior
decision makers and acknowledge as a sustainable tool to support their strategies. A number
of factors have contributed to this positive perception from the management team. Firstly, the
development of a high-level process model prior to the development of the simulation model
has greatly helped in the collection of relevant information on the operation of the system
(i.e. data collection) and, therefore, reduced the effort and time consumed to develop the
simulation model. The utilization of IDEF for process modeling has not only improved the
quality of the simulation model but also it enhanced the communication levels among
decision makers and the staff (e.g., doctors and nurses) through modeling the underlined
24
work flow, decision points, and processes in a hierarchical form. This hierarchical structure
kept the model scope within the boundaries represented by breaking down processes into
smaller sub-functions. Such organizational strategy allowed the system to be easily refined
into more details until the model is as descriptive as necessary for the decision maker.
Secondly, the integration between simulation modeling and balanced scorecard contributed to
the alleviation of BSC limitations in terms of its measurement capabilities and the lack of
inferring the causal-effect among performance measures. Further, this integration established
a clear link between the strategic objectives of the organization and the daily activities within
the department, which gave decision makers deep insights regarding performance bottlenecks
and potential corrective plans. Finally, the combination of multi-criteria decision analysis
tools along with simulation and BSC contributed significantly in the decision making process
by explicitly dealing with priorities and trade-offs between different performance indicators.
This was achieved by not only considering the preference of decision makers regarding the
achievement levels of performance measures, but also by incorporating the imprecise and
incomplete information about their preferences.
Although the proposed framework has successfully encompassed many factors that
affect decision making, there is still a room to improve. The key limitation of the proposed
framework is the cost factor of the decision. The study objectives set at the beginning of the
project highlighted the importance of the cost element in the strategic level of decisions,
however incorporating the cost element was not possible in this study due to two main
reasons; (1) lack of cost related information to support the analysis phase and, (2) the
variability in the cost model in various public hospital in Ireland created a high level of
complexity to model the financial element. Nevertheless, there is currently a new project
launched within the group to analysis the cost model in the public hospital and attempt to
standardize the financial model so that analysis can be conducted. It is also worth mentioning
that the output related to the staff utilization was approximated to union and confidentiality
issues. To work around this, the analysis was done based on the percentage of time that the
staff spent with patients and that was agreed with the ED managers. The proposed framework
is also limited to discrete-event simulation (DES), other simulation and modeling methods such
as system dynamics (SD) and agent-based simulation (ABS) are emerging as potential tools for
analyzing the inter-connected relationships between healthcare components at the macro-level of
the system.
25
Conclusion
With healthcare costs continuing to rise and funds continuing to decline, there is an urge of
effective process improvement methodologies to address the reported inefficiencies in
healthcare delivery. Most of the existing methodologies are described by managers as lacking
strategy-linked measures. Owning the fact that stakeholder’s engagement is crucial, a key
element in developing robust framework is working close to the process owners and teams. In
this paper, a simulation-based decision support system is presented for healthcare process
improvement. The framework integrates simulation with balanced scorecard (BSC) as a
performance management technique which plays an important role in communicating
objectives and actions between decision makers and other staff, monitoring achievements,
and driving related corrective actions. Further, multi-criteria decision analysis (MCDA) tools
are used to incorporate decision makers’ preferences in the evaluation of these actions. The
framework is demonstrated through a real-world case study in an emergency department of
one of leading hospitals in Republic of Ireland.
By using the model, the ED managers were able to reveal that enforcement of the
national benchmark of 6-hour boarding limit for EDs would have a significantly greater
impact on reducing average length of stay for all ED patients than increasing medical staff or
assessment cubicles. Access block therefore, has been shown by the model to have the
highest impact on prolonged average length of stay for patients and successful strategies are
available to reduce hospital access block especially in situations of ED surge and reduced
hospital bed capacity. The proposed framework was helpful and well-received by the ED
managers; it did directly contribute to plan the new strategies in the ED and also in the
planning phase of the new ED. Moreover, the recommendations of the framework have also
been considered by the executive board of the partner hospital where framework is currently
used to model other hospital processes that affect the flow of patients to achieve the required
alignment and coordination between hospital units.
References
Ahmed, M. A., & Alkhamis, T. M. (2009). Simulation optimization for an emergency
department healthcare unit in Kuwait. European Journal of Operational Research,
198, 936-942.
26
Balci, O. (1997). Verification validation and accreditation of simulation models. In S.
Andradóttir, K. J. Healy, D. H. Withers & B. L. Nelson (Eds.), Proceedings of the
1997 Winter Simulation Conference (pp. 135-141). Atlanta, Georgia, USA.
Benneyan, J. C. (1997). An introduction to using computer simulation in healthcare: patient
wait case study. Journal of the Society for Health Systems, 5, 1-15.
Brailsford, S., & Vissers, J. (2011). OR in healthcare: A European perspective. European
Journal of Operational Research, 212, 223-234.
Ceglowski, R., Churilov, L., & Wasserthiel, J. (2006). Combining data mining and discrete
event simulation for a value-added view of a hospital emergency department. Journal
of the Operational Research Society, 58, 246-254.
Colquhoun, G., Baines, R., & Cro, S. (1993). A state of the art review of IDEF0.
International Journal of Computer Integrated Manufacturing, 6, 252-264.
Connelly, L. G., & Bair, A. E. (2004). Discrete event simulation of emergency department
activity: a platform for system level operations research. Academic Emergency
Medicine, 11, 1177-1185.
Cronin, J. (2003). The introduction of the Manchester triage scale to an emergency
department in the Republic of Ireland. Accident and Emergency Nursing, 11, 121-125.
Dittus, R. S., Klein, R. W., DeBrota, D. J., Dame, M. A., & Fitzgerald, J. F. (1996). Medical
resident work schedules: design and evaluation by simulation modeling. Management
Science, 42, 891-906.
Duguay, C., & Chetouane, F. (2007). Modeling and improving emergency department
systems using discrete event simulation. Simulation, 83, 311-320.
Dyer, J. S., Fishburn, P. C., Steuer, R. E., Wallenius, J., & Zionts, S. (1992). Multiple criteria
decision making, multiattribute utility theory: the next ten years. Management
Science, 38, 645-654.
Eldabi, T., Paul, R., & Young, T. (2006). Simulation modelling in healthcare: reviewing
legacies and investigating futures. Journal of the Operational Research Society, 58,
262-270.
Forero, R., Hillman, K., McCarthy, S., Fatovich, D., Joseph, A., & Richardson, D. (2010).
Access block and ED overcrowding. Emergency Medicine Australasia: EMA, 22,
119-35.
Graff, L. (1999). Overcrowding in the ED: an international symptom of health care system
failure. The American Journal of Emergency Medicine, 17, 208-209.
27
Harper, P., Shahani, A., Gallagher, J., & Bowie, C. (2005). Planning health services with
explicit geographical considerations: a stochastic location-allocation approach.
Omega, 33, 141-152.
Ingolfsson, A., Erkut, E., & Budge, S. (2003). Simulation of single start station for Edmonton
EMS. Journal of the Operational Research Society, 54, 736-746.
Ismail, K., Abo-Hamad, W., & Arisha, A. (2010). Integrating balanced scorecard and
simulation modeling to improve emergency department performance in Irish
hospitals. In B. Johansson, S. Jain, J. Montoya-Torres, J. Hugan & E. Y¨ucesan
(Eds.), Proceedings of the 2010 Winter Simulation Conference (pp. 2340-2351).
Baltimore, Maryland, USA.
Jun, J., Jacobson, S., & Swisher, J. (1999). Application of discrete-event simulation in health
care clinics: a survey. Journal of the Operational Research Society, 50, 109-123.
Kaplan, R. S., & Norton, D. P. (1992). The balanced scorecard–measures that drive
performance. Harvard Business Review, 70, 71-79.
Kaplan, R. S., & Norton, D. P. (2001). Transforming the balanced scorecard from
performance measurement to strategic management: part I. Accounting Horizons, 15,
87-104.
Khare, R. K., Powell, E. S., Reinhardt, G., & Lucenti, M. (2009). Adding more beds to the
emergency department or reducing admitted patient boarding times: which has a more
significant influence on emergency department congestion?. Annals of Emergency
Medicine, 53, 575-585.
Kim, S. C., Horowitz, I., Young, K. K., & Buckley, T. A. (1999). Analysis of capacity
management of the intensive care unit in a hospital. European Journal of Operational
Research, 115, 36-46.
Kim, S. H., & Jang, K. J. (2002). Designing performance analysis and IDEF0 for enterprise
modelling in BPR. International Journal of Production Economics, 76, 121-133.
Liberatore, M. J., & Nydick, R. L. (2008). The analytic hierarchy process in medical and
health care decision making: A literature review. European Journal of Operational
Research, 189, 194-207.
Lindstedt, M. R. K., Hämäläinen, R. P., & Mustajoki, J. (2001). Using intervals for global
sensitivity analyses in multiattribute value trees. In Proceedings of the 15th
International Conference on Multiple Criteria Decision Making. Ankara, Turkey.
28
Litvak, N., Van Rijsbergen, M., Boucherie, R. J., & Van Houdenhoven, M. (2008). Managing
the overflow of intensive care patients. European Journal of Operational Research,
185, 998-1010.
Massey, F. J. (1951). The Kolmogorov-Smirnov test for goodness of fit. Journal of the
American Statistical Association, 46, 68-78.
Oddoye, J., Jones, D., Tamiz, M., & Schmidt, P. (2009). Combining simulation and goal
programming for healthcare planning in a medical assessment unit. European Journal
of Operational Research, 193, 250-261.
Rosen, L. S., Harmonosky, M. C., & Traband, T. M (2008). Optimization of systems with
multiple performance measures via simulation: Survey and recommendations.
Computers and Industrial Engineering, 54, 327-339.
Rowe, B. H., Bond, K., Ospina, M. B., Blitz, S., Afilalo, M., Campbell, S. G., & Schull, M.
(2006). Frequency, determinants, and impact of overcrowding in emergency
departments in Canada: a national survey of emergency department directors.
Academic Emergency Medicine, 13, S27.
Saaty, T. L. (1990). How to make a decision: the analytic hierarchy process. European
Journal of Operational Research, 48, 9-26.
Salo, A. A., & Hamalainen, R. P. (2001). Preference ratios in multiattribute evaluation
(PRIME)-elicitation and decision procedures under incomplete information. IEEE
Transactions on Systems, Man and Cybernetics, Part A: Systems and Humans, 31,
533-545.
Samaha, S., Armel, W. S., & Starks, D. W. (2003). The use of simulation to reduce the length
of stay in an emergency department. In S. Chick, P. J. Sánchez, D. Ferrin & D. J.
Morrice (Eds.), Proceedings of the 2003 Winter Simulation Conference (Vol. 2, pp.
1907-1911), New Orleans, Louisiana, USA.
Saunders, C. E., Makens, P. K., & Leblanc, L. J. (1989). Modeling emergency department
operations using advanced computer simulation systems. Annals of Emergency
Medicine, 18, 134-140.
Schafermeyer, R. W., & Asplin, B. R. (2003). Hospital and emergency department crowding
in the United States. Emergency Medicine, 15, 22-27.
Sinreich, D., & Jabali, O. (2007). Staggered work shifts: a way to downsize and restructure
an emergency department workforce yet maintain current operational performance.
Health Care Management Science, 10, 293-308.
29
Stahl, J. E., Rattner, D., Wiklund, R., Lester, J., Beinfeld, M., & Gazelle, G. S. (2004).
Reorganizing the system of care surrounding laparoscopic surgery: a cost-
effectiveness analysis using discrete-event simulation. Medical Decision Making, 24,
461-71.
Thorwarth, M., Arisha, A., & Harper, P. (2009). Simulation model to investigate flexible
workload management for healthcare and servicescape environment. In M. D.
Rossetti, R. R. Hill, B. Johansson, A. Dunkin & R. G. Ingalls (Eds.), Proceedings of
the 2009 Winter Simulation Conference (pp. 1946-1956). Austin, Texas, USA.
Vasilakis, C., Sobolev, B., Kuramoto, L., & Levy, A. (2006). A simulation study of
scheduling clinic appointments in surgical care: individual surgeon versus pooled
lists. Journal of the Operational Research Society, 58, 202-211.
Voelker, K. E., Rakich, J. S., & French, G. R. (2001). The balanced scorecard in healthcare
organizations: a performance measurement and strategic planning methodology.
Hospital Topics, 79, 13-24.
Wierzbicki, A. P. (2007). Modelling as a way of organising knowledge. European Journal of
Operational Research, 176, 610-635.
Zelman, W. N., Pink, G. H., & Matthias, C. B. (2003). Use of the balanced scorecard in
health care. Journal of Health Care Finance, 29, 1-16.
30
Appendix A: Preference Ratios in Multi-Attribute Evaluation (PRIME)
The preferences of decision maker(s) in PRIME are assumed to have an additive structure so
that the overall value of an alternative (i.e., scenario) equals the sum of its attribute-specific
scores,
Eq. (A. 1)
where, N in the number of leave nodes (i.e., twig-level KPIs that have not been decomposed
into further lower level attributes in the value tree), xi is the achievement level of alternative x
with regard to the i-th attribute (i.e., KPI), and vi(xi) is the single-attribute score associated
with the achievement xi on the i-th attribute. The process of decision maker preference
elicitation consists of two phases; score elicitation and weight elicitation. The goal of the
score elicitation phase is to rank consequences ordinally with respect to the least and most
preferred achievement levels xi0 and xi
* for each attribute i. The same process has been
repeated for all the twig-level KPIs. Ordinal rankings become linear constraints of the form:
Eq. (A. 2)
where xi j is more preferred than xi
k in the i-th attribute for alternatives j and k respectively.
Following ordinal ranking of achievement levels, further score information is obtained
through interval-valued statements about ratios of value differences. The decision maker sets
lower and upper bounds [L,U] on the ratio between the value difference from xi0 to the
achievement level xi j and the value difference from xi
0 to xi* :
Eq. (A. 3)
For the weight elicitation phase, trade-off information about the relative importance of
attributes is elicited through interval-valued (i.e., ratio) judgments. A reference attribute (e.g.,
the most important one) is selected and one hundred points are assigned to it. The decision
maker is then asked to assign a range of points [L,U] to the other attributes in accordance
with the perceived importance of these attributes:
Eq. (A. 4)
31
This process is repeated for each level in the value tree where the relative importance of KPIs
is specified by the decision maker. PRIME converts the imprecise preference model then into
preference synthesis structure and consists of; 1) weight intervals of the attributes, 2) value
intervals for the alternatives, and 3) dominance structures and decision rules for the
alternatives comparison. Weight intervals are obtained by solving the linear constraints
imposed on weights (represented by lower and upper inequalities). Following weight interval
calculation, value intervals are calculated for each alternative:
Eq. (A. 5)
While are the linear programs that give bounds for the
weight of the i-th attribute. The dominance structure for alternatives is then inferred.
Computationally, alternative x j is preferred to x k in the sense of absolute dominance if and
only if the smallest value of x j exceeds the largest value of x k, i.e.,
Eq. (A. 6)