Embed Size (px)
Citation preview
Development and Evaluation of a Web-‐based Electronic Medical Record System Without Borders
Jose Francisco Saavedra
A dissertation
submitted in partial fulfillment of the requirements for the degree of
Doctor of Philosophy
University of Washington
2012
Reading Committee:
Meliha Yetisgen-‐Yildiz, Chair
Barry A Aaronson
Craig S Scott
Program Authorized to Offer Degree:
Biomedical Informatics and Medical Education
All rights reserved
INFORMATION TO ALL USERSThe quality of this reproduction is dependent upon the quality of the copy submitted.
In the unlikely event that the author did not send a complete manuscriptand there are missing pages, these will be noted. Also, if material had to be removed,
a note will indicate the deletion.
Microform Edition © ProQuest LLC.All rights reserved. This work is protected against
unauthorized copying under Title 17, United States Code
ProQuest LLC.789 East Eisenhower Parkway
P.O. Box 1346Ann Arbor, MI 48106 - 1346
UMI 3552856Published by ProQuest LLC (2013). Copyright in the Dissertation held by the Author.
UMI Number: 3552856
Abstract
Development and Evaluation of a Web-‐based Electronic Medical Record System Without Borders
Jose Francisco Saavedra
Chair of the Supervisory Committee: Professor Fredric M. Wolf
Department of Biomedical Informatics and Medical Education Adjunct Professor of Health Services and Epidemiology
Despite implementation of electronic medical record (EMR) systems in the United States
and other countries, EMRs often lack global access, standardization, an efficient interface,
and effective knowledge-‐based tools at the point of care. Consequently, the information
needs of patients, practitioners, administrators, researchers, and policymakers often go
unmet, leaving providers especially dissatisfied. To address this multifaceted problem, a
novel EMR design referred to as “Electronic Medical Records Without Borders” (EMR WB)
was created to ensure that the most vital pieces of patient clinical records are available to
make health care decisions. A web-‐based, standardized, family medicine, clinical history
model was developed and evaluated as an EMR clinical core that integrates state-‐of-‐the-‐art
terminology, peer-‐reviewed, evidence-‐based protocols with real-‐time access to diagnostic
decision support systems and the biomedical literature, using a unified, navigable, intuitive
computer interface. This project aimed to facilitate structured clinical documentation,
usability, global access, and decision-‐making processes to better address not only local,
clinical, and psychosocial primary care problems in targeted underserved communities in
the state of Guanajuato, Mexico, but also to mitigate transnational migration health issues
based on information exchange among primary care settings. A survey of post-‐exposure
EMR WB use indicated a measurable, positive effect was made on provider satisfaction
compared with a previously used, paper-‐based record system. Data were analyzed using
descriptive statistics.
ACKNOWLEDGEMENTS
I would like to express my most sincere appreciation to the faculty and staff of the
Department of Biomedical Informatics and Medical Education for their support, and most
notably to Professor Fredric M. Wolf for his knowledge, guidance and patience. I would also
like to thank Professor Meliha Yetisgen-‐Yildiz, for serving as Co-‐Chair of my committee and
providing valuable insight and leadership. Also, I would like to recognize the efforts of
Clinical Associate Professor Barry Aaronson, Professor Craig Scott, and Professor Kelly
Edwards for their encouragement and very valuable advice. I am also very grateful to
Professor Robert Dickow for his invaluable help with the programming. Lastly, this
dissertation would not have been possible if it were not for the encouragement and
devotion of family and friends.
DEDICATION
As I learned throughout this journey, it takes quite a bit of isolation and solitude to
undertake a project of this scope including research, implementation, evaluation, and of
course, writing. My wife Merrie was exceptionally patient and supportive during this
whole process. I am so grateful to her, as well as the committee and faculty members who
all helped to make this exciting and meaningful endeavor a reality.
TABLE OF CONTENTS
Page
List of Figures ...................................................................................................................................iii List of Tables .....................................................................................................................................iv Chapter I: Introduction....................................................................................................................1 1.1. Statement of Problem...............................................................................................2 1.1.1. Historical context…………………………………………………………….2 1.2. Proposed Solution…...................................................................................................9 1.2.1. EMR WB project background………................................................11 1.3. Research Statements..............................................................................................25 1.3.1. Thesis statement....................................................................................25 1.3.2. Research questions…...........................................................................25 1.3.3. Objectives..................................................................................................25 1.4. Significance.................................................................................................................26 1.5. Research Goals……………………………………………………………………………27 1.5.1. Aim 1...........................................................................................................27 1.5.2. Aim 2...........................................................................................................30 1.6. Establishing Feasibility ........................................................................................31 1.7. Resource Sharing Plans.........................................................................................33 1.8. Summary……………………………………………………………………………………33 Chapter II: Methods—Designing the Solution………………………………………………..36 2.1. Access............................................................................................................................37 2.1.1. Strategy……………………………………………………………………...…37 2.1.2. Server hosting………………………………………...……………………..38 2.1.3. Cloud computing………………………………………………………..….38 2.1.4. Administrative management………………………………………….38 2.1.5. Privacy………………………………………………………………………….39 2.1.6. Concurrency…………………………….……………………………………40 2.1.7. Scalability……………………………………………………………..………41 2.1.8. Expandability………………………………………………………………..41 2.2. Programming Language………………………………………………………...……43 2.3 Standardization.........................................................................................................45 2.3.1. Strategy……………………………………………….………………………..46 2.3.2. Terminology…………………………………………..……………………...47 2.4. Interface.......................................................................................................................51 2.5. Decision Support...................................................................................................... 52 2.6. EMR WB Definition..................................................................................................56 2.7. Summary......................................................................................................................59 Chapter III: Evaluation..................................................................................................................60 3.1. Testing EMR WB Versions....................................................................................61
i
3.2. Focus Group and Survey.......................................................................................63 3.2.1. Focus group..............................................................................................63 3.2.2. Survey.........................................................................................................66 3.3. Subject population.................................................................................................. 71 3.4. Analysis........................................................................................................................ 72 3.4.1. Confidentiality………………………………………………………………77 3.5. Results..........................................................................................................................77 3.5.1. Participants…………………………………………………………………..77 3.5.2. User satisfaction with EMR WB…………………………..………….79 3.6. Discussion...................................................................................................................85 3.7. Summary.....................................................................................................................89 3.8. Limitations of Study……………………………………………………………………91 3.9. Suggestions for Further Research………………………………………………..93 Chapter IV: Ethical Dilemmas....................................................................................................95 Chapter V: Conclusions................................................................................................................97 5.1. Conflicts of interest…………………………………………………………………….98 References.........................................................................................................................................99 Appendix A: Focus Group Study. Outline of Question Sequence.............................104 Appendix B: The Evaluation Tool. EMR WB Questionnaire for User Satisfaction……………………………………….....................................................105 Appendix C: The National Center for Biomedical Ontology BioPortal..................110 Appendix D-‐1: Standardization Process Sample: History of Present Illness...............................................................................................................................111 Appendix D-‐2: Standardization Process Sample: Contents of the History of Present Illness...........................................................................................112 Appendix E: Example of Standardized HPI Content within the Clinical Core…………………………………………………………….....……………………………..113
ii
LIST OF FIGURES
Figure Number Page
1. Two Anonymous Examples of Handwritten
Medical Record Documentation…………………………..……………………………...5
2. The EMR WB Interface………………………………………………… …………………...52
3. Clinical Core-‐centered EMR Definition……………………………………………….58
iii
LIST OF TABLES
Table Number Page
1. Table 1: Aspects of currently available web-‐based EMRs
compared to EMR WB.…………………………………..……………………………………9
2. Table 2: Philosophical approach to standardization……………………………48
3. Table 3: Focus group. ……………………………………………………………………..…69
4. Table 4: Demographic characteristics of study participants………………...73
5. Table 5: Evaluation summary for participants..………..…………………………74
6. Table 6: User satisfaction question groupings ……………………………………78
iv
1
Chapter I: Introduction
The patient record is the principal repository for information concerning a patient’s health
care [1,9,10,12]. It affects virtually everyone associated with providing, receiving, or
managing health care services. Despite the many technological advances in health care over
the past few decades, the typical patient record of today is remarkably similar to the
patient record of 60 years ago [9,57]. This failure of patient records to evolve is now
creating additional stress within the already burdened US health care system and in other
countries as the information needs of patients, practitioners, administrators, researchers,
and policymakers often go unmet.
Transferring accurate data from the clinical encounter into the medical record, making
health care decisions, and sharing and retrieving patient information is a difficult
undertaking, but all are instrumental to the effective practice of medicine and biomedical
research. Without accurate data, patients cannot be properly diagnosed and treated, and
without structured data, computer tools such as diagnostic decision support systems
cannot parse that data and assist health care providers. Research in translational medicine
also depends on the ability to document patient medical records so they can be related to
the vast genome data [18,41]. Current electronic medical records (EMRs) have been unable
to provide evidence that clinical documentation and practice is more efficient than in the
past [1,3,4,65]. In addition, many studies indicate low levels of satisfaction with overall
EMR use and deficiencies in the record-‐keeping process as currently practiced by health
care providers, which leads to medical records that often fail to include and share critical
information [6,9,17,57].
2
1.1. Statement of Problem
The literature indicates that the present EMR structure fails to help providers effectively
document, store, share, and access clinical findings; achieve independent thinking and
integration of knowledge; or improve clinical care workflows and decision-‐making
[1,2,3,4,13,57]. Also, mixed results from the implementation of EMRs in specialized settings
suggest that their use has had a limited effect on quality improvement in US hospital care
[3]. Despite the expense, time, and broad implementation of EMRs, there is little hard
evidence of benefits in the form of provider satisfaction, patient outcomes, or standardized
terminology [4,9]. The importance of the system proposed here, Electronic Medical
Records Without Borders (EMR WB), is centered on the premise that current EMRs lack
open global access [1,4,11], standardization [5,10,45], efficient interface [4,10,13], effective
knowledge-‐based tools at the point of care [21-‐27,36], and a clear definition [5,10,19,45].
Consequently, the information needs of patients, practitioners, administrators, researchers,
and policymakers often go unmet. These critical barriers have a negative impact on
provider and patient satisfaction, quality of life, health outcomes, clinical care workflows,
and health care costs [4,5,7,8,17,57,64].
1.1.1. Historical context
To best understand the nature of current EMRs, it is important to review their foundation
and evolution. Many current EMRs still follow the traditional organization and
characteristics of the original paper-‐based system and have simply automated that system.
Narrative documentation, for example, is far more prevalent than standardized structured
text [65].
3
The beginnings of medical record-‐keeping
While the central idea of this project is to bring patient records out of the ink-‐and-‐paper
era and into the state-‐of-‐the-‐art computer age, the paper-‐based record is not all negative.
The notion that an EMR is inherently better than a paper one is weak [19]. The content in
the record clearly has a large impact on a record’s effectiveness, independent of the
medium of delivery. The most vital aspect of the medical record is well-‐defined and
structured documentation [45,65]. Knowing this, an efficient design has to take into
consideration the well-‐known, mature, structured, and clinically-‐oriented paper-‐based
system that has been supporting the health care system for centuries. Nearly 2,500 years
ago, Hippocrates called for the careful recording and sharing of evidence about patients
and their illnesses. He developed the first known medical record in the fifth century BC1
with two goals: to 1) accurately reflect the course of disease and 2) indicate the probable
cause of disease. While this charge sits at the foundation of modern medicine, large
amounts of valuable health care data still go unrecorded. These goals are still appropriate,
but electronic health record (EHR) systems can also provide additional functionality, such
as standardized structured documentation [65], interactive alerts, interactive flow sheets,
and computerized physician order entry (CPOE)2 systems [1,19]. Information systems
capturing increasing amounts of health care data have ignited interest in the exciting
possibilities ahead for leveraging large quantities of patient and hospital data to increase
the quality of care while reducing costs.
1 J.H. van Bemmel and M.A. Musen, ed., Handbook of Medical Informatics, Springer, The Netherlands, 1997, p. 99. 2 CPOE or medical order entry systems are computer-‐assisted information networks that enable providers to initiate medical procedures, prescribe medications, etc. These systems support medical decision-‐making and error reduction during patient care. NLM-‐MeSH Year introduced: 2006.
4
Although the physical nature of the paper record has remained relatively the same, the
formal structure of the information contained within the record has undergone
considerable change in the last 60 years [2,19,51]. The first step to creating effective EMRs
should consider simply digitizing a well-‐structured paper medical record. After this has
been proven successful, the next area of focus should be ways to facilitate providers
sharing insights into the complexities of modern medicine to support specific clinical
decisions. It is also important to be aware that EMRs and paper-‐based records are different
artifacts that support different cognitive processes, and therefore do not have the same
usability problems [19,51,52].
To fully appreciation these differences, it is necessary to have a clear notion of the physical
aspects of the paper record. Paper is portable and access is direct, self-‐contained, and
practical (no power supply, network, information technology [IT] support systems, or
internet access required) [19]. It is a highly familiar method of recording information (no
special training needed as compared with a computer system). Another advantage of paper
is how little structuring it demands because it is unconstrained in both form and content in
a relatively informal medium (e.g., a long detailed history with marginal annotations,
physical exam with drawing of the affected area, lab results using special characters, and
handwriting diagrams), and it can be classified as a general-‐purpose tool for capturing data
(see Figure 1) [5,14,19].
Despite its benefits, a paper record can only be used for one task at a time, has access
constraints (e.g., lab and X-‐ray results or clinical notes might be in a different office),
consumes space, can be physically cumbersome, is susceptible to damage and degradation
5
over time, and is dependent on production practices that are not ecologically sound (e.g.,
deforestation and water pollution). Also, because its creation and modification can be very
personal, it may be difficult to understand for those other than the writer (see Figure 1). In
addition, the lack of any formal structure to guide record creation increases the possibility
for errors, misunderstandings, and omissions of relevance [5,16,51].
Figure 1: Two anonymous examples of handwritten medical record documentation.
EMRs as the next step
The most important motivations for creating EMRs are the drawbacks of paper-‐based
records outlined above. In simple terms, the current EMR is the computer replacement for
the paper medical record system. It provides a mechanism for capturing information
6
during the clinical encounter, stores it in a secure and organized fashion, and permits use of
that information only by those with a clinical need [19]. The EMR has a number of powerful
attributes that make it ideal for data capture and storage, including the enormous quantity
of data that is possible to store in a small space (e.g., cloud computing), and the ease of
creating copies, backups, and secure data sharing. Also, it facilitates more structured data,
portability, access, availability, interoperability, and structured care around automated
protocol-‐guided care (e.g., e-‐prescribing, alerts, reminders, task-‐specific views of data, and
clinical audit and outcomes assessment) [17].
The current EMR allows an institution to replace its paper charts with electronic charts.
This offers the potential for improving productivity and efficiency. Some EMR systems
provide reminders and alerts, CPOE [14], and links to data for improved health care, but
the impacts on health indicators are mixed or not well understood. Most EMR systems in
place are evolving concepts that tend to respond to the dynamic nature of the health care
environment and take advantage of technological advances and the recent financial
motivation for their implementation and meaningful use [10,14,15].
Current EMR system limitations
As W.E. Hammond pointed out in 2009 [9], “For much of the world, truly productive and
functional EMRs remain an elusive goal of the future.” Ellwood added that since 1988, “the
intricate machinery of our health care system can no longer grasp the threads of
experience… Too often, payers, physicians, and health care executives do not share
common insights into the life of the patient… The health care system has become an
organism guided by misguided choices; it is unstable, confused, and desperately in need of
7
a central nervous system that can help it cope with the complexities of modern medicine”
[2].
Hammond [9] also noted that “After a history of almost 60 years, the development of the
use of computers in health care—Healthcare Information Technology (HIT)—is still lacking
in delivering its perceived potential. The reasons are many: 1) the lack of definition and
clarity of what is important and required in health care; 2) the lack of consistency in the
practice of medicine; 3) the variety in evidence in an evidenced-‐based-‐medicine delivery
model; 4) the lack of a business model that proves a return on investment for HIT; 5) lack
of standards to support interoperability across various sites, domains, and models for
health care; and 6) the lack of an infrastructure enabling the necessary connectivity to
support the bringing together of data about a single patient.” His overall assessment is that
“For the most part, today’s EMR is not even a good representation of the paper record,
which is still unorganized and contains unstructured content.”
According to Kush et al. [18], “We can travel almost anywhere in the world and find a
machine that will dispense local currency, taking the money from our home account with
the use of a bank card. Yet, when we go from a primary care physician to a specialist in our
hometown, we must begin at square one, providing the new doctor's office with all our
demographic and medical information, often by completing paper forms.” Also, they
pointed out “if we were traveling abroad and needed access to our health information, we
would face formidable difficulties.” Financial institutions in particular have for years
developed and used standards for electronic exchange, but not the health care industry
[1,18].
8
Finally, low usability, concerns about privacy, and inability to use secondary information
for research represent additional major current limitations to EMR functionality and
adoption across institutions not only in the United States, but globally [55,57].
The ideal EMR system
Based on Mandl and Kohane’s 2012 [64] assessment of what is still needed to update the
management of health care, the ideal EMR system would affordably provide efficient means
of communication among providers and with their patients, secure private storage of
health care data in the cloud, standardized documentation tools, integrated workflow of
multi-‐disciplinary users, clinical decision support, and interoperability.
Some medical institutions in the United States are approaching this ideal by adapting
emerging technologies to their health care systems. Harvard Medical School-‐affiliated Beth
Israel Deaconess Medical Center (BIDMC) in Boston was selected as the most innovative US
user of business technology in 2012 by InformationWeek magazine as part of their 24th
annual ranking based on BIDMC’s IT department creating “Clinical Query,” a search engine
tied to a database of patient records, along with a Web-‐based, cloud-‐hosted EMR system
that runs on tablets, called Online Medical Record, which allows both physicians and
patients access to medical information.3
3 Beth Israel Deaconess Medical Center Embraces Analytics BIDMC, the No. 1 company in the InformationWeek 500 2012. http://www.informationweek.com/global-‐cio/interviews/beth-‐israel-‐deaconess-‐medical-‐center-‐emb/240006766
9
1.2. Proposed Solution
This dissertation describes the approach to how the most critical problems with
conventional EMR systems were resolved with the design, implementation, and evaluation
of a new EMR system (see Table 1). The proposed changes are aimed at contributing to the
EMR of the future, which would first positively impact provider satisfaction. The ultimate
goal is higher quality of life as a result of improved health outcomes.
Table 1: Aspects of currently available web-‐based EMRs compared to EMR WB.
Themes OpenEMR, OpenMSR
Practice Fusion
Epic, Epocrates
EMR WB
1 Access. Openly accessible on a web-‐based system from any platform.
Limited Yes Limited Yes
2 Standardization. Consistently includes patient histories as part of a standardized clinical family medicine core EMR.
No No No Yes
3 Interface. Compiles and condenses an all-‐inclusive intuitive EMR interface into one navigation system.
No No No Yes
4 Decision Support. Uses knowledge-‐based point-‐of-‐care systems to support health care decision-‐making processes.
Limited Limited Limited Limited
5 Structure. Design based on a thorough definition centered around patient histories.
No No No Yes
6 Communication Range. Available in more than one language (i.e., English and Spanish)*
No No No Yes*
7 Multidisciplinary. Medicine, psychology, nutrition, nursing, and social work.
No No No Yes
Web pages http://www.oemr.org, and http://openmrs.org/
http://www.practicefusion.com
http://www.epic.com/, and http://www.epocrates.com/
http://www.emrwithoutborders.com
*EMR WB is a fully bilingual system, and the structure and terminology is available in English and Spanish. The user can indicate the language of preference when logging in. The system does not automatically translate from one language to another.
10
EMR WB builds the technical capability to 1) make EMRs accessible on a web-‐based system
from any platform; 2) consistently include patient histories as part of standardized clinical
core EMR; 3) compile and condense an all-‐inclusive intuitive EMR interface onto one
navigation system; and 4) provide effective knowledge-‐based tools at the point of care to
support health care decision-‐making processes. These features will provide access to
scientific knowledge and build specific needed infrastructure to better address not only
local, clinical, and psychosocial primary care problems in targeted underserved
communities around the globe, but transnational migration health issues based on data
transfer of clinical information in primary care settings, in particular free clinics serving
patients traveling between Mexico and the United States.
By making EMRs accessible on a web-‐based system from any platform—PC, Mac, tablet, or
smartphone, each with their corresponding operating systems—there will be a substantial
simplification and expansion of the ways in which health care providers access and input
clinical information. They will switch from local, restricted onsite use of paper-‐based
medical records or EMRs to a system that takes advantage of cloud computing. This change
will allow them to input, retrieve, store, share, and access patient medical records
anywhere (rural or urban settings) when needed [1,9]. Also, they will be able to remotely
access and use knowledge-‐based tools at the point of care to support clinical decision-‐
making. With this new service, the patient, as the owner of their personal medical record
and its access code, can travel almost anywhere in the world and find a provider that can
access their EMR, thereby avoiding the need to reenter demographic and medical
information onto paper forms. This represents a potentially significant savings in time and
11
frustration, and improvement in health care quality for patients who seek medical
attention while traveling or living abroad [18].
The application of this project clearly challenges the current EMR paradigm by shifting
away from the fragmented, difficult-‐to-‐use EMR system that has been unable to provide
evidence that clinical practice is more efficient than in the past [1,3,4,5]. This novel
proposed design counteracts the predominant health care system paradigm that prioritizes
the quantity of care rather than the quality of it. EMR WB represents the cost-‐saving health
care innovation that drug companies and private insurers have failed to provide [1,4,27].
EMR WB is patient-‐centered rather than system-‐centered, based on the premise that a
standardized, easy-‐to-‐use, well-‐defined, and accessible EMR that integrates best evidence,
provider clinical expertise, and patient choice will improve health care. It is a network of
tools that recognizes the primary information in the EMR—clinical documentation—is the
keystone on which the health care decision-‐making process is based. Provider interaction
with such an integrated system is expected to be a satisfying experience. A system that is
centered on efficiency and quality of information will be one way to reduce health care
costs, as the evidence points out [8,17,27,30,65]. In addition, this model could be
generalized as a valid frame of reference for other similar populations (primary care
centers aimed at providing services in limited resource settings).
1.2.1. EMR WB project background
The research, development, implementation, and evaluation of EMR WB required
international, multidisciplinary collaboration. This approach was based on an established
need to share clinical information in a global context, starting with specific populations on
12
both sides of the US and Mexican borders as part of a pilot project in a well-‐structured
primary care setting in the city of Leon, Mexico. A series of progressive steps was carried
out that involved the identification of specific problems with sharing information using
current EMR systems.
International collaborative research
In December 2009, the University of Guanajuato School of Medicine Department of
International Affairs in Mexico, the University of Pittsburgh Center for Global Health,4 and
the University of Pittsburgh-‐affiliated community clinics in the metropolitan area (i.e.,
Squirrel Hill Health5) approached me to participate in a three-‐day collaborative
international research conference titled “Workshop of Transnational Migration and Global
Health: The Case of Guanajuato and Pittsburgh, 2010.” The rationale for this meeting was to
discuss and analyze the quality of health care for migrant workers traveling between
Pittsburgh, Pennsylvania, and the urban community of León, as well as rural communities
in the state of Guanajuato (Ocampo and Los Lorenzos), Mexico.
The basis of this request to participate in the April 2010 global health workshop was a
series of successfully implemented collaborative research and exchange programs between
the UW School of Medicine and University of Guanajuato medical students and faculty over
five years. When the idea to develop a project that would address the issues raised at the
workshop was proposed to José Luis Lucio-‐Martínez, PhD, President of the University of
Guanajuato Campus León, and Ana María Chávez-‐Hernández, PhD, Director of the
4 University of Pittsburgh Center for Global Health: http://www.globalhealth.pitt.edu/education/International-‐scholars.php 5 Squirrel Hill Health Center: http://www.squirrelhillhealthcenter.org/
13
University of Guanajuato Center for Health Services (CUSS), both officially gave their
support to the plan in July 2010 with regard to collaboration, funding, and advising.
My participation was centered on researching the following question: What would be the
best approach to sharing clinical information across borders (i.e., between Leon, Mexico,
and Pennsylvania, Pittsburgh)? Initially, the plan was to include the University of
Pittsburgh-‐affiliated community clinics and several free clinics in the state of Guanajuato,
Mexico. The identified need for sharing clinical information on an international scale was
born, primarily based on the documented occurrence of patients moving one or several
times a year between these communities, and using the local health clinic services. Health
care providers were isolated in terms of patient data sharing, and dependent upon what
patients could provide during office visits to the clinics. Vital information was lost, such as
the description, severity, evolution, and prognosis for specific conditions; medication types,
doses, and regimens; responses to treatments; and the type of and results for laboratory
and image studies performed—essentially entire health records. The preliminary
conclusions were that rural and urban public clinics in the state of Guanajuato, Mexico, lack
a modern, standardized EMR system, open global access, efficient interface, and
knowledge-‐based point-‐of-‐care systems to efficiently collect, store, share, manage, and
access patient files. As a result, the vital information needs of patients, practitioners,
administrators, researchers, and policymakers were often unmet, and negative financial
and health outcomes were escalating. The same problem has to be addressed at the
University of Pittsburgh-‐affiliated clinics, and they have to be prepared to access the EMR
WB to mitigate transnational migration health issues based on information exchange
between the CUSS and corresponding primary care settings in Pittsburgh. It is expected
14
that University of Pittsburgh-‐affiliated primary care clinics would implement the EMR WB
project after the completion of the pilot project at the CUSS, in particular the Squirrel Hill
Health Center.6
Based on these findings, combined with the knowledge that interlinked communities
across borders is a common situation all over the United States and Mexico as well as in
other countries, the creation of a research project originally titled “A Web-‐based EMR
Regional Primary Care Network System for Urban and Rural Public Clinics in the State of
Guanajuato, Mexico: A Pilot Project” was proposed. This international, multidisciplinary,
multinstitutional collaborative research effort involved Ana María Chávez-‐Hernández, José
Luis Lucio-‐Martínez, and Brigitte Lamy, PhD, Professor of Sociology at the University of
Guanajuato Campus León, as the key network collaborators in Mexico. Other collaborators
included 1) Robert Dickow, PhD, Associate Professor, University of Idaho; 2) Kathleen
Musante DeWalt, PhD, Director of the University of Pittsburgh Center for Latin American
Studies and Professor of Anthropology and Public Health; 3) Joanne L. Russell, MPPM, RN,
CCRC, Director of the University of Pittsburgh Center for Global Health and Assistant
Professor of Global Health; 4) Kenneth S. Thompson MD, Medical Director of the Center for
Mental Health Services at the US Department of Health and Human Services; 5) Sergio
Aguilar-‐Gaxiola, MD, PhD, Director of the Center for Reducing Health Disparities at the
University of California-‐Davis and Professor of Internal Medicine; 6) Edward P. Hoffer, MD,
Associate Clinical Professor of Medicine at Harvard Medical School and researcher at the
Massachusetts General Hospital Laboratory of Computer Sciences; 7) Andrea Fox, MD,
MHP, Associate Professor of Family Medicine at the University of Pittsburgh School of
6 Squirrel Hill Health Center: http://www.squirrelhillhealthcenter.org/
15
Medicine and Medical Director, Internist, and Geriatrician at Squirrel Hill Health in
Pittsburgh; and 8) Alfonso Barquera, local Latino leader and active member of the Hispanic
Outreach Project of the Community Social Justice Project in Pittsburgh. The roles of the
collaborators were diverse, from direct clinical care of migrant patients on both sides of the
border, to research in fields such as migrant health, anthropology, sociopsychological
factors of migration, economics, and global biomedical informatics.
The EMR theoretical model that was developed incorporated a comprehensive concept of
multidisciplinary standardized family medicine clinical history that integrates state-‐of-‐the-‐
art terminology (controlled vocabularies, syntaxes, ontologies, and semantics) and
evidence-‐based (EB)7 clinical protocols and guidelines into an ethnically-‐oriented and
culturally-‐sensitive English-‐Spanish language EMR system. The goal was to better address
not only the current local and public health problems, but also transnational migration
health problems based on the need for data transfer of clinical information to improve
provider satisfaction and health outcomes in the targeted populations. Although EMRs
were being used in many areas of medicine and research indicated they had potential to
improve patient quality of life and health outcomes, their implementation, standardization,
and use in Latino populations was limited, particularly in underserved rural and migrant
sectors.
Due to the international nature of this research and the need to implement a pilot project,
two international summer independent research programs were applied for through UW
International Programs and Exchanges (IPE) in 2010 and 2011. Both were accepted and
7 Evidence-‐based: Characterized by “the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients and populations integrating individual clinical expertise and patients’ choice/preference.” (Sackett DL. BMJ. 1996 Jan 13;312(7023):71-‐2)[28]
16
credit was granted to travel, begin a feasibility study, and implement a research pilot
project at the University of Guanajuato.
During visits to Mexico, EMR WB was developed with the intent to provide a user-‐friendly
and efficient computerized system for recording patient medical histories in a global
context. This is a multidisciplinary, standardized, state-‐of-‐the-‐art EMR using cloud
computing to enable accessibility anywhere from any platform (PCs, Macs, tablets, and
smart phones with their respective operating systems). Using this model, a pilot project
was implemented at the University of Guanajuato Center for Health Services (CUSS), in the
City of Leon, state of Guanajuato, Mexico. Beginning in the summer of 2010, this pilot
project was led in collaboration with the University of Washington Department of
Biomedical Informatics and Medical Education (UW BIME) under the supervision of Fredric
M. Wolf, PhD, and the University of Guanajuato.
During the summer of 2010, onsite work on project feasibility was begun by setting up a
focus group with CUSS end users (health care providers). The results showed that potential
issues with EMR WB acceptability would need to be addressed because CUSS providers
were used to a paper-‐based system for creating and maintaining health records. The new
system was therefore adapted to their clinical environment using existing information
technology architecture within the University of Guanajuato: in particular, the same
hardware, internet access, and local technical support from their Information Technology
Department. Modifications that made the system unique to the CUSS included
computerization of the clinical data gathering and access to it in their own clinical context.
To achieve these basic features, the services of several companies that could potentially
17
program the EMR and provide cloud services were evaluated. Based on these preparations,
research questions were constructed and project feasibility was corroborated with the
providers and university administrators.
After completing this preliminary work, the interface design was begun, which included
standardization of clinical histories (for the domains of medicine, psychology, nursing,
nutrition, and social work) using shared terminologies with controlled vocabulary (terms),
ontologies (relations), semantics (meaning), and syntaxes (structure). EB and peer-‐
reviewed clinical protocols and knowledge-‐based diagnostic decision support tools at the
point of care were also included. Next, a contract was set up with the most promising US
company (Arvixe Web Hosting) for the cloud services. Implementation and beta testing
began on May 25, 2011. Robert Dickow, PhD, Associate Professor at the University of Idaho,
with more than 30 years of truly exceptional multi-‐language programming experience,
executed and programmed EMR WB in its totality.
The next step of the project was conducted during a trip to León in summer 2011 when the
implementation and onsite use of the EMR WB system at the CUSS was evaluated. During
this second international summer independent research program, use of the actual
working system by the CUSS health care providers in their own settings was witnessed, and
discussions were held with them on several occasions to discuss their experiences. This
included a series of talks about the standardized EMR pilot software used, demonstrations
of how EMRs can facilitate the clinical workflow, and feedback on encountered limitations.
Several very productive weekly working meetings were also conducted with Dr. José Lucio
Martínez and Dr. Ana María Chávez-‐Hernández about the project progress and adoption
18
specifics. The beta testing approached its final phase in December of 2011 based on
exhaustive feedback obtained from the CUSS health care providers.
Although the feedback from the EMR WB use evaluation at the CUSS was generally positive,
some onsite technical aspects and logistics were reconfigured as part of a risk management
approach and follow-‐up study. These included looking into the internet access to ensure
that even with some variants in speed, the overall functionality was not compromised.
Fields of free text were added or expanded (in particular with nutrition and nursing) with
free text, made available across all modules, and deleted if users reported them as
duplicates when describing “bugs” in the presentation and structure of the record from
different domains (mainly psychology and social work).
Related EMR WB projects and recognition include the following:
• A paper based on an observational study in León, Mexico, in collaboration with Dr.
Ana María Chávez-‐Hernández, which will be submitted for publication to JAMIA as
“Perception of implementation and use of a standardized family medicine web-‐
based system among providers in urban clinics in the state of Guanajuato, Mexico.”
• A story of my experience in Mexico implementing EMR WB published by the UW
School of Public Health at http://sph.washington.edu/news/fellows.asp.
• Several talks, poster presentations, and seminars regarding the project, the field of
biomedical informatics, and EMRs as a whole, in different settings at the University
of Guanajuato (in July 2010, May 2011, August 2011, and January and February
2012).
19
• An abstract and poster at the Symposium of the Program for Education and
Research in Latin America (PERLA) by the UW Department of Global Health in
April 2011.
• A keynote speech at the XXVI National Congress in Medical Research in Monterrey,
Mexico, September 2011 (http://www.congresobiomedico.org.mx/).
• A poster at the UW Department of Global Health Open House in January 2012.
• The 2011-‐2012 Thomas Francis, Jr. Global Health Fellowship award from the UW
Department of Global Health.
• A focus session presentation to represent the University of Washington at the 2012
National Library of Medicine (NLM) Informatics Training Conference.
This collaborative, multinstitutional research experience has been instrumental in the
overall research, development, implementation, and evaluation of EMR WB, aimed at
supporting the clinical services provided by the University of Guanajuato to those who
need it most: underserved and migrant populations. This research was intended to address
one of the most significant problems in migrant health by offering health care providers in
the United States and Mexico the ability to access and transfer data to patients’ medical
records across international borders. EMR WB collaborative partners believe that the
system has the potential to be successfully applied anywhere in the world.
20
The CUSS in the context of Mexican health care
As with most other middle-‐income countries, Mexico does not provide universal access to
health care for its population. About half of Mexico's population does not have health
insurance under the current system, as reported by the international Organization for
Economic Co-‐operation and Development (OECD).8
Health care in Mexico is provided via public institutions, private entities, and private
physicians. Health care delivered through private health care organizations and physicians
operates entirely on the free market system, so it is only available to those who can afford
it. Public health care delivery, on the other hand, is provided via an elaborate system put in
place by the Mexican federal government. The CUSS is part of the public sector and
supported through the state government as part of the public university system.
Mexico has a fractionalized health insurance system with a variety of public programs, but
coverage is restricted based on employment status and if the person works for the
government or in the private sector. There is also a private insurance market, mostly used
by wealthy residents. A social security-‐administered system (IMSS) covers those who are
employed unless they have private employer-‐sponsored insurance, while the “Seguro
Popular” program, created in 2003, was set up to help cover the uninsured population.
Poor families can participate in Seguro Popular for free. People who do not participate in
the insurance program can still access services through the Ministry of Health free clinics
(which includes the CUSS), though sometimes with difficulty because these clinics are
limited across Mexico. An additional hurdle is that the different public systems and private
8 OECD Reviews of Health Systems – Mexico. OECD Publishing, 07 Jul 2005. 158 pages. Language: EnglishISBN: 9264008934
21
insurers all use different medical facilities and providers, each with a wide range in service
quality at all levels such as primary, secondary, and tertiary care, as reported by OECD.
The social security-‐based insurance in Mexico provides broad coverage for medical
services, including primary, acute, ambulatory, and hospital care, as well as medications.
Seguro Popular provides access to an established set of essential medical services and the
needed drugs for those conditions, as well as 17 high-‐cost interventions such as breast
cancer treatment. These services are provided through government facilities, usually run
by the state.
Out-‐of-‐pocket payments by patients represent over half of the financing for the Mexican
health care system, according to the World Health Organization (WHO).9 Government
expenditures account for 44 percent of health spending. The public entities, including the
Mexican Institute for Social Security and the Institute of Social Security for Government
Employees, are financed through general taxes. Payments are determined by salary level.
The Seguro Popular program is funded through taxes, contributions from state and federal
government, and payments by families as a percentage of income. Participants in Seguro
Popular pay nothing at the time of service.
EMR WB in the CUSS workflow
EMR WB was implemented at the CUSS, a multidisciplinary primary care clinic for
underserved and migrant populations in the city of Leon, state of Guanajuato, Mexico. The
CUSS is a research and training outpatient clinic on the University of Guanajuato Campus
Leon that employs a total of 50 health care providers in 24 offices distributed across the
9 WHO-‐Mexico: http://www.who.int/countries/mex/en/
22
different services offered: medicine, psychology, nursing, nutrition, and social work. As a
teaching facility, students perform their clinical rotations under close supervision of the
providers in their corresponding disciplines.
The CUSS can be defined as a multidisciplinary primary health care institution which
provides integrated and accessible health care services by providers who are accountable
for addressing a large majority of personal health care needs, developing a sustained
partnership with patients, and practicing in the context of family and community.10 In this
context, the workflow of the CUSS is based on a pattern of recurring procedures that
include notifications, decisions, and actions to influence quality of life and local public
health outcomes.11
The front office. When a patient enters the CUSS, they must go directly to the front desk and
ask for a consultation in one of the five areas of services provided, according to their own
identified needs. The front desk receptionist provides information and instructs the patient
regarding the options. At this time, the receptionist inputs patient demographic
information into the identifying data—ID (patient name, age, gender, marital status, etc.)
and motive for consultation—MC (or reason for the visit). The front desk either creates (in
the case of a new patient) a new chart within the EMR WB system using the initial data
input, or opens the patient’s existing chart within the EMR WB system using the patient’s
full name and unique identifying number, and corroborates that the patient matches the
data.
10 Primary Health Care. Care which provides integrated, accessible health care services by clinicians who are accountable for addressing a large majority of personal health care needs, developing a sustained partnership with patients, and practicing in the context of family and community. (JAMA 1995;273(3):192) ; MeSH-‐Year introduced: 1974(1972) 11 workflow is defined as the description of pattern of recurrent functions or procedures frequently found in organizational processes, such as notification, decision, and action [NLM/MeSH, Year introduced: 2010].
23
The back office. After the patient has been registered, the nurse takes the patient to the
exam or consultation room, sits the patient (on the examination table in the case of
medicine, nutrition, or nursing), opens the patient’s chart within the EMR WB system,
verifies the MC with the patient, and corroborates, adds, or inputs their professional
opinion about the MC. At this point, the nurse also checks the file for refills, screenings, or
preventive services that might be needed and then flags the chart so the primary health
care provider can specify those services during the visit. Then the nurse takes vital signs as
appropriate for the consultation in the fields of medicine, nutrition, or nursing (blood
pressure—B/P, pulse/heart rate—HR, respiratory rate—RR, body temperature—T,
height—HT, weight—WT, and head circumference—HC in children, etc.). Next, the nurse
asks the patient to change into a gown if needed, and informs him/her that the physician or
nutritionist will enter the room shortly, or the patient continues with the nurse if it is a
nurse consultation or health education session. In the case of psychology and social work,
the vital signs and physical preparation of the patient are omitted. During this process, the
computer screen (mainly desktops) is not in sight of other personnel to safeguard privacy,
but the patient is able to see what the clinician sees. Next the provider enters the room,
interacts with the patient, and opens the EMR WB system where the ID and MC appear on
the front page. If the patient is new, the provider begins the interview to gather the history
of present illness—HPI, past medical history—PMH, family history—FH, social history—
SH, review of systems—ROS, and health maintenance—HM, as appropriate. The final steps
are performing the physical exam (this part is skipped by psychologists and social
workers); ordering any needed laboratory or diagnostic image studies; and providing plans
for treatment (e.g., prescription for medications or physical therapy, referral to another
24
provider). The provider also has the option to add clinical or follow-‐up notes to the
patient’s EMR WB.
The idea behind using EMR WB in the clinical encounter at the CUSS is that the health care
providers are able to interview patients while simultaneously inputting the clinical data
into the medical record; the file remains open and accessible for consultation and
additional input throughout the physical exam, diagnosis, prognosis, treatment, and plans.
When the patient returns to the front desk, the receptionist or nurse uses EMR WB to
verify/provide some piece(s) of information such as follow-‐up visits, referrals, or payment
arrangements (usually a suggested fee if the patient has the financial means), and the file is
then saved and closed.
In the CUSS context, the EMR WB design is centered on a one-‐page, simplified data entry
interface. This format allows providers to see the whole EMR and navigate among the
different fields without losing sense of where they are in the chart. Based on the
preliminary observational study, providers want and need an easier way to enter patient
data while they are interacting with the patient without negatively affecting the patient-‐
provider relationship. EMR WB addresses this issue by providing an interface that is easy
to use and navigate, does not interfere with patient interaction during the clinical
encounter, and facilitates all the steps of the clinical workflow as defined by the clinicians.
25
1.3. Research Statements
1.3.1. Thesis statement
Currently, EMRs often lack global access, standardization, an efficient interface, and
effective knowledge-‐based point-‐of-‐care tools. Consequently, the information needs of
patients, practitioners, administrators, researchers, and policymakers often go unmet,
leaving providers especially dissatisfied.
1.3.2. Research questions
Aim 1. Will development of a standardized, web-‐based family medicine EMR WB system
that is accessible from any platform, functions with an intuitive interface, and provides
decision support tools for the University of Guanajuato Center for Health Services (CUSS)
be achievable?
Aim 2. Will use of the EMR WB system have a positive effect on provider satisfaction
compared with a paper-‐based system?
1.3.3. Objectives
1. Develop a standardized, web-‐based family medicine EMR WB system that is accessible
from any platform, functions with an intuitive interface, and provides decision support
tools for the CUSS.
2. Evaluate CUSS provider satisfaction with EMR WB compared to the paper-‐based system
as measured by a quantitative post-‐implementation follow-‐up survey.
26
1.4. Significance
EMR WB is unique for its global yet particularized approach. It was designed to be highly
functional in a diverse, global context by providing a familiar structure (clinical core),
which was standardized to better address the needs of patient documentation for both
unstructured and structured input [65]. Because EMR WB was implemented in free clinics,
where services are government-‐funded and provided for either low or no cost, the coding
intended for billing and other administrative avenues does not apply. The structure of EMR
WB shifts the priority from administration to the patient, thereby supporting health care
decision-‐making processes.
In addition, EMR WB provides support for a thorough blend of multidisciplinary primary
care aimed to work globally. Primary care, in this context, means the integration of
medicine, psychology, nursing, nutrition, and social work by sharing longitudinal
collections of patient data across these different fields. This approach is ideal for the
primary care environment in underserved global settings, as it covers the majority of the
population’s health needs in a seamless environment.
EMR WB, as a web-‐based system, is accessible from any platform for its input and output,
so providers in limited resource settings will be able to use it with practically any device
they have on hand. With the e-‐Mobile technology (mobile phones) currently used for
specific tasks (e.g., infectious diseases, public health data gathering) in many resource-‐
limited settings across the world, EMR WB can provide comprehensive access to an EMR
with multidisciplinary primary health care data gathering and sharing capabilities, which is
not currently available [9,10,41].
27
EMR WB is available in both English and Spanish, allowing the provider to use the language
of choice. This capability was incorporated for several reasons: 1) Mexico is the primary
place for the pilot implementation, where Spanish is the native language; and 2) Spanish is
the second most common natively-‐spoken language in the United States and the world
(500 million speakers) after Mandarin Chinese.12 The Spanish-‐language EMR WB system
will not be a translation, but an integration of linguistics with ethnically-‐oriented and
culturally-‐sensitive components that have been incorporated in the clinical core to
facilitate the input of information.
1.5. Research Goals
The purpose of this project was to create and evaluate a novel multidisciplinary (medicine,
psychology, nursing, nutrition, and social work) primary care EMR system that includes
global access, standardization, and knowledge-‐based tools at the point of care, all with an
efficient interface. This project aims to facilitate structured clinical documentation,
interoperability, global access, and decision-‐making processes to better address not only
local, clinical, and psychosocial problems encountered by patients at the CUSS, but also to
prepare CUSS to support transnational migration health issues based on information
exchange among primary care settings in the future.
1.5.1. Aim 1. Develop a family medicine web-‐based EMR WB system that is
standardized, accessible from any platform, functions with an intuitive interface, and
provides decision support tools. An EMR design aimed at working with underserved
12 Spanish language, facts: http://en.wikipedia.org/wiki/Spanish_language
28
populations in a global context will be created that integrates four essential strategies: 1)
make EMRs accessible on a web-‐based system from any platform; 2) consistently include
patient histories as part of a standardized clinical core EMR; 3) compile and condense an
all-‐inclusive intuitive EMR interface onto one navigation system; and 4) create access to
effective knowledge-‐based tools to be used at the point of care to support health care
decision-‐making processes. The EMR WB design will be based on both a standardized
family medicine clinical history and the terminology outlined by the NLM for medical
subject headings (MeSH) to index biomedical articles.
Aim 1.1. Access: Make EMRs openly accessible on a web-‐based system from
any platform. A tool will be created that provides a response to the needs for
information exchange and interoperability across institutions, platforms,13 and
borders within the medical care system. This solution is integrated into an efficient
web-‐based system using cloud computing. In this setting, the provider will be able
to access the EMR through any platform, anywhere, anytime. Patients will be
empowered by guaranteed ownership of their medical record via its access code,
allowing them to choose any provider anywhere for informed individualized care.
This EMR code will allow the provider to have full privileges to access, retrieve, and
add information to the clinical file. The patient, as the owner of the medical record,
will be able to access their file and input information regarding demographics,
family history, personal medical history (e.g., allergies), current prescription
medication, and/or over-‐the-‐counter supplements.
13 Computer platform: The basic technology of a computer system's hardware and software that defines how a computer is operated and determines what kinds of software can be used. Source: http://www.webopedia.com/search/platform. For the purpose of this project, the different platforms that will be considered are PC, Mac, tablets, and smartphones, with their corresponding operating systems.
29
Aim 1.2. Standardization: Consistently record patient histories as the EMR
clinical core using standardized terminology. The use of family medicine clinical
history, treated here as the clinical core of the EMR, will be standardized by
integrating state-‐of-‐the-‐art terminology (controlled vocabulary, ontologies,
semantics, and syntaxes) with EB [28] and peer-‐reviewed clinical protocols. This
model will be used to ensure patient histories are consistently recorded as the EMR
clinical core with structured and unstructured clinical documentation. Additionally,
another four primary care14 domains—psychology, nursing, nutrition, and social
work—will be included using the same principles of standardization in each
respective domain. These domains fulfill the multidisciplinary needs of patients at
the primary care level. Standardization of the EMR in this project has three specific
objectives: to improve 1) the accuracy of primary unstructured and structured
clinical documentation, 2) interoperability, and 3) information exchange within the
EMR WB system.
Aim 1.3. Interface: Compile and condense an all-‐inclusive, intuitive, and
efficient EMR interface into one navigation system. An intuitive EMR interface
will be created that allows providers to see the whole EMR and its different domains
in two top menu bars while navigating among the different fields and applications
without losing sense of where they are (i.e., all fields and applications are within the
reach of one or two clicks). The patient data entry portion will be intuitive,
structured by providing standardized clinical histories for each of the four primary
14 Primary health care: Integrated, accessible medical services by clinicians who are accountable for addressing a majority of personal health care needs, developing a sustained partnership with patients, and practicing in the context of family and community. (JAMA 1995;273(3):192) NLM—Year introduced: 1974(1972)
30
care domains identified in Aim 1.2. Rate-‐limiting data entry applications will be
created for each different field of the clinical history (e.g., history of present
illness—HPI or review of systems—ROS) so that providers can indicate positive and
negative findings and use either structured data or free text. The interface will allow
users to easily and accurately retrieve, seek, gather, encode, transform, organize,
and manipulate pertinent information to accomplish desired tasks.
Aim 1.4. Decision support: Integrate infobuttons as knowledge-‐based point-‐of-‐
care tools. A decision support application system will be included that relies on
“infobuttons” [21-‐27,36]15 for access and use of web-‐based point-‐of-‐care systems.
An infobutton interface will be set up to have direct access to knowledge-‐based
point-‐of-‐care systems in real time. The provider will be able to use the infobutton
functions to generate and send queries to electronic health information resources
using data manually extracted from the EMR. The infobutton will be displayed to the
right of the top menu bar in the EMR interface. When clicked, the infobutton will
formulate a menu with three options: 1) DXplain, 2) EB Medicine Calculators, and 3)
Knowledge Translation Tools.
1.5.2. Aim 2. Evaluate the family medicine web-‐based EMR WB system to determine if
it has a positive effect on provider satisfaction compared with a paper-‐based medical
record system. An evaluation of EMR WB will be conducted via both qualitative (pre-‐
implementation exploratory focus group) and quantitative (post-‐implementation survey)
analyses centered on provider perceptions of its use compared to other EMR systems and
15 Infobuttons: Decision support tools that offer links to information resources based on the context of the interaction between a clinician and an electronic EMR system. [J Biomed Inform. 2008 Aug;41(4):655-‐66. Epub 2007 Dec 8]
31
paper-‐based records. The focus group will be used as an exploratory pre-‐implementation
tool to gain knowledge of how providers perceive EMRs in general. This information will be
used to 1) create a schema of coding data to identify specific needs (needs analysis); 2)
establish the feasibility of the project (support and willingness to use the system); 3) obtain
input for the EMR WB design; and 4) create the post-‐implementation questionnaire survey
design.
The post-‐implementation survey will be focused on provider opinion of EMR WB use. A
tailored survey will be created to evaluate end user satisfaction based on the themes and
codes obtained from the focus group. The Questionnaire for User Interaction Satisfaction
[35,38,39] (QUIS)16 and IBM Computer Usability Satisfaction Questionnaires17 will be used
as frames of reference for its design. The survey will be given to CUSS health care
providers, who will be asked to make a decision on their level of agreement with
statements assessing their satisfaction with EMR WB use compared to paper-‐based
records. A 7-‐point Likert scale will be used to measure the data obtained.
1.6. Establishing Feasibility
A focus group was performed in August 2010 aimed at better understanding the potential
end user (health care provider) of the proposed EMR. The participants indicated a
willingness to participate in the pilot project because of its inherent potential to improve
their current work using a paper-‐based system, and hence their overall satisfaction with
16 QUIS Main page: http://lap.umd.edu/quis/ 17 IBM Computer Usability Satisfaction Questionnaires: http://drjim.0catch.com/usabqtr.pdf
32
patient care. The focus group was conducted to confirm with a high degree of certainty that
providers would participate in the project. (See Chapter III: Evaluation, 3.2. Focus Group
and Survey.)
In addition, a well-‐defined professional relationship was established with the top
administrator of the University of Guanajuato Campus Leon, Dr. José Luis Lucio-‐Martínez,
who has authority for direct decision-‐making related to my project implementation and
evaluation. An appointment was set up in the summer of 2010, and several follow-‐up
meetings were held during my stay. He enthusiastically welcomed the project, and agreed
to its implementation and evaluation. By coincidence, Dr. Lucio-‐Martínez was in the
process of building the CUSS, which allowed me to set up the implementation of EMR WB at
the same time that they opened the clinic to the public.
Dr. Lucio-‐Martínez then introduced me to the clinic’s director, Dr. Ana María Chávez-‐
Hernández, who openly welcomed the concept and has ever since been an invaluable
collaborator and colleague. A Memorandum of Understanding was established, in addition
to the verbal agreement, which allowed management of any potential high-‐risk aspects of
the proposed work such as internet access and local onsite technical support provided by
the University of Guanajuato. In addition, coordination for the contracted services that
programmed the EMR and cloud computing was provided.
The project, from its early stages in May 2011 when the beta testing was initiated, has
gradually advanced towards its full completion, which included 1) Set-‐up of the EMR home
page and commercial service in the cloud to support web-‐based access, available at
www.emrwithoutborders.com; 2) Standardization and programming of the clinical core,
33
including the domains of medicine, psychology, nutrition, nursing, and social work; 3)
Establishment of an interface using one navigation system; and 4) Access to knowledge-‐
based decision support tools at the point of care such as an EB medicine calculator,
knowledge translation tools, and the decision support system DXplain, all of which are
currently available online.
1.7. Resource Sharing Plans
Funding for the development and implementation of EMR WB was provided by the
University of Guanajuato Campus León, UW BIME, University of Pittsburgh, UW Office of
Financial Aid, NLM, and a Thomas Francis Jr. Global Health Fellowship. The Laboratory of
Computer Sciences at Massachusetts General Hospital, Harvard Medical School, offered
additional support in the form of free access and use of the diagnostic decision support
system DXplain. Access to the knowledge-‐based tools used in this proposal (EB medicine
calculator and knowledge translation tools) are free of charge.
1.8. Summary
Rural and urban public clinics in the state of Guanajuato, Mexico, lack a modern,
standardized EMR system with global access, an efficient interface, and knowledge-‐based
point-‐of-‐care tools that allow providers to efficiently collect, store, share, manage, and
access patient files. As a result, the vital information needs of patients, practitioners,
administrators, researchers, and policymakers often go unmet, which leads to negative
34
provider satisfaction and poor quality health care. Also, despite implementation of EMR
systems in the United States and other countries, the same limitations and unmet needs
exist.
To address this multifaceted problem, a novel EMR design was created as a pilot project to
ensure that the most vital pieces of patient EMRs are available to make the best health care
decisions for anybody in need. A bilingual web-‐based EMR family medicine model was
developed with an efficient, intuitive interface that integrates state-‐of-‐the-‐art terminology
(controlled syntaxes, ontologies, and semantics), EB clinical protocols centered on a
patient’s clinical history, and knowledge-‐based point-‐of-‐care tools that provide real-‐time
access to a diagnostic decision support system. The newly designed EMR system, EMR WB,
was implemented in a public, multidisciplinary family medicine clinic at the University of
Guanajuato in an underserved urban community in the city of Leon, Mexico. The CUSS is
equipped with computers and/or mobile devices with access to the internet.
The objectives of this independent research project are to 1) Highlight the importance of
and need to better understand the logistics of bringing patient records out of the ink-‐and-‐
paper era and into the state-‐of-‐the-‐art computer age, and 2) Learn about provider
satisfaction with EMR WB use in comparison with the paper-‐based system. During this
onsite, hands-‐on experience, the focus was on answering “why” and “how-‐to” questions
regarding the definition, design, implementation, interoperability, and evaluation of EMR
WB.
The focus group was used to establish the feasibility of implementing EMR WB at the CUSS.
Evaluation of the system was conducted via the EMR WB Questionnaire for User
35
Satisfaction (EMR WB QUS) survey focused on provider satisfaction. It was hypothesized
that EMR WB use would have an overall positive measurable effect on provider
satisfaction.
36
Chapter II: Methods—Designing the Solution
The possibility that technology can help providers accurately capture unstructured and
structured data from the clinical encounter, transfer it into the medical record, and then
share and retrieve it offers a solution to the current EMR design that represents a paradigm
shift in health care. This change means moving away from the fragmented, difficult-‐to-‐use
EMR system [1,4] towards well-‐defined, standardized, easy-‐to-‐use processes with decision
support and universal access.
The overall strategy used to accomplish the specific aims of the project as described in
Section 1.5 was focused on building a system from the beginning, without borrowing codes
from open sources or extending existing open EMR systems (e.g., OpenMRS18 and
OpenEMR19). This decision was made because the open-‐source software available carried
the same functional limitations in their design that most EMRs have as outlined in this
study: 1) Minimal accessibility, 2) Incomplete and inconsistent clinical documentation, 3)
Lack of connectivity between medical record categories, and 4) Limited or no decision
support tools at the point-‐of-‐care. The focus of most original EMR developers and users
was to solve specific administrative problems, which included providing service-‐related
functions for admission, discharge, and billing. The central idea of the EMR WB design is a
system with a well-‐structured, standardized foundation that will support taking accurate
data from the clinical encounter into the medical record, sharing and retrieving it, and
integrating it into the practice of medicine and biomedical research.
18 OpenMRS offical website: http://openmrs.org/ 19 OpenEMR ofical website: http://www.open-‐emr.org/
37
2.1. Access
A tool was created to provide a response to needs for information access, exchange, and
interoperability across an unlimited number of institutions, communication platforms, and
political borders. This solution was integrated into a web-‐based system using cloud
computing, which refers to subscription-‐based, fee-‐for-‐service utilization of computer
hardware and software over the internet [20]. The model is gaining acceptance for
business IT applications because it allows capacity and functionality to increase on the fly
without major investment in infrastructure, personnel, or licensing fees [5,9]. Large IT
investments can be converted to a series of smaller operating expenses. Cloud
architectures could potentially be superior to traditional EMR designs in terms of economy,
efficiency and utility. Also, the cloud computing model can achieve acceptable privacy and
security through cloud providers that specify compliance requirements, performance
metrics, and liability sharing [9,20]. In the CUSS setting, providers were able to access EMR
WB through any platform (mostly desktop PCs), anywhere, anytime through internet
access provided by the University of Guanajuato and local cell phone network using cloud
computing.
2.1.1. Strategy
The ability to approach, enter, exit, communicate with, and make use of EMRs across
diverse computer and smart phone platforms was created by contracting with commercial
services to securely host EMRs “in the cloud,” an efficient web-‐based system. Providers
were thus able to access EMRs through any platform anywhere, anytime.
38
2.1.2. Server hosting
The hosting company maintaining the web server software and computers was selected
based on the provision of reliable and low-‐cost web services 24 hours a day, seven days a
week. Arvixe Web Hosting (http://www.arvixe.com) also provides data backup and
technical servicing of the software and hardware in a large facility with state-‐of-‐the-‐art
equipment and staffing. Arvixe.com is currently considered one of the best hosting services
available worldwide and has received numerous web hosting awards
(https://www.arvixe.com/awards.php).
2.1.3. Cloud computing
The concept of the “cloud” is an extension of existing computing practices, but places more
content and program code in the hands of remote servers rather than the user’s local
computer [20]. In the case of the EMR WB system, the web server is providing the storage
for medical/clinical records and the principal software application (the web application
software on the site’s server) that manages these records for users.
2.1.4. Administrative management
Access to the application is via the World Wide Web. This makes the application open to
patients, providers, and administrators through simple use of web browsers. The user roles
are defined as follows: 1) Patient: view clinical records and notes, but may only change
certain personal information such as phone numbers and addresses; 2) Administrator
(Admin): create and edit patient records, but cannot access deeper levels of system control.
Admin cannot edit or create other user roles, for example, nor can they change system
39
configuration settings or delete patient records; 3) SuperAdmin: edit, view, delete patient
records, and have similar control over other Patient, Admin, and SuperAdmin user
accounts; 4) SuperDooperAdmin: manage problems that may appear in the system to
modify, update, and develop the software and/or help other roles recover lost accounts or
other information.
The EMR WB site automatically sends different page data according to specific user roles,
access devices, and browsers. Testing indicates that most devices such as iPhones and
iPads are able to scale the page or arrange it in the display appropriately enough to use.
The emulators20 are more valuable at this point for developing applications that are
targeted to run on those particular devices. So far, no device-‐specific codes have been
necessary. At this stage of version 1.5, monitoring indicates that EMR WB is working
appropriately. Users have access to the electronic mail addresses of technical support as
well as the project supervisor at the EMR WB front page (by clicking “About Us”).
2.1.5. Privacy
Security and privacy of the role-‐based account system relies on the Microsoft (Active
Server Pages) ASP.NET membership classes in the Common Language Runtime (CLR)21 of
the .NET system. It uses encrypted cookies and other methods that ensure a malicious user
cannot forge or decode information to access someone else’s account. Using secure socket
layers (SSL) protects user privacy on the internet by encoding the information sent back
20 In computing, an emulator is hardware or software or both that duplicates (or emulates) the functions of a first computer system (the guest) in a different second computer system (the host) so that the emulated behavior closely resembles the behavior of the real system. This focus on exact reproduction of behavior is in contrast to some other forms of computer simulation in which an abstract model of a system is being simulated (http://en.wikipedia.org/wiki/Emulators). 21 The CLR is a special runtime environment that provides the underlying infrastructure for Microsoft's .NET framework. This runtime is where the source code of an application is compiled into an intermediate language called CIL, originally known as MSIL (Microsoft Intermediate Language). When the program is run, the CIL code is translated into the native code of the operating system using a just-‐in-‐time (JIT) compiler (http://en.wikipedia.org/wiki/Common_Language_Runtime).
40
and forth from a user’s browser [37]. Consequently, EMR WB is fully compliant with the
Health Insurance Portability and Accountability Act (HIPAA) and privacy standards
recommended by the United States and Mexico [60].
2.1.6. Concurrency
Achieving concurrency of information requires managing information in the data set that
may undergo modification from multiple users in different locations. In the case of EMR
WB, usage is typically limited to a single health practitioner modifying a record, such as
during a consultation or shortly afterward. In the rare instance another user is modifying
the same record, the last record entered will overwrite the other user’s input. A more
complicated scenario is if user 1 uploads a document that was modified by user 2 while
user 1 was disconnected from the internet, then user 1’s upload could be labeled as a
conflicted copy. Further research into concurrency will be needed to address this problem.
Concurrency is a difficult issue in all multi-‐user environments. Solutions vary, depending
on the context and requirements. Database tables can be locked, or a given patient record
could be locked for exclusive use by the user who opens the patient record. However, on
the web, it is difficult to assess when the “owner” has closed the need for a reservation on
the patient record. An open or locked record might exist if a user walks away from a
terminal for some reason and forgets to close the patient account. This would prevent
another legitimate user from modifying that record.
41
An “atomic”22 update for any given page is currently being used to achieve concurrency for
EMR WB use. This includes a tracking method that identifies who modified a record most
recently, which allows secondary users to communicate about patient records directly with
the user who was responsible for recording the information in question. Ways to improve
this method are being considered, such as notifying users if a record was updated after they
opened it.
2.1.7. Scalability
The EMR pilot project runs on a single server machine. In a production environment with
many thousands or millions of records, the system would need to be expanded using server
farms. Web requests would be routed to machines with the capacity to accept additional
load. The appearance of the system for the public would be that of a single website entity,
and the access speed would remain unchanged.
2.1.8. Expandability
In addition to the adaptations mentioned above, other features could be added to expand
the practical application of the EMR WB system. For example, the web has the disadvantage
that some people may not have consistent access to it. The system may also be subject to
failures of various kinds, such as temporary outages due to hardware or software failures,
electric storms, etc. One proposed solution is to provide multiple methods to access the
database. A stand-‐alone software application running on a user’s local machine could serve
as a proxy for the web application, allowing temporary local storage of patient data. When
22 Atomicity (database systems): It is an operation during which a processor can simultaneously read a location and write it in the same bus operation. This prevents any other processor or I/O device from writing or reading memory until the operation is complete. Atomic implies indivisibility and irreducibility, so an atomic operation must be performed entirely or not performed at all. Source, Webopedia: http://www.webopedia.com/TERM/A/atomic_operation.html
42
access to the central database becomes available, the local application could synchronize
the records, for example. This application would allow the provider to work offline when
internet access is interrupted or otherwise not accessible temporarily, to continue their
work uninterrupted when offline, and reconnect automatically when internet access is
reestablished. Applications such as these will also be designed for tablet computing
platforms or smart phones to allow for data entry in remote or undeveloped locations.
The EMR WB design supports the development of ideas about how practical application of
the software can be expanded to be more comprehensive, including meaningful use of
certified EMR technology as defined by the US Department of Health and Human Services in
the Health Information Technology for Economic and Clinical Health (HITECH) Act.
“Meaningful use” of HIT is an umbrella term for rules and regulations that hospitals and
physicians must follow to qualify for federal incentives under the American Recovery and
Reinvestment Act of 2009 (ARRA). ARRA authorizes the Centers for Medicare and Medicaid
Services (CMS) to reimburse eligible professionals and hospitals that meet meaningful use
criteria for certified EMR technology. This includes using an EMR for functions that both
improve and demonstrate the quality of health care, such as e-‐prescribing, electronic
exchange of health information, and submission of quality measures to CMS.23 Also,
international health care informatics interoperability standards developed by Health Level
Seven International (HL7) that are recommended by the America Academy of Family
Physicians (AAFP)24 will be incorporated. Based on these, for example, EMR WB could
interface to or incorporate billing and accounting record-‐keeping systems, appointment
23 http://www.gpo.gov/fdsys/pkg/FR-‐2010-‐01-‐13/html/E9-‐31217.htm 24 AAFP 2011 HL7 list of EHR functions: http://www.centerforhit.org/online/chit/home/cme-‐learn/tutorials/ehrcourses/ehr120/basicfunctions/hl7.html
43
management, scheduling, and other record management as part of the mainstream
administrative capabilities of most EMRs in use today.
2.2. Programming Language
The application for EMR WB was written using Microsoft ASP.NET C# programming
language for server software. It runs under the Microsoft IIS 7 web server and relies on the
.NET platform, version 4.0, at this time. The easiest way to understand what the .NET
platform does is to think of it as a standard set of functions that are available for various
programming languages to call up when needed. It was intended to be multi-‐platform, like
the Java Virtual Machine, and can therefore allow the same programs to run without
change on various operating systems and devices. The .NET now runs on Microsoft
Windows and MONO operating environments. The EMR WB application also relies on
client-‐side software, which runs directly in a user's browser software on their local
computer. For this, the system uses jQuery javascript and Ajax. The specific programming
model used is the Microsoft Web Application model. In this scenario, the executable code
resides in a central dynamically-‐linked library (DLL). The code supports the functionality in
the various pages, and is executed on demand when pages are requested from a web
server. The programming code is compiled during the software development stage and
then stored on the server in the DLL file storage area. In this model, known as a “code-‐
behind” model, the C# programming logic is separated from the page scripts, which are
principally responsible for visual presentation of content to the user.
44
The programming for EMR WB was based on the following process:
(1) Initial Planning. Define the problem and general goals of the project.
(a) Consider the advantages and disadvantages of possible approaches to arrive at a
solution. It was at this stage where a web application was chosen over third-‐party
cloud-‐based and client software application solutions, for example.
(b) Consider the goals for user interaction. At this stage user interactions were
considered in terms of certain desirable functional goals. Avoiding problems with
existing software due to complex, multi-‐layered user navigation was an important
goal, as was ease of use based on an intuitive interface approach.
(2) Application Design Assessment. What would be the best way to solve the problem?
(a) Consider the platform, programming language, and development tools.
(c) Consider the needs for multi-‐language functionality.
(d) Consider the data storage requirements and database platform. MySql was
chosen due to performance, ease of use, low cost, and availability.
(e) Consider the development time-‐line. How long would the project take? Set dates
for phased testing, deployment, and assessment.
(3) Development. For this a mix of top-‐down and bottom-‐up programming strategies were
used. The top-‐down approach involved sketching out a rough framework, including setting
up web pages for editing patient records, administrative management, and user account
access. This framework was then gradually fleshed out with user interface details and code-‐
45
behind logic to support functional behaviors. The bottom-‐up approach involved doing
some detailed functionality at a low level, including developing software object classes for
abstraction of database access functions.25
The following activities formed the development process, mostly in a cyclical manner (i.e.,
the debugging process returned me to the programming stage recurrently).
(a) Programming.
(b) Debugging and testing.
(c) Deployment and user testing. This was the beta testing phase in which potential
clients used the software, and problems were discovered and fixed.
2.3. Standardization
Standardization is the process of establishing a technical specification that is understood
globally.26 Standardization is a very relevant concern for EMRs because there has been no
synchronization of terms, relationships, meaning, construction of sentences, and linguistics,
which can compromise the quality of health care [9,50].
25 Class (computer programming): In object-‐oriented programming, a class is a construct that is used to create instances of itself – referred to as class instances, class objects, instance objects or simply objects. A class defines constituent members which enable its instances to have state and behavior. Data field members (member variables or instance variables) enable a class instance to maintain state. Other kinds of members, especially methods, enable the behavior of class instances. Classes define the type of their instances. Source: http://en.wikipedia.org/wiki/Class_(computer_programming)#CITEREFGamma.2C_Helm.2C_Johnson.2C_Vlissides1995 26 http://en.wikipedia.org/wiki/Standardization
46
2.3.1. Strategy
The strategy for standardization of EMR WB was based on the premise that the best quality
of service is determined by a clear understanding of the patient, an accurate diagnosis, and
treatment planned according to the latest EB medical knowledge, experience, and patient
choice [27,28]. The framework was based on the belief that the milestone of best quality of
service in clinical medicine can be achieved using a standardized clinical history [5,45]. The
use of state-‐of-‐the-‐art technology to computerize the clinical history, integrate an easy-‐to-‐
use interface, and access knowledge facilitated achieving this milestone. As Rector et al.
pointed out back in 1991, “structure is essential if the system is to be active in organizing
information for the clinician as well as to support valid aggregation of data” [45].
The goal of developing a standardized clinical history model was accomplished by creating
an architecture to guide unstructured and structured information pertinent to the process
of clinical care. The clinical core is the guidepost that keeps all the different elements of the
EMR system moving down the same track. The family medicine clinical history consists of a
high-‐yield condensing of elements from other core clinical and surgical medical specialties;
in particular, internal medicine, pediatrics, obstetrics, gynecology, geriatrics, and general
surgery. The clinical core was thus structured to incorporate state-‐of-‐the-‐art terminology
using 1) controlled vocabulary (terms), 2) ontologies (relationships), 3) semantics
(meaning), 4) syntaxes (constructing sentences), and linguistics (technical English and
Spanish); then two additional elements were included: 5) EB literature and 6) peer-‐
reviewed clinical protocols.27 In this context, the master problem list28 is placed at the
27 e.g., National Institutes of Health (NIH) Consensus Development Program. NIH State-‐of-‐the-‐Science Conference: Family History and Improving Health (2009, http://consensus.nih.gov/2009/familyhistorystatement.htm)
47
beginning of the clinical history because it represents patient problems that serve as an
index to their record.
2.3.2. Terminology
Rector et al. [50,51] pointed out that clinical terminologies are at the center of efforts to
develop and integrate EMR systems. In the standardized EMR context, a terminology is a
set of words, expressions, designations, or symbols used in a domain of knowledge that is
defined primarily through natural language [46,47]. A terminology describes the concepts
within a particular subject and is oriented towards coding and annotating data rather than
inference [47]. A terminology generally consists of the technical terms or expressions used
in a specific field. The terminology may be organized according to a hierarchy29 or
categories,30 and terminology developers and curators may use ontology tools and
methods for organization and management (see Table 2 and Appendix E). Although
terminologies such as the National Cancer Institute Thesaurus (NCIt)31 can be machine-‐
readable and used for inference, the ability to represent and infer knowledge is not as
expressive as an ontology. Terminologies are not necessarily intended to be machine-‐
interpretable or executable models for inference. In this context, inference may simply be
about properties in a hierarchy of relationships [46,47,48].
28 The MASTER PROBLEM LIST is an index system of record-‐keeping in which a list of the patient’s problems is made. All of the problems are numbered and the approximate date of onset, date problem recorded, active problems, inactive/resolved problems, and date resolved are documented for each problem placed under the heading. 29 A hierarchy can be defined as an arrangement of items (objects, names, values, categories, etc.) in which the items are represented as being "above," "below," or "at the same level as" one another. Abstractly, a hierarchy is an ordered set or an acyclic graph. A hierarchy can link entities directly, indirectly, vertically, or horizontally (source: Wikipedia). 30 A category is a class or division of people or things regarded as having particular shared characteristics (source: Oxford dictionaries). 31 http://ncit.nci.nih.gov/ncitbrowser/
48
Standard medical terminology
common in primary medical
domains was used as the
vocabulary for accurately
describing the clinical core of
EMR WB in a systematic,
science-‐based manner. This
approach involved reviewing
each word and phrase
(syntaxes) of the clinical
histories and formats described in the literature in a comprehensive and orderly fashion to
come up with state-‐of-‐the-‐art clinical content for EMR WB. Häyrinen et al. systematically
reviewed the research dealing with EMR content in 2008 due to increased concern about a
lack of structure leading to ineffective clinical documentation [5,65]. Hence, one of the most
important features of the EMR WB clinical core standardized structure design is its ability
to consistently facilitate structured clinical documentation to fill this gap and mainstream
limitation.
MeSH32 were consulted as the primary knowledge source for EMR WB, along with Unified
Medical Language System (UMLS);33 Systematized Nomenclature of Medicine—Clinical
Terms (SNOMED CT: English and Spanish versions);34 Generalized Architecture for
32 http://www.ncbi.nlm.nih.gov/mesh 33 http://www.nlm.nih.gov/research/umls/Snomed/snomed_main.html 34 http://www.nlm.nih.gov/research/umls/Snomed/snomed_faq.html#what
Table 2: Philosophical approach to standardization. The standardized approach used a multidisciplinary family medicine clinical history as a backbone, called “the clinical core” of the EMR WB system, which integrated state-‐of-‐the-‐art terminology (controlled vocabulary, ontologies, semantics, and syntaxes) with EB and peer-‐reviewed clinical protocols. Integration of State-‐of-‐the-‐art Terminology
• Controlled vocabulary (terms) • Ontologies (relationships) • Semantics (meaning) • Syntaxes (constructing sentences) • Linguistics (technical English and Spanish)
Clinical History
• EB literature and guidelines • Peer-‐reviewed clinical formats and protocols
49
Languages, Encyclopedias, and Nomenclatures in Medicine (GALEN);35 Logical Observation
Identifiers Names and Codes (LOINC);36 National Cancer Institute Thesaurus Terminology
(NCIt);37 International Classification for Disease (ICD)—9 and 10;38 Diagnostic and
Statistical Manual of Mental Disorders, Fourth Edition (DSM-‐IV);39 Medical Dictionary for
Regulatory Activities (MedDRA);40 and The Clinical Care Classification (CCC) System41 to
check the accuracy of the terminology used to construct the clinical core. In addition,
Medpedia,42 Medicalopedia,43 and the Merck Manual Online Professional Edition,44 medical
textbooks considered the gold standards in the field of clinical medicine by leading medical
schools in the United States and Mexico,45 were consulted to verify the formal clinical
history and physical exam protocols, formats, properties of concepts, and relationships for
consistency and accuracy. Such leading textbooks and specialized encyclopedias provided
the best approach in the context of this EMR design to 1) identify the specific meaning of
terms used based on linguistic semantics to express meaning through natural languages;
and 2) use syntax as a means to apply principles and rules for constructing sentences in
English and Spanish (see Appendix D-‐1,2: Standardization Process Sample). 35 http://www.opengalen.org/ 36 http://loinc.org/ 37 https://wiki.nci.nih.gov/display/VKC/NCI+Thesaurus+Terminology 38 http://apps.who.int/classifications/apps/icd/icd10online/ 39 http://www.psychiatry.org/practice/dsm (the fifth edition will be released on May 2013). 40 MedDRA is a medical terminology used to classify adverse event information associated with the use of biopharmaceuticals and other medical products (e.g., medical devices and vaccines). Coding these data to a standard set of MedDRA terms allows health authorities and the biopharmaceutical industry to more readily exchange and analyze data related to the safe use of medical products. Source: http://www.meddramsso.com/ 41 The Clinical Care Classification (CCC) System is a standardized, coded nursing terminology that identifies the discrete elements of nursing practice. CCC provides a unique framework and coding structure for capturing the essence of patient care in all health care settings. Source: http://www.sabacare.com 42 The Medpedia Project is a long-‐term, worldwide project to evolve a new model for sharing and advancing knowledge about health, medicine and the body among medical professionals and the general public. This model is founded on providing a free online technology platform that is collaborative, interdisciplinary and transparent (http://www.medpedia.com/). 43 Medicalopedia is a medical wiki written collaboratively by healthcare professionals all around the world. It was founded in December 2010 by Burhan Ahmed with the aim of collecting all medical information and making it available at a single portal (http://en.medicalopedia.org). 44 http://www.merckmanuals.com/professional/index.html 45 1) Bates' Guide to Physical Examination and History Taking / Edition 10, December 2008; 2) The Clinical Encounter: A Guide to the Medical Interview and Case Presentation / Edition 2 August 1998, by J. Andrew Billings, John D. Stoeckle; 3) DeGowin's Diagnostic Examination, Ninth Edition / Edition 9, October 2008 by Richard F. LeBlond, Donald D. Brown, Richard L. DeGowin; 4) Physical Examination and Health Assessment / Edition 6 by Carolyn Jarvis; and 5) Introduction to clinical medicine. Janice L. Willms, and Judy Lewis. National Medical Series. January 1991. Lippincott Williams & Wilkins. Paperback , 260pp.
50
Open-‐source ontologies, mostly BioPortal of the National Center for Biomedical Ontologies
supported by the National Institutes of Health (NIH)46 and Foundational Model of Anatomy
(FMA),47 were also consulted because they provide a shared understanding of a domain, in
this case the clinical history and anatomy, by defining relationships among clinical terms.
The terms denote important concepts (classes of objects) in the domain (e.g., History of
Present Illness—HPI; see Appendix E: Example of Standardized HPI Content within the
Clinical Core) and the relationships that typically include hierarchies of classes (see
Appendix C: The National Center for Biomedical Ontology BioPortal).
Controlled vocabularies were used to build the terminology for EMR WB. First, a draft of
the clinical history template was compiled from a variety of protocols in leading US (e.g.,
UW School of Medicine, and Harvard Medical School), and Mexican (Autonomous
University of Nuevo Leon School of Medicine, and the University of Guanajuato School of
Medicine) medical schools, and the gold standard textbooks and peer-‐reviewed literature.21
After that, the terminology that represents the structure of the clinical history was
systematically compared against the controlled vocabulary tools outlined above, creating
the state-‐of-‐the-‐art clinical content of the medical record and the other key primary care
domains (psychology, nursing, nutrition, and social work) considered in EMR WB.
46 BioPortal is an open repository of biomedical ontologies that provides access via Web browsers and Web services to ontologies. BioPortal supports ontologies in OBO format, OWL, RDF, Rich Release Format (RRF), Protégé frames, and LexGrid XML. Functionality includes the ability to browse, search and visualize ontologies as well as to comment on, and create mappings for ontologies. http://bioportal.bioontology.org/ 47 The Foundational Model of Anatomy Ontology (FMA) is a reference ontology for the domain of anatomy. http://sig.biostr.washington.edu/projects/fm/AboutFM.html
51
2.4. Interface
With the recent explosive growth of electronic medical information, the user interface
design of EMRs has become a crucial issue. Many interfaces in current EMRs are barriers
between users and tasks [4,10]. By re-‐implementing these interfaces in functionally
equivalent but representationally different interfaces, the barriers can be removed or
minimized via direct interaction.
For EMR WB, the interface was constructed so that users could easily and accurately
retrieve, seek, gather, encode, transform, organize, and manipulate pertinent information
to accomplish desired tasks. Two top menu bars are always visible across the screen
regardless of where the user is working. The upper bar shows the different domains as
buttons (medicine, psychology, nursing, nutrition, and social work).
The user can click once at the desired domain (e.g., medicine), and then the second bar
immediately below will open to display a series of buttons that represent the subfields of
this specific domain (e.g., Identifying data—ID, Chief Complaint—CC, History of Present
Illness—HPI). In addition, the content of the specific chosen subfield is displayed as
subheadings across the screen below the bars. Each subheading is executable with one
click for input or data gathering. User activities can direct interaction with the task
domains rather than generating secondary tasks that demand extra cognitive resources
[13]. All the fields are readily accessible without having to move up or down a hierarchy of
navigation menu items (see Figure 2).
52
2.5. Decision Support
Including this function in EMR WB was based on studies that show providers have a high
need for information to answer the multitude of questions that arise during clinical care
[2,21-‐27]. These questions often go unanswered due to lack of time or readily available
resources. Estimates of information needs are as high as four questions per patient
encounter [26]. Failure to answer these questions may result in patient care error and
Figure 2: The EMR WB interface. A screenshot of EMR WB showing two top menu bars that are always visible. The upper bar shows the different domains as buttons (medicine, psychology, nursing, nutrition, and social work). The second bar immediately below opens to display a series of buttons that represent the subfields of each specific domain (medicine e.g., Identifying data—ID, Chief Complaint—CC, History of Present Illness—HPI, etc.). (Note: this is an anonymous sample for demonstration purposes only).
53
adverse outcomes. Linking to knowledge-‐based resources at the point of care could be an
effective means for addressing these needs and supporting evidence and guideline-‐based
practice.
Infobuttons were considered as a potential solution because they have been used to
provide context-‐sensitive links between clinical information systems (e.g., EMRs) and
online knowledge resources [27,49]. "Context" in this sense is what the user is doing at
some point in time in the EMR during a clinical encounter. Infobuttons make it easier for
providers to access the information they need at the point of care [22].
One study at Partners HealthCare system showed the KnowledgLink medication infobutton
to answer 84% of clinicians’ questions, thereby altering 15% of patient care decisions [22].
Another study showed that 74% of infobutton users responding to a survey felt the system
had a positive impact on their patient care decisions [23]. A recent review of several
methods for providing decision support to nurses at the point of care emphasized the need
to integrate such contextual information links into clinical workflow in order to effectively
support EB nursing practice [24].
Another recent study from Intermountain Healthcare [25] showed a use of infobuttons in
the hospital setting steadily increasing over a four-‐year period. Intermountain Healthcare
(“Intermountain”) is a not-‐for-‐profit integrated delivery system of 21 hospitals; over 70
outpatient clinics, more than 500 physicians, and an insurance plan located in Utah and
southeastern Idaho have access to a web-‐based EMR called HELP2. HELP2 offers access to
a wide variety of data and functions, including laboratory results, clinical notes, problem
lists, and medication order entry. Infobuttons were first released in HELP2 in September of
54
2001 in the medication ordering (outpatient), problem list, and laboratory results modules
[25].
The EMR WB system uses an infobutton interface for direct access to knowledge-‐based
tools at the point of care in real time. The infobuttons feature three web links that
providers need to manually open to input and tie clinical information to health resources
such as knowledge-‐based clinical decision support tools. In contrast, automated systems
such as infobutton managers are capable of matching what the user is doing (i.e., "context")
to appropriate information resources [21-‐23]. During the pilot project, EMR WB did not use
information managers because a manual infobutton system provided practical, accessible,
and useful decision support.
The three tools added to EMR WB for decision support are:
1. DXplain: This is a diagnostic decision support system tool that analyzes a patient's
historical data, physical findings, laboratory tests, and x-‐rays, and then generates a
ranked list of diseases that can explain these findings. DXplain is a web-‐based
system accessed in real time through the Laboratory of Computer Science at
Massachusetts General Hospital, Harvard Medical School48
(http://dxplain.org/dxp/dxp.pl). DXplain requires that the provider open the site
and enter their password in order to access and use the site.
48 1) Barnett GO, Famiglietti KT, Kim RJ, Hoffer EP, Feldman MJ. DXplain on the Internet. Proc AMIA Symp. 1998:607-‐11; 2) Bond WF, Schwartz LM, Weaver KR, Levick D, Giuliano M, Graber ML. Differential Diagnosis Generators: an Evaluation of Currently Available Computer Programs. J Gen Intern Med. 2011 Jul 26; and Katie Hafner. For Second Opinion, Consult a Computer? Software Programs Help Doctors Diagnose, but Can’t Replace Them. The New York Times. December 3, 2012. http://www.nytimes.com/2012/12/04/health/quest-‐to-‐eliminate-‐diagnostic-‐lapses.html?src=me&ref=general&_r=0
55
2. EB Medicine Calculator: This device helps providers make proper diagnoses,
devise the best testing plan, choose the best treatment and methods of disease
prevention, and develop guidelines for patients
(http://ktclearinghouse.ca/cebm/practise/ca/calculators/statscalc).
3. Knowledge Translation Tools: These are EB, dynamic, and interactive literature
access tools by which relevant information is made available to support health care
decision-‐making processes. They provide two different sets of tools for querying
primary studies that represent first-‐generation and second-‐generation knowledge
(http://ktclearinghouse.ca/tools/practicing).
Version 1.5 of the EMR WB system doesn’t include or interface with medication catalogs
(e.g., Epocrates), but instead provides free text to input treatments because free text is a
familiar and easy-‐to-‐use environment in a global context and the systems currently
available are costly, time-‐consuming, unreliable overall, and not offered in both English and
Spanish [36]. In addition, because the point-‐of-‐care system concept is commonly defined in
the literature as the use of diagnostic and laboratory testing provided to patients at their
bedside (MeSH),49 it is important to clarify that the knowledge-‐based tools used by EMR
WB to support the decision-‐making process are aimed at the point of care as distinct from a
patient’s bedside.
49 MeSH. Point-‐of-‐Care Systems. Laboratory and other services provided to patients at the bedside. These include diagnostic and laboratory testing using automated information entry. Year introduced: 1996
56
2.6. EMR WB Definition
One of the most relevant barriers that impede the development of EMRs is the lack of a
clear definition [5,9,60]. This is why design of EMR WB was delayed until a thorough
definition could be constructed. Although work on a common definition of a computerized
medical records system [3], also referred to in the literature as EMRs, CPRs (computerized
patient records), or EHRs [10,60], is under way by various groups, a universal
understanding of the concepts embodied in an EMR does not exist. Without a clear
understanding, users have a difficult time selecting systems that will meet their needs and
vendors have difficulty supplying such systems.
EMRs have been defined as "electronically stored information about an individual’s lifetime
health status and health care;" a “software that allows you to create, store, edit, and
retrieve patient charts on a computer;” and “any information related to the past, present, or
future physical/mental health, or condition of an individual which resides in electronic
system(s) used to capture, transmit, receive, store, retrieve, link, and manipulate
multimedia data for the primary purpose of providing health care and health-‐related
services”[60].
Beyond the definitions mentioned above, however, few details have been worked out and
agreed upon. For example, there is no common data model for an EMR, no common set of
data elements, no common vocabulary, and no common set of scenarios that are supported.
These requirements are fundamental if researchers and developers are to create a patient-‐
centered EMR that links care across different sites, specialties, and circumstances [5,10].
Even the terms “meaningful use” and “certified” in the HITECH Act are not yet clearly
57
defined, and they are neither user-‐friendly nor designed to meet the ambitious goal of
improving quality and efficiency in the health care system as outlined by David Blumenthal,
the former national coordinator of HITECH [17]. For the purpose of this project, the EMR
WB has been defined as the standardized, multidisciplinary, family medicine clinical
history as the EMR clinical core that provides the frame to input structured and
unstructured documentation.
Based on the need to have a definition upon which to base the EMR WB design, one of the
strategies was to focus on its clinical core. To do this, NLM MeSH was used because it
provides a unique and comprehensive systematic historic description of the concept, from
its paper-‐based roots to the latest concept that involves the integration of state-‐of-‐the-‐art
technology. According to MeSH, medical records consist of pertinent information
concerning a patient's illness or illnesses.50 Computerized medical record systems are
defined as a computer-‐based arrangement for input, storage, display, retrieval, and printing
of information contained in a patient's medical history (year introduced: 1991).51 EMR is
defined as the media that facilitates transportability of pertinent information concerning a
patient's illness across varied providers and geographic locations (year introduced:
2011).52 These definitions describe the natural evolution of the concept from a paper-‐based
version to a computerized format, leading to sharing across platforms and distance. EMR
WB is based on the latest definition, and centered on the clinical history that is oriented
towards primary health care (see Figure 3).
50 http://www.ncbi.nlm.nih.gov/mesh/68008499 51 http://www.ncbi.nlm.nih.gov/mesh/68016347 52 http://www.ncbi.nlm.nih.gov/mesh/68057286
58
EMR WB was designed with health information exchange (HIE) in mind in two steps: as a
pilot project and as a fully functional EMR. During the pilot project, the HIE had a similar
connotation as the meaningful use explained in Section 2.1. At the moment, EMR WB does
not rely on Health Level Seven (HL7) standards and hence Clinical Document Architecture
(CDA) and Continuity of Care Document (CCD) specifications. For the purposes of this
project, the objective of EMR WB was not to be interoperable with other systems, but
independently functional. In the future, however, EMR WB will adopt interoperability
standards aimed at fulfilling the meaningful use outlined by the US federal government
HITECH Act [17].
Figure 3: Clinical core-‐centered EMR definition. A definition of EMR WB was created to reflect its structure and standardization. A series of steps was followed that integrated the three EMR definitions available in the NLM medical subject headings (MeSH), the original description of a medical record (1966), the first EMR definition (1991), and the latest EMR definition (2011). This information was then combined with a description of the clinical core used as the design foundation for EMR WB.
59
2.7. Summary
EMR WB was implemented at CUSS, a public, multidisciplinary family medicine clinic at the
University of Guanajuato in an underserved, urban community in the city of Leon, Mexico.
The central idea was to address the multifaceted problem of the lack of a modern,
standardized EMR system with global access, comprehensive interface, and knowledge-‐
based point-‐of-‐care tools to efficiently collect, store, share, manage, and access patient files.
This system enabled providers from multiple disciplines (medicine, nursing, nutrition,
psychology, and social work) at CUSS to create, share, and maintain records of patient care
that were standardized (using controlled vocabulary—terms, semantics—meaning,
ontologies—relations, syntaxes—construction of sentences, with EB, peer-‐reviewed
clinical protocols), linked to diagnostic decision support (DXplain), and knowledge-‐based
(EB Medicine Calculator and Knowledge Translation Tools) point-‐of-‐care systems
functionally connected with an all-‐inclusive, intuitive interface, and accessible anywhere
from any platform (PCs, tablets, and mobile phones) using cloud computing. This EMR WB
system aimed to better address local, clinical, and psychosocial health challenges, as well as
transnational migration health issues based on systematic documentation, interoperability,
and transfer of clinical information among primary care providers in targeted, underserved
communities.
60
Chapter III: Evaluation
There is a large gap between the postulated and empirically demonstrated benefits of
EMRs [33]. In light of the scarce evidence for improvements in patient health outcomes,
provider satisfaction, and cost effectiveness, it is vital that future EMRs are evaluated with
both quantitative and qualitative tools [6,32].
EMR WB evaluation aimed to determine the level of satisfaction with use of the system at
the CUSS. First, to understand the potential end users of the proposed EMR system, a focus
group was held on August 5, 2010, to gather perceptions, interpretations, beliefs, and
attitudes towards EMR use. The focus group served as an exploratory tool for creating a
schema of coding data that provided the themes used to develop the structure of the
questionnaire for user satisfaction, the EMR WB QUS (see Appendix B). Since one of the
major aims of this research project is to evaluate provider satisfaction with the
implemented EMR system, an evaluation tool that examines end user perceptions was
necessary. A survey was formulated into a series of 15 questions aimed at obtaining
specific answers related to the perceived satisfaction of the providers with EMR WB
compared to the CUSS paper-‐based medical record system they previously used. Two other
sections were added to collect demographic data, amount of use of the system, and
suggestions for improvements (a total of five questions). These questions reflected on the
definition of “user information satisfaction” outlined by Ives et al. in1983 [12] as “the
extent to which users believe the information system available to them meets their
information requirements.” A 7-‐point Likert scale (or summated scale) was used to
quantify user satisfaction with features such as access, interface, standardization,
61
knowledge-‐based tools, and overall design. The scale was from “Much worse” to “Much
better,” with -‐3 as the lowest rating and +3 as the highest rating. The total score (i.e.,
evaluation) was reported as a sum of the responses.
The timeline criteria for administering the survey was at least three months after the beta
testing took place. EMR WB was initially implemented via beta testing in May 2011 at the
CUSS, and version 1.5 was implemented at the same facility in January 2012 (the version
used to evaluate EMR WB). Providers had the flexibility of answering the online survey
over a three-‐day period. The web-‐based, one-‐time, post-‐implementation evaluation survey
took between 20 and 30 minutes to complete.
3.1. Testing EMR WB Versions
Beta testing of the first EMR WB version began at the CUSS on May 25, 2011, and was
completed in December 23, 2011. Version 1.5 was implemented in January 06 2012 and
evaluated in July 13-‐16, 2012.
CUSS providers were informed about the introduction and capabilities of the EMR WB
system via staff meetings, e-‐mail, and video conferencing (using Adobe Connect) that
included step-‐by-‐step scenarios demonstrating how to use the new system. In addition, a
web casting video was created for clinicians to observe how EMR WB might work in their
own disciplines; this included a demo mode with fictional patients so providers could
familiarize themselves with the system to see if it fit their needs. The demo application is
currently available on the EMR WB home page at http://www.emrwithoutborders.com.
62
The vast majority of clinicians at the CUSS used the EMR WB system to support their daily
patient care practices for seven months. In response to demand for increased functionality,
EMR WB was upgraded to its current version 1.5. The redesign allowed providers to use a
system without identifiable “bugs,” better response time, more reliable and seamless
functionality, and faster and full access to all available medical literature information
(infobutton: Knowledge Translation Tools) while they were in clinical encounters with
patients by using the EB tools (infobutton: EB Medicine Calculator) and fully incorporated
diagnostic decision support system DXplain.
EMR WB is still in operation at the CUSS, where it has proven to be stable and reliable. EMR
WB has achieved the goals for Aim 1 as initially outlined:
(1) It is globally accessible on a web-‐based system from any platform (PCs, Macs, tablets,
and mobile phones).
(2) It is standardized across primary care multidisciplinary core domains (medicine,
nursing, psychology, nutrition, and social work) and uses shared terminologies, data
models, controlled vocabulary, ontologies, semantics, syntaxes, pragmatics, and EB peer-‐
reviewed protocols.
(3) It uses an all-‐inclusive, intuitive EMR interface into one navigation system.
(4) It uses knowledge-‐based, point-‐of-‐care systems to support health care decisions.
(5) It is completely bilingual in English and Spanish.
(6) It is network-‐based (web/cloud).
63
(7) It is intuitive, easy to use and learn, and does not require formal training.
(8) It supports underserved populations in the urban setting of the city of León, Mexico.
3.2. Focus Group and Survey
Evaluations were conducted via both qualitative (pre-‐implementation exploratory focus
group) and quantitative (post-‐implementation survey) analyses focused on provider
perceptions of EMRs in general and EMR WB in particular, respectively.
3.2.1. Focus group
To better understand the potential end users of the proposed EMR system, a focus group
was conducted. This qualitative research tool facilitated the collection of perceptions,
interpretations, beliefs, and attitudes towards EMR use [34]. As outlined by the literature,
an advantage of the focus group methodology is that it reveals the evolution of perceptions
in a social context, which tends to be more objective than the isolation of a typical survey or
individual interview, hence the results are richer, more complete, and more revealing [59].
However, there are risks involved such as “group think” and consensus-‐based reasoning.
On August 5, 2010, a group of health care providers was brought together from most of the
primary care domains who would be end users of the proposed EMR system. They were
asked to provide their opinions of EMRs before EMR WB implementation and beta testing.
This same group would also participate in the post-‐evaluation survey. The focus group was
held for one and one-‐half hours with six providers, approximately 12% (6 out of total of
50) of the existing CUSS providers. The participants were recruited by the CUSS director
64
and myself. The topic was introduced by providing a brief description of the EMR WB
project. A discussion guide was used to encourage an open yet focused dialogue. A focus-‐
questioning route (see Appendix A) was used based on the research aims that included
opening comments about the topic of EMRs, introductory open-‐ended questions to engage
the participants in the topic, transition questions related to use of EMR WB in their
respective clinical settings, key questions regarding the perceived benefits and limitations
of its use, and ending questions to summarize the discussions and confirm the main points.
At each stage of questioning, sufficient time for all the participants to share their views was
allowed. The focus group served as an exploratory tool more than a data-‐gathering tool,
although the results were used to create a schema of coding data for modifications to EMR
WB.
All six participant providers were interested in using the emerging EMR technology in their
own practices. The overall perception was that EMRs are the computerization of paper
records. They perceived this kind of technology to be a useful tool for both them and their
patients. They expressed wanting an EMR that is easy to use and read, accessible from
different places, supports accuracy of free text and structured documentation, and permits
sharing among members of the primary care team. They perceived the input of free text as
quite useful based on their paper-‐based experience with structured clinical forms and
guidelines. They commented that a mixture of free text with structured documentation
would bring about the needed freedom to create accurate and individualized clinical notes.
They also wanted a system that provides continuity of care with systematic follow-‐ups. In
addition, since EMR WB would serve in a teaching environment, they wanted it to facilitate
access to supervised work.
65
There were some overall concerns about privacy and confidentiality, such as how to
restrict unauthorized access to information, including the limiting of access to other
providers when pertinent (e.g., mental health-‐sensitive issues, without compromising the
safety of others). EMR implementation and use was seen as a challenge, too, because of its
demand for new forms of thinking, ethics, and work. A concern of possible intrusion of the
computer in the patient-‐provider relationship during the clinical encounter was also
expressed.
Among the perceived benefits, a big savings of materials (paper) and time were
emphasized. Although the web-‐based technology was welcomed, previous negative
experience with a local system was used as a frame of reference. Apparently they had been
unable to access information from other settings because of dependence on fixed desktops.
Limited time spent onsite entering patient data in the clinical encounter was perceived as a
helpful feature. Cross sharing of information among other primary care domains (e.g.,
social work with psychology) was a feature that they wanted to see for comprehensive
management of health care information. When asked, “What do you think would be the
most important uses of the information?” they expressed interest in clinical decision-‐
making as a primary use, and supporting interventions in the community and research
purposes as secondary uses.
The data obtained from the focus group was assessed using content analysis based on
coding, categorizing, and counting the types of responses. The lists of codes with some
representative themes quoted from participants, recommendations for the design of the
66
system, and suggested themes for the post-‐implementation survey are condensed in Table
3.
Limitations
The sample size was small, but it was representative of the total end user population
diversity. The focus group met just once. The other limitation was inherent to the technique
itself, namely that providers might not say what they actually do or think because of “group
think” (e.g., people expressing an opinion which is in agreement with the rest of the group
even if that opinion is at odds with their own personal perceptions, interpretations, or
beliefs). This factor was not perceived by the moderator to be present. An additional
possible limitation is that respondents were inclined to give the answers they thought their
recruiter wanted to hear, which would be consistent with her bias in favor of EMR WB.
3.2.2. Survey
Survey development
Through a series of four well-‐defined stages, an instrument was developed to evaluate end
user satisfaction of EMR WB compared with a paper-‐based record system, as tested by the
health care providers at the CUSS. The central idea behind developing this tool was to
create a tailored questionnaire incorporating and optimizing both the accepted literature
definition of end user satisfaction as well as the philosophical approach of successfully
tested measurement instruments [6]. The instrument was used to determine 1) the onsite
professional experiences and approaches associated with using a paper-‐based system; and
2) the perceptions, interpretations, and beliefs regarding onsite use of the EMR WB system.
67
Stage I. The Focus Group
As mentioned in Section 3.2.1, the focus group was used as an exploratory tool for
creating a schema of coding data for the themes that structured the EMR WB QUS
(see Table 3 and Appendix B). The paper-‐based system previously used by the CUSS
health care providers served as a frame of reference for comparison with EMR WB.
Stage II. End-‐User Satisfaction Definition
The next step to constructing an instrument for measuring satisfaction with EMR
WB involved examining “satisfaction” based on how the literature defines it [29].19
According to Ives et al. (1983) [12], “User information satisfaction is the extent to
which users believe the information system available to them meets their
information requirements.” Synonymous terms are “system acceptance” (Igersheim,
1976) [20], “perceived usefulness” (Larcker and Lessig, 1980) [31], “management
information system appreciation” (Swanson, 1974) [42], and “positive feelings
about an information system” (Maish, 1979) [43]. Ang and Koh (1997) [44]
described user information satisfaction as “a perceptual or subjective measure of
system success.” This means that the phenomenon will differ in meaning and
significance from person to person. In other words, users who are equally satisfied
with the same system according to one definition or measure may not be equally
satisfied according to another.53 And, according to Brender et al. [29], “User
satisfaction measures the end users’ response to the use of the input and output of
the EMR and its attributes.” The definition provided by Ives et al. [12] outlined
53 http://en.wikipedia.org/wiki/Computer_user_satisfaction
68
above was the primary basis for development of the EMR WB satisfaction
measurement tool.
Stage III. Frames of Reference: the QUIS and IBM Computer Usability
Satisfaction Questionnaires
In addition to the themes and codes obtained from the focus group performed on
August 5, 2010 (see Table 3), the University of Maryland QUIS Version 7.0 and IBM
Computer Usability Satisfaction Questionnaires were used as frames of reference to
design the user satisfaction questionnaire for EMR WB.
In 1988, researchers from the Human Computer Interaction Laboratory at the
University of Maryland developed the QUIS as a standardized general user
evaluation instrument for interactive computer systems. The QUIS relies on
psychological test construction methods to ensure empirical validity and reliability
[39].
The IBM Computer Usability Satisfaction Questionnaires54 address evaluation at
both a global overall system level and a more detailed scenario level. The series of
four questionnaires rely on psychometric characteristics (assessment of
psychological variables—thoughts, feelings, and behaviors—using mathematical
procedures) to measure satisfaction with computer system usability.
54 Lewis, J. R. (1995) IBM Computer Usability Satisfaction Questionnaires: Psychometric Evaluation and Instructions for Use. International Journal of Human-‐Computer Interaction, 7:1, 57-‐78.
69
Table 3: Focus group. The focus group served as an exploratory tool for creating a schema of coding data for EMR WB design recommendations and the themes to develop the structure of the end-‐user satisfaction survey.
Code Summary of Relevant Themes Recommendations for EMR Design
Survey Themes
1 Adoption/ Feasibility
“I am very interested in using the current technology.” “I believe that the EMR will be a useful tool for both providers and patients.” “Implementation will be a challenge at the beginning.”
EMR WB will be implemented as a pilot project Providers will participate
Acceptability, Achievability
2 Documentation “I believe that the EMR will be a very useful technology to improve documentation in the chart.” “I would like to have well defined formats… similar to the ones that I use in my charts (paper based).” “I would like to have the freedom to input both free text and structured information.” “Extensive typing not needed during the clinical encounter.”
Structure Standardization Documentation guidelines Flexible: use free and structured data
Completeness of documentation
3 Ease of use “I would like to have a system that is easy to use.” “I don’t want to add more work.”
Usable No added work Friendly interface
Easy to use
4 Access “I would like to have an EMR that I can access anywhere.” “I welcome the web-‐based technology.”
Open access Web-‐based Accessible
Ability to access from any location
5 Privacy
“I am concerned about overall privacy, confidentiality, and how the system will restrict unauthorized access to information.” “Restrict access to sensitive information among providers without compromising safety. ”
Privacy Confidentiality Safety
Safeguards, privacy and confidentiality
6 Communication “I would like to share information with the primary care team.”
Information exchange / sharing
Ease of communication with other members of the clinical team
7 Quality improvement
“It could improve overall patient care.” “I am concerned about the possible intrusion of the EMR in the patient-‐provider relationship.” “A system that supports follow-‐ups.”
Improve workflow Continuity of care Not intrusive
Work Flow
8 Decision support
“Supports overall care decisions.” “I’d like to have access to literature or some guidelines.”
Decision support tools Support care decisions
9 Research “Use data from EMR as a base for interventions in the community.”
“Research purposes.”
Secondary use of data Facilitate use of data for quality and research purposes
70
Stage IV: The Evaluation Tool—EMR WB QUS
The last set of components used to create the EMR WB QUS (see Appendix B) were
the variables of the study—global access, standardization, interface, knowledge-‐
based access tools, and design—which were cross-‐referenced with the research
questions for consistent correlation with the aims. The CUSS providers were asked
to use a 7-‐point Likert scale to assess agreement with what they thought of EMR WB
as compared to the paper-‐based system on a scale from much worse (-‐3) to much
better (+3) over a series of 15 questions. Dumas (1999) [61] claims, “This is the
most commonly used question format for assessing participants' opinions of
usability and satisfaction.”
In addition to the 15 questions that evaluate user satisfaction with EMR WB (Part 2),
another two series of questions (Parts 1 and 3) explored demographic data, system
use, and recommendations for improvement. Part 1 considered 1) age, 2) gender,
and 3) CUSS professional field (medicine, psychology, nutrition, nursing and social
work); while Part 3 asked users to specify 4) how long they had used the EMR WB
system and 5) what suggestions they had to improve it. In total, the survey thus
consisted of 20 questions (Appendix B). The web-‐based, one-‐time, post-‐
implementation user satisfaction survey took approximately 20 minutes to
complete, but respondents had the flexibility of answering the questions over a
three-‐day period.
71
3.3. Subject Population
The population recruited for the EMR WB QUS was defined as the total adult (aged 18 and
older) female and male health care providers from all primary care domains (medicine,
psychology, nutrition, nursing, and social work) identified from the CUSS that used EMR
WB for at least one month or 20 hours (i.e., inclusion criteria). Forty (out of a total of 50, or
80%) self-‐reported health care providers fulfilled the inclusion criteria, representing the
selected subset of EMR WB users surveyed. A total of 36 subjects started the survey, with a
total of 33 of the 40 surveyed (82.5%) finishing the survey [62]. The three who did not
finish the survey were unable to complete most of the questions, including the
demographic information. They did not make any comments; hence the information
provided had very limited value in the survey and was not considered. The reasons why
they did not finish the survey were not stated.
Inclusion Criteria: CUSS providers (physicians, psychologists, nurses, nutritionists, and
social workers, age 18 and older) who used EMR WB for at least one month or 20 hours.
Exclusion Criteria: CUSS providers (physicians, psychologists, nurses, nutritionists, and
social workers age, 18 and older) who used EMR WB for less than one month or 20 hours.
Data representing all five primary care domains at the CUSS were taken as of equal
importance to match and obtain information from each of the EMR WB corresponding
professional domains. Each domain within the overall CUSS population was not sampled
independently because there was no control over the respondents to make the subgroup
sample sizes proportional to the total population, but responses from all professional
domains were obtained.
72
3.4. Analysis
The specific survey data analysis steps were 1) making sure that the surveys returned were
complete and valid (n = 33); 2) checking for response bias; 3) conducting a descriptive
analysis; 4) condensing items into scales; and 5) checking for reliability of the scales.
Interpretation of data was based on respondent scoring of satisfaction with EMR WB use
on a Likert scale. An overall satisfaction score was obtained from Question 15.
Descriptive statistics were used to quantitatively explain the main features of the results.
Descriptive statistics are more informative and appropriate than testing a hypothesis in
this study, but the results only generalize to other populations similar to the one studied
here [62]. Tables 4 and 5 below show the sample size, demographics, and proportion of
subjects’ satisfaction with EMR WB use compared to the paper-‐based system.
Table 4 summarizes the demographic characteristics of the sample: gender, age, and
profession. Categorization of the 15 survey questions is shown in Table 5. Grouping the raw
data from the study population in a contingency table using ranges of values reduced the
number of categories to yield interpretable results.
In Table 5, the quantities that summarize the distribution of results (evaluation summary
statistics) are expressed as measures of central tendency, representing a "typical value" for
a member of a population. The two measures considered are the mean and median, in
addition to two measures of dispersion of the distribution, standard deviation (SD), and
range. In the statistical approach to the satisfaction questionnaire and analysis, a frequency
distribution was used to arrange all of the variables (i.e., responses) measured in the
sample (n = 33).
73
Table 4: Demographic characteristics of study participants (n=33).
Total Number of Participants Started Survey: 36. Total Number of Participants Finished Survey: 33 (91.7% completion rate)
The rationale behind managing and operating from frequency-‐tabulated data is that it is
much easier to understand than raw data. Having frequency distribution data allowed the
calculation of central tendency via the mean from the survey results, along with measures
of variability or statistical dispersion such as the SD and confidence interval (CI) [63].
The mean satisfaction score for each question was reported as a measure of central
tendency. While the categories are not necessarily equally spaced, the mean nonetheless
conveys information about the positive or negative trend in the responses for each
question. Hence, the middle of the scale was coded as 0 so that the range was from -‐3 to 3.
This made it easier to interpret the results (i.e., by looking at whether numbers were
positive or negative), and provided the needed symmetry.
1 Gender n % Male 10 30% Female 23 70% 2 Age 20 – 30 14 42% 31 – 40 7 21% 41 – 50 5 15% 51 – 60 5 15% 61 + 2 7% 3 Profession Physician 6 18% Psychologist 14 42% Nutritionist 3 9% Nurse 6 18% Social Worker 4 12%
74
75
The survey was sent to the total population of the study that fulfilled the inclusion criteria
(N = 40), followed by three consecutive electronic mail reminders, to procure as many
responders as possible and thus results that would accurately reflect future prospective
end users [63]. Once the data were obtained, missing responses were checked, which
revealed a total of 36 subjects started the survey and a total of 33 finished the survey
(82.5% of the total population); 91.7% of those surveyed were thus included in the study
(n = 33).
For the EMR WB QUS survey, CIs were used to determine the reliability of the observed
sample mean responses to survey questions compared to the expected results from the
population. On the EMR WB QUS survey, Question 15, “Overall Satisfaction,” the mean score
was 2.4 (95% CI=2.0, 2.8) on a 7-‐point Likert scale ranging from -‐3 to +3. A skewed upward
distribution was obtained because the majority of health care providers reported a higher
level of satisfaction with EMR WB use compared with the paper-‐based system. Thus, for the
majority of respondents, the rating for overall satisfaction with EMR WB was higher than
the mean rating.
The information in Table 5 summarizes the total scores given a mean and standard
deviation from 1.8 (0.5) to 2.8 (1.5). On average, for most questions, the ratings were
“Better” to “Much better,” indicating that a large quantity of CUSS health care providers
reported favorable levels of satisfaction with EMR WB use compared with the paper-‐based
system.
A "not applicable" option was not included in the survey questionnaire because all
questions were pertinent to all providers. This conclusion was based on the research
76
conducted previously to prepare the survey, including the pre-‐implementation focus group.
However, not all participants answered each question. The rationale for including
questions such as “Facilitating my use of information for quality control" (Q. 12)55 and
"Helping me obtain information for research" (Q. 13) was based on specific goals related to
the local research-‐oriented setting of the University of Guanajuato where all providers are
required to 1) conduct research and 2) provide the university feedback for quality control.
3.4.1. Confidentiality
The design of the one-‐time, post-‐implementation evaluation survey ensured the anonymity
of the respondents because their identities were not linked to their responses, and the data
was not uniquely identifiable. Also, an internet-‐based SurveyMonkey system was used with
the anonymous setting to administer the survey.
Although some of the CUSS health care providers who participated in the survey met me,
the CUSS director was in charge of organizing the survey participants onsite, including
sending three anonymous email reminders. The online survey was conducted without
knowing the identities of the individuals (see Appendix B). The UW Institutional Review
Board determined that EMR WB research qualified for exemption status under 45 CFR
46.101 (b) (2) from all 45 CFR requirements (exemption #42999). This is because the EMR
WB research posed little to no risk for those human subjects involved, and falls within one
or more of six federally-‐defined categories deemed as “exempt from IRB review.”56
55 The “Q.” means “Question” of the survey used to evaluate end user satisfaction. 56 Office for Human Research Protections (OHRP) -‐ Categories of Research: http://www.hhs.gov/ohrp/policy/expedited98.html
77
3.5. Results
3.5.1. Participants
Part I of the EMR WB QUS is presented in a tabular form as a data set representing the
demographic information (Table 4). The first column represents the particular variables of
the sample, including age, gender, and all the professional fields considered at the CUSS.
Each row corresponds to a given member of the data set in question. It lists values for each
of the variables expressed as datum in quantity of occurrences and its corresponding
percentages.
Participant characteristics are summarized in Table 4. The mean age of the respondents
was 35.5 years (SD 4.5) and ranged from 31 to 40 years old. The distribution of sexes was
10 males (30%) and 23 females (70%). Respondents were frequent users of the previous
paper-‐based system (4–10 hours/week) for at least one year prior to the study. Of the five
different professional fields, psychologists represented the majority with 14 (42%),
followed by 6 (18%) physicians, 6 (18%) nurses, 3 (9%) nutritionists, and 3 (9%) social
workers. On average, participants saw 6.5 patients per week and used EMR WB for at least
20 hours (4–6 hours a week) to assist with clinical encounters with new patients, multiple
visits with the same patients, and follow-‐ups (reported as the most common encounter,
especially among psychologists).
Table 4 indicates high proportions of females, younger subjects, and psychologists. There
are several possible explanations for these findings. According to current trends in
demographics and education in Mexico, women represent more than 50% of those
graduating from college, and are especially prevalent in the field of psychology. In terms of
78
age, Mexico has a predominantly young population that impacts the young age overall
among professions.57 Of the total of 50 health care providers at the CUSS, psychologists
represent 40% (n=20). Thus, it was expected that a high proportion of participants would
be psychologists because they represent the largest proportion of health care providers at
CUSS, following by physicians (n=10), nurses (n=9), nutritionists (n=6), and social workers
(n=5).
Table 6: User satisfaction question groupings.
57 Source: Index Mundi—Mexico Demographics Profile 2012: http://www.indexmundi.com/mexico/demographics_profile.html.
Categorization of 15 EMR WB user satisfaction questions into 8 different clusters. The questions (Q) described below are grouped based on the similarity of their mean satisfaction scores, and were organized in hierarchical order from the highest (1) to the lowest (8).
Hierarchy Questions x̅
1 Q7. Sharing patient information with my fellow clinical team members 2.8
2 Q13. Helping obtain information for research 2.7
3 Q8. Ease of access to files, Q10. Improving the continuity of patient care
2.6
4 Q1. Helping me complete my tasks Q9. Improving workflow Q11. Helping make effective evidence-‐based care decisions
2.5
5 Q5. Effectively review patient problems Q15. Overall satisfaction
2.4
6 Q2. Facilitating my input of information Q6. Safeguarding patient privacy, Q2 Facilitating my input of information Q12. Facilitating use of information for quality control Q14. Appropriateness for use in my care setting
2.3
7 Q4. Ease of use 2.2
8 Q3. Learning how to use 1.8
79
3.5.2. User satisfaction with EMR WB
The survey results described below are based on the percentage of the survey respondents
for each question (Q), along with the average score of the mean and the CI in a hierarchy
from the highest (1) to the lowest (8). (See Table 6.)
(1) In the area of "Sharing patient information with my fellow clinical team members" (Q7),
82% of survey respondents indicated that EMR WB was “Much better” than the paper-‐
based system, which had an average score of 2.8 (95% CI: 2.6–3.0). Open-‐ended comments
from participants on the survey included “EMR WB allows for effective interdisciplinary
work, as each health care professional can access the information when needed;” “Yes,
because it compiles the information from the health care team in different interventions;” and
“I have been practicing professionally in several institutions, and this is the first time that I
have seen the work for the end user performed in an integral manner, across disciplines, and
including the field of social work. I am very pleased to see that the same importance has been
incorporated to include all fields (medicine, psychology, nutrition, nursing, and social work). I
have carried out several consultations with the same patients many times across different
fields in the EMR WB system, and in this same way, other professionals from different fields
have also consulted my patients many times.”
(2) In the area of "Helping obtain information for research" (Q13), 77% of survey
respondents indicated that EMR WB was “Much better” than the paper-‐based system,
which had an average score of 2.7 (95% CI: 2.4–2.9). This rating can be correlated to
comments such as “With this kind of platform, we can have quick access to specific data and
80
information for research” and “Yes, because basically we have the recording, evidence, and
other fundamental data that are necessary to perform research, epidemiology, and
prospective and retrospective studies.” Because the CUSS is a teaching and research facility
in a university setting, these reactions seem especially meaningful.
(3) In the areas of a) "Ease of access to files" (Q8), and b) "Improving the continuity of
patient care" (Q10), 69–76% of survey respondents indicated that EMR WB was “Much
better” than the paper-‐based system, which had an average score of 2.6 (95% CI: 2.3–2.9).
3a) Related comments for "Ease of access to files" (Q8) were “Quite convenient accessing
files from any place and platform;” “As we commented, knowing the access codes and
password;” and “When the system has been designed perfectly.” This is consistent with the
opinion stated by various clinical system designers over the years that "ease of use" is the
single most important determinant of user satisfaction [12,38,58].
3b) Some supporting comments for "Improving the continuity of patient care" (Q10)
included “Only with EMR WB can health care providers document and therefore support the
continuity of integral care;” “Yes, because absolutely no intervention is lost;” and “If we use
the system appropriately, other health care professionals can provide further care.” Since the
CUSS provides continual health care beginning with the initial contact following the patient
through all phases of multidisciplinary care (medicine, psychology, nutrition, nursing, and
social work), the continuity of patient care is one of the elements that providers value the
most, as they also outlined during the focus group.
(4) In the areas of a) "Improving workflow" (Q9), b) “Helping me complete my tasks” (Q1),
and c) “Helping make effective evidence-‐based care decisions” (Q11), 63–67% of survey
81
respondents indicated that EMR WB was “Much better” than the paper-‐based system,
which had an average score of 2.5 (95% CI: 2.2–2.8).
4a) With regard to "Improving workflow" (Q9), some related comments were “Yes, based
on the availability from any space, city, clinic, etc.;” “Yes, as soon as the system becomes
ubiquitous with all health care professionals, better access to the internet, and becomes a
standard of care;” and “Although the first visit could be a time-‐consuming task inputting all
required information, but this also happens with the paper-‐based system, anyway.” Such
feedback represented a positive response to the design of EMR WB as all-‐inclusive and
multidisciplinary across the most important primary care domains that parallel the same
workflow in use at the CUSS.
4b) Some of the favorable comments for “Helping me complete my tasks” (Q1) included
“The EMR WB allows for working easily with the information resources provided in the
system, and you’re aware of the work of the other health care providers;” “It's all there with a
few clicks, available anytime, anywhere;” “It simplifies the visit, documentation and storage;”
and “I was dependent upon the training of how to efficiently manage the EMR.”
4c) Regarding “Helping make effective evidence-‐based care decisions” (Q11), some
supporting comments were “Yes, because we have access to the evidence when we need it”
and “Yes, in particular, when we are dealing with uncertainties, we need to be better
informed.”
(5) In the areas of a) "Overall satisfaction" (Q15), and b) "Effectively review patient
problems" (Q5) 67–68% of survey respondents indicated that EMR WB was “Much better”
than the paper-‐based system, which had an average score of 2.4 (95% CI: 2.0–2.8).
82
5a) With regard to "Overall satisfaction" (Q15), it was associated with the following survey
comments: “After working with EMR WB, I realized that this is an excellent tool in order to
have a better control of the clinical files, working in a multidisciplinary environment in a
timely manner, and facilitating our contact with supervisors;” “EMR WB is an excellent system
to provide the users of the system with complete care. I truly congratulate the creators of this
project;” and “I believe that this is an excellent project. It would contribute to the appropriate
management of health information to establish an appropriate control of health care
interventions. It would establish a guarantee of overall quality.”
5b) Some related comments for "Effectively review patient problems" (Q5) included
“Definitively, only with EMR WB is it possible to work in an integral and multidisciplinary
manner because the information is accessible when it’s needed” and “Yes, because after we get
into the system, we can review all fields.” This reflects the potential benefit that health care
providers could receive from using EMR WB to gather data from patients to review
problems and support diagnoses and treatment plans.
(6) In the areas of a) “Appropriateness for use in my care setting” (Q14), b) “Safeguarding
patient privacy” (Q6), c) “Facilitating my input of information” (Q2), and d) “Facilitating use
of information for quality control” (Q12), 55–67% of survey respondents indicated that
EMR WB was “Much better” than the paper-‐based system, which had an average score of
2.3 (95% CI: 2.0–2.7).
6a) With regard to “Appropriateness for use in my care setting” (Q14), they expressed that
EMR WB would be appropriate for use in the CUSS due to its primary care multidisciplinary
approach that permits sharing among members of the primary care team. Some related
83
comments were “Yes, because it facilitates the accessibility of the needed health
interventions” and “Yes, but we all are dependent upon internet connectivity.”
6b) With regard to “Safeguarding patient privacy” (Q6), some related comments included
“This has to be reinforced, in particular with multidisciplinary health care professionals’
access” and “This is fundamental because everyone in the world can have access to it.”
6c) With regard to “Facilitating my input of information” (Q2), some favorable comments
were “EMR WB offers options to fill out information that facilitates the time and coverage of
the documentation” and “Yes, it facilitates the input of information because you have clear
options at your disposal.” Such responses lend support to success with the design of EMR
WB to better address the need for standardized patient documentation using both
unstructured and structured input.
6d) With regard to "Facilitating use of information for quality control" (Q12). Some
favorable comments were “Yes, because it’s possible to compare with health indicators,
standards of care, etc. that could be useful as quality control” and “Yes, we can use the data
from the system as we have previously envisioned, to perform statistical analysis based on
diverse pathologies and treatments.”
(7) In the area of "Ease of Use" (Q4), 55% of survey respondents indicated that EMR WB
was “Much better” than the paper-‐based system, which had an average score of 2.2 (95%
CI: 1.7–2.6). This contrasted with the opinion that various clinical system designers have
stated over the years that "ease of use" often is the single most important determinant of
user satisfaction [12,38,58]. Some related comments were “Only if I can learn how to access
84
it;” “It requires more time to create a new file, as with following up with the patient;” and “As
soon as you get to use it, it’s easy to manage.”
(8) In the area of "Learning how to use" (Q3), 48% of survey respondents indicated that
EMR WB was “Much better” than the paper-‐based system, which had an average score of
1.8 (95% CI: 1.2–2.3). This was the eleventh and lowest scored response. Some users
commented that “It takes time to learn how to use the EMR” and “To become competent in its
management is an achievable but time-‐consuming task.”
Suggested improvements for EMR WB
Provider feedback to Q20 (“Do you have any comments or suggestions to improve EMR
WB?”) is summarized here, immediately followed by related verbatim comments:
1. Include a button for additions to patient problems: “Have a button ‘add to problem
list’ on progress notes to facilitate updating the problem list when a new diagnosis is
made.”
2. Expand the variables that are specific to the nursing clinical file: “In the field of
nursing, the variables that are specific to the nursing clinical file need to be expanded.
These additions would impact the nursing side of the management of the patient.”
3. Add some guidelines or a tutorial on the home page: “I consider it important to add
some guidelines on the home page to show how to use EMR WB.”
4. Expand to hospital settings: “I think if physicians could access EMR WB in a private
hospital setting it would be a great asset.”
5. Add Spanish-‐language EB sources of information: “Have access to sources of
information in Spanish because not all of us have mastered English.”
85
6. Use better hardware: “I don’t have any suggestions for improving EMR WB, but
having better hardware would facilitate the speed of inputting information into the
system.”
3.6. Discussion
As W.E. Hammond pointed out in 2009 [9], “For much of the world, truly productive and
functional EMRs remain an elusive goal of the future.” However, opportunities for
improvement abound from the availability of HIT funding in the United States, Mexico, and
other countries. EMR WB underscored the importance of and need to better understand
the logistics of bringing patient records out of the ink-‐and-‐paper era and into the state-‐of-‐
the-‐art computer age in a global context. This project focused on answering the “why” and
“how-‐to” questions regarding the basic functions needed to ensure user satisfaction with
EMRs.
The EMR WB QUS evaluation tool shed light on the system’s interface, standardization,
access, and offering of knowledge-‐based tools compared to the paper-‐based system. The
results showed that a majority of respondents found the EMR WB system overall more
satisfying than the paper-‐based system with regard to these aspects by indicating “Much
better” on the “Overall satisfaction” question of the survey. The novelty effect was
considered as a possible explanation for this finding at the CUSS, where providers may
have initially reacted favorably to the institution of new technology rather than to any
actual improvement in workflow with EMR WB in comparison with the paper-‐based
system. While it is possible that the higher satisfaction reported with the use of EMR WB
86
via the EMR WB QUS can be attributed to a novelty effect, informal monitoring of user
satisfaction after one year indicates the level of interest in EMR WB is still greater than that
in the paper-‐based system. CUSS providers continue to display effort and persistence with
the new system, which demonstrates the novelty effect is no longer applicable.
Implementation of EMR WB in the CUSS primary care setting was associated with
perceived overall improvements and satisfaction in data sharing across all five primary
care domains, as indicated by both the survey responses specifying the highest level of
satisfaction reported (Table 5) and associated comments. Respondents believed that EMR
WB data sharing allowed them to deliver more effective patient care. These findings were
not particularly surprising for two reasons. First, during the focus group CUSS health care
providers had expressed great interest in multidisciplinary collaborative work where all
different professionals (physicians, psychologists, nutritionists, nurses, and social workers)
are able to input and review all available information for the same patient. Second, EMR
WB was designed to be all-‐inclusive and multidisciplinary across the most important
primary care domains that parallel the same workflow in use at the CUSS, where the
professional health care provider could input and share information as a member of a
patient’s health team, as was done with the paper-‐based system.
Perhaps the most revealing findings from the survey were that the second and third highest
scored areas (out of 14), “Helping obtain information for research” (Q13), followed by
“Ease of access to files” (Q8), contrasted with the score for "Ease of use" (Q4, 13th overall
place). Various clinical system designers have stated over the years that "ease of use" is the
single most important determinant of user satisfaction [12,38,58]. However, our findings
87
indicated that overall user satisfaction correlated best with the questions that related to
the health care providers’ ability to use the system to carry out their assigned tasks in
terms of access to the clinical information they needed “from any place, any time, and any
platform,” as one respondent added in the comments at the end of this question.
A possible explanation for the high rating for "Improving the continuity of patient care"
(69%, Q10, and 4th overall place) is that health care providers at the CUSS found that the
capabilities of the EMR WB system (e.g., electronic data storage, access, sharing
information electronically, etc.) offered better support than the paper-‐based system when
following the patient through all phases of health care.
One of the possible explanations for the relatively low rating for "Helping me complete my
tasks" (63%, Q1) is that health care providers at the CUSS found the content of EMR WB to
be very familiar and therefore similar to the paper-‐based system previously used. Another
related possibility is that the paper-‐based system was perceived as very efficient at CUSS
clinical documentation, and therefore there was no need to replace it. As outlined above in
Section 2.2 on standardization, EMR WB was specifically designed to simulate the structure
and clinical contents of the CUSS’ paper-‐based system, so these results can be positively
interpreted (i.e., correlated with ease of use).
The explanation for the low satisfaction reported for "Facilitating use of information for
quality control" (55%, Q12) may be similar to that for "Helping me complete my tasks"
(Q1) since CUSS health care providers already had used the paper-‐based system (for at
least one year) for this purpose, as requested by the CUSS. In this context, EMR WB
88
represented an investment of time needed to obtain all the information that users could
previously see and corroborate easily on paper for quality control.
In contrast, the overall weak support for decision support tools (61%, Q11) may be due to
their newness to the clinical workflow. However, there were enough positive responses to
confirm previous reports demonstrating favorable acceptance of decision support tools in
global adult primary care and support continued adoption in this environment [9,55, 58].
Computer experience was not evaluated because all providers reported being
knowledgeable in basic computer skills, such as the ability to use electronic mail and a
word processor, as expressed in the initial focus group. Due to the assumed simplicity of
the EMR WB interface, its similarity with the paper-‐based system, and user familiarity with
the clinical contents, no training was considered necessary. However, it seems there is a
clear need for training, as only 48% (the lowest rating) of respondents indicated EMR WB
was “Much better” than the paper-‐based system with regard to ”Learning how to use” (Q3).
This negative rating might be attributed to the short time period (about two hours) in
which health care providers had to become familiarized with the new system. The
introduction of EMR WB, with its documentation functions and new interface, brought a
radically changed interaction structure and represented a new challenge in terms of
learning a different system for regular clinical work. A demo is available upon request, but
it requires the user to spend some time becoming familiar with the system, and it does not
include step-‐by-‐step instructions.
Although specific aspects of system speed were not addressed in the EMR WB QUS, users
were asked about this during follow-‐up onsite visit seven months after EMR WB
89
implementation. None of the CUSS health care providers expressed concerns about speed,
and the majority considered the system to be adequately fast to accomplish needed tasks
such as information input and retrieval, and to take advantage of decision support tools.
The data indicates that relative to other system features, respondents were the least
satisfied with learnability (48%, Q3), and most satisfied with information sharing (82%,
Q7). EMR WB facilitation of research (77%, Q13), file access (76%, Q8), and continuity of
patient care (69 %, Q10) were also highly rated. Thus, the system may be difficult to learn
for some professionals, but once the user is familiar with the system, its biggest payoffs
appear to be information sharing, research, ease of access to files, and continuity of care.
3.7. Summary
Apart from the effort to build an all-‐inclusive interface using one navigation system and the
standardization of the clinical core to facilitate the input of structured and unstructured
(free text) information, what remains one of the greatest challenges to achieving a highly
rated EMR WB user interface is balancing the providers’ requirements with an integrated
and fully functional system in the global context. In many instances, difficulties in learning
how to use the system prevented health care providers from gaining a rapid knowledge of
the system capabilities. Too much information and too many functions on a single screen
may impede task efficiency rather than improve it. However, less information and fewer
functions on a single screen require more screens and thus lead to a more complex user-‐
interaction structure. Fortunately, minor changes in screen design and interaction
structure (e.g., making the all-‐inclusive main menu bars visible all the time) addressed all of
90
these issues and had a positive impact on usability based on direct observation during the
onsite visits and comments gathered from the providers. This was achieved with version
1.5, which required an in-‐depth review focused on interface and documentation structure.
The issues discovered in this project related to the primary care context. Interaction with
the system among diverse professions treating the same patient can be linked back to the
importance of user-‐centered system design. For the interaction structure to be effective,
the tasks and procedures that users may perform with the EMR WB system need to be
organized in a logical and consistent manner [38]. This means that the system functions
should specifically correspond with the goals that the health care provider sets for
performing tasks, including the order in which the provider wants to attain these goals.
Also, the order of information presented should match the order in which a user processes
this information, which means clustering related data elements on computer screens
across the different primary care domains considered [38, 51].
Technical support is another essential element of successful EMR WB implementation.
Follow-‐up inquiries indicated CUSS users felt they had adequate support from both the
University of Guanajuato Information Technology Department, and my team because of the
quick response they obtained when faced with onsite technical difficulties such as
interrupted internet connections or changes in speed, as well as the rapid and tailored
updates to their feedback during the beta testing (i.e., version 1.5). User satisfaction with
an EMR system may obviously vary according to the availability and delivery of associated
support, especially in global settings where EMR adoption has been less prevalent.
91
This project demonstrates that despite the associated challenges, we can move from a
paper system to an empowering system based on state-‐of-‐the-‐art-‐technology (e.g., cloud
computing and cellular networks now widely available). EMR WB needs to be recognized
for its potential to become an intelligent, active partner with the health care provider and
patient to enhance health care services globally, without borders [9].
3.8. Limitations of Study
This research has several important limitations. Probably the most significant is that
because EMR WB was tested in only one clinic, the CUSS, post-‐implementation user
evaluations reflected the proposed system’s functionality and usability in only that
environment. A power calculation was not performed with a random sampling of EMR WB
users to determine the sample size needed to be able to claim support for generalizability
because the study was focused on one setting that represented local, clinical, and
psychosocial primary care problems in targeted underserved communities in the state of
Guanajuato, Mexico, as well as transnational migration health issues [62]. The main
objective of the study was to obtain information about provider satisfaction with the new
EMR WB system compared to the old paper-‐based one by using descriptive statistics rather
than hypothesis testing. Therefore, not all conclusions based on how EMR WB worked in
the CUSS may be generalized to other countries or institutions. However, presenting
information on the study setting may help others apply lessons from the CUSS experience
to similar contexts.
92
A related limitation is the small size of the pre-‐implementation focus group (6 health care
providers), but it is representative of the diverse groups in the total end user population
included in the study (i.e., CUSS physicians—1, psychologists—2, nutritionists—1,
nurses—1, and social workers—1). Another limitation in this context is that the focus
group met just once (for 90 minutes). Other possible limitations are inherent to the
technique itself, namely that providers might not have said what they actually think
because of “group think” (e.g., people expressing an opinion which is in agreement with the
rest of the group even if that opinion is at odds with their own personal perceptions,
interpretations, or beliefs) [59]. Similarly, respondents may have been inclined to give the
answers they thought their recruiter wanted to hear, which would be consistent with her
bias in favor of EMR WB. The moderator did not perceive these factors to be present.
Despite these limitations, this was the first study assessing user satisfaction with an
entirely new multidisciplinary, standardized web-‐based EMR system compared to a paper-‐
based record system in a global primary care setting. At a time when governments, health
care organizations, and stakeholders are focused on efficacy, development of a system such
as EMR WB that satisfies its users is an important step toward a design supported by solid
evidence showing it may actually improve overall provider satisfaction, and eventually
patient quality of life as well. Innovations in software technologies, and more particularly
the development of cloud and mobile computing, will increasingly encourage the use of
EMR research in similar global settings in the future.
93
3.9. Suggestions for Further Research
Although the study offered important insights about the level of satisfaction with use of
EMR WB compared to paper-‐based records as expressed by health care providers, several
related critical questions remain unanswered. One of these is how patient satisfaction is
affected during clinical encounters with health care providers who use EMR WB. Also, to
what extent does use of EMR WB in the same and similar settings impact patients’ overall
quality of life, specific health indicators, and health outcomes? Fortunately, these
unanswered questions have become more focused because of the study. The next
suggested step for further research is centered on answering the level of patient
satisfaction as it relates to health outcomes and treatment of prevalent chronic conditions
such as type 2 diabetes mellitus,58 hypertension, cardiovascular diseases, and depression as
facilitated by EMR WB. These research initiatives should take place in parallel with the
software updates and setting expansions in primary care settings not only in Mexico, but
the United States and other countries. Additional implementation of EMR WB in
multispecialty primary health clinics of varying sizes needs to be done to better understand
the functionality that is most associated with high levels of satisfaction with its use.
Several ideas are already being developed to update ongoing future improvements: 1)
Meaningful use certification in the United States; 2) Information exchange and
interoperability with diverse EMRs; 3) an mHealth-‐based EMR WB system (mHealth
unification initiative); 4) Global expansion (Mesoamerica Health Initiative 201559; United
58 Medscape Outpatient EHRs May Improve Diabetes Management (http://www.medscape.com/viewarticle/775233?src=nl_topic). Tenforde M, Nowacki A, Jain A, Hickner J. The association between personal health record use and diabetes quality measures. J Gen Intern Med. 2012 Apr;27(4):420-‐4. 59 Belize, El Salvador, Guatemala, Nicaragua, Honduras, Panama, Costa Rica and the Mexican state of Chiapas
94
States—pilot project in Pittsburgh60; Africa—Tanzania; and Europe—the European EMR
WB model-‐pilot project at Heidelberg University, Germany); 5) Personalized medicine
integration (e.g., genome infobuttons); 6) Integration of machine translation tools (for
English and Spanish); 7) EMR WB new modules (hospital, medical, and surgical
subspecialties; public health; and a surveillance-‐based EMR WB system); 8) EMR WB Lab-‐
on-‐a-‐chip (portable medical office lab project); and the EMR WB-‐based Office of the
Future—Integrating Information in the Care System project (e.g., machine learning,
knowledge discovery, and data mining).
I look forward to the diffusion and expansion of a ubiquitous, state-‐of-‐the-‐art EMR system
to improve global health outcomes in a cost-‐effective manner.
60 Squirrel Hill Health Center and the University of Pittsburgh Center for Global Health.
95
Chapter IV: Ethical Dilemmas
EMRs are increasingly being used in many developing countries, several of which have
moved beyond isolated pilot projects to mainstream components of their local (as in the
case with EMR WB) and national health strategies. Despite growing enthusiasm for
adopting EMRs in middle income and poor countries, almost no attention has been paid to
the ethical dilemmas associated with their implementation and use [52]. These ethical
issues should be addressed now to avoid problems with maintaining appropriate
application of EMRs and their future adoption.
The development and implementation of EMR WB was guided by the HITECH Act, a widely
accepted ethical framework currently in use in the United States and Mexico. HITECH has
now extended some privacy and security protections as described in the Health Insurance
Portability and Accountability Act (HIPAA) to EMR health care provider users.
Despite their promise as a technological innovation capable of reforming health care, many
currently unanswered ethical questions threaten the widespread adoption of EMRs. The
literature points out that privacy, security, and confidentiality issues appear to be the most
common ethical concerns with EMR implementation and adoption [37,52,53]. In addition,
there are other concerns that may arise with respect to disclosing clinical information
stored electronically [54,56]. One is related to the use of EB tools. By definition, EB implies
the use of the best evidence available, the experience of the provider, and patient’s
preference or choice [28]. The problem arises when the evidence competes with the
patient’s decision-‐making process, preference, or best interest, even with good intentions.
The lack of provider experience perhaps is a predisposition that has to be considered.
96
Imposing knowledge-‐based tools at the point of care over a patient’s best interest is a
concern that remains to be studied, but because EMR WB uses these tools, it is important to
point out the potential ethical issues involved and create the needed awareness. Another
concern is related to copying and pasting of clinical notes or information from other
patients or other sources that has the potential to add misinformation, create mistakes, and
build clinically unrelated files [40,52, 53].61
Because the need for access is one of the primary reasons for establishing the proposed
EMR system, restriction of access, time of access, place of access, and even provision of
access remain sources of ethical concern when dealing with a patient’s data across borders.
To address these critical areas, the system was designed to be HIPAA compliant, obey the
International Organization for Standardization (ISO) confidentiality62 and privacy rules,
and work seamlessly for a patient’s best interest. An EMR culture of ethics was created
where patients’ best interests are the norm, which reflects the overall patient-‐centered
design. Future plans include setting up lines of research to better understand the intricate
ethical issues related to the nature of this project and implement better ethical and
technological solutions related to overall clinical data management across borders.
61 Leora Horwitz. Shortcuts in Medical Documentation: Electronic medical records make some things too easy. The New York Times. OP-‐ED CONTRIBUTOR. November 22, 2012. 62 ISO definition: Confidentiality is a characteristic that applies to information. To protect and preserve the confidentiality of information means to ensure that it is not made available or disclosed to unauthorized entities. In this context, entities include both individuals and processes. (http://www.praxiom.com/iso-‐27001-‐definitions.htm#Confidentiality)
97
Chapter V: Conclusions
EMR WB attempted to highlight the importance of and need to better understand the
logistics of bringing patient records out of the ink-‐and-‐paper era and into the state-‐of-‐the-‐
art computer age in a global context. This project focused on answering the “why” and
“how-‐to” questions regarding the interface, access, standardization, decision support,
definition, and evaluation of the proposed concept. Hence, it was expected that EMR WB
would have a measurable, positive effect on provider satisfaction compared with the paper-‐
based system.
The implementation of EMR WB in a primary care setting, the CUSS, was associated with
perceived overall improvements in information sharing, research, access to files, continuity
of care, workflow, and primary documentation capabilities. Users valued remote access and
secure sharing of clinical data across multidisciplinary primary care disciplines in
particular. These results confirm previous reports demonstrating favorable acceptance of
EMRs in adult primary care, and support continued adoption in this environment [41].
The central idea of this effort was to provide a research-‐based medical record system with
state-‐of-‐the-‐art biomedical informatics applications that would improve provider
satisfaction and health care for populations in Mexico with limited resources. This included
efficient documentation, the transfer and sharing of technical and clinical information,
decision support, technology, and organizational skills.
98
5.1. Conflicts of interest
One of the priorities for EMR WB is to make it available for free. The intention after
completion of the pilot project is to find an institution in the United States and/or abroad to
support its updates and expansion (e.g., migration to farm computing when needed) under
the same “open source for all” philosophy. No current or potential conflicts of interest have
arisen.
99
References
1. O'Malley AS. Tapping the unmet potential of health information technology. N Engl J Med. 2011 Mar 24;364(12):1090-‐1. 2. Romano MJ, Stafford RS. Electronic health records and clinical decision support systems: impact on national ambulatory care quality. Arch Intern Med. 2011 May 23;171(10):897-‐903. 3. Jones SS, Adams JL, Schneider EC, Ringel JS, McGlynn EA. Electronic health record adoption and quality improvement in US hospitals. Am J Manag Care. 2010 Dec;16(12 Suppl HIT):SP64-‐71. 4. Bewley S, Fawdry R, Cummings G, Perry H. Electronic health records. The naked truth. BMJ. 2010 Oct 13;341:c5637. 5. Häyrinen K, Saranto K, Nykänen P. Definition, structure, content, use and impacts of electronic health records: a review of the research literature. Int J Med Inform. 2008 May;77(5):291-‐304. 6. Doll WJ, Torkzadeh G. The measurement of end user computing satisfaction. MIS Quart. 1988 June;12(2):258-‐74. 7. O’Malley AS, Grossman JM, Cohen GR, Kemper NM, Pham HH. Are electronic medical records helpful for care coordination? Experiences of physician practices. J Gen Intern Med 2010;25:177-‐85. 8. Mandl KD, Kohane IS. No small change for the health information economy. N Engl J Med. 2009 Mar 26;360(13):1278-‐81. 9. Hammond WE. Realizing the potential of healthcare information technology to enhance global health. Stud Health Technol Inform. 2009;150:8-‐13. 10. Center for Enterprise Modernization. Electronic health records overview [Internet]. McLean, VA: The MITRE Corporation; 2006 Apr. 30 p. Supported by the National Institutes of Health, National Center for Research Resources. Available from: http://www.himss.org/content/files/Code%20180%20MITRE%20Key%20Components%20of%20an%20EHR.pdf 11. Ball MJ, Peterson H, Douglas JV. The computerized patient record: a global view. MD Comput. 1999 Sep-‐Oct;16(5):40-‐6. 12. Ives B, Olson, MH, Baroudi JJ. The measurement of user information satisfaction. Commun ACM. 1983 Oct;26(10):785-‐93.
100
13. Cimino JJ, Teich JM, Patel VL, Zhang J. What is wrong with EMR? Panel proposal for: AMIA Annu Symp Proc. 1999 Nov 6-‐10; Washington, DC. 14. Hsieh CT. Electronic medical record system: current status use to support universal healthcare system. Commun IIMA. 2010 Aug 1;10(3). 15. Chen K, Hellerstein JM, Parikh TS. Data in the first mile. CIDR Conf Proc. 2011 Jan 9-‐12:203-‐6; Asilomar, CA. 16. Efthimiadis EN, Hammond KW, Laundry R, Thielke SM. Developing an EMR simulator to assess users’ perception of document quality. HICSS Conf Proc. 2010 Jan 5-‐8:1-‐9. 17. Blumenthal D, Tavenner M. The “meaningful use” regulation for electronic health records. N Engl J Med. 2010;363:501-‐4. 18. Kush RD, Helton E, Rockhold FW, Hardison CD. Electronic health records, medical research, and the Tower of Babel. N Engl J Med. 2008 Apr 17;358(16):1738-‐40. 19. Coiera E. Guide to health informatics. 2nd ed. New York: Oxford University Press; 2003. Chapter 10, The EMR; p. 111-‐23. 20. Rosenthal A, Mork P, Li MH, Stanford J, Koester D, Reynolds P. Cloud computing: a new business paradigm for biomedical information sharing. J Biomed Inform. 2010 Apr;43(2):342-‐53. Epub 2009 Aug 26. Review. 21. Del Fiol G, Haug PJ. Infobuttons and classification models: a method for the automatic selection of on-‐line information resources to fulfill clinicians' information needs. J Biomed Inform. 2008 Aug;41(4):655-‐66. Epub 2007 Dec 8. 22. Maviglia SM, Yoon CS, Bates DW, Kuperman G. KnowledgeLink: impact of context-‐sensitive information retrieval on clinicians' information needs. J Am Med Inform Assoc. 2006 Jan-‐Feb;13(1):67-‐73.
23. Cimino JJ. Use, usability, usefulness, and impact of an infobutton manager. AMIA Annu Symp Proc. 2006:151-‐5.
24. Bakken S, Currie LM, Lee NJ, Roberts WD, Collins SA, Cimino JJ. Integrating evidence into clinical information systems for nursing decision support. Int J Med Inform. 2008 Jun;77(6):413-‐20.
25. Del Fiol G, Rocha RA, Clayton PD. Infobuttons at Intermountain Healthcare: utilization and infrastructure. AMIA Annu Symp Proc. 2006:180-‐4.
26. Osherhoff SA, Forsythe DE, Buchanan BG, Bankowitz RA, Blumenfeld BH, Miller RA. Physicians’ information needs: analysis of questions posed during clinical teaching. Ann Intern Med. 1991; 114:576-‐81.
101
27. Cimino JJ, Johnson SB, Aguirre A, Roderer N, Clayton PD. The MEDLINE Button. Proc Annu Symp Comput Appl Med Care. 1992:81-‐5.
28. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996 Jan 13;312(7023):71-‐2.
29. Brender J, Nohr C, McNair P. Research needs and priorities in health informatics. Int J Med Inf. 2000 Sep 1;58–59:257–89.
30. Hassey A, Gerrett D, Wilson A. A survey of validity and utility of electronic patient records in a general practice. BMJ. 2001 Jun 9;322(7299):1401-‐5.
31. Larcker DF, Lessig VP. Perceived usefulness of information: a psychometric examination. Decision Sci 1980 Jan;11(1):121-‐34.
32. Wyatt JC, Sullivan F. ABC of health informatics: what is health information? BMJ 2005 Sep 8;331:566-‐8. Clinical review.
33. Black AD, Car J, Pagliari C, Anandan C, Cresswell K, Bokun T, McKinstry B,Procter R, Majeed A, Sheikh A. The impact of eHealth on the quality and safety of health care: a systematic overview. PLoS Med. 2011 Jan 18;8(1):e1000387. Review.
34. Yan H, Gardner R, Baier R. Beyond the focus group: understanding physicians' barriers to electronic medical records. Jt Comm J Qual Patient Saf. 2012 Apr;38(4):184-‐91.
35. Chin JP, Diehl VA, Norman KA. Development of an instrument measuring user satisfaction of the human-‐computer interface. ACM CHI Proc. 1988:213-‐8.
36. Cimino JJ, Li J, Bakken S, Patel VL. Theoretical, empirical and practical approaches to resolving the unmet information needs of clinical information system users. AMIA Annu Symp Proc. 2002:170-‐4
37. Peng C, Kesarinath G, Brinks T, Young J, Groves D. Assuring the privacy and security of transmitting sensitive electronic health information. AMIA Annu Symp Proc. 2009:516-‐20.
38. Jaspers MW, Peute LW, Lauteslager A, Bakker PJ. Pre-‐post evaluation of physicians' satisfaction with a redesigned electronic medical record system. Stud Health Technol Inform. 2008;136:303-‐8.
39. Sittig DF, Kuperman GJ, Fiskio J. Evaluating physician satisfaction regarding user interactions with an electronic medical record system. AMIA Annu Symp Proc. 1999:400-‐4.
40. Payne TH, Hirschmann JV, Helbig S. The elements of electronic note style. J AHIMA. 2003 Feb;74(2):68, 70.
41. Klann JG, Szolovits P. An intelligent listening framework for capturing encounter notes from a doctor-‐patient dialog. BMC Med Inform Decis Mak. 2009 Nov 3;9 Suppl 1:S3.
42. Swanson, EB. Management and information systems: an appreciation and involvement. Manage Sci. 1974;21(2):178-‐88.
102
43. Maish AM. A user’s behavior towards his MIS. MIS Quart 1979;3(1):37-‐52.
44. Ang J, Koh S. Exploring the relationships between user information satisfaction and job satisfaction. Int J Inform Manage 1997;17(3):169-‐77.
45. Rector AL, Nowlan WA, Kay S. Foundations for an electronic medical record. Methods Inf Med. 1991 Aug;30(3):179-‐86.
46. Jimeno-‐Yepes A, Jiménez-‐Ruiz E, Berlanga-‐Llavori R, Rebholz-‐Schuhmann D. Reuseof terminological resources for efficient ontological engineering in Life Sciences. BMC Bioinform. 2009 Oct 1;10 Suppl 10:S4.
47. Gruber T. A translation approach to portable ontology specifications. Knowl Acquis. 1993;5:199-‐220.
48. Rubin DL, Shah NH, NF Noy. Biomedical ontologies: a functional perspective. Brief Bioinform. 2007;9(1):75-‐90.
49. Cimino JJ. An integrated approach to computer-‐based decision support at the point of care. Trans Am Clin Climatol Assoc. 2007;118:273-‐88.
50. Rector AL. Thesauri and formal classifications: terminologies for people and machines. Methods Inf Med. 1998 Nov;37(4-‐5):501-‐9. 51. Rector AL, Nowlan WA, Kay S, Goble CA, Howkins TJ. A framework for modeling the electronic medical record. Methods Inf Med. 1993 Apr;32(2):109-‐19. 52. Were MC, Meslin EM. Ethics of implementing Electronic Health Records in developing countries: Points to Consider. AMIA Annu Symp Proc. 2011:1499-‐505. Epub 2011 Oct 22. 53. Sittig DF, Singh H. Legal, ethical, and financial dilemmas in electronic health record adoption and use. Pediatrics. 2011 Apr;127(4):e1042-‐7. Epub 2011 Mar 21. 54. Balka E, Tolar M. Everyday ethical dilemmas arising with electronic record use in primary care. Stud Health Technol Inform. 2011;169:285-‐9. 55. Williams F, Boren SA. The role of the electronic medical record (EMR) in care delivery development in developing countries: a systematic review. Inform Prim Care. 2008;16(2):139-‐45. Review. 56. Cushman R, Froomkin AM, Cava A, Abril P, Goodman KW. Ethical, legal and social issues for personal health records and applications. J Biomed Inform. 2010 Oct;43(5 Suppl):S51-‐5. 57. Greenhalgh T, Potts HW, Wong G, Bark P, Swinglehurst D. Tensions and paradoxes in electronic patient record research: a systematic literature review using the meta-‐narrative method. Milbank Q. 2009 Dec;87(4):729-‐88. Review.
103
58. Joos D, Chen Q, Jirjis J, Johnson KB. An electronic medical record in primary care: impact on satisfaction, work efficiency and clinic processes. AMIA Annu Symp Proc. 2006:394-‐8. 59. Brodigan DL. Focus group interviews: applications for institutional research. Tallahassee, FL: AIR Professional File No. 43;1992 Win. 6 p. 60. Institute of Medicine. The computer-‐based patient record: an essential technology for health care. Committee on Improving the Patient Record, Division of Health Care Services. Washington, DC: National Academy Press; 1997. 256 p. 61. Dumas JS, Redish JC. A practical guide to usability testing. Rev. ed. Portland, OR: Intellect Books; 1999. 404 p. 62. Bartlett JE II, Kotrlik JW, Higgins CC. Organizational research: determining appropriate sample size in survey research. ITLPJ. 2001;19(1):43-‐50. 63. Gardner MJ, Altman DG. Confidence intervals rather than P values: estimation rather than hypothesis testing. Br Med J (Clin Res Ed). 1986 Mar 15;292(6522):746-‐50. 64. Mandl KD, Kohane IS. Escaping the EHR trap—the future of health IT. N Engl J Med. 2012 Jun 14;366(24):2240-‐2. 65. Linder JA, Schnipper JL, Middleton B. Method of electronic health record documentation and quality of primary care. J Am Med Inform Assoc. 2012 Nov 1;19(6):1019-‐24.
104
Appendix A
Focus Group Study Outline of Question Sequence
Opening comments: Welcome, brief description of the project EMR WB, and statements regarding the purpose of the study, focus group procedures, and ethical issues.
Opening comment: Please tell us a little bit about yourself.
Introductory question: The use of EMRs in the clinical setting to collect, store, share, and access patient medical records promises overall vast improvements in health care; many providers are currently using them. In thinking about your daily professional life, what does the use of an EMR mean to you?
Transition questions: Are EMRs a negative tool in your clinical setting? If so, explain how they are negative. Are EMRs a positive tool in your clinical setting? If so, explain how they are positive.
Key questions (issues for interactive participation): How do you define EMRs? What do you expect from EMRs? What do you think are the major differences between EMRs and paper-‐based medical records? What functions would you like to see? What do you think are the major, current problems of EMRs? What fundamental changes can EMRs make in health care practice? How do you describe the ideal EMR system? Is an ideal EMR possible? Among the usability problems with EMRs, which is the most serious one? Ending questions: All things considered, what would you say is the major potential benefit of EMR adoption? Is there anything about the implementation and use of EMRs that I haven’t talked about that you would like to raise before I leave?
105
Appendix B The Evaluation Tool
EMR WB Questionnaire for User Satisfaction (EMR WB QUS)
SURVEY PART 1 of 3. Demographic Data. Please complete:
PART 2. User Satisfaction Questionnaire. For each of the following features, please mark the rating that best describes how you think of Electronic Medical Record Without Borders (EMR WB) in comparison with the paper-‐based system (PBS). 4) EMR WB in comparison with the PBS: (Q1) Helping me complete my tasks
Comments: ____________________________________________________________________________________________ ____________________________________________________________________________________________ 5) EMR WB in comparison with the PBS: (Q2) Facilitating my input of information
Comments: ____________________________________________________________________________________________ ____________________________________________________________________________________________
1) Age: 20 -‐ 30 31 -‐ 40 41 -‐ 50 51 -‐ 60 60 + ☐ ☐ ☐ ☐ ☐
2) Gender: M F ☐ ☐ 3) Field: Medicine Psychology Nutrition Nursing Social Work ☐ ☐ ☐ ☐ ☐
Much worse -‐3
Worse -‐2
Somewhat worse -‐1
About the same 0
Somewhat better 1
Better 2
Much better 3
☐ ☐ ☐ ☐ ☐ ☐ ☐
Much worse
Worse
Somewhat worse
About the same
Somewhat better
Better Much better
☐ ☐ ☐ ☐ ☐ ☐ ☐
106
6) EMR WB in comparison with the PBS: (Q3) Learning how to use
Comments: ____________________________________________________________________________________________ ____________________________________________________________________________________________ 7) EMR WB in comparison with the PBS: (Q4) Ease of use
Comments: ____________________________________________________________________________________________ ____________________________________________________________________________________________ 8) EMR WB in comparison with the PBS: (Q5) Effectively review patient problems
Comments: ____________________________________________________________________________________________ ____________________________________________________________________________________________ 9) EMR WB in comparison with the PBS: (Q6) Safeguarding patient privacy
Comments: ____________________________________________________________________________________________ ____________________________________________________________________________________________ 10) EMR WB in comparison with the PBS: (Q7) Sharing patient information with my fellow clinical team members
Much worse
Worse
Somewhat worse
About the same
Somewhat better
Better Much better
☐ ☐ ☐ ☐ ☐ ☐ ☐
Much worse
Worse
Somewhat worse
About the same
Somewhat better
Better Much better
☐ ☐ ☐ ☐ ☐ ☐ ☐
Much worse
Worse
Somewhat worse
About the same
Somewhat better
Better Much better
☐ ☐ ☐ ☐ ☐ ☐ ☐
Much worse
Worse
Somewhat worse
About the same
Somewhat better
Better Much better
☐ ☐ ☐ ☐ ☐ ☐ ☐
107
Comments: _____________________________________________________________________________________ ___________________________________________________________________________________________________ 11) EMR WB in comparison with the PBS: (Q8) Ease of access to files
Comments: ____________________________________________________________________________________________ _____________________________________________________________________________________________ 12) EMR WB in comparison with the PBS: (Q9) Improving workflow63
Comments: ____________________________________________________________________________________________ ____________________________________________________________________________________________ 13) EMR WB in comparison with the PBS: (Q10) Improving the continuity of patient care64
Comments: ____________________________________________________________________________________________ _____________________________________________________________________________________________ 14) EMR WB in comparison with the PBS: 63 Workflow: Pattern of recurrent functions or procedures frequently found in organizational processes, such as notification, decision, and action (NLM-‐MeSH—Year introduced: 2010). 64 Continuity of Patient Care. Health care provided on a continuing basis from the initial contact, following the patient through all phases of medical care. [NLM-‐MeSH—Year introduced: 1991(1975)]
Much worse
Worse
Somewhat worse
About the same
Somewhat better
Better Much better
☐ ☐ ☐ ☐ ☐ ☐ ☐
Much worse
Worse
Somewhat worse
About the same
Somewhat better
Better Much better
☐ ☐ ☐ ☐ ☐ ☐ ☐
Much worse
Worse
Somewhat worse
About the same
Somewhat better
Better Much better
☐ ☐ ☐ ☐ ☐ ☐ ☐
Much worse
Worse
Somewhat worse
About the same
Somewhat better
Better Much better
☐ ☐ ☐ ☐ ☐ ☐ ☐
108
(Q11) Helping make effective evidence-‐based care decisions
Comments: ________________________________________________________________________________ ______________________________________________________________________________________________ 15) EMR WB in comparison with the PBS: (Q12) Facilitating use of information for quality control
Comments: ____________________________________________________________________________________________ ____________________________________________________________________________________________ 16) EMR WB in comparison with the PBS: (Q13) Helping obtain information for research
Comments: ____________________________________________________________________________________________ ____________________________________________________________________________________________ 17) EMR WB in comparison with the PBS: (Q14) Appropriateness for use in my care setting
Comments: ____________________________________________________________________________________________ ____________________________________________________________________________________________ 18) EMR WB in comparison with the PBS: (Q15) Overall satisfaction
Much worse
Worse
Somewhat worse
About the same
Somewhat better
Better Much better
☐ ☐ ☐ ☐ ☐ ☐ ☐
Much worse
Worse
Somewhat worse
About the same
Somewhat better
Better Much better
☐ ☐ ☐ ☐ ☐ ☐ ☐
Much worse
Worse
Somewhat worse
About the same
Somewhat better
Better Much better
☐ ☐ ☐ ☐ ☐ ☐ ☐
Much worse
Worse
Somewhat worse
About the same
Somewhat better
Better Much better
☐ ☐ ☐ ☐ ☐ ☐ ☐
Much worse
Worse
Somewhat worse
About the same
Somewhat better
Better Much better
109
Comments: ____________________________________________________________________________________________ ____________________________________________________________________________________________ PART 3. Use of EMR WB and Recommendations. 19) Approximately how many patient/client visits did you have at the CUSS while using EMR WB?* *Please include total encounters—new patients, multiple visits/follow-‐ups of the same patient. Comments: ____________________________________________________________________________________________ ____________________________________________________________________________________________ 20) Do you have any comments or suggestions to improve EMR WB? ______________________________________________________________________________________________ ______________________________________________________________________________________________
☐ ☐ ☐ ☐ ☐ ☐ ☐
1 -‐ 2 3 -‐ 5
6 -‐ 9 10 +
☐ ☐ ☐ ☐
110
Appendix C The National Center for Biomedical Ontology BioPortal
Screen shots of an electronic medical record showing its relationships
111
Appendix D-‐1 Standardization Process Sample
History of Present Illness
Here is an example of the standardization process to create the medical history clinical core using one of its components, the History of Present Illness (HPI). It incorporated a comprehensive state-‐of-‐the-‐art terminology consisted in 1) controlled vocabularies (terms), 2) semantics (meaning); 3) ontologies (relations), and 4) syntaxes (sentences). In addition, the backbone of its structure was based on peer-‐reviewed evidence-‐based clinical protocols and gold standard clinical guidelines into an English and Spanish language EMR system.
112
Appendix D-‐2 Standardization Process Sample
Contents of the History of Present Illness
This table below shows the specific meaning of terms used in the contents of HPI based on linguistic semantics to express meaning through natural languages, and the syntax used as a means to apply principles and rules for constructing sentences in English and Spanish. These definitions have been applied to all primary care fields (medicine, psychology, nutrition, nursing and social work).
113
Appendix E
Example of Standardized HPI Content within the Clinical Core
65 HPI is the logical continuation and detailed interview prompted by the chief complaint (CC) or presenting symptom, and represents the central aspect of the clinical history itself.
HISTORY OF PRESENT ILLNESS—HPI:65
a. Characterization of each problem (chronologic order).
b. Symptom modifiers (or dimensions to characterize symptoms): 1) Location; 2) Quality or character; 3) Quantity (severity or amount); 4) Chronology —onset, duration, frequency, periodicity, temporal characteristics; 5) Setting (under what circumstances it occurs); 6) Aggravating-‐alleviating factors; 7) Associated symptoms / manifestations; 8) Disability & Adaptation; and 9) Attributions.
c. Pertinent positives & pertinent negatives.
114
VITA
Jose Francisco Saavedra was born in the city of León, state of Guanajuato, Mexico. He has
lived in many places in Mexico, England and the United States. Currently he calls Seattle his
home. He earned a Doctor of Medicine degree at the Autonomous University of Nuevo León
(UANL), a specialty in Family Medicine at the National Institutes of Health in Mexico, and a
fellowship in Hospital Medicine at the UANL. In 2012 he earned a Doctor of Philosophy in
Biomedical Informatics at the University of Washington.
115
©Copyright 2012
Jose Francisco Saavedra
