Example Of Doctoral Dissertation Outline

DOCTORAL DISSERTATION

DESIGN, IMPLEMENTATION, USER ACCEPTANCE, AND

EVALUATION OF A CLINICAL DECISION SUPPORT SYSTEM FOR

EVIDENCE-BASED MEDICINE PRACTICE

A dissertation submitted to the

H. JOHN HEINZ III SCHOOL OF PUBLIC POLICY AND MANAGEMENT

in partial fulfillment for the requirements for the degree of

DOCTORAL OF PHILOSOPHY

in

INFORMATION SYSTEMS AND HEALTH INFORMATICS

by

Kai Zheng

Carnegie Mellon UniversityH. John Heinz III School of Public Policy and Management

Pittsburgh, Pennsylvania 15213

September 2006

Medical decision support tools were essentially formulated from a tech-nical capability perspective and this view has met limited adoption andslowed down new development as well as integration of these importantsystems into patient management work flows and clinical information sys-tems. The science base of these systems needs to include evidence-basedmedicine and clinical practice guidelines and the paradigms need to beextended to include a collaborative provider model, the users and the or-ganization perspectives. The availability of patient record and medicalterminology standards is essential to the dissemination of decision sup-port systems and so is their integration into the care process. To build newdecision support systems based on practice guidelines and taking into ac-count users preferences, we do not so much advocate new technologicalsolutions but rather suggest that technology is not enough to ensure suc-cessful adoption by the users, the integration into practice workflow, andconsequently, the realization of improved health care outcomes.

– Fieschi et al., 2003

Abstract

Evidence-based medicine is the “conscientious, explicit, and judicious use of cur-rent best evidence in making medical decisions about the care of individual patients”(Sackett et al., 1999 [163]). There has been a general consensus that continuous, com-prehensive practice of evidence-based medicine has tremendous potential to improvequality of care and reduce practice variation. However, there is also a widely acknowl-edged gap between physicians’ awareness of these care standards and physicians’consistent application of them in practice. While mounting evidence has shown thatclinical decision support systems can improve physician guideline compliance, andthus patient health, widespread use of such systems has not become available due tonumerous technological, behavioral, and organizational barriers. These facts motivatethe present research.

In this thesis, I report findings from a seven-year effort to design, implement, andevaluate a clinical decision support system, called Clinical Reminder System (CRS), intwo ambulatory primary care clinics at the Western Pennsylvania Hospital. CRS aimsto improve quality of care by providing clinicians just-in-time alerts and advisories us-ing evidence-based medicine guidelines. First, I describe the technical aspects of thesystem and introduce a computational ontology that enables structured acquisitionand automated execution of evidence-based medicine guidelines. A mixed evalua-tion approach is presented next, which combines quantitative developmental trajec-tory modeling with qualitative assessments to examine the users’ technology adoptionand acceptance behavior. Synthesizing the findings, I critique and extend the Technol-ogy Acceptance Model (TAM)—a widely used theory for studying technology diffu-sion in the information systems area. I show that while TAM explains certain usagebehavior at discrete times, methodological pluralism illustrated in this thesis helpsreveal and understand more subtle, longitudinal behavior that spans the entire tech-nology diffusion process. Next, I take into account the social context in which theusers of CRS are situated. I find that social contagion, particularly through structuralequivalence of friendship networks, has great impact on the users’ level of adoption.Finally, I use sequential pattern analysis and a first order Markov chain model to an-alyze the temporal event sequences recorded in CRS. The results lead to a softwaredesign pattern for system reengineering, which calibrates the system’s user interfaceso that the within-application workflow is aligned with clinicians’ mental model inmedical problem-solving. In the last chapter, I present future research extensions andlong-term plans for evaluating the system’s effectiveness on physician guideline com-pliance and patient health outcomes.

I conclude that this research enhances our understanding of medical, technologi-cal, behavioral, and institutional challenges associated with diffusion of decision sup-port technologies into health care practice. The methods and findings may also pro-vide methodological and practical insights into design, implementation, and evalu-ation issues of other health informatics applications, as well as information systemsmore generally.

Acknowledgement

First I wish to express my deep gratitude to my advisors, Dr. Rema Padman, Dr.Michael Johnson, Dr. Herbert Diamond, and Dr. David Krackhardt, for their in-valuable advice and encouragement throughout these many years. I am also gratefulto Dr. John Engberg and Mr. Andrew Garvin who worked on prototyping the re-minder system at early stage, and Dr. Daniel Nagin who provided valuable feedbackon the development trajectory analysis. My special thanks go to the two pioneers atthe Western Pennsylvania Hospital, Dr. Herbert Diamond, Chairman of the Depart-ment of Medicine, and Dr. Michelina Fato, Associate Program Director of AmbulatoryMedicine, for their effort in initiating this project and their unflinching support eversince.

Dr. Elliot Goldberg, Dr. Richard Weinberg, Dr. Alejandro Gonzalez, Dr. NanditaJain, and Dr. Sandeep Arora in the Department of Medicine at the Western Pennsylva-nia Hospital have been instrumental in converting vast volumes of medical literatureinto easily understood visual diagrams. Their contributions are greatly appreciated.Many colleagues from the West Penn Allegheny Health System participated in thisproject and provided great administrative and technical support. They are: Project Co-ordinators—Pamelasue Kozlowski, Lisa Summit, Cindy Carlson, Joan Vuljanic; ClinicManagers—Nancy Scenna, Lynn Chilton, Jennifer Rusiewicz, Gina Musser; IT SupportStaff —Siu Peng Chan, Lorraine Malloy, Renee Blucas, Sherrie Rizza; System Interfac-ing Group—Mark Caro, Mark Bailey, Alan Oley, Mary Louise Lollo, Tim Ruff; andPatricia Corbin from the Institutional Review Board. This work could never have beendone without the commitment and participation of all of them. I am also thankful toVelma Payne, Sharique Hasan, Kelly Stenhoff, Karen Chen, and Samuel Tzeng, whowere involved in this project at various stages. I am indebted to countless people inthe West Penn Medical Associates clinic who have spent enormous time and effort onlearning and using the reminder system. To name a few: Dr. Kofi Clarke, Dr. AndreasAchilleos, Dr. Mary Lynn Sealey, Dr. Maria Tranto, Dr. Kenneth Plowey, Dr. XuongLu, Dr. Sarah Stussy, Dr. Lindsay Ledwich, Ms. Janet Rodgers, Ms. Ann Myers, andMs. Jessica Williams. They all deserve my deep thanks.

I would like to thank all my student fellows for making my graduate life at CarnegieMellon a fun and memorable experience. I want to especially thank those who ran intome in the past two years in HbH and whose faces showed why this guy is still here?,without that I may have spent many more years to bring a closure to this work. Fi-nally, my gratitude to my parents, Guanghai Zheng and Quanzhen Zhou. No wordscan express my love to you.

This work is supported in part by Grant D28HP10107 from Health Resources and Ser-vices Administration (HRSA), Rockville, Maryland, USA, and indirectly by a varietyof funds including research support from the West Penn Allegheny Health System,Merck & Co., Inc., and Proctor & Gamble Company.

Acronyms

ADT Admissions, Discharge, TransferCCR Continuity Of Care RecordCDSSs Clinical Decision Support SystemsCRS Clinical Reminder SystemGREM Guideline Representation and Execution ModelCPoE Computerized Physician Order EntryCPT Current Procedural TerminologyDOI Diffusion of Innovations TheoryDTA Developmental Trajectory AnalysisEHR Electronic Health RecordEMR Electronic Medical RecordGLIF Guideline Interchange FormatICD International Statistical Classification of DiseasesICD-9-CM ICD Ninth Revision, Clinical ModificationLOINC Logical Observation Identifiers, Names, and CodesHL7 Health Level 7MeSH Medical Subject Headings ClassificationNDC National Drug Code DirectoryPEoU Perceived Ease of UsePU Perceived UsefulnessRIM HL7 Reference Information ModelSEM Structural Equation ModelingSNA Social Network AnalysisSNOMED CT Systematized Nomenclature of Medicine – Clinical TermsSPA Sequential Pattern AnalysisTAM Technology Acceptance ModelTPB Theory of Planned BehaviorTRA Theory of Reasoned ActionUMLS Unified Medical Language SystemUTAUT Unified Theory of Acceptance and Use of TechnologyvEMR Virtual Electronic Medical Record System

Medical Terminology

CCR CONTINUITY OF CARE RECORDA standard specification used to foster and improve continuity of pa-tient care, which includes patient’s health status, provider informa-tion, insurance information, recent care provided, recommendationsfor future care, and the reason for referral or transfer.

CPOE COMPUTERIZED PHYSICIAN ORDER ENTRY SYSTEMComputer applications that allow orders of medication, laboratoryor radiology test, or procedure to be transmitted electronically to theappropriate individuals, departments, or organizations, so they canbe carried out.

CPT CURRENT PROCEDURAL TERMINOLOGYA directory of descriptive terms and identifying codes for reportingmedical services and procedures under public and private health in-surance programs. CPT4, the 4th edition in this series, is most widelyadopted edition.

EHR ELECTRONIC HEALTH RECORD SYSTEMA subset of EMR, presently assumed to include summaries, suchas continuity of care record and information from pharmacy bene-fit management firms, reference labs and other organizations aboutthe health status of a patient. It contains patient input and accessspanning episodes of care across multiple units within a community,region, or state.

EMR ELECTRONIC MEDICAL RECORD SYSTEMAn application environment composed of clinical data repository,clinical decision support system, controlled vocabulary, computer-ized physician order entry, and pharmacy and clinical documentationapplications. It is used by healthcare practitioners to document, mon-itor and manage care delivery within the care delivery organization.

HL7 HEALTH LEVEL 7HL7 refers to Health Level Seven, Inc., an all-volunteer, not-for-profitorganization that is involved in development of international health-care standards. The term HL7 is also used to refer to some of thespecific standards created by the organization, for example HL7 v2.x,HL7 v3.0, HL7 Reference Information Model, etc.

ICD INTERNATIONAL STATISTICAL CLASSIFICATION OF DISEASESStandard diagnostic classification used by WHO member states forcompilation of mortality and morbidity statistics. In the UnitedStates, ICD-9-CM (ICD 9th Revision, Clinical Modification) is widelyused to enable the storage and retrieval of diagnostic information.

LOINC LOGICAL OBSERVATION IDENTIFIERS, NAMES, AND CODESUniversal identifiers developed to facilitate the transmission and stor-ing of clinical laboratory results. It is an endorsed standard by theAmerican Clinical Laboratory Association and the College of Ameri-can Pathologists.

MESH MEDICAL SUBJECT HEADINGS CLASSIFICATIONA controlled vocabulary thesaurus maintained by National Library ofMedicine. It consists of sets of terms naming descriptors in a hierar-chical structure that permits searching at various levels of specificity.

NDC NATIONAL DRUG CODE DIRECTORYA directory that lists universal product identifier for human drugs. Itscurrent edition is limited to prescription drugs and insulin products.

SNOMED SYSTEMATIZED NOMENCLATURE OF MEDICINEA clinical terminology that provides unique meanings and formallogic-based definitions for diseases, clinical findings, and procedures.It helps to structure and computerize the medical record, reducing thevariability in the way data is captured, encoded and used for clinicalcare of patients and research.

UMLS UNIFIED MEDICAL LANGUAGE SYSTEMA software tool designed to facilitate the development of computersystems for use of terminologies of biomedicine and health. It is usedby system developers in building or enhancing electronic informationsystems that create, process, retrieve, and aggregate biomedical andhealth data and information.

VEMR VIRTUAL ELECTRONIC MEDICAL RECORDAn abstract collection of patient care related information. It is awidely adopted concept to enable data exchange across health careapplications, and platform-independent application development.

Table of Contents

Abstract 3

Acknowledgement 4

Acronyms 5

Medical Terminology 6

Table of Contents 8

List of Tables 13

List of Figures 15

1 Clinical Decision Support Systems 171.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

1.1.1 Motivation of the Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . 191.1.2 Clinical Decision Support Systems . . . . . . . . . . . . . . . . . . . . . . 191.1.3 Evidence-Based Medicine Practice . . . . . . . . . . . . . . . . . . . . . . 201.1.4 The Clinical Reminder System Project . . . . . . . . . . . . . . . . . . . . 21

1.2 Study Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 221.2.1 Medical Knowledge Engineering . . . . . . . . . . . . . . . . . . . . . . . 231.2.2 Integration with Healthcare Information Systems . . . . . . . . . . . . . 231.2.3 Integration into Routine Clinical Workflow . . . . . . . . . . . . . . . . . 241.2.4 IT Adoption and Acceptance by Healthcare Professionals . . . . . . . . . 251.2.5 Designing an Effective User Interface . . . . . . . . . . . . . . . . . . . . 25

1.3 Structure of this Thesis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

2 The Clinical Reminder System 302.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 312.2 History of the CRS Project . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 322.3 System Functional Requirements . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

2.3.1 Decision-Making Algorithms . . . . . . . . . . . . . . . . . . . . . . . . . 332.3.2 Security and Confidentiality . . . . . . . . . . . . . . . . . . . . . . . . . 362.3.3 Operational Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

2.4 System Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 382.4.1 High-Level Application Design . . . . . . . . . . . . . . . . . . . . . . . . 382.4.2 Relational Database Design . . . . . . . . . . . . . . . . . . . . . . . . . . 382.4.3 Reminder Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 392.4.4 User Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

2.5 System Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432.5.1 Database Schema . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 432.5.2 Reminder Generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

TABLE OF CONTENTS 9

2.5.3 User Interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 442.5.4 Data Exchange Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

2.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

Appendices 472.A A Brief User Manual . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 472.B Screenshots of Main Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

2.B.1 The Prototype System (1999–2000) . . . . . . . . . . . . . . . . . . . . . . 512.B.2 Version 1: Visual Basic Interface (2000–2004) . . . . . . . . . . . . . . . . 522.B.3 Version 2: The Web Prototype (2004–2005) . . . . . . . . . . . . . . . . . 542.B.4 Version 3: Web-Enabled Interface (2005–present) . . . . . . . . . . . . . . 55

2.C CRS Key Data Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612.D Sample Usage Report (continued next page) . . . . . . . . . . . . . . . . . . . . 63

3 Guideline Representation and Execution Model 663.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 673.2 Existing Guideline Ontologies . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

3.2.1 The Evolution of Guideline Ontologies . . . . . . . . . . . . . . . . . . . 683.2.2 Major Ontologies in Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

3.3 Two Ontologies in Depth . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 733.3.1 Analysis Dimensions and Evaluation Criteria . . . . . . . . . . . . . . . 743.3.2 GLIF3 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 753.3.3 PROforma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

3.4 Guideline Representation and Execution Model . . . . . . . . . . . . . . . . . . 833.4.1 Methodology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 853.4.2 Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 883.4.3 GREM Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

3.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103

Appendices 1053.A Sample WSDL for the Flu Plan . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105

4 User Acceptance: A DTA Approach 1064.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1074.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109

4.2.1 Study Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1094.2.2 Measure Metrics of System Usage . . . . . . . . . . . . . . . . . . . . . . 1104.2.3 Developmental Trajectory Analysis . . . . . . . . . . . . . . . . . . . . . 1124.2.4 Classical Cluster Analysis Methods . . . . . . . . . . . . . . . . . . . . . 1154.2.5 Evaluation of Clustering Similarity . . . . . . . . . . . . . . . . . . . . . . 115

4.3 Data Analysis and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1164.3.1 Aggregated Usage and Simple Grouping Scenarios . . . . . . . . . . . . 1164.3.2 Developmental Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . 1174.3.3 Group Profiling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1204.3.4 Analysis of Truncated Usage . . . . . . . . . . . . . . . . . . . . . . . . . 1234.3.5 Comparison with Classical CA Methods . . . . . . . . . . . . . . . . . . 1264.3.6 Impact on Patient Encounters . . . . . . . . . . . . . . . . . . . . . . . . . 126

4.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1294.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132

TABLE OF CONTENTS 10

Appendices 1344.A Classical Cluster Analysis Methods . . . . . . . . . . . . . . . . . . . . . . . . . . 134

4.A.1 Partitioning Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1344.A.2 Hierarchical Clustering . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134

4.B Cork’s Instrument . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1364.C IBM User Satisfaction Questionnaire . . . . . . . . . . . . . . . . . . . . . . . . . 138

5 User Acceptance: Qualitative Assessments 1395.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1405.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141

5.2.1 User Satisfaction Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1415.2.2 Structured Interviews . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1415.2.3 Analysis of System Recorded Clinical Notes . . . . . . . . . . . . . . . . 141

5.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1415.3.1 Positive Themes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1425.3.2 Negative Themes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1425.3.3 Discrepancy across Usage Groups . . . . . . . . . . . . . . . . . . . . . . 143

5.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1445.4.1 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1445.4.2 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1445.4.3 Comparison with Existing Literature . . . . . . . . . . . . . . . . . . . . . 1455.4.4 Lessons Learned . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1455.4.5 Implications for System Reengineering . . . . . . . . . . . . . . . . . . . 146

5.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 147

6 User Acceptance: A Revised TAM Model 1486.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1496.2 Theoretical Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150

6.2.1 Social Psychology Theories . . . . . . . . . . . . . . . . . . . . . . . . . . 1506.2.2 The Technology Acceptance Model . . . . . . . . . . . . . . . . . . . . . . 152

6.3 Limitation of the Existing Models . . . . . . . . . . . . . . . . . . . . . . . . . . . 1586.3.1 Model Operationalization . . . . . . . . . . . . . . . . . . . . . . . . . . . 1596.3.2 Self-Reported Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1606.3.3 Type of Information Systems Studied . . . . . . . . . . . . . . . . . . . . 1626.3.4 Temporal Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1626.3.5 Behavioral Intention, Actual Behavior, and Performance Gains . . . . . 164

6.4 Assessing Physician IT Acceptance . . . . . . . . . . . . . . . . . . . . . . . . . . 1646.4.1 Professional Context of Physicians . . . . . . . . . . . . . . . . . . . . . . 1656.4.2 Social and Organizational Issues . . . . . . . . . . . . . . . . . . . . . . . 1666.4.3 Review of the Existing Studies . . . . . . . . . . . . . . . . . . . . . . . . 1676.4.4 Limitations of the Existing Studies . . . . . . . . . . . . . . . . . . . . . . 167

6.5 Goals of the Present Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1716.6 Research Model and Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . 171

6.6.1 New Conceptualization of IT Acceptance . . . . . . . . . . . . . . . . . . 1716.6.2 Research Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 172

6.7 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1756.7.1 Operationalization of Model Constructs . . . . . . . . . . . . . . . . . . . 1766.7.2 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177

6.8 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 179


6.8.1 Developmental Trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . 1796.8.2 Psychometric Properties of Revised TAM Scales . . . . . . . . . . . . . . 1816.8.3 TAM Analyses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 181


7 User Acceptance: Interface Optimization 1897.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1907.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 193

7.2.1 Sequential Pattern Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . 1937.2.2 First-Order Markov Chain Analysis . . . . . . . . . . . . . . . . . . . . . 1957.2.3 Study Site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196

7.3 Data Analysis and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1967.3.1 Data Preparation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1967.3.2 Frequency of Feature Access . . . . . . . . . . . . . . . . . . . . . . . . . 1987.3.3 Consecutive Sequential Patterns . . . . . . . . . . . . . . . . . . . . . . . 1987.3.4 First Order Markov Chain Analysis . . . . . . . . . . . . . . . . . . . . . 200


8 User Acceptance: Social Contagion and IT Adoption 2068.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2078.2 Social Network Analysis and Theories of Social Influence . . . . . . . . . . . . . 2088.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 209

8.3.1 Study Context . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2098.3.2 Hypotheses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2108.3.3 Social Network Analysis Models . . . . . . . . . . . . . . . . . . . . . . . 2108.3.4 Instrument Development . . . . . . . . . . . . . . . . . . . . . . . . . . . 2118.3.5 Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212

8.4 Data Analysis and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2138.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 216

8.5.1 Influence of Social Contagion . . . . . . . . . . . . . . . . . . . . . . . . . 2168.5.2 Local Effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

8.6 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 219

Appendices 2218.A Social Network Survey Instrument . . . . . . . . . . . . . . . . . . . . . . . . . . 2218.B Informational Script . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222

9 Future Work and Long Term Objectives 2249.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2259.2 Continuous Development of CRS . . . . . . . . . . . . . . . . . . . . . . . . . . . 2259.3 Evolving Social Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2269.4 Long Term Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 226

Appendices 2289.A Clinical Performance and Patient Outcome Measures . . . . . . . . . . . . . . . 228

9.A.1 Preventive Care . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2289.A.2 Chronic Disease Management . . . . . . . . . . . . . . . . . . . . . . . . . 228

9.B Chart Review Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232


9.B.1 Influenza Vaccine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2329.B.2 Pneumococcal Vaccine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2329.B.3 Steroid-Induced Osteoporosis . . . . . . . . . . . . . . . . . . . . . . . . . 2329.B.4 Breast Cancer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2339.B.5 Asthma . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2339.B.6 Diabetes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2349.B.7 Hyperlipidemia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2359.B.8 Hypertension . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236

References 237

Index 249

List of Tables

1.1 Early Experimental Decision Support Systems . . . . . . . . . . . . . . . . . . . 201.2 History of the Clinical Reminder System Project . . . . . . . . . . . . . . . . . . 21

2.1 CRS Key Data Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 612.2 Sample Usage Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 642.3 Sample Data Entry Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.1 Descriptive Traversed Pathways . . . . . . . . . . . . . . . . . . . . . . . . . . . 943.2 Evolvement of the Function Repository . . . . . . . . . . . . . . . . . . . . . . . 101

4.1 About the Study Site . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1114.2 Reminder Response Categorization . . . . . . . . . . . . . . . . . . . . . . . . . . 1124.3 Group Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1184.4 Summary of System Use Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1204.5 Group Profiles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1204.6 Impact of User Characteristics on the Group Probabilities . . . . . . . . . . . . . 1224.7 Comparison between DS10, DS5 and Complete Dataset . . . . . . . . . . . . . . 1234.8 Sample Match Matrix: K-means fixed . . . . . . . . . . . . . . . . . . . . . . . . 1274.9 Sample Match Matrix: Ward . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1274.10 Cluster Similarity Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1284.11 Response Rate by Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1284.12 Sample Notes Recorded in CRS System . . . . . . . . . . . . . . . . . . . . . . . 1304.13 Proportion of Different Themes in Reminder Notes . . . . . . . . . . . . . . . . 1304.14 Proportion by Disease Type Treated by Usage Groups . . . . . . . . . . . . . . . 1314.15 Response Rate by Usage Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . 131

5.1 User Satisfaction Survey Results . . . . . . . . . . . . . . . . . . . . . . . . . . . 142

6.1 Summaries of TAM Studies on Physicians’ IT Acceptance . . . . . . . . . . . . . 1686.7 Revised TAM Scales . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1776.8 Operationalization of Variables and Labels . . . . . . . . . . . . . . . . . . . . . 1786.9 Group Composition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1806.10 Summary of Actual Usage Measures . . . . . . . . . . . . . . . . . . . . . . . . . 1806.11 Factor Loadings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1816.12 Correlation Matrices of Main Model Constructs . . . . . . . . . . . . . . . . . . . 1826.13 Regression Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1836.14 Multinomial Logit Regression on Usage Group Membership . . . . . . . . . . . 1846.15 Results of Hypothesis Testing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 185

7.1 The Study Site and Usage of CRS . . . . . . . . . . . . . . . . . . . . . . . . . . . 1967.2 Major Features and Overall Frequency of Access . . . . . . . . . . . . . . . . . . 1997.3 Maximal Sequential Pattern Discovered . . . . . . . . . . . . . . . . . . . . . . . 200

LIST OF TABLES 14

7.4 Recurring Patterns within Encounters . . . . . . . . . . . . . . . . . . . . . . . . 2007.5 Markov Chain Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2017.6 Markov Chain Transition Matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . 205

8.3 QAP Network Correlation Test Results . . . . . . . . . . . . . . . . . . . . . . . . 2138.4 Network Effects Model for Direct Tie Networks . . . . . . . . . . . . . . . . . . 2158.5 Network Effects Model for Structural Equivalence . . . . . . . . . . . . . . . . . 2168.6 Structural Equivalence in Friendship Network Segments . . . . . . . . . . . . . 2178.7 Impact of Attending Physicians’ Usage . . . . . . . . . . . . . . . . . . . . . . . 218

9.1 Preventive Care Measures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2289.2 Chronic Disease Management Patient Outcome Measures . . . . . . . . . . . . . 229

List of Figures

2.1 Sample Decision Tree: Asthma Guideline . . . . . . . . . . . . . . . . . . . . . . 352.2 Steroid-Induced Osteoporosis Guideline . . . . . . . . . . . . . . . . . . . . . . . 372.3 Data Model of CRS v1.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 402.4 Data Model of CRS v2.0 and 3.0 . . . . . . . . . . . . . . . . . . . . . . . . . . . . 412.5 A Sample Data Flow Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 422.6 A Sample HL7 Message . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 452.7 User Interface of the Prototype System . . . . . . . . . . . . . . . . . . . . . . . . 512.8 Version 1: Physician Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 522.9 Version 1: Staff Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 532.10 Version 2: User Interface of The Web Prototype . . . . . . . . . . . . . . . . . . . 542.11 Version 3: Prototype . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 552.12 Version 3: Login Window and New Patient Registration . . . . . . . . . . . . . . 562.13 Version 3: Patient Search and New Appointment Scheduling . . . . . . . . . . . 572.14 Version 3: Recording Vital Signs . . . . . . . . . . . . . . . . . . . . . . . . . . . . 582.15 Version 3: Main Physician Work Space . . . . . . . . . . . . . . . . . . . . . . . . 592.16 Version 3: Diagnosis and Medication Data Entry . . . . . . . . . . . . . . . . . . 60

3.1 Sample Guideline Diagram Modeled with GLIF3 . . . . . . . . . . . . . . . . . . 763.2 Sample Guideline Diagram Modeled with PROforma . . . . . . . . . . . . . . . 823.3 On the Fly Reminders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 873.4 Sample Preliminary Flowchart . . . . . . . . . . . . . . . . . . . . . . . . . . . . 893.5 Sample Worksheet for Medical Terminology Mapping . . . . . . . . . . . . . . . 903.6 Diagram of Traversed Pathway Generated . . . . . . . . . . . . . . . . . . . . . . 923.7 Use of Sub Plans and Connecting Nodes . . . . . . . . . . . . . . . . . . . . . . . 973.8 Inference Engine Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

4.1 Aggregated System Usage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1164.2 Simple Grouping Scenario . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1164.3 Sample Raw Usage Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1174.4 Best Fit Model: Three Distinct Groups . . . . . . . . . . . . . . . . . . . . . . . . 1184.5 Developmental Trajectories of Two Groups . . . . . . . . . . . . . . . . . . . . . 1194.6 Developmental Trajectories of Four Groups . . . . . . . . . . . . . . . . . . . . . 1194.7 Mean Group Membership Probabilities for Categorical Variables . . . . . . . . . 1214.8 Membership Probability as a Function of Computer Literacy . . . . . . . . . . . 1244.9 Trajectories based on DS10 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1254.10 Trajectories based on DS5 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1254.11 Rate by Response Type Across Time (All Users) . . . . . . . . . . . . . . . . . . . 1294.12 Favorable Response Rate by Usage Group . . . . . . . . . . . . . . . . . . . . . . 1324.13 Skipped Rate by Usage Group . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1324.14 Unavorable Rate by Usage Group . . . . . . . . . . . . . . . . . . . . . . . . . . . 133

6.1 Theory of Reasoned Action and Theory of Planned Behavior . . . . . . . . . . . 151

LIST OF FIGURES 16

6.2 Original Technology Acceptance Model . . . . . . . . . . . . . . . . . . . . . . . 1536.3 Revised Technology Acceptance Model . . . . . . . . . . . . . . . . . . . . . . . 1536.4 Technology Acceptance Model 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . 1576.5 Unified Theory of Acceptance and Use of Technology . . . . . . . . . . . . . . . 1586.6 Best Fit Model: Three Distinct Groups . . . . . . . . . . . . . . . . . . . . . . . . 1796.7 Results of the New Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 186

7.1 Example of User Interface Usability Testing Experiment . . . . . . . . . . . . . . 1917.2 Screenshot of the Pilot User Interface . . . . . . . . . . . . . . . . . . . . . . . . . 1937.3 Screenshot of the New Web Interface . . . . . . . . . . . . . . . . . . . . . . . . . 1947.4 Distribution of Event Sequence Length . . . . . . . . . . . . . . . . . . . . . . . . 1977.5 Usage Distribution among Users . . . . . . . . . . . . . . . . . . . . . . . . . . . 1987.6 Markov Chain of Traverse Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2027.7 Feature Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 204

8.1 Usage Distribution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2138.2 Professional Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2148.3 Friendship Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2148.4 Perceived Influence Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2148.5 Structural Equivalence Dendrogram (Professional Network) . . . . . . . . . . . 214

CHAPTER 1

Clinical Decision Support Systems

This chapter begins with an introduction to the health informatics field, the con-cept and applications of clinical decision support systems, and the evidence-basedmedicine principle that forms the foundation of the Clinical Reminder System pre-sented in this thesis. In the second part I analyze barriers that may have inhibitedwidespread diffusion of decision support technologies into healthcare, followed byan elaboration on the five objectives of this research: a) to design a computational on-tology that enables structured acquisition and automated execution of evidence-basedguidelines; b) to elicit best practices for integrating decision support technologies withheterogeneous healthcare information systems; c) to develop effective system designand implementation strategies that allow computer applications work seamlessly asan integral part of routine clinical workflow; d) to examine and identify solutions toissues arising from user resistance by healthcare professionals; and e) to propose soft-ware design patterns for building effective healthcare application user interface. Atthe end of this chapter I present an overview of the thesis.

1.1. INTRODUCTION 18

“Despite many years of research and millions of dollars of expenditures onmedical diagnostic systems, none is in widespread use at the present time.”

– Miller, 1991 [136]

1.1 Introduction

Clinical decision support systems (CDSSs), with the potential to minimize practicevariation and improve patient care, have begun to surface throughout the healthcareindustry [214]. Clinical decision support methodologies and applications form a cor-nerstone of the health informatics field, which is the study and application of methodsto improve the management of patient information, clinical knowledge, populationdata, and other information relevant to patient care and community health [216]. Itis an integrative discipline that arises from the synergistic application of computa-tional, informational, cognitive, organizational, and other sciences whose primary fo-cus is the acquisition, storage, and use of information in the health and biomedicaldomain [95, 215].

As health care costs continue to spiral upward, healthcare institutions are underenormous pressure to create a cost-effective system by controlling operating costswhile maintaining quality of care and services. Health informatics applications, pow-ered by cutting-edge information technologies, provide considerable promise for achiev-ing this goal through managing information, reducing costs, and facilitating totalquality management and continuous quality improvement programs. While devel-opment and acquisition of these applications has traditionally consumed a less signif-icant portion of capital budgets, this share is sure to rise as pressures from governmentbodies and insurers combine to make the use of such applications mandatory in theforeseeable future.

Doctors are increasingly transitioning from a 1-to-1 view of medical practice inwhich the “fee for service” model has dominated, to a 1-to-n, population-based view.Bodenheimer (1999) notes that the trends towards disease management have placedgreater emphasis on a smaller, sicker population rather than on the larger populationwith chronic illnesses at lower immediate health risk [28]. Greenlick (1992) outlinesa population-based clinical practice model that addresses allocation of appropriatemedical resources to perform outreach to populations that are underserved by clinicalpractices [88]. He further argues that special education in population-based medicinewill be necessary for the many physicians who work in large organizations [89].

Health maintenance organizations, medical researchers, and health practitionersrealize that in an environment that rewards short-term cost savings and emphasizesmanagement of current diseases, increased use of personalized reminders can signifi-cantly improve delivery of preventive care services and treatment for chronic illnesses.UnitedHealth Group, a large health maintenance organization, has found that use ofbilling information to design patient reminder treatment strategies for patients withdiabetes, heart-attack survivors and women at risk for breast cancer has increased the


rate of utilization of simple interventions in all of these areas [37]. Leninger, et al.(1996) presents a methodology for designing office information systems to improvedelivery of preventive care services that span written practice policies, through im-plementing a computer system and monitoring progress [121].

1.1.1 Motivation of the Study

The three trends listed above: a) the increased importance of health informatics appli-cations throughout the medical enterprise, b) the practice of population-based medicinepracticed in larger organizations, and c) the use of computer information systems toprompt physicians and patients to execute steps associated with prevention of dis-eases or treatment of chronic illnesses, reach a synthesis in a parallel stream of medicalresearch that is over thirty years old. This research domain comprises the design ofcomputer-based decision support systems to assist preventive care and disease man-agement through generation of personalized recommendations using evidence-basedmedicine, combined with individual clinical expertise, to make decisions about thecare of individual patients [163]. Such personalized recommendations provide sug-gestions regarding treatment plans to physicians who without computer assistancemight not make appropriate or timely treatment decisions.

The research represented by this thesis is therefore well-motivated, not just by thefindings listed above that make a strong case for computer-assisted, population-basedmedical practice, but also by the actual paucity of commercially-available systemsthat generate such reminders. The goal of this thesis is to design, develop, and imple-ment a computer-based decision support system that uses evidence-based medicine,in particular, protocols for application of preventive care and chronic disease treat-ment strategies, to provide solicited suggestions for physician review that allow bet-ter monitoring the treatment of patients. The long-term objective of this stream ofresearch is to evaluate the effectiveness of such reminders on physicians’ guidelinecompliance, and ultimately, the impact on patient health outcomes.

1.1.2 Clinical Decision Support Systems

Decision support systems (DSS) aid the process of decision making [73]. It is “aninteractive, flexible, and adaptable computer-based information system, especiallydeveloped for supporting the solution of a non-structured management problem forimproved decision making. It utilizes data, provides an easy-to-use interface, andallows for the decision maker’s own insights” [201]. In the past several decades in-tensive research in DSS has occurred in many areas, such as executive informationsystems, group decision support systems, organizational decision support systems,data warehousing and on-line analytical processing, and new web-based analyticalapplications.

In the medical arena, clinical decision support systems (CDSSs) are a class of “ac-tive knowledge systems which use two or more items of patient data to generate case-specific advice” [215]. Such advice takes the form of alerts and reminders, diagnostic


Table 1.1: Early Experimental Decision Support Systems

De Dombal 1972 De Dombal is a decision support tool that uses naive Bayesianapproach to provide automated reasoning under uncertainty fordiagnosis of acute abdominal pain and the need for surgery. Itwas developed at the Leeds University.

INTERNIST I 1974 INTERNIST I is a rule-based expert system for diagnosing com-plex problems in general internal medicine, developed at theUniversity of Pittsburgh. A successive commercial system, QMR,is still in use today.

MYCIN 1976 MYCIN is a rule-based expert system developed at Stanford Uni-versity. It was used to diagnose and recommend treatment forcertain blood infections and other infectious diseases.

DXplain 1984 DXplain is a decision support tool developed and maintained atMassachusetts General Hospital, which uses a modified form ofBayesian logic to produce a ranked list of diagnoses which mightexplain or be associated with the clinical manifestations.

assistance, therapy critiquing and planning, prescribing decision support, informa-tion retrieval, and image recognition and interpretation [51]. Clinical decision supportsystems have formed an increasingly significant part of the field of clinical knowledgemanagement technologies, through their capacity to support the clinical process anduse of the best known medical knowledge [215].

Research into the use of artificial intelligence in medicine produced many experi-mental systems. Table 1.1 lists several influential pioneering attempts since 1970’s. Anextensive archive of these early experimental systems and the current systems that arein routine clinical use is maintained by Open Clinical1.

1.1.3 Evidence-Based Medicine Practice

The work presented in this thesis mainly concerns a special type of CDSSs, evidenceadaptive decision support systems, which provide decision aid with a knowledge basethat is constructed from and continually adapts to new research based and prac-tice based evidence, or evidence-based medicine [180]. According to Sackett et al.(1996), evidence-based medicine is the “conscientious, explicit, and judicious use ofcurrent best evidence in making medical decisions about the care of individual pa-tients” [163]. It has been a general consensus that continuous, comprehensive practiceof evidence-based medicine has tremendous potential to improve quality of care andreduce practice variation [164]. However, there is also a widely acknowledged gapbetween physicians’ awareness of these care standards and physicians’ consistent ap-plication of them in practice. Solberg, et al. (1998) has studied organized processes to

1URL: http://www.openclinical.org.

http://www.openclinical.org


Table 1.2: History of the Clinical Reminder System Project

1999–2000 A prototype system was created as a result of two masters project coursesin the H. John Heinz III School of Public Policy and Management atCarnegie Mellon University. The prototype demonstrated the capacity ofusing computer systems to generate physician-directed reminders at thepoint of care.

2000–2004 I developed the first production version of the CRS from 2000 to 2002. Theimplementation took place in February 2002 in the Medical AmbulatoryCare Clinic (MACC) at the Western Pennsylvania Hospital. The MACC is aprimary care clinic that serves as a rotation site for the hospital’s residents.

2004–present A new, web-enabled version of the CRS was developed from 2004 to 2005,based on the research findings and experiences learned from the earlierMACC implementation. The new system was installed in August 2005 inthe West Penn Medical Associates clinic, which replaced the MACC in theWestern Pennsylvania Hospital during an organization restructuring.

provide preventive care services in primary care clinics and finds that most clinics didnot have use a complete suite of preventive care processes [183]. McGlynn et al. (2003)assess 439 indicators of quality of care for 30 acute and chronic conditions. They findthat that adults in the United States receive only about half of recommended care, andthe deficits in adherence to recommended processes for basic care pose serious threatsto the health of the American public [134].

Clinical decision support systems have been considered as a viable means to im-prove physicians’ compliance of the evidence-based medicine practices. The use ofCDSSs to facilitate the practice of evidence-based medicine promises to substantiallyimprove health care quality [181], and beneficial outcomes have been reported alonga number of dimensions, including compliance with treatment standards, reducedtreatment costs, and improved population health outcomes [57, 71, 133, 147, 174, 199].However, to date widespread use of clinical decision support systems that supportevidence-based medicine practice has not yet become available due to numerous tech-nological, behavioral, and organizational barriers. In this chapter I identify these bar-riers and propose research studies to elicit solutions to address them.

1.1.4 The Clinical Reminder System Project

Over the past several years, researchers in the H. John Heinz III School of Public Policyand Management at Carnegie Mellon University and practitioners from the WesternPennsylvania Hospital have worked together to develop a clinical decision supportsystem, called Clinical Reminder System (CRS), that aims to improve quality of careby providing clinicians just-in-time alerts and recommended actions using evidence-based medicine guidelines. Since its launch in 1999, this project has undergone three

1.2. STUDY OBJECTIVES 22

major phases, listed in Table 1.2.The current version of CRS is a web-based, “lite” electronic medical record system

that implements evidence-based medical guidelines to generate physician directedreminders at the point of care. In order to generate such reminders, CRS stores orretrieves in real time from many other hospital information systems a wide varietyof patient information including demographics, laboratory test results, disease diag-noses, and vital signs recorded during medical staff encounters. The reminders gen-erated by CRS take the form of recommendations to have certain tests performed, toreceive vaccinations, or to discuss the pros and cons of alternative treatments. At thepresent time CRS is intended to improve medical practice of four chronic diseases:asthma, diabetes, hypertension, and hyperlipidemia; and five preventive care cate-gories: breast cancer, cervical cancer, influenza, pneumonia, and steroid-induced os-teoporosis. Three additional chronic diseases management guidelines: cardiovasculardiseases, lower back pain, and osteoporosis, are currently under development.

Since February 2002 the system has been used in routine practice in the MedicalAmbulatory Care Clinic (2002–2003) and the West Penn Medical Associates (2003–present) at the Western Pennsylvania Hospital. The clinical staff use CRS for manag-ing appointments, patient check-in and check-out, and recording of vital signs. Theresidents and attending physicians use the system for documenting clinical obser-vations, prescribing new orders, and generating reminders to aid in their decision-making during patient encounters.

1.2 Study Objectives

Although clinical decision support systems provide considerable potential to improvepatient care and reduce practice variation, many implementations failed due to un-foreseen costs, unfulfilled promises, and disillusionment [12]. As Classen (1998) notewhile glowing predictions about the all-encompassing and beneficial role of decisionsupport systems in medicine have appeared with increasing frequency in the scien-tific literature since the 1970’s, this optimistic vision has not yet been realized almost25 years later [50].

Coiera (2003) summarize four causes for the failure of clinical decision supportsystems to be used clinically: 1) dependence on an electronic medical record systemto supply their data, 2) poor human interface design, 3) failure to fit naturally into theroutine process of care, and 4) reluctance or computer illiteracy of some healthcareworkers [51]. Kawamoto et al. (2005) also identify several key features that are criti-cal to CDSSs’ success: 1) automatic provision of decision support as part of clinicianworkflow, 2) provision of recommendations rather than just assessments, 3) provi-sion of decision support at the time and location of decision making, and 4) computerbased decision support [110].

The design of the present study is inspired by these findings and the early resultsfrom a series of field implementation attempts of CRS. Accordingly I structure theresearch studies constituting this thesis to address issues and challenges that arise


from each of the following five facets:

a. Medical Knowledge Engineering;

b. Integration with Heterogeneous Healthcare Information Systems;

c. Integration of Computer Applications into Routine Clinical Workflow;

d. IT Adoption and Acceptance by Healthcare Professionals;

e. Designing an Effective Healthcare Application User Interface.

1.2.1 Medical Knowledge Engineering

Sackett et al. (1996) define the concept of evidence-based medicine and note thatevidence-based medicine relies on large amount of data from a variety of practiceareas, generated using a number of research designs and must be combined with indi-vidual clinical expertise [163]. As a result, “medical knowledge engineering” becomesan important component of the Clinical Reminder System, and a challenge of devel-oping and extending CRS will be the extent to which embedded treatment algorithmsmay be modified to reflect changes in current best medicine practices. In responseto this challenge I design a computational ontology that enables structured acquisi-tion and automated execution of evidence-based medicine guidelines. This ontologymodel is presented in Chapter 3—Guideline Representation and Execution Model:Ontology and Tools.

1.2.2 Integration with Healthcare Information Systems

Clinical decision support systems rely on other healthcare information systems to op-erate. In particular, they must be integrated with electronic medical record (EMR)systems to acquire critical patient care data for effective decision-making. Unfortu-nately despite increasing demand the EMR adoption rate remains low in the UniteStates. A recent study surveys more than 3,300 medical group practices. It finds thatin 2005 less than 15% of healthcare institutions has fully functional EMR implemented,and this rate is even lower for smaller practices2. A 2005 CDC report reveals similarresult: only one in ten of the physicians surveyed is considered to be using an EMRthat meets minimal functional requirements [35].

The Western Pennsylvania Hospital where the Clinical Reminder System is de-ployed does not currently have a fully functional EMR. As a result CRS has evolvedinto a “lite” EMR over time that provides necessary functionalities for managing clinicworkflow and recording patient health conditions. Researchers have worked closelywith the IT department of the Western Pennsylvania Hospital to create interfaces thatallow CRS to retrieve patient data in real time from a variety of sources—test resultsfrom a laboratory information system, diagnoses from an electronic billing system,

2URL: http://www.mgma.com/press/EHR-adoptionstudy.cfm.

http://www.mgma.com/press/EHR-adoptionstudy.cfm


and patient demographic data from a hospital-wide ADT application. CRS’ EMR ar-chitecture and connectivity to other healthcare information systems are documentedin Chapter 2—The Clinical Reminder System: A Clinical Decision Support System forEvidence-Based Medicine Practice.

1.2.3 Integration into Routine Clinical Workflow

As Kawamoto et al. (2005) note a key feature critical to CDSSs’ success is automaticprovision of decision support as part of clinician workflow. They urge that givenits close correlation to successful outcomes (P < .00001) this feature should be im-plemented if at all possible [110]. CRS is carefully designed to provide automatedreminding at the point of care, however, ensuring its use during busy patient encoun-ters is found to be a real challenge.

In designing and implementing the Clinical Reminder System several precautionsand user demanded enhancements have taken place to ensure a seamless integrationof CRS with routine clinical workflow. Intensive training was provided to the nursingand clerical staff prior to introducing the system into examination rooms. This trainingassured that resident and attending physician users receive adequate administrativeand data entry assistance from their support colleagues. In its application and userinterface design, CRS also allows enough flexibility when system designers’ expectedworkflow may not be met. For example although it is highly recommended to recordclinical observations during a patient encounter, CRS accepts late documentation upto 24 hours given during busy sessions a user may not be able to complete all neces-sary data entry. CRS also allows clinicians to start a patient encounter with missingvital signs readings, which occurs often when the clinic’s nursing staff is operating intheir full capacity. Finally based on user feedback a “Quick Switch” function is pro-vided that enables switching user identity while preserving the current workspace.This feature is particularly welcomed when attending physicians need to use the sys-tem to audit residents’ practices, and residents need to log back into the system againto work on what attending physicians may have just suggested.

The workflow flexibility allowed by CRS provides a solution to minimizing thedisturbance to the routine workflow by the use of the system; however this is a com-promised solution that may introduce new problems, such as inaccurate remindersbeing generated due to missing vital signs data. Unfortunately this research, limitedby its scope, cannot identify a full solution to solve all the issues arising from workflowintegration. It will be a future research topic to be closely studied in implementing thenext generation of CRS.

It is worth noting that, despite the precautions taken and enhanced features addedduring the implementation, the workflow integration issue remains a top complaintby the CRS users. For example two common themes emerge from the qualitative datacollected in post-implementation user satisfaction surveys: the presence of computersystems in an examining room incurs a) decreased clinicians’ productivity and b) di-minished quality of physician-patient interaction. The qualitative results as reportedin Chapter 5—User Acceptance: Qualitative Assessments.


1.2.4 IT Adoption and Acceptance by Healthcare Professionals

Information systems not improve performance if they are not used (e.g., Davis et al.,1989). It have also been well recognized that end users are often unwilling to useavailable systems that, if used, would generate significant performance gains [192,178, 146]. There is also growing realization that professionals who develop, imple-ment, and evaluate information systems such as CDSSs, frequently address only thetechnical aspects of these systems, whereas the success of implementation and utiliza-tion depends on integration of the computer systems into a complex organizationalsetting [12].

Unfortunately, most of the existing CDSS evaluation literature either focuses onaccuracy and relevance of the computer-generated recommendations, or uses exper-imental designs to assess system or clinical performance [107]. Few involve a natu-ralistic design in routine settings with real patients [106]. As a result it is not clearwhether a CDSS that has been shown to be effective in a laboratory setting will befully utilized by its end-users in clinical environments.

In this thesis I use four distinct research designs, each focusing on a different di-mension, to study the IT adoption and acceptance behavior by healthcare workers3.I use a developmental trajectory analysis (DTA) approach to delineate the CRS users’longitudinal, evolving IT adoption and acceptance behavior (Chapter 4), a qualitativeassessment to explain the behavior revealed by DTA (Chapter 5), a study based on theTechnology Acceptance Model, a widely used theory in the information systems field,to find antecedents that determine the usage of CRS (Chapter 6), and finally a socialnetwork study to examine technology adoption and acceptance in the social contextwhere the users are situated (Chpater 8). The methods and findings of these four stud-ies may provide methodological and practical insights into design, implementation,and evaluation issues of other health informatics applications, as well as informationsystems more generally.

1.2.5 Designing an Effective User Interface

Medical practice is a complex process. Large amounts of data must be accessed andanalyzed at the point of care to inform proper medical decision-making. Using paper-based patient records, clinicians flip through stacks of paper charts to look for desiredinformation. Using electronic systems largely facilitates this information retrievingprocess, however, they introduce new problems. Two paper documents, for instance,can be laid out side by side for cross reference, while on a computer screen it is usuallyimpractical to have two windows both visible at the same time. How to preservethe easy “look-and-feel” of paper charts while constrained by available displayingcapacity (e.g., size and resolution of a computer screen) is a real challenge for healthapplication designers. Unfortunately, except a handful user interface (UI) usabilitytesting studies conducted on handheld devices, very little attention has been paid in

3In this thesis I usually use the word adoption to describe the behavior of “initiative to start using anIT”, and acceptance as “incorporating an adopted IT into a user’s everyday practice”.

1.3. STRUCTURE OF THIS THESIS 26

the health informatics community to designing elegant and effective user interfaces.Consequently, systems are created ad hoc, users are dissatisfied, and often systems areabandoned [104].

In CRS I implement a set of UI usability monitoring procedures. These procedurescapture the users’ navigation path across different UI elements as temporal event se-quence. With the availability of actual usage data, I design a pattern recognition thatuses sequential pattern analysis and first order Markov chain model to discover fre-quent, reoccurring traversed pathways. The results inform a sequential design patternfor software reengineering, which calibrates the user interface layout of CRS so thatthe within-application workflow is aligned with physicians’ mental model in medicalproblem-solving. This study is presented in Chapter 7—User Acceptance: A DesignPattern for Interface Optimization.

1.3 Structure of this Thesis

This introductory chapter presents the background of clinical decision support sys-tems, barriers that inhibit their widespread use, and the accordingly designed researchstudies. An executive summary of Chapter 2 through Chapter 9 is provided below.

Chapter 2. THE CLINICAL REMINDER SYSTEM: A CLINICAL DECISION SUPPORTSYSTEM FOR EVIDENCE-BASED MEDICINE PRACTICE (page 30)

BRIEF: This chapter documents the technical aspects and the implementation processof the Clinical Reminder System, including its functional requirements, system ar-chitecture, database schema, and interfacing strategies with heterogeneous healthcareinformation systems.

RESULTS: Since February 2002, CRS has been successfully deployed and used in rout-ing practice in two ambulatory primary care clinics at the Western Pennsylvania Hos-pital. Preliminary evaluation results show that the system has great potential to in-crease physician guideline compliance along with beneficial outcomes on many otherdimensions.

Chapter 3. GUIDELINE REPRESENTATION AND EXCHANGE MODEL: ONTOLOGYAND TOOLS (page 66)

BRIEF: This chapter presents the Guideline Representation and Execution Model (GREM).GREM is a computational ontology that enables structured acquisition and automatedexecution of evidence-based medicine guidelines. It also provides the promise of shar-ing guideline representations in CRS with other clinical decision support systems.

METHOD: The ontology model is developed based on several existing models and theexperience in implementing six chronic disease management and five preventive care


procedure guidelines in CRS.

FINDINGS: Comparing to other ontology models, the Guideline Representation andExecution Model provides a practical solution for creation, execution, and dissem-ination of evidence-based medicine guidelines. It is especially suitable for model-ing chronic disease management and preventive care procedure guidelines. GREMhas been used to support several guideline-based, reminder-generating algorithms inCRS.

Chapter 4. USER ACCEPTANCE: A DEVELOPMENTAL TRAJECTORY ANALYSIS AP-PROACH (page 106)

BRIEF: This chapter examines the CRS users’ longitudinal, evolving adoption and ac-ceptance behavior.

METHOD: 10-month actual usage data generated by 41 users are analyzed using de-velopmental trajectory analysis. The users’ computer literacy is assessed using Cork’sinstrument for measuring physicians’ use of, knowledge about, and attitudes towardscomputers [55].

FINDINGS: Three categories of users are identified who demonstrate distinct usagebehavior: ever-increasing (“Heavy”), ever-decreasing (“Moderate”), and always-low(“Light”). Multinomial logit analysis further finds users with higher computer opti-mism are more likely “Heavy” users, whereas users with better computer knowledgehave an increased probability of being “Light” users.

Chapter 5. USER ACCEPTANCE: QUALITATIVE ASSESSMENTS (page 139)

BRIEF: This chapter uses qualitative assessments to explain the adoption and accep-tance behavior observed in the previous chapter.

METHOD: 16 residents participated in onsite interviews and 37 residents returnedquestionnaire surveys (29 valid responses were received). In both instruments rep-resentatives from each of the three usage groups were present. The data are analyzedusing constant comparative method.

FINDINGS: Users of different usage groups have distinct perception about the system.Across the entire user population five common themes emerge: iterative advisories,heavy data entry duty, detrimental to efficiency, diminished quality of physician-patient communication, and lack of guidance in user interface.

Chapter 6. USER ACCEPTANCE: A REVISED TECHNOLOGY ACCEPTANCE MODEL(page 148)


BRIEF: Synthesizing the findings of Chapter 4 and Chapter 5, this chapter critiques andextends the Technology Acceptance Model (TAM), a widely used theory for studyingtechnology diffusion.

METHOD: Data collected in Chapter 4 are reanalyzed using the TAM model. The re-sults are compared to the findings of a new model that accommodates longitudinal,developmental usage behavior.

FINDINGS: TAM is poor in predicting IT adoption and acceptance behavior over time.The self-reported usage measure of TAM does not relate to the actual usage recordedin the system. Based on these findings this chapter proposes a new conceptualizationof IT adoption and acceptance by introducing two new constructs: saturated usageand developmental pattern.

Chapter 7. USER ACCEPTANCE: A DESIGN PATTERN FOR INTERFACE OPTIMIZA-TION (page 189)

BRIEF: This chapter uses data mining techniques to discover reoccurring navigationpatterns within CRS. The goal is to identify an optimal sequential order to presentdifferent clinical information elements on a computer screen.

METHOD: Temporal event sequences constructed from 10-month actual usage dataare analyzed using sequential pattern analysis and first order Markov chain model.

FINDINGS: Among the 17 main features of CRS, “History of Present Illness” → “SocialHistory” → “Assessment and Plan” is the most frequented navigation path. Severallocal consecutive patterns are also discovered. These findings lead to a software designpattern for building more effective healthcare application user interfaces.

Chapter 8. USER ACCEPTANCE: SOCIAL CONTAGION AND IT ADOPTION (page 206)

BRIEF: This chapter takes into account the clinic’s social context in which the CRSusers are situated. The objective is to study the influence of social contagion on theusage level of the system.

METHOD: Social contagion for technology adoption is modeled using social networkanalysis approach. Resident users of CRS were asked to report their relationships tothe other clinicians in the clinic. The survey instrument reveals three major socialstructures in the clinic that may be relevant to technology adoption: professional, per-sonal, and network based on perceived influence from others on a user’s intention touse CRS.


FINDINGS: Structural equivalence of the friendship network, particularly within thesegment between residents and attending physicians, is a strong indicator of the resi-dent users’ recorded usage levels. None of the three networks composed of direct tiesare found to have significant impact. Self-reported social influence or subjective normis therefore a poor predictor of actual usage. Several local factors including computerknowledge, computer optimism, and perceived ease of use of CRS are also found toinfluence usage.

Chapter 9. FUTURE WORK AND LONG TERM OBJECTIVES (page 224)

BRIEF: This concluding chapter describes future research extensions and long-termplans for evaluating the system’s effectiveness on physician guideline compliance andpatient health outcomes.

CHAPTER 2

The Clinical Reminder System: A ClinicalDecision Support System for

Evidence-Based Medicine Practice1

The Clinical Reminder System is a web-based, “lite” electronic medical record systemthat implements evidence-based medical guidelines to generate physician directed re-minders at the point of care. Since February 2002, CRS has been successfully deployedand used in routine practice in two ambulatory primary care clinics at the WesternPennsylvania Hospital. This chapter documents the technical aspects and the imple-mentation process of CRS, including its functional requirements, system architecture,database schema, and interfacing strategies with heterogeneous healthcare informa-tion systems. A brief user manual, screenshots of main features, key data dictionaries,and sample usage reports are provided in the Appendix.

1This chapter is based on Zheng K., Padman R., Johnson M.P., Engberg J.B., and Diamond H.S. (2005),Clinical Reminder System: a relational database application for evidence-based medicine practice, HeinzSchool Working Paper Series 2005–30 [221].

CHAPTER 3

Guideline Representation and ExecutionModel: Ontology and Tools

A guideline representation ontology is a specification of conceptualizations that con-stitute the practice of evidence-based medicine. It represents the elements of a clinicalguideline by specifying its attributes and defining the relationships that hold amongthem. A rigorously defined computational ontology also provides promise of produc-ing computable representations that can be visualized, edited, executed, and sharedusing computer-based systems. A uniformly acknowledged ontology, or standardrepresentation schema, is the key to facilitating the dissemination of guidelines acrosscomputer systems and healthcare institutions.

I begin the chapter by describing the evolution of ontology research in clinicalguideline representation. I review and discuss several representative ontologies within-depth analyses of two popular models: GLIF (Guideline Interchange Format) andPROforma. The second part of the chapter presents a computational ontology, theGuide Representation and Execution Model (GREM). I show that with distinctivemethodologies GREM successfully achieves a simple yet comprehensive represen-tation of guidelines, with a minimal set of constructs. Comparing to other existingtechnologies the GREM provides a more practical solution for creation, execution,and dissemination of evidence-based clinical practice guidelines using clinical deci-sion support systems.

CHAPTER 4

User Acceptance: A DevelopmentalTrajectory Analysis Approach1

In this chapter, I assess user acceptance and adoption of the Clinical Reminder Systemthat is implemented in an ambulatory care environment. I use a novel developmentaltrajectory approach for data clustering. This group-based, semi-parametric statisti-cal modeling method identifies distinct groups, following distinct usage trajectories,among those who recorded use of the reminder system during a 10-month evalua-tion period. System use was traced within these groups over time using computer-generated logs and user satisfaction surveys. The trajectory analysis delineates threecategories of users who demonstrate distinctive adoption behavior. The membershipprobabilities of these categories can be further related to clinicians’ demographics andcomputer literacy backgrounds. These user categories are also correlated with re-minder compliance, and these reminders have triggered follow-up actions that couldbe otherwise missed. I conclude that this developmental trajectory approach has con-siderable promise to provide new insights into system usability and technology adop-tion issues that may benefit CDSSs as well as information systems more generally.

1This chapter is based on a) Zheng, K., Padman, R., Johnson, M.P., Engberg, J.B., and Diamond, H.S.(2004). An adoption study of a clinical reminder system in ambulatory care using a developmentaltrajectory approach. Medinfo 2004, pages 1115–1120 [220]; and b) Zheng K., Padman R., and JohnsonM.P. (2005). User acceptance and adoption of a clinical reminder system: interpreting the effects ofuser characteristics, In Trauth E.M. et al., editor, Encyclopedia of Gender and Information Technology,Hershey. Idea Group Publishing [217].

CHAPTER 5

User Acceptance: Qualitative Assessments1

Evaluation studies of clinical decision support systems (CDSS) have tended to focuson assessments of system quality and clinical performance in a laboratory setting. Rel-atively few studies have used field trials to determine if CDSSs are likely to be usedin routine clinical settings and whether reminders generated are likely to be actedupon by end-users. Moreover, such studies when performed tend not to identify dis-tinct user groups, nor to classify user feedback. In Chapter 4 I presented a group-based, semi-parametric statistical modeling method to identify distinct groups, withdistinct usage trajectories. In this chapter I use qualitative instruments of usabilityand satisfaction surveys and structured interviews to validate insights derived fromusage trajectories, and to provide explanations to the observed behavior. Quantita-tive analysis delineates three types of user adoption behavior: “Light”, “Moderate”and “Heavy” usage. Qualitative analysis reveals that clinicians of distinct types tendto exhibit views of the system consistent with their demonstrated adoption behavior.Drawbacks in the design of the CDSSs identified by users of all types (in differentways) motivate a redesign based on current clinician workflows. I conclude that thismixed methodology has considerable promise to provide new insights into system us-ability and adoption issues that may benefit clinical decision support systems as wellas information systems more generally.

1This chapter is based on Zheng K., Padman R., Johnson M.P., and Diamond H.S. (2005). Understand-ing technology adoption in clinical care: clinician adoption behavior of a point-of-care reminder system.International Journal of Medical Informatics, 74(7–8):535–543 [219].

CHAPTER 6

User Acceptance: A Revised TechnologyAcceptance Model

In order to better predict, explain, and increase the usage of IT, it is of vital importanceto understand the antecedents of end users’ IT adoption decisions. This chapter firstreviews the theoretical background of intention models that have been widely used tostudy factors governing IT acceptance, with particular focus on the technology accep-tance model (TAM)—a prevalent technology adoption theory in the area of informa-tion system research. Although TAM has been extensively tested and shown to be arobust, powerful, and parsimonious model, its limitations have also been recognized.The second part of this chapter analyzes these limitations and discusses possible pre-cautions of potential pitfalls. The third part of this chapter specifically addresses theapplicability of the technology acceptance model in the professional context of physi-cians, with a review of available studies that have applied TAM to the technologyadoption issues in healthcare. TAM is then tested in an empirical study using lon-gitudinal, computer-recorded usage data generated by physicians’ use of a clinicaldecision support system used. The results show that while TAM has been extensivelyapplied in this stream of research with proven validity and power, it has limited ca-pacity to explain actual acceptance behavior over time. Using these insights, I extendTAM to accommodate actual, longitudinal usage metrics.

CHAPTER 7

User Acceptance: A Design Pattern forInterface Optimization

Many information technology-enabled healthcare applications have failed becausetheir interfaces are difficult to use. Unfortunately, little attention has been paid inthe health informatics community to designing effective user interfaces that are ac-ceptable to healthcare professionals. This chapter illustrates a method for improvingapplication interface usability by applying sequential pattern analysis to analyze tem-poral event sequences recorded in an electronic medical record system. Such event se-quences, or clickstreams, reflect clinicians’ navigation patterns in their everyday inter-actions with the computer system. The identified patterns have been used by softwaredevelopers to calibrate the user interface of the system, so that the within-applicationworkflow is better aligned with clinicians’ mental model of medical problem-solving.Such inferred patterns may also help to modify clinicians’ suboptimal practice behav-ior components, as manifested through their actual usage of this point-of-care elec-tronic system.

CHAPTER 8

User Acceptance: Social Contagion and ITAdoption

This chapter applies social network analysis to studying technology adoption behav-ior by healthcare professionals. The empirical study was conducted in an ambulatory,primary care clinic where the clinicians use a clinical decision support system to treatpatients. A survey instrument was developed assessing three network structures inthe clinic that may be relevant to technology adoption: 1) network based on consulta-tion of patient-care related matters (professional network); 2) network based on per-sonal proximity (friendship network); and 3) network based on perceived influencefrom others on a user’s intention to use the system (perceived influence network).Strong evidence of social contagion is found in the structural equivalence derivedfrom the friendship network, particularly within the segment between resident usersand their attending physicians. None of the cohesion networks is found to have sig-nificant impact on usage.

CHAPTER 9

Future Work and Long Term Objectives

This chapter outlines the future plans of the Clinical Reminder System project. I fo-cus on three major areas: a) continuous development of CRS and future expansions;b) research extensions, in particular, a follow-up social network study that examinesthe formation and evolution of social networks among health professionals; and c)long-term plans for evaluating the system’s effectiveness on physician guideline com-pliance and patient health outcome.

References

[1] D. A. ADAMS, R. R. NELSON, AND P. A. TODD, Perceived usefulness, ease of use, and usage of infor-mation technology: A replication, MIS Quarterly, 16 (1992), pp. 227–247. 154

[2] R. AGARWAL, Individual adoption of new information technology, in Framing the Domains of IT Man-agement Research: Glimpsing the Future through the Past, R. W. Zmud, ed., Pinnaflex, 2000,pp. 85–104. 172

[3] R. AGARWAL AND E. KARAHANNA, Time flies when you’re having fun: Cognitive absorption and beliefsabout information technology usage, MIS Quarterly, 24 (2000), pp. 665–694. 155

[4] R. AGARWAL AND J. PRASAD, The role of innovation characteristics and perceived voluntariness in theacceptance of information technologies, Decision Sciences, 28 (1997), pp. 557–582. 149, 155

[5] , Are individual differences germane to the acceptance of new information technologies?, DecisionSciences, 30 (1999), pp. 361–391. 108, 110, 155, 173

[6] I. AJZEN, From intentions to actions: a theory of planned behavior, in Springer series in social psychol-ogy, J. Kuhl and J. Beckmann, eds., Springer, Berlin, 1985, pp. 11–39. 150, 151, 164

[7] , The theory of planned behavior, Organizational Behavior and Human Decision Processes, 50(1991), pp. 179–211. 151, 164

[8] I. AJZEN AND M. FISHBEIN, Understanding Attitudes and Predicting Social Behavior, Prentice-Hall,Englewood Cliffs, NJ, 1980. 150, 172, 207

[9] D. ALBARRACIN, B. T. JOHNSON, M. FISHBEIN, AND P. A. MUELLERLEILE, Theories of reasonedaction and. planned behavior as models of condom. use: A meta-analysis, Psychological Bulletin, 127(2001), pp. 142–161. 152

[10] C. ALEXANDER, Timeless Way of Building, Oxford University Press, 1979. 190

[11] C. ALEXANDER, S. ISHIKAWA, M. SILVERSTEIN, M. JACOBSON, I. FIKSDAHL-KING, AND S. AN-GEL, A Pattern Language, Oxford University Press, 1977. 190

[12] J. G. ANDERSON, C. E. AYDIN, AND S. J. JAY, Evaluating Health Care Information Systems, SAGEPublications, Thousand Oaks, CA, 1994. 22, 25, 107, 166, 170

[13] J. G. ANDERSON, S. J. JAY, H. M. SCHWEER, M. M. ANDERSON, AND D. KASSING, Physiciancommunication networks and the adoption and utilization of computer applications in medicine, in Useand Impact of Computers in Clinical Medicine, J. G. Anderson and S. J. Jay, eds., Springer-Verlag,New York, 1987, pp. 185–199. 166

[14] C. J. ARMITAGE AND M. CONNER, Efficacy of the theory of planned behaviour: a meta-analytic review,British Journal of Social Psychology, 40 (2001), pp. 471–499. 152

[15] C. E. AYDIN, Occupational adaptation to computerized medical information systems, Journal of Healthand Social Behavior, 30 (1989), pp. 163–179. 166

[16] C. E. AYDIN AND R. E. RICE, Social worlds, individual differences and implementation: predicting atti-tudes toward a medical information system, Information & Management, 20 (1991), pp. 119–136. 145

[17] E. A. BALAS, S. M. AUSTIN, J. A. MITCHELLAND, B. G. EWIGMAN, K. D. BOPP, AND G. D.BROWN, The clinical value of computerized information services, Archives of Family Medicine, 5 (1996),pp. 271–78. 107, 140

[18] A. BANDURA, Social Foundations of Thought and Action: A Social Cognitive Theory, Prentice Hall,Englewood Cliffs, New Jersey, 1986. 150, 157

REFERENCES 238

[19] , Social cognitive theory, in Annals of Child Development Vol. 6, R. Vasta, ed., Jai Press Ltd,Greenwich, CT, 1989, pp. 1–60. 157

[20] H. BARKI AND J. HARTWICK, Measuring user participation, use involvement, and user attitude, MISQuarterly, 18 (1994), pp. 59–82. 155

[21] D. W. BATES, G. J. KUPERMAN, E. RITTENBERG, J. M. TEICH, J. FISKIO, N. MA’LUF, A. ONDER-DONK, D. WYBENGA, J. WINKELMAN, T. A. BRENNAN, A. L. KOMAROFF, AND M. TANASIJEVIC,A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests,American Journal of Medicine, 106 (1999), pp. 144–150. 191

[22] L. BECK AND I. AJZEN, ’predicting dishonest actions. using the theory of planned behavior, Journal ofResearch in Personality, 25 (1991), pp. 285–301. 152

[23] A. BEN-DOR, R. SHAMIR, AND Z. YAKHINI, Clustering gene expression patterns, Journal of Compu-tational Biology, 6 (1999), pp. 281–297. 115

[24] A. BENDIX, T. BENDIX, K. BAEGTER, C. LUND, L. FROLUND, AND L. HOLM, Multidisciplinaryintensive treatment for chronic low back pain: a randomized, prospective study, Cleveland Clinic Journalof Medicine, 63 (1996), pp. 62–69.

[25] D. BERGER, I. BORTEY, M. BROWN, C. CARTER, K. CHANDRASEKAR, R. DESAI, R. GROCE,T. JOHNSON, E. LEE, AND B. RODRIGUEZ, Physician reminder system for the practice of evidence-based medicine at western pennsylvania hospital. Pittsburgh: H. John Heinz III School of Public Policyand Management, Carnegie Mellon University, 2000. 32

[26] E. S. BERNER AND R. S. MAISIAK, Influence of case and physician characteristics on perceptions of de-cision support systems, Journal of the American Medical Informatics Association, 6 (1999), pp. 428–434. 108, 110

[27] E. BLAIR AND S. S. BURTON, Cognitive process used by survey respondents to answer to behavioralfrequency questions, Journal of Consumer Research, 14 (1987), pp. 280–288. 160

[28] T. BODENHEIMER, Disease management - promises and pitfalls, The New England Journal ofMedicine, 340 (1999), pp. 1202–1205. 18

[29] A. A. BOXWALA, M. PELEG, S. W. TU, O. OGUNYEMI, Q. T. ZENG, D. WANG, V. L. PATEL, R. A.GREENES, AND E. H. SHORTLIFFE, GLIF3: a representation format for sharable computer-interpretableclinical practice guidelines, Journal of Biomedical Informatics, 37 (2004), pp. 147–161. 75

[30] A. A. BOXWALA, S. W. TU, M. PELEG, Q. ZENG, O. OGUNYEMI, R. A. GREENES, E. H. SHORT-LIFFE, AND V. L. PATEL, Toward a representation format for sharable clinical guidelines, Journal ofBiomedical Informatics, 34 (2001), pp. 157–169. 75

[31] N. M. BRADBURN, J. HUTTENLOCHER, AND L. HEDGES, Telescoping and temporal memory, in Au-tobiographical memory and the validity of retrospective reports, N. Schwarz and S. Sudman, eds.,Springer Verlag, New York, 1994, p. 203C216. 161

[32] N. M. BRADBURN, L. J. RIPS, AND S. K. SHEVELL, Answering autobiographical questions: the impactof memory and inference on surveys, Science, 236 (1987), pp. 157–161. 161

[33] A. BRAZMA, I. JONASSEN, J. VILO, AND E. UKKONEN, Pattern discovery in biosequences, in ICGI,1998, pp. 257–270. 193

[34] R. BREIGER, S. BOORMAN, AND P. ARABIE, An algorithm for clustering relational data with applica-tions to social network analysis and comparison with multidimensional scaling, Journal of MathematicalPsychology, 12 (1975), pp. 428–438. 211

[35] C. W. BURT, E. HING, AND D. WOODWELL, Electronic medical record use by office-based physicians:United States, 2005. Division of Health Care Statistics, 2005. 23

[36] R. S. BURT, Social contagion and innovation: Cohesion versus structural equivalence, The AmericanJournal of Sociology, 92 (1987), pp. 1287–1335. 166, 170, 208, 209

[37] T. M. BURTON, UnitedHealth gets physicians to improve medical care, The Wall Street Journal, Thurs-day, August 19, (1999), p. B4. 19

REFERENCES 239

[38] C. CARLEN, H.-H. LAU, Q. SHEN, I.-S. SON, X. WU, AND H. XIA, Physician reminder system for thepractice of evidence-based medicine at western pennsylvania hospital- summer 2000 extensions. Pittsburgh:H. John Heinz III School of Public Policy and Management, Carnegie Mellon University, 2000. 32

[39] G. CARTER, M. MURRAY, R. WALKER, AND W. WALKER, Organizational Decision Support Systems,Academic Press, Inc, Boston, MA, 1992. 38

[40] P. CHANG, Y. S. HSU, Y. M. TZENG, Y. Y. SANG, I. C. HOU, AND W. F. KAO, Development ofintelligent, triage-based mass-gathering emergency. medical service PDA support systems, The Journal ofNursing Research, 12 (2004), pp. 227–236. 168

[41] P. Y. K. CHAU AND P. J. H. HU, Information technology acceptance by professionals: A model comparisonapproach, Decision Sciences, 32 (2001), pp. 699–719. 167, 168, 218

[42] , Examining a model of information technology acceptance by individual professionals: An ex-ploratory study, Journal of Management Information Systems, 18 (2002), pp. 191–229. 167, 168

[43] , Investigating healthcare professionals’ decisions on telemedicine technology acceptance: An empiri-cal test of competing theories, Information & Management, 39 (2002), pp. 297–311. 167, 168

[44] C. CHEN, M. CZERWINSKI, AND R. MACREDIE, Individual differences in virtual environments - intro-duction and overview, Journal of the American Society for Information Science, 51 (2000), pp. 499–507. 162

[45] W. W. CHIN, The measurement and meaning of IT usage: reconciling recent discrepancies between selfreported and computer recorded usage, in Proceedings of the Administrative Sciences Association ofCanada, Information Systems Division, Montreal, Quebec, Canada, 1996, pp. 65–74. 160, 184

[46] W. G. CHISMAR AND S. WILEY-PATTON, Test of the technology acceptance model for the internet inpediatrics, in Proc AMIA Symp, Hanley & Belfus, 2002, pp. 155–159. 167, 168

[47] W. G. CHISMAR AND S. WILEY-PATTON, Does the extended technology acceptance model apply to physi-cians, in HICSS ’03: Proceedings of the 36th Annual Hawaii International Conference on SystemSciences (HICSS-36), Washington, DC, USA, 2003, IEEE Computer Society, p. 160. 165, 167, 168

[48] M. CHRIST, R. KRISHNAN, D. NAGIN, R. KRAUT, AND O. GONTHER, Trajectories of individualWWW usage: implications for electronic commerce, in HICSS ’01: Proceedings of the 34th AnnualHawaii International Conference on System Sciences (HICSS-34) Volume 7, Washington, DC, USA,2001, IEEE Computer Society, p. 7066. 112, 172

[49] K. E. CLARKE AND A. AISH, An exploration of health beliefs and attitudes of smokers with vasculardisease who participate in or decline a smoking cessation program, Journal of Vascular Nursing, 20 (2002),pp. 96–105. 152

[50] D. C. CLASSEN, Clinical decision support systems to improve clinical practice and quality of care, Journalof the American Medical Informatics Association, 280 (1998), pp. 1360–1361. 22

[51] E. COIERA, The Guide to Health Informatics, Arnold, London, 2003. 20, 22

[52] J. COLEMAN, E. KATZ, AND H. MENZEL, The diffusion of an innovation among physicians, Sociome-try, 20 (1957), pp. 253–270. 208

[53] , Medical Innovation: A Diffusion Study, Bobbs-Merrill, New York: NY, 1966. 2nd. 166, 170,208

[54] R. B. COOPER AND R. W. ZMUD, Information technology implementation research: a technological dif-fusion approach, Management Science, 36 (1990), pp. 123–139. 163

[55] R. D. CORK, W. M. DETMER, AND C. P. FREIDMAN, Development and initial evaluation of an in-strument to measure physicians’ use of, knowledge about, and attitudes toward computers, Journal of theAmerican Medical Informatics Association, 5 (1998), pp. 164–176. 27, 110, 177, 212

[56] A.-M. CROTEAU AND D. VIERUL, Telemedicine adoption by different groups of physicians, in HICSS’02: Proceedings of the 35th Annual Hawaii International Conference on System Sciences (HICSS-35) Volume 6, Washington, DC, USA, 2002, IEEE Computer Society, p. 151. 168

REFERENCES 240

[57] K. CURTIN, B. D. HAYES, C. L. HOLLAND, AND L. A. KATZ, Computer-generated intervention forasthma population care management, Effective Clinical Practice, 1 (1998), pp. 43–46. 21, 107, 140

[58] F. D. DAVIS, A technology acceptance model for empirically testing new end-user information systems:theory and results, PhD thesis, Sloan School of Management, Massachusetts Institute of Technology,1986. 152, 207

[59] , Perceived usefulness, perceived ease of use, and user acceptance of information technology, MISQuarterly, 13 (1989), pp. 319–342. 149, 150, 152, 161, 164, 165, 176

[60] , User acceptance of information technology: system characteristics, user perceptions and behavioralimpacts, International Journal of Human-Computer Studies, 38 (1993), pp. 475–487. 154

[61] F. D. DAVIS, R. P. BAGOZZI, AND P. R. WARSHAW, User acceptance of computer technology: a com-parison of two theoretical models, Management Science, 35 (1989), pp. 982–1003. 152, 165, 176, 182

[62] D. DEKKER, D. KRACKHARDT, AND T. A. SNIJDERS, Sensitivity of MRQAP tests to collinearity andautocorrelation conditions. Working Paper Series on Research in Relationship Management, 2005.211

[63] T. J. DEMAIO, Social desirability and survey measurement: A review, in Surveying Subjective Phenom-ena, C. F. Turner and M. E, eds., Russell Sage, New York, 1984, pp. 257–281. 161

[64] S. DEVARAJ AND R. KOHLI, Information technology payoff in the health-care industry: A longitudinalstudy, Journal of Management Information Systems, 16 (2000), pp. 41–68. 108

[65] , Performance impacts of information technology: Is actual usage the missing link?, ManagementScience, 49 (2003), pp. 273–289. 108

[66] D. R. DIXON AND M. STEWART, Exploring information technology adoption by family physicians: sur-vey instrument valuation, in Proc AMIA Symp, Hanley & Belfus, 2000, pp. 185–189. 169

[67] W. J. DOLL AND M. U. AHMED, Managing user expectations, Journal of Systems Management, 34(1983), pp. 6–11. 163

[68] P. DOREIAN, Estimating linear models with spatially distributed data, in Critical issues in informationsystems research, S. Leinhardt, ed., Jossey-Bass, San Francisco, CA, 1981. 211

[69] A. F. DOWLING, Do hospital staff interfere with computer system implementations?, Health Care Man-age Review, 5 (1980), pp. 23–32. 140, 145

[70] J. M. EISENBERG, D. S. KITZ, AND R. A. WEBBER, Development of attitudes about sharing decision-making: a comparison of medical and surgical residents, Journal of Health and Social Behavior, 24(1983), pp. 85–90. 170

[71] R. S. EVANS, S. L. PESTOTNIK, D. C. CLASSEN, T. P. CLEMMER, L. K. WEAVER, J. F. ORME, J. F.LLOYD, AND J. P. BURKE, A computer-assisted management program for antibiotics and other antiinfec-tive agents, The New England Journal of Medicine, 338 (1998), pp. 232–238. 21

[72] B. EVERITT, Cluster Analysis, Halsted Press of John Wiley & Sons Inc., New York and Toronto,3rd ed., 1993. 115

[73] P. N. FINLAY, Introducing decision support systems, Blackwell, Cambridge, MA, 1st ed., 1994. 19

[74] M. FISHBEIN AND I. AJZEN, Beliefs, Attitude, Intention and Behavior: An Introduction to Theory andResearch, Addison-Wesley, Reading, MA, 1975. 150, 207

[75] E. B. FOWLKES AND C. L. MALLOWS, A method for comparing two hierarchical clusterings, Journal ofthe American Statistical Association, 78 (1983), pp. 553–569. 115

[76] J. FOX AND J. BURY, A quality and safety framework for point-of-care clinical guidelines, in Proc AMIASymp, Hanley & Belfus, 2000, pp. 245–249. 79

[77] J. FOX, N. JOHNS, A. RAHMANZADEH, AND R. THOMSON, Disseminating medical knowledge: thePROforma approach, Artificial Intelligence in Medicine, 14 (1998), pp. 1857–1816. 79

REFERENCES 241

[78] J. FOX, N. JOHNS, A. RAHMANZADEH, R. THOMSON, AND P. WILSON, PROforma: a general tech-nology for clinical decision support systems, Comput Methods Programs Biomed, 54 (1997), pp. 59–67.79

[79] L. C. FREEMAN, D. R. WHITE, AND A. K. ROMNEY, Research methods in social network analysis,Transaction Publishers, New Brunswick, NJ, 1992. 208

[80] M. P. GAGNON, G. GODIN, C. GAGNE, J. P. FORTIN, L. LAMOTHE, D. REINHARZ, ANDA. CLOUTIER, An adaptation of the theory of interpersonal behaviour to the study of telemedicine adoptionby physicians, International Journal of Medical Informatics, 71 (2003), pp. 103–115. 171

[81] C. A. GAITHER, R. P. BAGOZZI, F. J. ASCIONE, AND D. M. KIRKING, A reasoned action approach tophysicians’ utilization of drug information sources, Pharmaceutical Research, 13 (1996), pp. 1291–1298.170

[82] E. GAMMA, R. HELM, R. JOHNSON, AND J. VLISSIDES, Design patterns: elements of reusable object-oriented software, Addison-Wesley Professional, 1995. 190

[83] R. M. GARDNER AND H. P. LUNDSCAARDE, Evaluation of user acceptance of a clinical expert system,Journal of the American Medical Informatics Association, 1 (1994), pp. 428–438. 140

[84] C. L. GATCH AND D. KENDZIERSKI, Predicting exercise intentions: the theory of planned behavior,Research Quarterly For Exercise and Sport, 61 (1990), pp. 100–102. 152

[85] D. GEFEN AND D. W. STRAUB, Gender difference in the perception and use of E-Mail: an extension tothe technology acceptance model, MIS Quarterly, 21 (1997), pp. 389–400. 155

[86] G. GODIN, P. VALOIS, L. LEPAGE, AND R. DESHARNAIS, Predictors of smoking behavior: an applica-tion of Ajzens theory of planned behavior, British Journal of Addition, 87 (1992), pp. 1335–1343. 152

[87] D. L. GOODHUE AND R. L. THOMPSON, Task-technology fit and individual performance, MIS Quar-terly, 19 (1995), pp. 213–236. 150

[88] M. GREENLICK, Educating physicians for population-based clinical practice, Journal of the AmericanMedical Association, 267 (1992), pp. 1645–1648. 18

[89] M. R. GREENLICK, Educating physicians for the twenty-first century, Academic Medicine, 70 (1995),pp. 179–185. 18

[90] T. R. GRUBER, A translation approach to portable ontology specifications, Knowledge Acquisition, 5(1993), pp. 199–220. 67, 68

[91] M. S. HAGGER, N. L. D. CHATZISARANTIS, AND S. J. H. BIDDLE, Meta-analysis of the theories ofreasoned action and planned behavior in physical activity: an examination of predictive validity and thecontri-bution of additional variables, Journal of Sport and Exercise Psychology, 24 (2002), pp. 3–32.152

[92] C. HARTLEY, M. BRECHT, P. PAGERLY, G. WEEKS, A. CHAPANIS, AND D. HOECKER, Subjectivetime estimates of work tasks by office workers, Journal of Occupational Psychology, 50 (1977), pp. 23–36. 160

[93] H. A. HEATHFIELD, D. PITTY, AND R. HANKA, Evaluating information technology in health care:barriers and challenges, British Medical Journal, 32 (1998), pp. 1959–1961. 107

[94] H. A. HEATHFIELD AND J. WYATT, Philosophies for the design and development of clinical decision-support systems, Methods of Information in Medicine, 32 (1993), pp. 1–8. 107, 145

[95] W. HERSH, Medical informatics—improving health care through information, Journal of the AmericanMedical Association, 288 (2002), pp. 1955–1958. 18

[96] J. HORSKY, G. J. KUPERMAN, AND V. L. PATEL, Comprehensive analysis of a medication dosing errorrelated to cpoe, Journal of the American Medical Informatics Association, 12 (2005), pp. 377–382.191

[97] G. HOTCHKISS, S. ALSTON, AND G. EDWARDS, Eye tracking study, June 2005. Released by Enquiro,EyeTools and Did-It, 2005. 190

REFERENCES 242

[98] P. HU, P. Y. K. CHAU, O. R. L. SHENG, AND K. Y. TAM, Examining the technology acceptance modelusing physician acceptance of telemedicine technology, Journal of Management Information Systems,16 (1999), pp. 91–112. 167, 168, 169

[99] L. HUBERT, Combinatorial data analysis: association and partial association, Psychometrika, 50 (1985),pp. 449–467. 211

[100] G. S. HUBONA AND P. H. CHENEY, System effectiveness of knowledge-based technology: the relationshipof user performance and attitudinal measures, in HICSS ’94: Proceedings of the 27th Annual HawaiiInternational Conference on System Sciences (HICSS-27) Volume 4, Washington, DC, USA, 1994,IEEE Computer Society, pp. 532–541. 154

[101] D. L. HUNT, R. B. HAYNES, S. E. HANNA, AND K. SMITH, Effects of computer-based clinical deci-sion support systems on physician performance and patient outcomes, Journal of the American MedicalAssociation, 280 (1998), pp. 1339–1346. 107, 140

[102] M. IGBARIA, T. GUIMARAES, AND G. B. DAVIS, Testing the determinants of microcomputer usage viaa structural equation model, Journal of Management Information Systems, 11 (1995), pp. 87–114. 155

[103] M. IGBARIA, J. IIVARI, AND H. MARAGAHH, Why do individuals use computer technology? a Finnishcase study, Information & Management, 29 (1995), pp. 227–238. 155

[104] C. M. JOHNSON, T. R. JOHNSON, AND J. ZHANG, A user-centered framework for redesigning healthcare interfaces, Journal of biomedical informatics, 38 (2005), pp. 75–87. 26, 177, 191

[105] B. KAPLAN, Addressing organizational issues into the evaluation of medical systems, Academy of Man-agement Journal, 4 (1997), pp. 94–101. 166

[106] B. KAPLAN, Evaluating informatics applications - clinical decision support systems literature review, In-ternational Journal of Medical Informatics, 64 (2001), pp. 15–37. 25, 107, 108, 140

[107] , Evaluating informatics applications - some alternative approaches: theory, social interactionism, andcall for methodological pluralism, International Journal of Medical Informatics, 64 (2001), pp. 39–56.25, 107, 108, 140, 165

[108] E. KARAHANNA, D. W. STRAUB, AND N. L. CHERVANY, Information technology adoption acrosstime: a cross-sectional comparison of pre-adoption and post-adoption beliefs, MIS Quarterly, 23 (1999),pp. 183–213. 162, 163, 164

[109] Y. KASHIMA, C. GALLOIS, AND M. MCCAMISH, Theory of reasoned action and cooperative behavior:it takes two to use a condom, British Journal of Social Psychology, 32 (1993), pp. 227–239. 152

[110] K. KAWAMOTO, C. A. HOULIHAN, E. A. BALAS, AND D. F. LOBACH, Improving clinical practiceusing clinical decision support systems: a systematic review of trials to identify features critical to success,British Medical Journal, 330 (2005), pp. 740–741. 22, 24

[111] P. KEEN, Shaping the future: business design through information technology, Harvard Business SchoolPress, Cambridge, MA, 1991. 140

[112] H. C. KELMAN, Compliance, identification, and internalization: three processes of attitude change?, Jour-nal of Conflict Resolution, 2 (1958), pp. 51–60. 156, 207

[113] D. KRACKHARDT, Predicting with networks: noparametric multiple regression analysis of dyadic data,Social Networks, 10 (1988), pp. 359–381. 211

[114] D. KRACKHARDT, The ties that torture: Simmelian tie analysis in organizations, Research in the Soci-ology of Organizations, 16 (1999), pp. 183–210. 211

[115] R. A. KUHN AND R. S. REIDER, A C++ framework for developing Arden Syntax compiler, Computersin Biology and Medicine, 24 (1994), pp. 365–370. 68

[116] T. H. KWON AND R. W. ZMUD, Unifying the fragmented models of information systems implementation,in Critical issues in information systems research, R. J. Boland and R. A. Hirschheim, eds., JohnWiley & Sons, Inc., Chichester, England, 1987, pp. 227–251. 163

[117] J. S. LEE, H. CHO, G. GAY, B. DAVIDSON, AND A. INGRAFFEA, Technology acceptance and socialnetworking in distance learning, Educational Technology & Society, 6 (2003), pp. 50–61. 174

REFERENCES 243

[118] Y. LEE, K. A. KOZAR, AND K. R. T. LARSEN, The technology acceptance model: past, present, andfuture, Communications of the AIS, 12 (2003), pp. 752–780. 154, 159, 163, 164, 170

[119] R. T. LEENDERS, Modeling social influence through network autocorrelation: constructing the weightmatrix, Social Networks, 24 (2002), pp. 21–47. 207, 209, 211

[120] F. LEGARE, G. GODIN, V. RINGA, S. DODIN, L. TURCOT, AND J. NORTON, Variation in the psy-chosocial determinants of the intention to prescribe hormone therapy prior to the release of the Women’sHealth Initiative trial: a survey of general practitioners and gynaecologists in France and Quebec, BMCMedical Informatics and Decision Making, 5 (2005), p. 31. 170

[121] L. S. LENINGER, L. FINN, AND D. L, An office system for organizing preventive services: a report by theamerican cancer society advisory group on preventive health care reminder systems, Archives of FamilyMedicine, 5 (1996), pp. 108C–115. 19

[122] J. R. LEWIS, IBM computer usability satisfaction questionnaires: psychometric evaluation and instructionsfor use, International Journal of Human-Computer Interaction, 7 (1995), pp. 57–78. 110, 141, 176,212

[123] D. K. LITZELMAN, R. S. DITTUS, M. E. MILLER, AND W. M. TIERNEY, Requiring physicians torespond to computerized reminders improves their compliance with preventive care protocols, Journal ofGeneral Internal Medicine, 8 (1993), pp. 311–317. 107, 140

[124] H. P. LUNDSGAARDE, P. A. FISCHER, AND D. J. STEELE, Human problems in computerized medicine,in Publications in Anthropology, 13, The University of Kansas, Lawrence, KS, 1981. 166

[125] J. MACQUEEN, Some methods for classification and analysis of multivariate observations, in Proc. 5thBerkeley Symp. on Mathematical Statistics and Probability, L. M. L. Cam and J. Neyman, eds.,vol. 1, University of California Press, Berkeley, CA, 1967, pp. 281–297. 115

[126] Y. MALHOTRA AND D. F. GALLETTA, Extending the technology acceptance model to account for socialinfluence: theoritical bases and empirical validation, in HICSS ’99: Proceedings of the 32nd AnnualHawaii International Conference on System Sciences (32), vol. 1, Washington, DC, USA, 1999,IEEE Computer Society, p. 1006. 160

[127] T. A. MASSARO, Introducing physician order entry at a major academic medical center: I. impact onorganizational couture and behavior, Academic Medicine, 68 (1993), pp. 20–25. 145, 149

[128] K. MATHIESON, Predicting user intentions: comparing the technology acceptance model with the theoryof planned behavior, Information Systems Research, 2 (1991), pp. 173–191. 154, 164

[129] K. MATHIESON, E. PEACOCK, AND W. W. CHIN, Extending the technology acceptance model: theinfluence of perceived user resources, ACM SIGMIS Database, 32 (2001), pp. 86–112. 155

[130] S. K. MAUE, R. SEGAL, C. L. KIMBERLIN, AND E. E. LIPOWSKI, Predicting physician guidelinecompliance: an assessment of motivators and perceived barriers, The American J of Managed Care, 10(2004), pp. 382–391. 170

[131] B. MAUGHAN, Developmental trajectory modeling: a view from developmental psychopathology, TheANNALS of the American Academy of Political and Social Science, 602 (2005), pp. 118–130. 172

[132] E. MAYO, The Human Problems of an Industrial Civilization, MacMillan, New York, NY, USA, 1933.161

[133] C. J. MCDONALD, S. L. HUI, D. M. SMITH, W. M. TIERNEY, S. J. COHEN, L. WEINBERGER,AND G. P. MCCABE, Reminders to physicians from an introspective computer medical record, Annals ofInternal Medicine, 100 (1995), pp. 130–138. 21, 107, 140

[134] E. MCGLYNN, S. ASCH, J. ADAMS, J. KEESEY, H. J., AND A. DECRISTOFARO, The quality ofhealth care delivered to adults in the United States, The New England Journal of Medicine, 348 (1993),pp. 1287–1335. 21

[135] S. B. MERRIAM, Qualitative research and case study applications in education, Jossey-Bass Publishers,San Francisco, 2nd ed., 1998. 141

REFERENCES 244

[136] A. MILLER, Medical diagnostic decision support systems - past, present, and future: a threaded bibliogra-phy and brief commentary, Journal of the American Medical Informatics Association, 1 (1994), p. 160.18

[137] B. MOBASHER, H. DAI, T. LUO, AND M. NAKAGAWA, Effective personalization based on associationrule discovery from web usage data, in Web Information and Data Management, 2001, pp. 9–15. 193

[138] A. L. MONTGOMERY, S. L, K. SRINIVASAN, AND J. C. LIECHTY, Modeling online browsing and pathanalysis using clickstream data, Marketing Science, 23 (2004), pp. 579–595. 193

[139] G. C. MOORE AND I. BENBASAT, Development of an instrument to measure the perceptions of adoptingan information technology innovation, Information Systems Research, 2 (1991), pp. 173–191. 156

[140] D. M. MORRISON, M. S. SPENCER, AND M. R. GILLMORE, Beliefs about substance use among preg-nant and parenting adolescents, Journal of Research on Adolescence, 8 (1998), pp. 69–95. 152

[141] H. J. MURFF AND J. KANNRY, Physician satisfaction with two order entry systems, Journal of theAmerican Medical Informatics Association, 8 (2001), pp. 499–509. 177, 212

[142] M. A. MUSEN, S. TU, A. DAS, AND Y. SHAHAR, EON: a component-based approach to automa-tion of protocol-directed therapy, Journal of the American Medical Informatics Association, 3 (1996),pp. 367–388. 71

[143] D. S. NAGIN, Analyzing developmental trajectories: a semi-parametric, group-based approach, Psycho-logical Methods, 4 (1999), pp. 139–157. 112, 113, 172

[144] D. S. NAGIN AND R. E. TREMBLAY, Trajectories of boys’ physical aggression, opposition and hyperactiv-ity on the path to physically violent and nonviolent juvenile delinquency, Child Development, 70 (1999),pp. 1181–1196. 112, 172

[145] , Analyzing developmental trajectories of distinct but related behaviors: a group-based method, Psy-chological Methods, 6 (2001), pp. 18–34. 112, 172

[146] R. S. NICKERSON, Why interactive computer systems are sometimes not used by people who might benefitfrom them, International Journal of Human-Computer Studies, 51 (1999), pp. 307–321. 25, 149

[147] D. S. NILASENA AND M. J. LINCOLN, A computer-generated reminder system improves physician com-pliance with diabetes preventive care guidelines, in Proc Annu Symp Comput Appl Med Care, vol. 3,American Medical Informatics Association, 1995, pp. 640–645. 21, 107, 140

[148] J. NUNNALLY AND I. BERNSTEIN, Psychometric Theory, McGraw Hill, New York, NY, 1994. 3rd.181

[149] L. OHNO-MACHADO, J. H. GENNARI, S. MURPHY, N. L. JAIN, S. W. TU, D. E. OLIVER,E. PATTISON-GORDON, R. A. GREENES, E. H. SHORTLIFFE, AND G. O. BARNETT, The guidelineinterchange format: a model for representing guidelines, Journal of the American Medical InformaticsAssociation, 5 (1998), pp. 357–372. 75, 76

[150] M. H. OLSON AND H. C. LUCAS, The impact of office automation on the organization: Some implicationsfor research and practice, Communications of the ACM, 25 (1982), pp. 838–847. 166

[151] D. PARKER, A. S. R. MANSTEAD, S. G. STRADLING, AND J. REASON, Intention to commit drivingviolations: An application of the theory of planned behavior, Journal of Applied Psychology, 77 (1992),pp. 94–101. 152

[152] V. L. PATEL, T. BRANCH, D. WANG, M. PELEG, AND A. A. BOXWALA, Analysis of the processof encoding guidelines: a comparison of GLIF2 and GLIF3, Methods of Information in Medicine, 41(2002), pp. 105–113. 75

[153] M. PELEG, A. A. BOXWALA, E. BERNSTAM, S. W. TU, R. A. GREENES, AND E. H. SHORTLIFFE,Sharable representation of clinical guidelines in GLIF: relationship to the arden syntax, Journal of Biomed-ical Informatics, 34 (2001), pp. 170–181. 73

[154] M. PELEG, A. A. BOXWALA, O. OGUNYEMI, Q. ZENG, S. W. TU, R. LACSON, E. BERNSTAM,N. ASH, P. MORK, L. OHNO-MACHADO, E. H. SHORTLIFFE, AND R. A. GREENES, GLIF3: theevolution of a guideline representation format, in Proc AMIA Symp, vol. 41, Hanley & Belfus, 2000,pp. 645–649. 76

REFERENCES 245

[155] M. PELEG, S. W. TU, J. BURY, P. CICCARESE, J. FOX, R. A. GREENES, R. HALL, P. D. JOHN-SON, N. JONES, A. KUMAR, S. MIKSCH, S. QUAGLINI, A. SEYFANG, E. H. SHORTLIFFE, ANDM. STEFANELLI, Comparing computer-interpretable guideline models: a case-study approach, Journal ofthe American Medical Informatics Association, 10 (2003), pp. 52–68. 68, 69, 73, 74

[156] L. PERREAULT AND J. METZGER, A pragmatic framework for understanding clinical decision support,Journal of Healthcare Information Management, 13 (1999), pp. 5–21. 31

[157] M. B. PRESCOTT AND S. A. CONGER, Information technology innovations: a classification by it locus ofimpact and research approach, SIGMIS Database, 26 (1995), pp. 20–41. 163

[158] J. PUZICHA, T. HOFMANN, AND J. M. BUHMANN, A theory of proximity based clustering: structuredetection by optimization, Pattern Recognition, 33 (1999), pp. 617–634. 115

[159] R. RICE AND W. RICHARDS, An overview of network analysis methods and programs, in Progress incommunication sciences, B. Dervin and M. Voight, eds., Ablex Publishing Co, Norwood, NJ, 1985,pp. 105–165. 208

[160] E. M. ROGERS, Diffusion of Innovations, The Free Press, New York, 1983. 3rd. 150, 163

[161] , Diffusion of Innovations, The Free Press, New York, 1995. 4th. 157

[162] N. ROUSSEAU, E. MCCOLL, J. NEWTON, J. GRIMSHAW, AND M. ECCLES, Practice based, longi-tudinal, qualitative interview study of computerised evidence based guidelines in primary care, BMJ, 326(2003), p. 314. 107, 140

[163] D. L. SACKETT, M. C. ROSENBERG, J. A. GRAY, R. B. HAYNES, AND W. S. RICHARDSON, Evidencebased medicine: what it is and what it isn’t?, BMJ, 312 (1996), pp. 71–72. 3, 19, 20, 23

[164] D. L. SACKETT, S. E. STRAUS, W. S. RICHARDSON, W. ROSENBERG, AND R. B. HAYNES, Evidence-based medicine: how to practice and teach EBM, Churchill Livingstone, London, 2000. 2nd. 20

[165] V. SAGA AND R. ZMUD, The nature and determinants of information technology acceptance, routinizationand infusion, in Diffusion, Transfer and Implementation of Information Technology, L. Levine, ed.,North-Holland, Amsterdam, 1994, pp. 67–86. 172

[166] J. J. SALEEM, E. S. PATTERSON, L. MILITELLO, M. L. RENDER, G. ORSHANSKY, AND S. M. ASCH,Exploring barriers and facilitators to the use of computerized clinical reminders, Journal of the AmericanMedical Informatics Association, 12 (2005), pp. 438–447. 191

[167] V. SAMBAMURTHY AND C. W. W, The effects of group attitudes toward alternative gdss designs on thedecision-making performance of computer-supported groups, Decision Sciences, 25 (1994), pp. 215–241.154

[168] D. C. SCHMIDT, R. E. JOHNSON, AND M. FAYAD, Guest editorial: Software patterns, Special Issue onPatterns and Pattern Languages, Communications of the ACM, 39 (1996), pp. 1–5. 190

[169] G. SCHWARTZ, Estimating the dimension of a model, The Annals of Statistics, 6 (1978), pp. 461–464.114

[170] N. SCHWARZ AND D. OYSERMAN, Asking questions about behavior: cognition, communication, andquestionnaire construction, American Journal of Evaluation, 22 (2001), pp. 127–160. 160, 161, 219

[171] J. SCOTT, Social Network Analysis: A handbook, Sage, London, 2000. 2nd. 208

[172] Y. SHAHAR, S. MIRKSCH, AND P. JOHNSON, The Asgaard Project: a task-specific framework for theapplication and critiquing of time-oriented clinical guidelines, Artificial Intelligence in Medicine, 14(1998), pp. 29–51. 72

[173] A. SHARMA, Professionals as agent: knowledge asymmetry in agengy exchanges, Academy of Manage-ment Review, 22 (1997), pp. 758–798. 165

[174] S. SHEA, W. DUMOUCHEL, AND L. BAHAMONDE, A meta-analysis of 16 randomized controlled trialsto evaluate computer-based clinical reminder systems for preventive care in the ambulatory setting, Journalof the American Medical Informatics Association, 3 (1996), pp. 399–409. 21, 107, 140

REFERENCES 246

[175] G. B. SHELLY, T. J. CASHMAN, AND H. J. ROSENBLATT, Systems Analysis and Design, Course Tech-nology, Cambridge, MA, 1998. 3rd. 42

[176] B. H. SHEPPARD, J. HARTWICK, AND P. R. WARSHAW, The theory of reasoned action: a meta-analysisof past research with recommendations for modifications and future research, Journal of Consumer Be-havior, 15 (1988), pp. 325–343. 152

[177] R. N. SHIFFMAN, Representation of clinical practice guidelines in conventional and augmented decisiontables, Journal of the American Medical Informatics Association, 4 (1997), pp. 382–393. 68

[178] D. E. SICHEL, The Computer Revolution: An Economic Perspective, Brookings, Washington, DC, 1997.25, 149

[179] G. P. C. SICOTTE AND H. JACQUES, The effects of creating psychological ownership on physicians’ ac-ceptance of clinical information systems, Journal of the American Medical Informatics Association, 13(2006), pp. 197–205. 167, 168

[180] I. SIM, P. GORMAN, R. GREENES, R. HAYNES, B. KAPLAN, H. LEHMANN, AND P. TANG, Clinicaldecision support systems for the practice of evidence-based medicine, Journal of the American MedicalInformatics Association, 8 (2001), pp. 527C–534. 20

[181] I. SIM, P. GORMAN, R. A. GREENES, R. B. HAYNES, B. KAPLAN, H. LEHMANN, AND P. C. TANG,Clinical decision support systems for the practice of evidence-based medicine, Journal of the AmericanMedical Informatics Association, 8 (2001), pp. 527–534. 21

[182] A. C. SIOCHI AND R. W. EHRICH, Computer analysis of user interfaces based on repetition in transcriptsof user sessions, ACM Transactions on Information Systems, 9 (1991), pp. 309–335. 193

[183] L. SOLBERG, T. KOTTKE, M. BREKKE, S. CONN, S. MAGNAN, AND G. AMUNDSON., The case ofthe missing clinical preventive services systems, Effective Clinical Practice, 1 (1998), pp. 33–38. 21

[184] D. STRANG AND N. B. TUMA, Spatial and temporal heterogeneity in diffusion, The American Journalof Sociology, 99 (1993), pp. 614–639. 208

[185] D. STRAUB, M. KEIL, AND W. BRENNER, Testing the technology acceptance model across cultures: athree country study, Information & Management, 33 (1997), pp. 1–11. 155

[186] D. STRAUB, M. LIMAYEM, AND E. KARAHANNA-EVARISTO, Measuring system usage: implicationsfor IS theory testing, Management Science, 41 (1995), pp. 1328–1342. 108, 160, 173, 184

[187] J. K. STROSS AND G. G. BOLE, Evaluation of a continuing education program in rheumatoid arthritis,arthritis and rheumatism, Arthritis and Rheumatism, 23 (1980), pp. 846–849. 170

[188] G. SUBRAMANIAN, A replication of perceived usefulness and perceived ease of use measurement, DecisionSciences, 25 (1994), pp. 863–874. 154

[189] M. J. SUCCI AND Z. D. WALTER, Theory of user acceptance of information technologies: an examinationof health care professionals, in HICSS ’99: Proceedings of the 32nd Annual Hawaii InternationalConference on System Sciences (HICSS-32), Los Alamitos, CA, 1999, IEEE Computer Society. 165,167, 169

[190] H. SUN AND P. ZHANG, The role of moderating factors in user technology acceptance, InternationalJournal of Human-Computer Studies, 64 (2006), pp. 53–78. 155, 156

[191] D. R. SUTTON AND J. FOX, The syntax and semantics of the PROforma guideline modeling language,Journal of the American Medical Informatics Association, 10 (2003), pp. 433–443. 79, 80

[192] E. B. SWANSON, Information System Implementation: Bridging the Gap between Design and Utilization,Irwin, Homewood, IL, 1988. 25, 149

[193] B. SZAJNA, Empirical evaluation of the revised technology acceptance model, Management Science, 42(1996), pp. 85–92. 108, 110, 152, 184

[194] P. C. TANG AND V. L. PATEL, Major issues in user interface design for health professional workstations:summary and recommendations, International Journal of Biomedical Computing, 34 (1994), pp. 139–148. 191

REFERENCES 247

[195] S. TAYLOR AND P. TODD, Understanding information technology usage: a test of competing model, In-formation Systems Research, 6 (1995), pp. 145–176. 154, 160

[196] R. L. THOMPSON, C. A. HIGGINS, AND J. M. HOWELL, Personal computing: toward a conceptualmodel of utilization, MIS Quarterly, 15 (1991), pp. 125–143. 150, 157

[197] , Influence of experience on personal computer utilization: testing a conceptual model, Journal ofManagement Information Systems, 11 (1994), pp. 167–187. 157

[198] J. TIDWELL, Designing interfaces, O’Reilly, 2005. 190

[199] W. M. TIERNEY, S. L. HUI, AND C. J. MCDONALD, Delayed feedback of physician performance versusimmediate reminders to perform preventive care: effects on physician compliance and medical care, MedCare, 24 (1986), pp. 659–666. 21, 107, 140

[200] L. TORNATZKY AND K. KLEIN, Innovation characteristics and innovation adoption implementation: ameta-analysis of findings, IEEE Transactions on Engineering Management, 29 (1982), pp. 28–45. 162

[201] E. TURBAN, Decision support and expert systems: management support systems, Prentice-Hall, NewJersey, 4th ed., 1995. 19

[202] M. L. TUSHMAN AND T. J. SCANLAN, Boundary spanning individuals: their role in information-transferand their antecedents, Academy of Management Journal, 24 (1981), pp. 289–305. 166

[203] J. VAN DER LEI, M. A, MUSEN, E. VAN DER DOES, A. J. M. IN ’T VELD, AND J. H. VAN BEMMEL,Comparison of computer-aided and human review of general practitioners’ management of hypertension,Lancet, 338 (1996), pp. 1504–1508. 107, 140

[204] V. VENKATESH, Determinants of perceived ease of use integrating control, intrinsic motivation, and emo-tion into the technology acceptance model, Information Systems Research, 11 (2000), pp. 342–365. 155,173

[205] V. VENKATESH AND F. D. DAVIS, A model of the antecedents of perceived ease of use development andtest, Decision Sciences, 27 (1996), pp. 451–481. 155

[206] , A theoretical extension of the technology acceptance model: four longitudinal field studies, Man-agement Science, 46 (2000), pp. 186–204. 154, 156, 160, 175, 182, 207, 208

[207] V. VENKATESH AND M. G. MORRIS, Why don’t men ever stop to ask for directions? gender, socialinfluence, and their role in technology acceptance and usage behavior, MIS Quarterly, 24 (2000), pp. 115–139. 155, 164

[208] V. VENKATESH, M. G. MORRIS, G. B. DAVIS, , AND F. D. DAVIS, User acceptance of informationtechnology: toward a unified view, MIS Quarterly, 27 (2003), pp. 425–478. 158, 164, 175, 208

[209] M. VINGRON AND P. ARGOS, A fast and sensitive multiple sequence alignment algorithm, ComputerApplications in the Biosciences, 5 (1989), pp. 115–122. 193

[210] E. M. VOORHEES, Implementing agglomerative hierarchic clustering algorithms for use in document re-trieval, Information Processing and Management, 22 (1986), pp. 465–476. 115

[211] D. WANG, M. P. M, S. W. TU, A. A. BOXWALA, O. OGUNYEMI, Q. ZENG, R. A. GREENES, V. L.PATEL, AND E. H. SHORTLIFFE, Design and implementation of the GLIF3 guideline execution engine,Journal of Biomedical Informatics, 37 (2004), pp. 305–318. 75

[212] D. WANG, M. PELEG, S. W. TU, A. A. BOXWALA, R. A. GREENES, V. L. PATEL, AND E. H.SHORTLIFFE, Representation primitives, process models and patient data in computer-interpretable clin-ical practice guidelines: a literature review of guideline representation models, International Journal ofMedical Informatics, 68 (2002), pp. 59–70. 68, 73

[213] S. WASSERMAN AND K. FAUST, Social network analysis, Cambridge University Press, Cambridge,1994. 208

[214] H. J. WONG, M. W. LEGNINI, AND H. H. WHITMORE, Diffusion of decision support systems in health-care: Are we there yet?, Journal of Healthcare Management, 45 (2000), pp. 240–249. 18

REFERENCES 248

[215] J. WYATT AND D. SPIEGELHALTER, Field trials of medical decision-aids: potential problems and solu-tions, in Proc Annu Symp Comput Appl Med Care, vol. 1, American Medical Informatics Associa-tion, 1991, pp. 3–7. 18, 19, 20

[216] J. C. WYATT AND J. L. Y. LIU, Basic concepts in medical informatics, Journal of Epidemiology andCommunity Health, 56 (2002), pp. 808–812. 18

[217] K. ZHENG, R. PADMAN, AND M. P. JOHNSON, User acceptance and adoption of a clinical remindersystem: interpreting the effects of user characteristics, in Encyclopedia of Gender and InformationTechnology, E. M. Trauth, ed., Idea Group Publishing, Hershey, PA, 2005. 106

[218] K. ZHENG, R. PADMAN, M. P. JOHNSON, AND H. S. DIAMOND, User acceptance and adoption of aclinical reminder system in ambulatory care: a developmental trajectory approach. Heinz School WorkingPaper Series 2003–14, 2003. 84

[219] , Understanding technology adoption in clinical care: clinician adoption behavior of a point-of-carereminder system, International Journal of Medical Informatics, 74 (2005), pp. 535–543. 139, 192

[220] K. ZHENG, R. PADMAN, M. P. JOHNSON, J. B. ENGBERG, AND H. S. DIAMOND, An adoption studyof a clinical reminder system in ambulatory care using a developmental trajectory approach, in Proceedingsof Eleventh World Congress on Medical Informatics, M. Fieschi, ed., Amsterdam, 2004, IOS Press,pp. 1115–1120. 84, 106, 192

[221] K. ZHENG, R. PADMAN, M. P. JOHNSON, J. B. ENGBERG, AND H. S. DIAMOND, Clinical remindersystem (CRS): A relational database application for evidence-based medicine practice. Heinz School Work-ing Paper Series 2005–30, 2005. 30, 191, 209

[222] R. D. ZIELSTORFF, Online practice guidelines: issues, obstacles, and future prospects, Journal of theAmerican Medical Informatics Association, 5 (1998), pp. 227–36. 68

Index

K-means algorithm, 122, 141

Arden Syntax, 75Asbru, 79Attitude, 159

Backus-Naur Form, 88Bayesian Information Criterion, 121Behavioral Intention, 161BI, see Behavioral IntentionBIC, see Bayesian Information CriterionBNF, see Backus-Naur Formbundled action, 209

CA, see Cluster AnalysisCDSSs, see Clinical Decision Support SystemsClinical Decision Support Systems, 24, 25Cluster Analysis, 122, 141Clustering Similarity, 122cohesion, 220complex information systems, 171Cork’s Instrument, 143, 186, 223

Decision Support Systems, 25design patterns, 200Developmental Trajectory Analysis, 119DTA, see Developmental Trajectory Analysis

EON, 78

First-Order Logic, 78, 87, 88FOL, see First-Order Logic

GELLO, 77, 85GLEE, see GLIF3 Guideline Execution EngineGLIF, see Guideline Interchange FormatGLIF3, see Guideline Interchange FormatGLIF3 Guideline Execution Engine, 77, 85GREM, see Guideline Representation and Execu-

tion ModelGUIDE, 80Guideline Interchange Format, 77, 82Guideline Representation and Execution Model, 90

Halo Effect, 170Hawthorne Effect, 170Hierarchical Clustering, 141HL7 Reference Information Mode, 85

IBM User Satisfaction Questionnaire, 145, 149, 185,223

Markov Chain Analysis, 205maximal sequential pattern, 205Medical Logic Modules, 75MLMs, see Medical Logic Modules

Ontology, 74

Partitioning Clustering, 141PBC, see Perceived Behavioral ControlPEoU, see Perceived Ease of UsePerceived Behavioral Control, 160Perceived Ease of Use, 161Perceived Usefulness, 161PFR, 117PPE, 117PRODIGY, 79PRODIGY3, see PRODIGYProtege, 77–79PROforma, 78, 86PTC, 118PU, see Perceived Usefulness

RIM, see HL7 Reference Information Mode

SAGE, see Sharable Active Guideline Environmentsequential pattern, 203Sequential Pattern Analysis, 203Sharable Active Guideline Environment, 77SN, see Subjective NormSNA, see Social Network AnalysisSocial Network Analysis, 219Social Psychology Theories, 159SPA, see Sequential Pattern Analysisstructural equivalence, 220Subjective Norm, 159

TAM, see Technology Acceptance ModelTAM2, 165Technology Acceptance Model, 161Temporal Dynamics, 171Theory of Planned Behavior, 160Theory of Reasoned Action, 159TPB, see Theory of Planned BehaviorTRA, see Theory of Reasoned Action

INDEX 250

Unified Theory of Acceptance and Use of Technol-ogy, 166

UTAUT, see Unified Theory of Acceptance and Useof Technology

vEMR, see Virtual Electronic Medical RecordVirtual Electronic Medical Record, 80, 86

Documents

Example Of Doctoral Dissertation Outline