www.elsevier.com/locate/yjbin

Journal of Biomedical Informatics 38 (2005) 173–175

Guest Editorial

Human-centered computing in health information systemsPart 2: Evaluation

This is the second part of two special issues on hu-man-centered computing in health information systems(see [1] for Part 1 Editorial). Human-centered computingis based on four types of analyses for users, functions,representations, and tasks [1,2]. It is also a process thatincludes work domain analysis, design, and evaluationas three major steps. Part 1 of the special issue focusedon the domain analysis and design components ofhuman-centered computing in health informationsystems. Part 2 focuses on the evaluation component.

Evaluation is always an important process for the de-sign and implementation of any health information sys-tem [3]. Part 2 of the special issue brings togetheroriginal research and methodology papers that are con-cerned primarily with how effective and efficient a sys-tem is, along user characteristics, such as ease of use,ease of learning, reduction of medical error, and usersatisfaction.

The first paper by Pizziferri et al. [4] examineswhether an electronic health record (EHR) system de-creases or increases physician times on clinical tasks.Time is one of the critical factors that often dictate thesuccess or failure of an EHR system. The authors useda time–motion study method to collect the times pri-mary care physicians spent on clinic tasks before andafter the implementation of an EHR system. They alsoconducted a survey to assess physicians� perceptionsregarding the EHR. The results show that there is nosignificant difference in times spent on clinic tasks beforeand after implementation. However, only about a thirdof the survey respondents reported that the EHR tookthe same amount or less time than paper records,although the majority of them believed that the EHR re-sulted in better care quality. Time–motion studies suchas the current one are valuable for the assessment ofthe efficiency and effectiveness of health information sys-tems. With more detailed observations that includetimes on nonclinical as well as clinical tasks, times ondifferent types of tasks, task steps, and errors, we cangain more insight into human-centered characteristicsof EHR systems.

1532-0464/$ - see front matter � 2005 Elsevier Inc. All rights reserved.

doi:10.1016/j.jbi.2004.12.005

The second paper by Patterson et al. [5] is a study ofthe barriers to the effective use of clinical reminders. Ad-vances in health information systems have made possi-ble many decision support systems such as clinicalreminders for recommended actions due for a patientand clinical warnings that immediate actions should betaken to avoid harm to patients. Despite evidence thatclinical reminder systems improve adherence to guide-lines, there are some challenges in having providers con-sistently use clinical reminders as intended. This paperdescribes a study using multiple methods, including eth-nographic observations and surveys, to triangulate anunderstanding of barriers to the effective use of clinicalreminders in the Veterans� Health Administration(VHA). Six barriers were identified from the first studyand four more identified from the second study. It isimportant to note that nearly all of these barriers arecaused by user, social, organizational, educational, andother nontechnology factors.

The third and fourth papers focus on patient safetyconsiderations in the medical device purchasing process.The paper by Laxmisan and colleagues [6] is the sister pa-per toMalhotra et al. [7] published in Part 1 of the specialissue that focused on analysis and design. Laxmisan et al.used three medical device error cases to create three sce-narios for the study of how people with different types ofexpertise evaluate patient safety and device-related med-ical errors in the decision process of medical device pur-chasing. They found that clinicians (nurses and doctors)focused on clinical and human aspects of errors, biomed-ical engineers focused on device-related errors, andadministrators focused on documentation and training.This study raises the question of whether the differencescaused by various types of expertise may reflect thebroader issues in institutional decision making such asinterventions of medical errors, medical device purchas-ing, and deployment of information technology. Thepaper by Ginsburg [8] is a clear demonstration of howa human factors evaluation could affect the decisionmaking in hospital procurement processes. In this studythree infusion pumps were evaluated for their usability

174 Guest Editorial / Journal of Biomedical Informatics 38 (2005) 173–175

problems in five clinical areas in two phases: heuristicsevaluation and user testing. The results were consistentand clearly favored one of the three infusion pumps.Due to the success of this study, the author recom-mended that a human factors evaluation should be per-formed to influence all hospital procurement decisionsabout medical devices to enhance patient safety.

The next two papers are concerned with the use of tele-health systems and wireless communication devices. Thepaper by Farzanfar et al. [9] describes a user evaluationstudy of an automated telephone-based system for healthpromotion and behavior interventions. It is generally as-sumed that adherence to the recommended schedule is re-lated to the impact of the system on users. However, it isnot clear what factors influence users� decisions on usingor not using the system. The results of this study showthat both user and system factors were responsible forthe underutilization or nonuse of the system. Studies suchas the current one have great potential for the identifica-tion of barriers to the use of and the improvement ofusability in telehealth systems, which have been growingin number, usage, and popularity. The paper by Reddy etal. [10] evaluates the impact of the introduction of a wire-less alert pager system on collaborative work practicesand information flows in a surgical intensive care unit.New technologies are often introduced with good inten-tions and with optimistic expectations. They can in manyways improve clinical work flows. However, new technol-ogies can also have negative, often unexpected, conse-quences on information flows. This is the dilemmaillustrated by the results of this study, which show thatthe paging system provided new routes of informationto clinical staff but in doing so also disrupted existingwork practices and information flows. This study onceagain shows that implementing a health information sys-tem is not a straight information technology project: it in-volves many nontechnology factors that often determinethe success or failure of the system.

The next paper by Wachter et al. [11] describes a userevaluation study of a pulmonary graphic display thatdepicts pulmonary physiological variables for intubated,mechanically ventilated patients in a graphical format.Traditional ICU medical monitors provide discrete dataand discrete alarms that alert clinicians to parametersoutside a set range, but they do not provide a compre-hensive representation of patient physiology by integrat-ing multiple data points. The authors developed anintegrated display that was designed to match users�mental models, increase situation awareness, and helpusers in assimilating information more rapidly and facil-itating efficient and timely medical interventions. Thisstudy evaluated how this new technology was integratedand accepted by users. The results, though preliminary,show that the system was monitored, used, and well per-ceived, and that the integrated display was considered asan accurate representation of respiratory variables.

The final paper by Despont-Gros et al. [12] is themethodology review paper for this special issue. Theauthors reviewed articles that are studies of user evalu-ation of clinical information systems. From this com-prehensive review they identified eight dimensionsused in the evaluations, which were then integratedwith the dimensions already identified in the human–computer interaction literature. From these integrateddimensions the authors developed an model for evalu-ation of clinical information systems. This model is anice bridge that connects the traditional methods usedin clinical information system evaluations and the hu-man–computer interaction methods that focus on hu-man dimensions.

The current Part 2 of the special issue on evaluationand the earlier Part 1 on analysis and design are two col-lections of original research and method papers on hu-man-centered computing in health informationsystems. We hope they will promote and advance hu-man-centered design in health information systems forimproving health care quality, increasing patient safety,and reducing health care costs.

Acknowledgments

I thank all the reviewers for their comprehensive re-views. Without their hard work and professional recom-mendations it would be an impossible task for me tocompile this special issue. Their constructive criticismsalso greatly helped the authors in their revisions. Iwould also like to give special thanks to Vimla Patelfor her encouragement and support in producing bothpart of the special issue.

References

[1] Zhang J. Human-centered computing in health informationsystems. Part 1: Analysis and design [Editorial]. J Biomed Inform2005;38:1–3.

[2] Zhang J, Patel VL, Johnson KA, Malin J, Smith JW. Designinghuman-centered distributed information systems. IEEE Intell Syst2002;17(5):42–7.

[3] Friedman CP, Wyatt JC. Evaluation methods in medical infor-matics. Berlin: Springer-Verlag; 1997.

[4] Pizziferri L, Kittler AF, Volk LA, Honour MM, Gupta G, WangT, et al. Primary care physician time utilization before and afterimplementation of an electronic health record: a time-motionstudy. J Biomed Inform 2005;38:176–88.

[5] Patterson ES, Boebbeling BN, Fung CH, Militello L, Anders S,Asch SM. Identifying barriers to the effective use of clinicalreminders: bootstrapping multiple methods. J Biomed Inform2005;38:189–99.

[6] Laxmisan A, Malhotra S, Keselman A, Johnson TR, Patel VL.Decisions about critical events in device-related scenarios as afunction of expertise. J Biomed Inform 2005;38:200–12.

[7] Malhotra S, Laxmisan A, Keselman A, Zhang J, Patel VL.Designing the design phase of critical care devices: a cognitiveapproach. J Biomed Inform 2005;38:34–50.

Guest Editorial / Journal of Biomedical Informatics 38 (2005) 173–175 175

[8] Ginsburg GE. Human factors engineering: a tool for medicaldevice evaluation in hospital procurement decision-making. JBiomed Inform 2005;38:213–9.

[9] Farzanfar R, Frishkopf S, Migneault J, Friedman R. Telephone-linked care for physical activity (TLC-PA): a qualitative evaluationof the use patterns of an information technology program forpatients. J Biomed Inform 2005;38:220–8.

[10] Reddy M, McDonald DW, Pratt W, Shabot MM. Technology,work, and information flows: lessons from the implementation ofa wireless alert pager system. J Biomed Inform 2005;38:229–38.

[11] Wachter B, Markewitz B, Rose R, Westenskow D. Evaluation ofa pulmonary graphical display in the medical intensive care unit:an observational study. J Biomed Inform 2005;38:239–43.

[12] Despont-Gros C, Mueller H, Lovis C. Evaluating user interac-tions with clinical information systems: a model based on human–computer interaction models. J Biomed Inform 2005;38:244–55.

Jiajie ZhangSchool of Health Information Sciences

University of Texas Health Science Center at HoustonHouston, TX 77030, USA

E-mail address: Jiajie.Zhang@uth.tmc.edu

Available online 19 January 2005