. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

RESEARCH Original article

Q The effect of ICU telemedicine on mortalityand length of stay

Benjamin A Kohl*, Margaret Fortino-Mullen, Amy Praestgaard,C William Hanson*, Joseph DiMartino and E Andrew Ochroch**Department of Anesthesiology and Critical Care, Perelman School of Medicine, University of Pennsylvania, Philadelphia, USA; Penn eLert

Telemedicine Program, University of Pennsylvania Health System, Philadelphia, USA; Department of Biostatistics and Epidemiology,

Perelman School of Medicine, University of Pennsylvania, Philadelphia, USA

Summary

We conducted a retrospective, observational study of patient outcomes in two intensive care units in the same hospital.

The surgical ICU (SICU) implemented telemedicine and electronic medical records, while the medical ICU (MICU) did

not. Medical charts were reviewed for a one-year period before telemedicine and a one-year period afterwards. In the

SICU, records were obtained for 246 patients before and 1499 patients after implementation; in the MICU, records

were obtained for 220 patients and 285 patients in the same periods. The outcomes of interest were ICU length of

stay and mortality, and hospital length of stay and mortality. Outcome variables were severity-adjusted using APACHE

scoring. A bootstrap method, with 1000 replicates, was used to assess stability of the findings. The adjusted ICU length

of stay, ICU mortality, and hospital mortality for the SICU patients all decreased significantly after the implementation

of telemedicine. There was no change in adjusted outcome variables in the MICU patients. Implementation of

telemedicine and electronic records in the surgical ICU was associated with a profound reduction in severity-adjusted

ICU length of stay, ICU mortality, and hospital mortality. However, it is not possible to conclude definitively that the

observed associations seen in the SICU were due to the intervention.

Introduction

Telemedicine has been practised in intensive care units

(ICUs) in the US for at least 20 years and there has been an

increase in the number of centres using telemedicine in the

last 10 years.1 Unfortunately, studies evaluating the impact

of remote ICU care on morbidity and mortality have not

been conclusive.25 Most ICU telemedicine studies have

employed a single-centre historical control design,

comparing outcomes after implementing telemedicine with

outcomes from the same ICU before the use of

telemedicine.68 Such a design cannot control for potential

confounding variables such as changes in the healthcare

system. As a result, it is difficult to know whether observed

changes are due to random variation, patient selection,

changes in staffing structures, new medicines or technology,

or the implementation of novel quality/safety initiatives2,9

While it is difficult, although not impossible, to conduct

randomized, double-blind, placebo-controlled studies with

a telemedicine intervention, there are a number of

techniques which can reduce confounding effects, such as

patient matching, stratification and/or propensity

analysis.10,11

In a previous study, a comparison of pre-and post

implementation data in our surgical ICU (SICU), suggested

that the introduction of telemedicine was associated with

reductions in mortality and length of stay, in addition to

major cost savings for the health system.12,13 However, the

telemedicine programme was implemented in tandem with

other hospital-wide quality improvement initiatives, such

as a hand hygiene campaign and the use of care bundles to

prevent deep vein thrombosis, ventilator associated

pneumonia and central line infections. The aim of the

present study was to compare mortality and length of stay

changes over time between the SICU (a unit with

telemedicine services) and a medical intensive care unit

(MICU), a unit without telemedicine services at the same

hospital.

Telemedicine

The telemedicine system was installed in the SICU in

November 2004. The software included an electronic

medical record system and videoconferencing (VISICU

Accepted 21 March 2012

Correspondence: Dr Benjamin A Kohl, Department of Anesthesiology and Critical

Care, Perelman School of Medicine, University of Pennsylvania 3400 Spruce

Street, Founders 5, SICU Administration, Philadelphia PA 19104, USA

(Fax: 1 215 614 0350; Email: Benjamin.Kohl@uphs.upenn.edu)

Journal of Telemedicine and Telecare 2012; 18: 282286 DOI: 10.1258/jtt.2012.120208

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

eICU remote monitoring system, Phillips Electronics,

Amsterdam, The Netherlands). This enabled the provision

of critical care services from an offsite central monitoring

facility.14 The system includes two-way audio conferencing,

one-way video conferencing (i.e. the telemedicine team can

view activity in the patients room by means of a remotely

controlled camera), an electronic medical record available

to both the telemedicine and bedside clinicians, and

continuous physiological monitoring that can detect trends

in vital signs and laboratory values as well as alert the

telemedicine staff if these numbers deviate from pre-defined

limits. Using this system, a small number of physicians and

critical care nurses assist the bedside care team from a

remote location, known as the Clinical Operations Room.

The remote ICU team consults on critical issues, monitors

patients for physiological deterioration and facilitates

communication between care providers. In addition,

telemedicine physicians have access to all radiology

examinations and continuous telemetry data. Details of all

ICU admissions are entered into the system by either the

telemedicine nurse or a trained data coordinator. At the

time of the present study, the telemedicine programme was

responsible for covering a total of five intensive care units

(69 beds) in three hospitals. The SICU analysed for the

present study was the largest (24 beds) unit covered by

telemedicine.

Two care providers in the Clinical Operations Room

monitor ICU patients 24 h/day, seven days a week. During

daytime hours (07:0019:00) there are two ICU nurses, and

in the evening (19:0007:00) there is one physician

(intensivist) and one ICU nurse. The telemedicine nurses

perform audits for benchmarking of outcomes, review

patient profiles for updates in the plans of care and respond

to clinical alarms and enquiries. In addition, they evaluate

and intervene on patient safety measures (e.g. redirecting a

delirious patient who is attempting to get out of bed) and

ensure compliance with best-care practices. Rounds involve

the evaluation of all new patient data and

videoconferencing into the patient room, and are

completed every 14 h based on need. Updates are

communicated by the day telemedicine nurse to the

incoming physician and nurse. The physicians

communicate frequently with the bedside team and are able

to assist as necessary. A button in each SICU room can be

pressed to alert the remote intensivist that there is a request

to assist with an emergency and they should activate the

camera. In addition, the tele-ICU team frequently initiate

contact with the ICU staff if there is a particular concern

regarding patient status or if they believe there should be a

change in management. Either party may call the other by

telephone to discuss any matters of concern more privately.

All telemedicine physicians are credentialled, but their

non-telemedicine clinical duties are entirely at the Hospital

of the University of Pennsylvania.

The staffing paradigms (i.e. standard of care) within the

MICU and SICU are similar but not identical. The MICU

follows a closed intensivist staffing model that is covered

overnight solely by residents. Both the attending

intensivist and the pulmonary fellow (training in critical

care) are available on-call at home. In-house coverage

overnight is provided by residents, in addition to a fellow

(training in critical care). The attending SICU intensivist is

available on-call at home. Implementation of the

telemedicine programme did not affect either staffing

paradigm. The MICU does not use an electronic medical

record.

Methods

We performed a retrospective, observational study using

medical chart review of patients in the SICU and MICU at

the Hospital of the University of Pennsylvania. The study

was approved by the appropriate ethics committee.

Specially trained critical care nurses conducted chart

reviews and extracted ICU admission day information.

APACHE scoring was performed using data from the first

24 h of ICU admission.15 The range for APACHE scores is

0299, higher scores indicating more severe illness. For the

pre-implementation phase, a list of all patients admitted to

the MICU and SICU between April 2003 and March 2004

was obtained. In general, the MICU patients were older and

had a greater number of co-morbidities when compared

with SICU patients (thus explaining their greater APACHE

scores). A minimum of 65 consecutive charts for each

quarter were reviewed for data abstraction to ensure the

availability of information needed to calculate the APACHE

scores. Pre-implementation records were selected at the

same starting point and were chosen as consecutive

admissions over a period of one year.

People trained in quality assurance performed audits of

abstracted data on 10% of the records to ensure accurate

data collection. Records that did not provide the necessary

data to permit APACHE calculation were excluded from the

study and the next admission was then reviewed. This

process resulted in 246 SICU patients and 220 MICU

patients for analysis in the pre-implementation phase.

Implementation of telemedicine and electronic medical

records occurred in the SICU in November 2004.

Post-implementation data were collected for the two units

from July 2005 to June 2006 and were collected in the

MICU with the same methodology described above,

resulting in 285 patients for analysis. Post-implementation

data were collected on all SICU patients admitted during the

12-month period via the newly implemented electronic

medical record (n 2100). Readmissions and off-servicepatients (primary neurosurgical or primary cardiac surgical

patients) were excluded from analysis. In addition, records

that were either incomplete or otherwise ineligible for

APACHE calculation were excluded. This resulted in a total

of 1499 SICU patients for the post-implementation phase.

Four outcomes were considered in the statistical analysis:

hospital and ICU length of stay, and hospital and ICU

mortality.

B A Kohl et al. ICU telemedicine

Journal of Telemedicine and Telecare Volume 18 Number 5 2012 283

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Standard packages were used for the statistical analysis

(STATA 11, StataCorp, TX, USA and SAS 9.1, SAS Institute

Inc. Cary, NC, USA). Mean APACHE III scores were

compared between ICUs by analysis of variance (ANOVA).

Analyses of covariance (ANCOVA) were used to compare pre

and post tele-ICU implementation with hospital length of

stay, using the APACHE III score as a covariate. A similar

ANCOVA model was fitted for the ICU length of stay

outcome. Pre- to post-implementation changes in hospital

and ICU mortality outcomes were analysed with logistic

regression. Finally, logistic regression was used to compare

the pre- and post-telemedicine implementation with

changes in ICU mortality. A bootstrap method, with 1000

replicates, was used to assess stability of the findings.

Results

Severity of illness

On average, MICU patients were more ill than SICU patients

(P, 0.0001), as indicated by higher APACHE scores, see

Table 1. While the mean MICU APACHE score decreased

from pre- to post-implementation (indicating that the

severity of illness decreased), the mean SICU APACHE score

increased from pre- to post-implementation (indicating that

the severity of illness increased). The ANOVA indicated

that both of these changes were significant (P, 0.0001 and

P 0.005 for MICU and SICU, respectively), and that thepre- to post-rates of change in the two units were

significantly different from each other (ANOVA interaction

P, 0.0001).

Hospital length of stay and mortality

The unadjusted and severity-adjusted hospital length of stay

and mortality results for both ICUs pre- and post-

telemedicine implementation are summarised in Table 1.

Unadjusted and severity-adjusted hospital length of stay

decreased for both the MICU and SICU, although neither

change was significant. Similarly, the MICU and SICU pre-

to post-rates of change were not significantly different from

each other. Unadjusted hospital mortality in the MICU

population decreased significantly. However this difference

was not significant after adjusting for severity of illness

(P 0.24). Hospital mortality in the SICU, decreased

significantly after telemedicine was implemented, both in

the unadjusted analysis as well as when adjusted for severity

of illness (0.13 to 0.04, OR 0.30, P 0.023). The MICU andSICU model-adjusted rates of change were not significantly

different from each other.

ICU length of stay and mortality

The unadjusted and severity-adjusted length of stay and

mortality results for both ICUs pre- and post- telemedicine

implementation are summarised in Table 2. Unadjusted

(4.9 to 5.9 d, P 0.08) and severity adjusted (5.3 to 6.1 d,P 0.62) ICU length of stay both increased in the MICUafter telemedicine; however neither of these findings were

significant. In contrast, both unadjusted (5.0 to 3.3d,

P, 0.001) and severity adjusted (6.3 to 3.9 d, P, 0.001)

ICU length of stay decreased significantly in the SICU after

implementation of telemedicine. The rates of change

between the two units over time were significantly different

from each other (ANCOVA interaction P 0.005). Whileunadjusted ICU mortality decreased significantly in the

MICU after the telemedicine intervention (0.80 to 0.57,

P, 0.001), there was no significant change in the MICU

mortality after adjusting for severity of illness. Both

unadjusted and severity adjusted ICU mortality for SICU

patients decreased significantly after implementation of

telemedicine (0.09 to 0.01, OR 0.15, P 0.003).

Discussion

A randomized, double-blind, placebo-controlled study

would provide the best evidence about whether or not

telemedicine affects outcomes. However, it would be

difficult to organize. As a result many centres, including our

own, have chosen to conduct observational studies by

comparing historical control data with

post-implementation data.4,5,6,8,12,16,17,18 Unfortunately, it

is not possible to control for all potential confounding

variables. To help mitigate some of these confounders, we

evaluated data from two ICUs in the same health system.

The results indicate an association between the

implementation of telemedicine and a decrease in ICU

length of stay, ICU mortality and hospital mortality. No

such associations were seen in the medical ICU not exposed

Table 1 Hospital length of stay and mortality

Control Telemedicine

MICU (Pre) MICU (Post) P-value SICU (Pre) SICU (Post) P-value

No of patients 220 285 246 1499

APACHE score mean (SEM) 100.7 (2.5) 80.1 (2.5) ,0.001 46.2 (1.5) 54.1 (0.6) 0.005

Unadjusted hospital length of stay mean d (SEM) 13.2 (1.0) 11.3 (0.7) NS 15.6 (0.9) 15.1 (0.6) NS

Severity adjusted hospital length of stay mean d (SEM) 12.5 (1.1) 10.9 (0.8) NS 19.0 (1.0) 16.7 (0.8) NS

Unadjusted hospital mortality mean (SEM) 0.88 (0.02) 0.65 (0.03) ,0.001 0.11 (0.02) 0.06 (0.01) 0.003

Severity-adjusted hospital mortality mean (SEM) 0.74 (0.05) 0.56 (0.04) NS 0.13 (0.03) 0.04 (0.01) 0.023

NS denotes P 0.05

B A Kohl et al. ICU telemedicine

284 Journal of Telemedicine and Telecare Volume 18 Number 5 2012

to telemedicine. Despite an increase in the severity of illness

scores (i.e. patients were more ill) after telemedicine

implementation, there was a profound decrease in ICU

length of stay and ICU mortality in the SICU. In the

non-intervention ICU, however, there was a decrease in

severity of illness scores (i.e. patients were less ill), with no

change in ICU length of stay. These results may be partly

explained by the initiation of the telemedicine service and

the ability to provide additional oversight with best

practices. Compliance with best care processes has been

shown to reduce ICU morbidity.19,20

The present study had certain limitations. The principal

drawback of all observational studies is that causal relations

cannot be established from observed associations. In

addition, the medical chart abstraction was

non-randomized and involved selecting consecutive charts

within each quarter. This could bias results if the start of

each quarter coincided with other confounding influences.

During the period of study, working time restrictions were

instituted for all house staff (residents and fellows). While it

is possible that these regulations may have had a differential

effect on mortality in the two ICUs, a comparison in

different medical specialties did not support this

contention.21

One important difference in the overnight staffing,

however, was that an advanced trainee (critical care fellow)

was present in the SICU and not the MICU. It is therefore

possible that the SICU had greater oversight during evening

hours, which may have contributed to the improved

outcomes. During the study period, there were no

significant changes in faculty staffing or in the level of

training for the residents who were providing patient care in

the two ICUs. Another limitation of the study was that

severity of illness in the populations being compared was

very different. In particular, the MICU patients were not

only significantly more ill (as indicated by their higher

APACHE scores) than the SICU patients, but their baseline

APACHE scores in the pre-intervention period were

significantly greater than most similar MICU

populations.19,22,23,24 Given this finding, however, one

would have expected the ICU length of stay to have

significantly decreased as the severity of illness scores

decreased. This did not occur.

In the present study there was a dramatic disparity in the

sizes of the post-implementation populations compared.

This was a result of the electronic medical record that was

implemented at the same time as telemedicine. Since the

software automatically calculates the APACHE score for the

first 24 h of ICU admission, we chose to include all patients

admitted to the SICU during the study period because

manual chart abstraction was no longer needed. Thus,

manual chart abstraction for the purposes of APACHE

scoring was undertaken for both pre-implementation

groups in addition to the post-implementation MICU

group. We used a bootstrap calculation to confirm the

stability of our findings.

A final limitation arises from the fact that our

telemedicine programme, from the beginning, consisted of

two key elements: (1) additional oversight by remote nurses

and physicians via bi-directional communication links and

(2) an electronic medical record. Thus it is not possible to

know whether the observed outcomes were due solely to the

additional oversight provided and what, if any, was the

independent effect of installing the electronic medical

record. Indeed, the marriage between most telemedicine

technologies and electronic medical records is a

confounding effect that must be considered in all such

studies.2

In conclusion, implementation of telemedicine in the

surgical ICU was associated with significant reductions in

severity-adjusted ICU length of stay and mortality, as well as

hospital mortality. Over the same period, and within the

same hospital, a medical ICU not using telemedicine had

no significant change in any of the measured outcomes

after adjusting for severity of illness. However, it is not

possible to conclude definitively that the observed

associations seen in the SICU were due to the intervention.

Further work is thus required to quantify the effect of

telemedicine on ICU outcomes.

References

1 Berenson RA, Grossman JM, November EA. Does telemonitoring of

patients the eICU improve intensive care? Health Aff (Millwood)

2009;28:w93747

2 Kahn JM, Hill NS, Lilly CM, et al. The research agenda in ICU

telemedicine: a statement from the Critical Care Societies Collaborative.

Chest 2011;140:2308

3 Young LB, Chan PS, Lu X, Nallamothu BK, Sasson C, Cram PM. Impact of

telemedicine intensive care unit coverage on patient outcomes: a

systematic review and meta-analysis. Arch Intern Med 2011;171:498506

4 Lilly CM, Cody S, Zhao H, et al. Hospital mortality, length of stay, and

preventable complications among critically ill patients before and after

tele-ICU reengineering of critical care processes. JAMA 2011;305:217583

5 Thomas EJ, Lucke JF, Wueste L, Weavind L, Patel B. Association of

telemedicine for remote monitoring of intensive care patients with

mortality, complications, and length of stay. JAMA 2009;302:26718

Table 2 ICU length of stay and mortality

Control Telemedicine

MICU (Pre) MICU (Post) P-value SICU (Pre) SICU (Post) P-value

No of patients 220 285 246 1,499

APACHE score mean (SEM) 100.7 (2.5) 80.1 (2.5) ,0.001 46.2 (1.5) 54.1 (0.63) 0.005

Unadjusted ICU length of stay mean d (SEM) 4.87 (0.42) 5.89 (0.40) NS 5.04 (0.48) 3.29 (0.13) ,0.001

Severity-adjusted ICU length of stay mean d (SEM) 5.27 (0.52) 6.09 (0.43) NS 6.25 (0.50) 3.86 (0.17) ,0.001

Unadjusted ICU mortality mean (SEM) 0.80 (0.03) 0.57 (0.03) ,0.001 0.08 (0.02) 0.03 (0.004) ,0.001

Severity-adjusted ICU mortality mean (SEM) 0.54 (0.06) 0.42 (0.04) NS 0.09 (0.02) 0.01 (0.003) 0.003

NS denotes P 0.05

B A Kohl et al. ICU telemedicine

Journal of Telemedicine and Telecare Volume 18 Number 5 2012 285

6 Breslow MJ, Rosenfeld BA, Doerfler M, et al. Effect of a multiple-site

intensive care unit telemedicine program on clinical and economic

outcomes: an alternative paradigm for intensivist staffing. Crit Care Med

2004;32:318

7 Marcin JP, Nesbitt TS, Kallas HJ, Struve SN, Traugott CA, Dimand RJ. Use

of telemedicine to provide pediatric critical care inpatient consultations

to underserved rural Northern California. J Pediatr 2004;144:37580

8 Rosenfeld BA, Dorman T, Breslow MJ, et al. Intensive care unit

telemedicine: alternate paradigm for providing continuous intensivist

care. Crit Care Med 2000;28:392531

9 Grigsby J, Bennett RE. Alternatives to randomized controlled trials in

telemedicine. J Telemed Telecare 2006;12(Suppl. 2):7784

10 Wunsch H, Linde-Zwirble WT, Angus DC. Methods to adjust for bias and

confounding in critical care health services research involving

observational data. J Crit Care 2006;21:17

11 Ho PM, Peterson PN, Masoudi FA. Evaluating the evidence: is there a rigid

hierarchy? Circulation 2008;118:167584

12 Kohl BA, Gutsche JT, Kim P, Sites FD, Ochroch EA. Effect of telemedicine

on mortality and length of stay in a university ICU. Crit Care Med

2007;35(Suppl. 12):A22

13 Kohl BA, Sites FD, Gutsche JT, Kim P. Economic impact of eICU

implementation in an academic surgical ICU. Crit Care Med

2007;35(Suppl. 12):A26

14 Breslow MJ. Remote ICU care programs: current status. J Crit Care

2007;22:6676

15 Knaus WA, Wagner DP, Draper EA, et al. The APACHE III prognostic

system. Risk prediction of hospital mortality for critically ill hospitalized

adults. Chest 1991;100:161936

16 Zawada ET Jr, Kapaska D, Herr P, et al. Prognostic outcomes after the

initiation of an electronic telemedicine intensive care unit (eICU) in a

rural health system. S D Med 2006;59:3913

17 Morrison JL, Cai Q, Davis N, et al. Clinical and economic outcomes of the

electronic intensive care unit: results from two community hospitals. Crit

Care Med 2010;38:28

18 McCambridge M, Jones K, Paxton H, Baker K, Sussman EJ, Etchason J.

Association of health information technology and teleintensivist

coverage with decreased mortality and ventilator use in critically ill

patients. Arch Intern Med 2010;170:64853

19 Gajic O, Afessa B, Hanson AC, et al. Effect of 24-hour mandatory versus

on-demand critical care specialist presence on quality of care and family

and provider satisfaction in the intensive care unit of a teaching hospital.

Crit Care Med 2008;36:3644

20 Hasibeder WR. Does standardization of critical care work? Curr Opin Crit

Care 2010;16:4938

21 Prasad M, Iwashyna TJ, Christie JD, et al. Effect of work-hours regulations

on intensive care unit mortality in United States teaching hospitals.

Crit Care Med 2009;37:25649

22 Digiovine B, Chenoweth C, Watts C, Higgins M. The attributable

mortality and costs of primary nosocomial bloodstream infections in the

intensive care unit. Am J Respir Crit Care Med 1999;160:97681

23 Khan H, Belsher J, Yilmaz M, et al. Fresh-frozen plasma and platelet

transfusions are associated with development of acute lung injury in

critically ill medical patients. Chest 2007;131:130814

24 Rosenberg AL, Hofer TP, Strachan C, Watts CM, Hayward RA. Accepting

critically ill transfer patients: adverse effect on a referral centers outcome

and benchmark measures. Ann Intern Med 2003;138:88290

B A Kohl et al. ICU telemedicine

286 Journal of Telemedicine and Telecare Volume 18 Number 5 2012

Copyright of Journal of Telemedicine & Telecare is the property of Royal Society of Medicine Press Limitedand its content may not be copied or emailed to multiple sites or posted to a listserv without the copyrightholder's express written permission. However, users may print, download, or email articles for individual use.