Effectiveness of Computerized Decision SupportSystems Linked to Electronic Health Records:A Systematic Review and Meta-Analysis

Based on: Moja L, Kwag KH, Lytras T, Bertizzolo L, Brandt L, Pecoraro V, Rigon G, Vaona A, RuggieroF, Mangia M, Iorio A, Kunnamo I, Bonovas S. Effectiveness of Computerized Decision SupportSystems Linked to Electronic Health Records: A Systematic Review and Meta-Analysis. Am J PublicHealth. 2014 Oct 16:e1-e11.

Systematic review Eligibility criteria

P: Population

I: Intervention

C: Comparisons

O: Outcomes

Population

Post-graduate health professionals (medical, nursing, and allied health) in primary, secondary, and tertiary care settings.

Only interventions that were implemented in a real, non-simulated, clinical setting were considered.

Definition of CDSS:

An information system aimed to support clinical decision-making, linking patient-specific information in Electronic Health Records (HER) with evidence-based knowledge to generate case-specific guidance messages through a rule- or algorithm-based software.

Moja et al., Am J Public Health 2014

Intervention

CDSS format

Patient informationfrom EHR

Evidence-basedknowledgefrom literature or a priorievidence (e.g., guidelines or point-of-care services)

Patient-specificguidancemessages

Point-of-care

Electronic media

Rule- or algorithm-basedsoftware

Moja et al., Am J Public Health 2014

Comparisons

Access to CDSSs according to our definition compared to:

1. Standard care with no access to CDSSs

2. CDSSs that does not generate advice

3. CDSSs that are not based on evidence

Trials comparing arms accessing the same CDSS at different intensities (e.g., one arm having guidance messages pushed to the health professional versus another arm having guidance message statically available in a folder) were not pooled together with the other trials in the quantitative analyses.

Outcomes

1. Mortality: We selected mortality as it is the most relevant and objective outcome, although there may exist variability across studies with regards to the time frame during which mortality is captured.

2. Morbidity: We selected and grouped clinically relevant patient outcomes such as occurrence of illness (e.g., pneumonia, myocardial infarction, stroke), progression of disease, and hospitalization.

3. Economic outcomes: Information about health care utilization (e.g., length of stay, emergency department visits, and primary care consultations), and costs.

Exclusion criteria

1. Non-RCTs

2. Systems that alter the guidance based on previous experience or average behaviors (i.e., expert systems)

3. Outcomes: a. Patient satisfaction

b. Measures of process

c. Health care professional activity/performance (e.g., adherence to guidelines, rates of screening and other preventive measures, provision of counselling, rates of appropriate drug administration, and identification of at-risk behaviors)

Summary of evidence search and selection

Moja et al., Am J Public Health 2014

Results: Mortality

16 RCTs

37,395 individuals• Intervention: 18,848

• Control: 18,547

Overall mortality rate:• Intervention: 6.2%

• Control: 6.0%

7 RCTs reported lowermortality in interventiongroup, while 8 reportedhigher.

Random-effects model RR=0.96 (95% CI: 0.85–1.08)Fixed-effect model RR=1.00 (95% CI: 0.92–1.08)

Moja et al., Am J Public Health 2014c

Results: Morbidity

9 RCTs

13,868 individuals

A weak inverse association between CDSS use and morbidity from any disease.

Random-effects model RR=0.82 (95% CI: 0.68–0.99)Fixed-effect model RR=0.91 (95% CI: 0.83–1.00)

Moja et al., Am J Public Health 2014

Results: Economic outcomes (qualitative)

• 17 RCTs

• RCTs highly heterogeneous in direction and magnitude of effects across costs and health service utilization (e.g., drug or test orders), although small in magnitude

• Across economic outcomes, interventions equipped with CDSSs did not consistently perform better than non-equipped ones

Moja et al., Am J Public Health 2014

Risk of bias assessment

• 18 RCTs included in meta-analyses (mortality and morbidity outcomes)

• 7 studies at high risk of bias (39%)

• 10 studies at unclear risk of bias (56%)

• 1 study (5%) at low risk of bias

• Majority of trials did not measure mortality as an outcome, but reported it as an additional information, often as a reason for loss to follow-up

• Risk of bias assessment did not evaluate studies based on their intended outcomes, but according to the two outcomes of our systematic review

Limitations

• We did not evaluate the quality of the evidence-based information supporting the CDSS recommendations.

• We accepted study authors’ description of a CDSS as evidence-based at face value, even if the authors did not explain the source of evidence or knowledge in detail.

• Limited number of trials, especially regarding the meta-analysis for the morbidity outcomes, increases the uncertainty of the findings and conclusions.

• Included trials were conceptually heterogeneous in terms of their design, setting, participants, interventions, and the definition/measurement of outcomes.

• Literature search was restricted to studies published in indexed journals. We did not search for unpublished studies or for source data.

• Trials included in this meta-analysis were not designed to specifically analyze the relationship between mortality and CDSS use.

Conclusion

• CDSSs do not result in substantial benefits or risks for patients in terms of mortality. • This effect, when it exists, is largely dependent on the disease and setting

characteristics. Focusing on subgroup analyses, however, can lead to misleading claims when the overall data are limited and unavoidably weak due to inherent design problems.

• Effects on morbidity may exist. • The magnitude of the effect, in the order of 10-20%, could be large enough to

impact on mortality if appropriate follow up is ensured.

• The results of this study may provide enough evidence to fuel the debate on the prospects of CDSSs.