2

Learning Objectives

• Describe the use of the Outcomes Impact Analysis model to assess economic impact of a CME activity

• Discuss the potential role of Outcomes Impact Analysis modeling in CME practice

3

What We’ve Been Hearing…

• “How do I convey the value of CME to non-CME administrators?”

• “I need to summarize an entire year of outcomes in a condensed format.”

• “I’ve been asked to justify my continuing medical education budget.”

• “How can I compare different outcomes from a number of activities?”

4

The Problems Confronting CME

• Funding and resources are decreasing– The core message of the value of CME is not

reaching internal and external stakeholders, policy makers, and society

– There are few resources for professionals to demonstrate the return on CME investment to nonmedical or noneducator audiences

5

Solutions

• Demonstrating the benefits of CME depends on effective communication

• Economic data provide a universally understood outcome measure—monetary impact

• Economic modeling of CME is a rapid, cost effective, and repeatable method of predicting cost impact

Outcomes Impact Analysis

7

Outcomes Impact Analysis: A Novel Analytic Framework

• Evaluating the economic impact of CME outcomes is based on the established principles of health economic modeling, decision analysis, probability theory, and statistical analysis

• No single existing economic analysis model is ideally suited for assessing CME outcomes

8

• CME symposium was attended by 133 participants

• Reviewed strategies for blood conservation and implementation of the Society of Thoracic Surgeons Blood Conservation Clinical Practice Guidelines

• 93.8% were committed to change their practice based on what they learned

• Major barriers to blood conservation cited were cost, administration buy-in, and inertia

Blood Conservation Symposium: Economic Implications

9

• Outcomes impact analysis was used to estimate the potential economic impact of applying CME-related learning to clinical practice for cardiac surgical outcomes:– Prevention of bleeding-related complications– Prevention of reoperation for bleeding in

coronary artery bypass graft (CABG) surgery• Costs averted were estimated from the

perspective of the healthcare provider

Evaluating the Economic Impact

10

Bleeding-related complications (BRC)• A study of 103,826 cardiac operations showed that mean

hospital costs were $12,128 higher in patients with a BRC than in those without a BRC; the additive costs from 142,533 thoracic operations was $15,8991

Reoperation for bleeding in CABG• Nineteen studies studies evaluated additive costs of

reoperation for bleeding following CABG (n=755,382); weighted mean was $24,0482-4

Assumptions• Operative volume

– The STS/AATS Workforce Report showed that adult cardiac surgeons (n=1211) performed a mean of 155 cardiac operations and 214 thoracic operations in 20095

Models and Parameters

1. Stokes ME et al. BMC Health Serv Res. 2011;11:135-148.2. Brown PP et al. Ann Thorac Surg. 2008;85:1980-1987.3. Herwaldt LA et al. Infect Control Hosp Epidemiol. 2003;24:44-50.

4. Speir AM et al. Ann Thorac Surg. 2008;85:1980-1987.5. Shemin RJ et al. Ann Thorac Surg. 2012;93:348-355.

11

Base Case Analysis• 3 in 10 participants changed clinical

practices as a result of CME learning in such a way as to prevent the outcome of interest (BRC or reoperation)

• Resulted in prevention of BRC and RFB in 2% and 1.5% of operations, respectively

• One year time period

Base Case

12

• One-way sensitivity analysis evaluated the potential savings for the base case when the proportion of participants who prevented outcomes of interest varied from 1 in ten to 5 in ten

• Probabilistic sensitivity analysis with second-order Monte Carlo simulation was used to evaluate robustness of the estimate to parameter uncertainty and to calculate a mean and confidence interval for the estimated value

Adjustments• Estimated values were standardized to 2012 $US

using the medical care component of the Consumer Price Index

Sensitivity Analysis

13

Results

ModelEstimated Value ($US 2012)

Base Case PSA Mean (95% CI)

Bleeding-related complications, cardiac 1,500,112 1,502,769 (869,860–2,359,068)

Bleeding-related complications, thoracic 2,715,104 2,715,246 (1,590,308-4,217,092)

Reoperation for bleeding, CABG 2,230,873 2,233,988 (1,223,901–3,648,719)

Ravyn D et al. J Cont Educ Health Prof. 2014;34(S1):S41-S46.

14

Dynamic Estimator

15

Outcomes Impact Analysis

Strengths Limitations

• Rapid, cost effective, and highly versatile

• Can be used in funding decision making or to evaluate outcomes after an activity

• Estimates of the monetary return on education

• Does not require patient-level or medical record data

• Independent of educational modality, audience, or therapy area

• Allows outcomes comparisons across heterogeneous activities

• Data, scenario, assumptions, and time horizon are tailored to the decision maker’s needs

• Results easily understood by nonmedical, noneducator, and lay audiences

• Sensitivity analysis demonstrates results across a wide range of plausible scenarios

• Cost data may be unavailable or unreliable

• Does not provide direct evidence of patient impact; outcomes inferred from utilization

• Serves as an adjunct to, not a replacement of educational outcomes

• Does not usually consider indirect or intangible costs

• Limited to expenditures, does not examine quality of life

• Requires expertise in the therapy area, health services, outcomes, and economic modeling

• As with all economic predictive models, it may be necessary to validate estimated values after implementation

16

Conclusions

• Plausible economic estimates suggest that CME-related learning favorably impacting practice yields substantial cost savings

• Model prediction of averted costs associated with CME allows estimation of the economic impact on outcomes in the absence of patient-level outcomes data related to CME activities

• This model may also be used to optimize CME resources by identifying areas of educational need most likely to result in cost savings

For more information contact Dana Ravyndravyn@cmeology.org