GETTING YOUR MONEY’S WORTH: RETURN ON INVESTMENT Craig Thomas, CDC-OSTLTS Greg Randolph, North Carolina Center for Public Health Quality Brynn Riley, Maine

GETTING YOUR MONEY’S WORTH: RETURN ON INVESTMENT

Craig Thomas, CDC-OSTLTS

Greg Randolph, North Carolina Center for Public Health Quality

Brynn Riley, Maine Department of Health

Craig Thomas, PhD

Division of Public Health Performance ImprovementOffice for State, Tribal, Local and Territorial Support

Centers for Disease Control and Prevention

Open Forum for Quality ImprovementJune 20, 2012

Measuring the Impact of Improvement: Lessons from the National Public Health

Improvement Initiative (NPHII)

Office for State, Tribal, Local, and Territorial Support

Division of Public Health Performance Improvement

National Public Health Improvement Initiative

Five year initiative for health departments

Adopt and institutionalize cross-cutting performance management and quality improvement methods

Improve accountability, efficiency, and effectiveness of public health programs and services

Advance health departments’ readiness for public health accreditation

NPHII Return On Investment Workgroup Why is ROI for improvement important?

Greater accountability in times of scare resources Stronger evidence base for what works to improve business

practices and service delivery for greater health impact Promotes a business culture of improvement in public

health

Purposes: Identify practical approaches to measuring the costs and

benefits of improvement with select NPHII grantees Provide technical assistance as requested Develop and share ROI methods, tools, and approaches

Analytic Approaches to ROI

Cost Benefit Analysis QI methods on health outcomes is important from

societal perspective Not always feasible

Budget Impact Analysis Offers practical approach to assessment of the costs and

benefits of QI methods• Calculates the cost and benefits of improving business

processes and service delivery• Focused on efficiency and effectiveness outcomes• Documents both tangible and intangible benefits of QI

Key QI Outcome Measures Efficiency: Doing something well with the least

amount of waste Time saved Money saved Reduced steps in a process

Effectiveness: Achieving a purpose or desired result Quality enhancement (e.g., completeness and accuracy of data) Increased customer service satisfaction Increased reach of services Implementation of evidence-based practices

Other QI Outcome Measures

Improvements that relate to both efficiency and effectiveness: Consolidation and sharing of services Leverage of additional funds / revenue streams Integration of networks Pilot testing and evaluation of innovative ideas

Overview of ROI Methodology for QI

Planning: Develop objectives; ‘AIM’ Statement, measurement and data

collection plans Data Collection:

Establish baseline, collect cost data and outcomes before, during, and after implementation of improvement effort

Data Analysis: Control for alternative hypotheses; convert hard date (e.g., time

and staff) to monetary value; calculate the ROI (net project benefits/project costs) times 100

Identify Intangible Benefits: Increased customer satisfaction; improved buy-in and

organization support; greater credibility within the community

NPHII: Strategies to Increase Efficiencies

Out of 74 grantees, the following have prioritized strategies to target efficiencies

Percent

Focus

70 Decreasing cost to deliver services

69 Decreasing time to deliver services

57.5 Decreasing time to award contracts

54 Decreasing staff allocation to deliver services

NPHII: Increasing Efficiencies through QI

Early examples in saving time and money

Lab data reporting to CDC: Reduced lag time from 14-21 days to 2–3 days

IT costs: Identified strategies to save over $1.2 million annually

NPHII: Strategies to Improve Effectiveness

Out of 74 grantees, the following are prioritizing strategies to strengthen effectiveness

Percent Focus

94.4 Using data to prioritize activities for maximum impact

94.5 Identifying and implementing evidence-based practices

92 Engaging in cross-jurisdictional partnerships

93 Using an organization-wide performance management system

NPHII: Improving Effectiveness thru QI

Early successes achieved by some grantees Increased enrollment in Medicaid family planning

program by 109 percent in one year

Streamlined clinic flow of family planning visits to improve patients’ experience

Increased referrals of pregnant women in WIC to smoking cessation programs

For more information please contact Centers for Disease Control and Prevention1600 Clifton Road NE, Atlanta, GA 30333Telephone, 1-800-CDC-INFO (232-4636)/TTY: 1-888-232-6348E-mail: [email protected] Web: www.cdc.gov

The findings and conclusions in this report are those of the authors and do not necessarily represent the official position of the Centers for Disease Control and Prevention.

Phillips, J., & Phillips, P. (2007). Show me the Money: The Use Phillips, J., & Phillips, P. (2007). Show me the Money: The Use of of

ROI in Performance Improvement, Part 1. Performance ROI in Performance Improvement, Part 1. Performance

Improvement; V. 46, No 9. Improvement; V. 46, No 9.

OSTLTS Performance Management / Quality Improvement OSTLTS Performance Management / Quality Improvement Resources: Resources: http://www.cdc.gov/stltpublichealth/Performance/index.html

Office for State, Tribal, Local and Territorial Support

Division of Public Health Performance Improvement

Resources

Application of ROI to Quality Application of ROI to Quality Improvement Projects Improvement Projects

Greg Randolph MD, MPHGreg Randolph MD, MPHNC Center for Public Health Quality

Why is ROI Important? Why is ROI Important?

• Need to improve performance given substantial burden of population health issues and health disparities in US

• PH agencies are increasingly applying QI methods to their work – involves a substantial upfront investment

• Sound business practices can help in a time of fiscal scrutiny and dwindling budgets/resources

How We Measure and Use ROI…How We Measure and Use ROI…

• ROI Model

• Template linked to aim statement’s measurable goals (see handout)

• Communications

17

ROI Process ModelROI Process Model

PlanningPlanning

Develop AIM and

Measurement Plan

------------------Think about

Potential Business Impact

Data Data CollectionCollection

Before Improvements

(Baseline)------------------

After Improvements-------------------Capture costs throughout

Data AnalysisData AnalysisMore CluesMore Clues

Waste identified via

VSM and observation

------------------

QI Tools used for

improvements

Convert Measurement and Benefits

Data to Monetary

Values

Calculate ROI Using Cost and Benefit Values

Identify Intangible Benefits

Generate Report with Conclusions and Lessons Learned

Communicate Results to Target Stakeholders

ReportingReporting

Lou Anne Crawley-Stout, MBA, CLSSBB, PMPLou Anne Crawley-Stout, MBA, CLSSBB, PMPAdapted from “ROI at Work” by Jack J. Phillips and Patricia P. PhillipsAdapted from “ROI at Work” by Jack J. Phillips and Patricia P. Phillips

Terminology and FormulaTerminology and Formula

18

ROI (return on investment): A performance measure used to evaluate the efficiency of an investment.

ROI = Benefits-Costs/Costs

EI (economic impact): Refers to costs and benefits of an activity.EI = Benefits-Costs

Example: NC Institute for Public Health (NCIPH)Project Benefits (annualized savings) Project CostsArchiving (paper/printing) $ 360 Staff time/travel $ 9850Rent elimination 1490 Archiving supplies 200Labor/time savings 16330 $10050Overtime reduction 2300

$20480

ROI: $20480-10050/10050 = $1.04 For every $1 invested in QI, NCIPH received in return $1.04 after costs.EI: $20480-10050 = $10430

Communication Tips Communication Tips • Do stakeholder analysis early and plan for

when/how/what will communicate to stakeholders – customize to specific stakeholder audience – Since ROI relatively new for public health, best to report face-to-

face

• Stress that ROI data should not be used to reduce budgets further – by cutting budget of organization generating substantial ROI,

would starving most innovative and effective organizations and halt spread of use of ROI

• Communicate how ROI illustrates you are better able to achieve your mission and other intangibles

Lessons Learned Lessons Learned • “Begin with the end in mind”

– Begin with your aim statement – Capture tangibles and intangibles throughout

• When converting tangible benefits into monetary values, be conservative– Vet with your team and selected stakeholders

• Intangible Benefits (non-monetary benefits) are extremely valuable

Additional ROI Results in Jan / Feb 2012 issue of Journal of Public Health Management & Practice

“Applying Lean Principles and Kaizen Rapid Improvement Events in Public Health Practice”

http://journals.lww.com/jphmp/toc/2012/01000http://journals.lww.com/jphmp/toc/2012/01000

21

Resources Resources

Resources Resources

• ROI at Work: Best-Practice Case Studies from the Real World; Jack J. Phillips and Patricia Pulliam Phillips

• www.ncpublichealthquality.org• www.ies.ncsu.edu/

Greg Randolph: [email protected]

Brynn RileyPerformance Improvement Manager, Maine

June 20, 2012

Quality Improvement (QI) teams have an opportunity to show projects’ return on investment (ROI) to stakeholders

QI teams should consider the: upfront investment before applying QI methods costs and benefits throughout the project’s

duration.

Accounting for costs and benefits will Help improve the organization’s case for undertaking a QI

projectHelp prioritize competing interests Help account for community or societal benefits where data

doesn’t exist

Include intangible benefits, which are a significant and growing part of an organization’s and community’s worth.

Developed by B. Riley - [email protected] and J. Moran – [email protected] 6/10/12

Consider how it translates into public health and community value.

Part of public health is preventing adverse health outcomes for a population, essentially creating “non-events”.

Including intangible benefits in cost benefit calculations account for the organizations added value to the community that may not otherwise be included.

Concerns: Intangible benefits cannot be easily measured in dollar

terms The benefits may not be viewed as valid by some people


Use the AIM statement as a springboard to hone the a QI Project’s focus and evaluate return on investment.


The Aim is sometimes called the Charter or Opportunity Statement. It is a goal.

A written discrete, measurable, and time-bound description.

Includes the scope of an improvement effort, as viewed by the organization and the team/individual striving for improvement.

It should be developed with input from Senior Leadership to ensure support for the Team and alignment with the strategic goals of the organization.

What will improve?

When will it improve?

How much will it improve?

For whom will it improve?


When developing the AIM statement stay at a strategic level - try not to get too operational.

Determine baseline measurements while framing the AIM statement

Outline the expected tangible and intangible benefits


AIM statements are not written once and set in cement.

A regular team task is to review the aim statement

"Remember, we’re here to increase immunizations by 50% within 12 months,"

How are we doing? Regular review of progress quantitatively

It is acceptable to tweak the aim, as the team learns more about their work or new information changes the team’s direction.


Section 1: Problem Description, Boundary, and Team Composition

Describe the problem or opportunity to be addressed:◦ Example: Physician/Immunizing Stakeholders are not utilizing the Immunization

Information System (IIS) tools provided, resulting in lower than acceptable immunization rates.

This process is important to work on now because of: (describe the impact this problem or opportunity is having on the agency, program, customers, employees, or the community

Example: Not utilizing the provided public health improvement tools in the IIS results in improperly immunized people, wastage of State supplied vaccine, client susceptibility to disease, and community risk of disease.

Team Sponsor: _______________________ Team Leader: _________________________ Team Members: ______________________ Area of Expertise:

________________ Developed by B. Riley - [email protected] and J. Moran – [email protected]

6/10/12

Section 1: Problem Description, Boundary, and Team Composition

The problem or opportunity starts with… and ends with… For Example: The problem or opportunity starts with the lack of utilization of IIS tools*

and with the assignment of staff to conceptualize and recommend an improvement plan.

…and ends with the publishing of a results document that demonstrates an increase in immunization rates via the developed methodologies and that also quantifies the cost of the improvement effort in order to identify the cost associated with sustainability

IIS* = Immunization Information System (ImmPact), Maine’s tracking system



Internal and

External

Tangible Benefits (cost benefit) Intangible Benefits

Are benefits produced by an investment that is immediately obvious and measurable. (i.e. IT software)

A tangible benefit can usually be expressed in terms of a monetary savings.

Are indirect benefits which cannot be felt or touched - such as improved health access, staff morale, or the image of the health department by stakeholders.

Section 2: Internal and External Benefit and Cost Description: This opportunity has the following estimated potential benefits internally and to the external community:


Section 2: Internal and External Benefit and Cost Description:

What is unique to many public health organizations is the need for a way to account for intangible benefits to the community being served.

For example, tangible benefits may be a decrease in the smoking rate, an increase in the immunization rate, reduced infant mortality, etc.

An intangible benefit may be a program that contributes to increased self-esteem resulting in greater use of preventive services and adherence to an asthma management plan.


Internal: Tangible Benefits (cost benefit) Intangible Benefits

Reduced vaccine wastage Increased number of doses

recorded within the IIS* Increased Immunization Rates

Increased Stakeholder awareness of the IIS*

Increased Stakeholder satisfaction with the IIS*

Increased use of IIS tools

External: Tangible Benefits (cost benefit) Intangible Benefits

Increased Client Visits Increased Services Rendered Increased Billing Activities Increased Immunization Rates

Additional opportunity to serve the client

Increased Community Marketing Opportunity

Improved Assessment Rates

Section 2: Internal and External Benefit and Cost Description: This opportunity has the following estimated potential benefits internally and to the external community:

IIS = Immunization Information System (ImmPact), Maine’s tracking system

Some questions to consider are: If we implement a solution to this problem what are the potential cost

categories that will be impacted and by how much?

How much will it cost to develop a solution and implement it?

What is the cost of doing nothing?

What is the cost of sustaining the solution?

Are there possible unintended consequences, and if so, what are the costs?

If we plan to use existing resources, what will we have to stop doing to divert resources to this problem/solution?


Section 3: Current State Performance and Desired Future State

Describe the current state of the problem or opportunity and its current performance (baseline data):◦ External Stakeholders are not using the integrated Public Health

Improvement tools resident in the IIS for their use. These tools include, but are not limited to; Immunization Coverage Reports (ICR), Reminder/Recall Letters (R/R), Client Vaccine Forecasting (ACIP Forecasting)

Describe the ideal future state:

What are the key driving and restraining forces impacting the current state and the ideal future state?


Section 4: Improvement Description: This effort should improve the current state by:

Describe the improvement goals to be achieved:

The timing of these improvements: We will measure improvements by:


Section 5: Internal and External Customer Identification:

For the following customers/clients (customers, staff or

those affected by the process under improvement)

For Example: Maine Immunization Program Staff

Consumer Services Staff IIS Support Staff Administrative Stuff

Physician’s Office

Nurse Staff Administrative Support Billing/Fiscal Support


This presentation is based on the article:

Calculating the Real Value of Process Improvement: Factoring in Intangible Benefits. J. Moran and B. Riley, Process Excellent Network, April 2012

Web Address when ready: Process Excellence Network April 2012 http://www.processexcellencenetwork.com/people-performance-and-change-in-process-

improveme/articles/calculating-the-real-value-of-process-improvement/&mac=SSIQ_OI_Featured_2011&utm_source=processexcellencenetwork.com&utm_medium=email&utm_campaign=HrOptIn&utm_content=4/12/12
