Upload
others
View
4
Download
0
Embed Size (px)
Citation preview
Estimating Return on Investment for Public Health Improvements Tutorial on Using the new Tool
Karl Ensign, Director of Evaluation
Association of State and Territorial Health Officials (ASTHO)
Beta Testers:
Brynn Riley, Maine
Josh Czarda, Virginia
Susan Logan, Connecticut
Tool Developed by Dr. Glen P. Mays, University of Kentucky
June 2013
Helps answer the following
questions:
Are we making the right
investments?
Are we becoming more efficient?
What bang are we getting for our
buck?
What is our budget
accomplishing?
Are we being good stewards?
Must be done thoughtfully
and carefully
ROI is one way of
measuring and
communicating public
health effectiveness in a
manner that is particularly
salient for policymakers,
funders, administrators and
the general public
ROI is a Specialized Form of Analysis
Outcomes are monetized:
Compares the costs of an intervention with its benefits – in
financial terms
Yields the net return on investment – over time
Sensitivity analysis can be conducted – different levels of
investments and their benefits
ROI Calculated:
Net benefit = Benefits – Costs
ROI = Benefits – Costs
Costs
Hypothetical values:
• $5 = ($400 + $500 + $300) – ($150 + $50)
($150 + $50)
Or …
“…a dollar spent on pediatric immunization is estimated to save $5 in treating preventable
illness.”
WHERE AND HOW HAS ROI BEEN USED IN PUBLIC HEALTH?
Public health programs (injury prevention, tobacco prevention and control)
Aggregate public health spending (Glen P. Mays, UKY)
Improvement projects undertaken by agencies (primary ASTHO focus)
• New data probably need to
be collected
• An investment may take
years to produce benefits
• Benefits may accrue outside
the agency (population
health)
• Benefits may be difficult to
link back to a specific public
health program(s) or
function(s)
Challenges from an Agency Perspective
ROI of QI – Advantages and Disadvantages
ROI is more immediate
Timeframe is shorter
ROI accrues more directly
to the agency
Tends to be narrower in scope
Return is more modest
Funding from Prevention & Public Health Fund, ACA
Measures ROI for improvement projects
QI projects undertaken through the National Public Health
Improvement Initiative (NPHII)
ASTHO led workgroup provided input and guidance
CDC, state and local agencies, foundations, academia
Developed by Glen Mays, University of Kentucky
Beta tested by Connecticut, Maine, and Virginia
Pathways to Realizing ROI for
QI
Reductions in standard operating
costs
• Greater efficiencies realized
Revenue enhancements
• Increased cost reimbursement
Increased productivity of agency
functions
• Increased service encounters
Decreased time to produce outputs
• Reduced cycle time process
Achievement of health outcomes
• Tool accommodates this
Cost Category Pre-
Implementation(
Baseline)
Post-
Implementation
(Year 1)
Year 2
…
Personnel Costs
Non-Personal
• Contracted Services
• Office Operations
• Facilities/Maint/Rent
• Communications
• Equipment
• Construction/Renovation
Other Direct Costs
Indirect Costs
Planning and Development Costs
Cost Category Pre-
Implementation(
Baseline)
Post-
Implementation
(Year 1)
Year 2
…
Personnel Costs
Non-Personal
• Contracted Services
• Office Operations
• Facilities/Maint/Rent
• Communications
• Equipment
• Construction/Renovation
Other Direct Costs
Indirect Costs
Routine Operating Costs
Outcomes/Outputs
Cost Category Pre-
Implementation
Post-
Implementation
(Year 1)
Year 2
…
Service Units Delivered
Required Production Time
Target Population Reach
Other Outcomes/Outputs
How the Tool Calculates ROI
ROI = Benefits – Costs
Costs Improvements in Routine Operations – Investment Costs
Investment Costs
(Routine Operations + Other Outcomes) – Investment Costs
Investment Costs
Incorporates Standard Accounting Practices in ROI Calculation
Amortization
The cost of an investment should not be
absorbed entirely in the first year
Amortization rate spreads the agency’s
cost/investment over the useful life of the
product
Present value
The relative worth of a single dollar changes over time
Accurate comparisons are made by applying a discount rate
(inflation) to
• Costs
• Returns
Tool Can Be Used throughout Project
Prospectively – Planning Phase
Implementation Phase
Retrospectively – Post Implementation
WHEN & HOW TO USE THE TOOL
Ask ourselves the Right Questions
Is ROI the corrective evaluative tool? Have outcomes been defined and agreed upon?
Have performance measures been set and tracked?
Has effectiveness been evaluated?
How will the ROI results be used? Internally
• Allocation
• Investment maximization
Externally
• Budget justification
Apply ROI Correctly
Build evaluation methods – including ROI – into
project inception
Clearly specify intended purpose and use of ROI
Think about how results will be communicated
sooner rather than later
Conduct ROI through a
transparent process
Conduct ROI through
an inclusive process
Implementers and end
users of the analysis
LET’S SEE HOW IT WORKS!
Testing the New ROI Tool in Maine: A Prospective
Look
Brynn Riley, MS
Maine Center for Disease Control and Prevention
June 7, 2013
6/19/2013 22
Maine
6/19/2013 23
• 31,000 square miles
• 1.3 million people
• Average: 43 people per square mile (ppsm). o US = 87 ppsm.
• 16 counties which comprise of 8 Districts.
• Each District has a District Liaison o Role: to increase collaboration for a greater consistency and equity in the
statewide delivery of public health.
o They serve as liaisons between state planning and public, non-profit and
business sectors.
Background on QI Initiative • AIM: The District Liaisons will reduce their overall
travel costs to and from meetings by 50% by March
1, 2014.
• Plan o Identify the causes of high travel costs
o Identify alternative solutions to travel that address the highest cost areas
o Apply the ROI tool to analyze the business case
o Put a plan in place that provides a sustainable approach
o Track the new travel costs
o Evaluate the effectiveness of the new approach
6/19/2013 24
Baseline Data
All Other
District
Liaison
RT mileage to
Augusta Mileage cost
Lodging &
per diem
Total cost no
overnight
Total cost w/
overnight
% of time
traveling
DL 1 500 233.00$ 141.00$ 233.00$ 374.00$ 60
DL 2 330 158.20$ 128.00$ 158.20$ 286.20$ 7
DL 3 400 176.00$ 128.00$ 176.00$ 304.00$ 25
DL 4 400 176.00$ 128.00$ 176.00$ 304.00$ 25
DL 5 150 66.00$ 66.00$ 66.00$ 10
DL 6 178 83.32$ 83.32$ 83.32$ 10
DL 7 80 37.70$ 37.70$ 37.70$ 15
DL 8 62 29.78$ 29.78$ 29.78$ 1
2100 960.00$ 525.00$
1-day with overnight stays
If all travel: 960.00$ 1,485.00$
Average salary with fringe: $90,000
Cost of Quarterly Trip to Augusta for DL meeting
DL travel costs
Walk-through of ROI Tool
and Definitions
Time Periods:
Pre-Implementation
(Plan), Implementation (Do and Check),
Post-implementation
(Act)
Gather Financial
Data:
Salaries, Supply costs,
Travel, etc.
Calculate Costs of Planning
and Intervention
Calculate Costs of
Operations
Calculate Costs/
Savings for Outcomes
ROI is Calculated
Using the ROI Tool
Project Investment Costs
27
From To From To
Cost Category Reporting Period: 1/1/2012 12/31/2012 1/1/2013 2/28/2013
Personnel (see below) List by name or by category FTE % Cost FTE % Cost
District Liaison Cumberland 12.00% 37,500.00$ 4,500.00$ 0.00% -$
District Liaison Downeast 5.00% 37,500.00 1,875.00 0.00% -
0.00% - 0.00% -
Office Manager 1 OHE Division 0.00% - 10.00% 4,166.00$ 416.60
Office Manager 2 PHS Division 0.00% - 10.00% 4,166.00$ 416.60
Office Manager 3 Chronic Disease Division 0.00% - 10.00% 4,166.00$ 416.60
Office Manager 4 Infectious Disease Division 0.00% - 10.00% 4,166.00$ 416.60
Office Manager 5 Environmental Health Division 0.00% - 10.00% 4,166.00$ 416.60
0.00% - 0.00% -
0.00% - 0.00% -
Office operations
T ravel/conferencing Parking Passes ($3 pp per day) 18.00
Equipment
Software Temp WebEx License for testing $50
Software Web Ex License - annual 550.00
Subtotal direct costs 6,443.00$ 2,633.00$
Indirect cost percentage (see below)
Indirect costs (see below) -$ -$
Notes:
Total costs of investment in strategy 6,443.00$ 2,633.00$
Years of expected utility from pre-implementation investments (see below) 1.00
Amortized Pre-Implementation Investment Costs $6,636.29 $6,636.29
Pre-Implementation - Baseline Implementation Period 1
Salary and
Fringe
Salary and
Fringe
Staff time devoted to the project
Web EX training time
Represents 5 months of pre-implementaion work spread out over 12 mo's
Represents 8 weeks of implementation 3/1-4/15
Routine Operating Costs
From To From To From To
Cost Category Reporting Period: 1/1/2012 12/31/2012 1/1/2013 12/31/2013 1/1/2014 12/31/2014
Personnel (see below) Name / Notes FTE % Cost FTE % Cost FTE % Cost
District Liaison York - travel time 10.00% 90,000.00$ 9,000.00$ 8.00% 15,000.00$ 1,200.00$ 5.00% 90,000.00$ 4,500.00$
District Liaison Cumberland tarvel time 7.00% 90,000.00$ 6,300.00 5.00% 15,000.00$ 750.00 3.50% 90,000.00$ 3,150.00
District Liaison Downeast travel time 25.00% 90,000.00$ 22,500.00 20.00% 15,000.00$ 3,000.00 12.00% 90,000.00$ 10,800.00
District Liaison Penquis travel time 25.00% 90,000.00$ 22,500.00 20.00% 15,000.00$ 3,000.00 12.00% 90,000.00$ 10,800.00
District Liaison Western travel time 10.00% 90,000.00$ 9,000.00 8.00% 15,000.00$ 1,200.00 5.00% 90,000.00$ 4,500.00
District Liaison Aroostook travel time 60.00% 90,000.00$ 54,000.00 50.00% 15,000.00$ 7,500.00 30.00% 90,000.00$ 27,000.00
District Liaison MidCoast travel time 15.00% 90,000.00$ 13,500.00 10.00% 15,000.00$ 1,500.00 7.00% 90,000.00$ 6,300.00
District Liaison Central travel time 1.00% 90,000.00 900.00 1.00% 15,000.00$ 150.00 1.00% 90,000.00$ 900.00
0.00% - 0.00% - 0.00% -
Office Manager 1 OHE Division 0.00% - 0.00% - 10.00% 25,000.00$ 2,500.00
Office Manager 2 PHS Division 0.00% - 0.00% - 10.00% 25,000.00$ 2,500.00
Office Manager 3 Chronic Disease Division 0.00% - 0.00% - 10.00% 25,000.00$ 2,500.00
Office Manager 4 Infectious Disease Division 0.00% - 0.00% - 10.00% 25,000.00$ 2,500.00
Office Manager 5 Environmental Health Division 0.00% - 0.00% - 10.00% 25,000.00$ 2,500.00
Office operations
Travel/Mileage $990.8 per DL staff trip to Augusta (990.8 x 12) 11,889.00 990.8 one per quarter (January) 990.80 990.8 x 4, one per quarter 3,963.20
T ravel/conferencingParking passes ($3 per pass x 12 mtgs x 8 DLs) 288.00
Equipment
Software Web Ex License 550.00 550.00
Subtotal direct costs, operating 149,877.00$ 19,840.80$ 84,963.20$
Indirect cost percentage (see below)
Indirect costs, operating (see below) -$ -$ -$
Total costs, operating 149,877.00$ 19,840.80$ 84,963.20$
Salary and
Fringe
Salary and
Fringe
Salary and
Fringe
Pre-Implementation - Baseline Implementation Period I Implementation Period 2
Represents a decreasing average time spread out over 12 mo's
Measuring Outputs
6/19/2013 29
Measures of Production Time
TRUE Description: output/event and time frameTotal
events
Average
time (days)
Total time
(days)
Total
events
Average time
(days)
Total time
(days)
Total
events
Average time
(days)
Total time
(days)
Building new collaborative
partnerships through training
sessions with key stakeholders
Overall average travel time from office to Augusta:
DL's monthly visit to Me CDC (note: total travel time for all
DL's is 36.2 hrs / 8 = 4.525 days of travel for one mtg in
Augusta) 12.00 4.525 54.30 4.00 4.525 18.10 4.00 4.525 18.10
Building new collaborative
partnerships through training
sessions with key stakeholders
Overall average travel time from office to local meetings:
Total DL's local meetings avg 80/mo, (80x12 = 960
mtgs); avg 30 miles R/T @ 60 mph, 30 minutes is
.06265 of an 8 hr day is 58 days. 960.00 0.630 604.80 480.00 6.300 3,024.00 480.00 6.300 3,024.00
Baseline Post Period - Year I Post Period - Year 2Check here to use these measures in ROI calcuations
The travel time saved is used to provide district partner trainings.
Post Period – Year 3 data is the same as year 2.
Saved staff time invested in other activities produces the following value
ROI Analysis
6/19/2013 30
I. Analysis EXCLUDING Output, Production Time and Outcome Factors
Discount Rate 3%
Pre-Implementation Total
Baseline 1 2 3 (All Periods)
Investment in Project
Amortized Pre-Implementation Costs $6,443.00
Ongoing Implementation Costs 2,633.00 - -$
Total Annual Implementation Costs 6,443.00 2,633.00 - -
x Present Value Factors (see below) 1.00 0.97 0.94 0.92
Total Discounted Annual Implementation Costs 6,443.00$ 2,556.31$ -$ -$ 8,999.31$
Financial Gain/Loss Attributable to Project
Estimated Changes in Operating Costs (130,036.20)$ (64,913.80)$ (64,913.80)$
Estimated Changes in Revenue -$ -$ -$
Total Net Financial Gain/Loss $130,036.20 $64,913.80 $64,913.80
x Present Value Factors (see below) 0.97 0.94 0.92
Total Discounted Annual Net Gain/Loss 126,248.74$ 61,187.48$ 59,405.32$ 246,841.54$
Return on Investment Summary
Undiscounted Annual Net Cash Flows (6,443)$ 127,403$ 64,914$ 64,914$ 250,788$
Cumulative ROI 14.03 20.83 27.43 27.43
Net Present Value 237,842$
Internal Rate of Return 1929%
Implementation Period
ROI Analysis II. Analysis INCLUDING Output and Outcome Measures (Production Time, Reach, and Health-Related Outcome Factors)
Total
Baseline 1 2 3 (All Periods)
Investment in Project
Amortized Pre-Implementation Costs $6,636.29 $6,636.29 $6,636.29 $6,636.29
Ongoing Implementation Costs $0.00 $2,633.00 $0.00 $0.00
Total Annual Implementation Costs $6,636.29 $9,269.29 $6,636.29 $6,636.29
x Present Value Factors (see below) $1.00 $0.97 $0.94 $0.92
Total Discounted Annual Implementation Costs $6,636.29 $8,999.31 $6,255.34 $6,073.15 $27,964.09
Financial Gain/Loss Attributable to Project
Estimated Changes in Operating Costs (130,036.20)$ (64,913.80)$ (64,913.80)$
Estimated Changes in Revenue -$ -$ -$
Total Net Financial Gain/Loss 130,036.20$ 64,913.80$ 64,913.80$
x Present Value Factors (see below) 0.97$ 0.94$ 0.92$
Total Discounted Annual Net Gain/Loss 126,248.74$ 61,187.48$ 59,405.32$ 246,841.54$
Non-Financial Gains/Losses Attributable to Project
Estimated Changes in Output Calculation not enabled - - -
Estimated Change in Cost per Unit Output - - -
Monetary Value of Output Gain/Loss -$ -$ -$
x Present Value Factors (see below) -$ -$ -$
Total Discounted Value of Output Gain/Loss -$ -$ -$ -$
Cumulative ROI including Value of Output -$ -$ -$ -$
Estimated Production T ime Gain/Loss (2,383.00) (2,383.00) (2,383.00)
Monetary Value of T ime Gain/Loss (121,135.83)$ (532,534.31)$ (532,534.31)$
x Present Value Factors (see below) 0.97$ 0.94$ 0.92$
Total Discounted Value of T ime Gain/Loss (117,607.61)$ (501,964.66)$ (487,344.33)$ (1,106,916.59)$
Cumulative ROI including T ime Cost 0.55$ (19.74)$ (30.76)$ (30.76)$
Implementation Period
Highlights of using the ROI Tool
• Benefits: o Used prospectively to assess potential ROI and build a business case for
the project
o Will be used throughout the project to compare estimates against actual
project costs
o Could be used to compare two multiple strategies
o Seems daunting at first, but once data is collected, calculations are done
for you
• Cautions: o Some of the data can be sensitive (i.e. salaries)
o Some data can be difficult to obtain
6/19/2013 32
Josh Czarda, Performance Improvement Manager, Virginia Department of Health
Example 1: Justifying Continuing Operations Increase Enrollment in Plan First – A Brief Background
(A Medicaid Family Planning Program)
34
Post Partum
Birth Control
Other – Family Planning
LHD
External Provider
Registration
WebVision
DSS
Individuals Providers Enrollment Process
Service
Billing
Service Provide
d
STD Testing
WIC Clinic
Demographic, Age, Income,
Insurance
Possible - Prompt for Plan First. Clerk
still must know parameters
Ask Questions to Determine Eligibility
If eligible – fill out application/ self populates some
fields
If Interested Individual Signs & Dates
LHD Submits
Printed App to DSS
Eligible
No More
Needed
Application Given to
Individual
Not Interested
Individual Follows Up
Eligibility
LHD Bills
Medicaid
Example 1: Justifying Continuing Operations Increase Enrollment in Plan First – A Brief Background
(A Medicaid Family Planning Program)
35
Post Partum
Birth Control
Other – Family Planning
LHD
External Provider
Registration
WebVision
DSS
Individuals Providers Enrollment Process
Service
Billing
Service Provide
d
STD Testing
WIC Clinic
Demographic, Age, Income,
Insurance
Possible - Prompt for Plan First. Clerk
still must know parameters
Ask Questions to Determine Eligibility
If eligible – fill out application/ self populates some
fields
If Interested Individual Signs & Dates
LHD Submits
Printed App to DSS
Eligible
No More
Needed
Application Given to
Individual
Not Interested
Individual Follows Up
Eligibility
LHD Bills
Medicaid
No Incentive
Title X & Payment Issues
System Understanding
& Function
Tracking & Follow Up Capability
Application Process/
Form Length
Auto-Enrollment & Payment
Knowledge
Knowledge
Knowledge
Eligibility Levels
Tracking/ Notificati
on
Background - Implementing Changes and Greater Challenges Realizing Health Outcome ROI
0
5000
10000
15000
20000
25000
30000
35000
40000
45000
Plan First Enrollment – Medicaid Family Planning
Women
Men
Total
Determining
ROI For Plan
First Project
Project
Implementation
Costs
New Staffing
Costs
Marketing
Costs
Increased
Operational
Time on
Enrollment &
Billing Staff
Initial PI Teams
Costs DMAS, DSS & Local Health District Staff
Hired 2 FTEs to Run Program
Marketing Outreach, Focus
Groups, etc
Additional Time, Billing & Enrolling
Step 1: Calculating All Costs Pre-Implementation – Implementation Phases
Step 1: Calculating All Costs Pre-Implementation – Implementation Phases
$-
$10,000.00
$20,000.00
$30,000.00
$40,000.00
$50,000.00
$60,000.00
$70,000.00
$80,000.00
ALEXA
ND
RIA
ALLEG
HA
NY
ARLIN
GT
ON
CEN
TR
AL S
HEN
AN
DO
AH
CEN
TR
AL V
IRG
INIA
CH
ESA
PEA
KE
CH
EST
ER
FIE
LD
CH
ICK
AH
OM
INY
CRA
TER
CU
MBER
LA
ND
PLA
TEA
U
EA
STER
N S
HO
RE
HA
MPTO
N
LEN
OW
ISC
O
LO
RD
FA
IRFA
X
LO
UD
OU
N
MO
UN
T R
OG
ER
S
NEW
RIV
ER
NO
RFO
LK
PEN
INSU
LA
PIE
DM
ON
T
PIT
TSYLV
AN
IA/D
AN
VIL
LE
PO
RTSM
OU
TH
PR
INC
E W
ILLIA
M
RA
PPA
HA
NN
OC
K
RA
PPA
HA
NN
OC
K-RA
PID
AN
RIC
HM
ON
D
RO
AN
OK
E
SO
UT
HSID
E
TH
OM
AS JEFFER
SO
N
TH
REE R
IVERS
VIR
GIN
IA B
EA
CH
WEST
PIE
DM
ON
T
WEST
ER
N T
IDEW
AT
ER
Measuring Increased Revenue from Medicaid Billing
Through May 2013
Total Paid 2011
Total Paid 2012
2013 Claims
Step 2: Measuring Immediate Impact
Initial View of Short Term ROI Calculations Using the Tool
Plan First Population Pregnancy Rate
12.9
5.07
Comparison of Pregnancy Rate Per Thousand
Average Virginia Rate vs. Plan First Population
Step 3. Measuring ROI of Long Term Health Outcome Impact
Step 3. Measuring ROI of Long Term Health Outcome Impact
• Working with DMAS to Determine Cost Per Pregnancy to Medicaid
• Calculating Variation in Pregnancy Rate to Determine Savings
• Calculating ROI Compares All Costs Including Medicaid Payments for Plan First
Traveler
Supervisor
Business Manager
Office Director
Executive Advisor
Deputy Commissioner
Commissioner
Secretary HHS
Time Spent Per Unit
Salary Cost Per Unit
X
Number of TARs
(1000)
X
Example 2: Proactively Determining Is A PI Project Worth It? Automating the VDH Travel Process
= 73K in
Time and Effort
VS.
Estimated $130,000
Programming Costs
Optimistic Future State
T&E $40,293
Travel Cost Increase
From Approval
Delay
+
=
Sample Routine Operating Cost Calculation
TAR Automation ROI
The tool helps you recognize unanticipated costs and can be great for both prospective and retrospective decision making
ROI Analyses Must be Dynamic ◦ PI Projects vary in complexity and metrics
◦ ROI Tools must be flexible ◦ Amortization of project costs can be challenging as change processes fluctuate
Illustrating & Showing ROI is Critical to Drive & Promote Continuous PI
Measuring Health Outcome ROI Remains A Critical Challenge
◦ ROI for public health remains un-quantified for Public Health ◦ A National Standardized Cost Analyses Calculating Cost Per Incident/Unit should be
established
Susan Logan, MS, MPH
Connecticut Department of Public Health
June 10, 2013
Testing the New ROI Tool in Connecticut: A Prospective Look
Improving the Connecticut Department of Public Health Databases
AIM: Make three DPH databases compliant with the CT DPH policy on collecting
sociodemographic data (similar to federal OMB-15 directive in 1997)
Increase the percentage of compliant databases from 4% to 10%
Put a plan in place to modify the remaining databases
Make sure that all newly-designed databases are aligned with the DPH data collection policy.
The modification process for the targeted databases was implemented and this strategy will be followed on a continuous quality improvement basis through 2014.
Background on Quality Improvement Initiative
6/10/2013 48
Walk-through of
ROI Tool and Review Definitions
Time Periods:
Pre-Implementation
(Plan), Implementation (Do and Check),
Post-implementation
(Act)
Gather Financial Data:
Salaries, Computer
costs, Supply costs, Travel,
etc.
Calculate Costs of Planning
and Intervention
Calculate Costs of
Operations
Calculate Costs/
Savings for Outcomes
ROI is Produced Based on
Prior Calculations
Process
6/10/2013 49
Calculate Costs of QI Planning
Calculate Costs of QI Intervention
Determining Investment Costs: QI Planning and Implementation
• Pre-implementation/Baseline Period • Project team mid-point annual
salaries x FTE spent (plus Fringe) • Include employees assisting with
planning • Office operations costs (printing,
supplies) • Travel for grant award
• Implementation and Post-Implementation Periods
• Project team mid-point annual salaries x FTE spent (plus Fringe) • Include employees assisting with
implementation • Cost of contracting out services • Office operations costs (printing) • Travel for grant award • Modifications to Databases • Communications (storyboards,
easels, DPH Newsletter)
6/10/2013 50
Approximately how many meetings did you attend regarding this QI project? ________
Approximately how much time did you spend on the following activities related to this
QI project?
Attending formal or informal meetings? hours
Making changes to data collection forms? hours
Making changes to databases? hours
Generating cost estimates for changes? hours
Other activities related to this QI project? hours
Please briefly describe these activities: ________________________________________________________________________________________________________________________________________
Customer Satisfaction Survey on QI Process
6/10/2013 51
Collected Data on Time Spent on Planning and Implementation Activities:
Salary Costs for QI Initiative Team
6/10/2013 52
Pre-Implementation - Baseline Implementation Period 1
From To From To
Cost Category Reporting Period: 4/1/12 8/19/12 8/20/12 11/30/12
Salary and
Fringe
Salary and
Fringe
Personnel (see below)
List by name or by
category FTE % Cost FTE % Cost
Project Sponsor - Epidemiologist 4
Project team member
5.00% 87,931.00 $ 1,721.98 3.00%
87,931.00 $ 725.43
Project leader-Epidemiologist 2 20.00% 53,143.00 $ 4,162.87 20.00%
53,143.00 $ 2,922.87
Epidemiologist 4 Project team member 4.00% 87,931.00 $ 1,377.59 2.00%
87,931.00 $ 483.62
Epidemiologist 3 Project team member 6.00% 76,884.00 $ 1,806.77 5.00%
76,884.00 $ 1,057.16
Section Chief: Practitioner Licensing Project team member 5.00% 100,000.00 $ 1,958.33 5.00%
100,000.00 $ 1,375.00
Manager Vital Statistics Project team member 5.00% 86,300.00 $ 1,690.04 5.00%
86,300.00 $ 1,186.63
Section Chief: IT Project team member 4.00% 100,000.00 $ 1,566.67 2.00%
100,000.00 $ 550.00
Takes into account time prds (4.7 mos baseline)
Salary Costs for QI Initiative Team: Implementation Prds 2 and 3
6/10/2013 53
Implementation Period 2 Implementation Period 3
From To From To
Cost Category 12/1/12 12/31/13 1/1/14 12/31/14
Salary and
Fringe
Salary and
Fringe
Personnel (see below) FTE % Cost FTE % Cost
Project Sponsor - Epidemiologist 4 6.00% 87,931.00 $ 5,715.52 6.00%
87,931.00 $ 5,275.86
Project leader-Epidemiologist 2 8.00% 53,143.00 $ 4,605.73 8.00%
53,143.00 $ 4,251.44
Epidemiologist 4 4.00% 87,931.00 $ 3,810.34 4.00%
87,931.00 $ 3,517.24
Epidemiologist 3 $ - $ -
Section Chief: Practitioner Licensing $ - $ -
Manager Vital Statistics 3.00% 86,300.00 $ 2,804.75 3.00%
86,300.00 $ 2,589.00
Section Chief: IT 3.00% 100,000.00 $ 3,250.00 3.00%
100,000.00 $ 3,000.00
Data quality committee reviewing databases and overseeing fixes (Impl Prds 2 - 3)
Non-Personnel Investment Costs: Planning and Implementation Periods
6/10/2013 54
Baseline - Pre-implementation Implementation - Year 1
From To From To
Cost Category Reporting Period: 4/1/12 8/19/12 8/20/12 11/30/12
Contracted services3
Other contracted services Modifications done - 5 hours at $40.00/hr - 200.00
Office operations4
Printing paper ($5/ream) and toner($602/crtrdg) 303.50 303.50
Office supplies/postage
flip charts (2 cartons at $41/crtn), pens (4 dzn at $11.82/dzn), markers
(4 at $2 ea), post-its ($15/18 pk) 76.26 76.26
Travel/conferencing
Dinners ($26 per) and Taxis( $25 per): Travel to Portland, OR in Apr.
and Charlotte, NC in Dec. 1,050.00 1,050.00
Training - 137.08
Other office operations frames for certificates of recognition/appreciation (11 at $11.70/ frame) 128.70
Communications 8 hours of writing article for DPH newsletter and creating story board - -
Equipment6
Subtotal direct costs $ 17,815.61 $ 17,466.42
Indirect cost percentage (if %
of direct costs)9
Indirect costs9 $ - $ -
Total costs of investment in
strategy $ 17,815.61 $ 17,466.42
Years of expected utility from pre-implementation investments (for amortization)10 1.00
Amortized Pre-Implementation
Investment Costs $17,815.61
Calculate Costs/Savings for Outcomes
ROI is Produced based on Prior Calculations
Determining Costs and Savings for Outcomes and Outputs: ROI Produced
• Baseline and Post Periods • Measures of Production Time • Time to analyze data with
standardized sociodemographic categories (expected reduction)
• Time to review databases (improves over time)
• Improve Health Outcomes: Based on finding new cases in sociodemographic groups
• Taking into account the Investment and Routine Operating Costs Only o Implementation Period 1
Benefit (savings) was 64 cents for every dollar spent
o Implementation Period 2 Cost was $2.21 for every dollar
spent • Adding in Outcomes
o Implementation Period 1 and Overall Savings was $17.05 ($18.16
overall) for every dollar spent
6/10/2013 55
Outcome/Output Measures: Production Time Costs and Savings
6/10/2013 56
Measures of Production Time Baseline Post Period - Year I
TRUE Description: output/event and
time frame
Total
events
Average
time
(days)
Total
time
(days
)
Total
event
s
Averag
e time
(days)
Total
time
(days)
Time to analyze data with standardized
sociodemographic categories
May see reductions in Post Periods-Year
2 and 3 when employees are trained and
databases were starting to be used for
quarterly and final reports: 1/1/2013 to
12/31/2014
7.00
3.00
21.00
7.00
2.50
17.50
Number of databases reviewed by project
sponsor and leader
Spent 1 hour per database review in
baseline and QA checks
8.00
0.20
1.60
8.00
0.03
0.24
Number of databases reviewed by data
quality committee
Will spend 1 hour per database review in
baseline and QA checks
-
-
Total 5 4.52 22.60 5 3.548 17.74
Outcome/Output Measures: Reach Outcomes: Rates
6/10/2013 57
Measures of Reach/Intermediate
Outcomes Baseline Post Period - Year I
FALSE
Description: target population
and outcome Numerato
r
Denominato
r Rate
Numerato
r
Denomin
ator Rate
Finding cancer cases based on self-
identified gender
Using SEER cancer registry as source
data 343 75,250 455.81 350 75,250 465.12
Finding cases of reportable disease
based on sociodemographic
characteristics (e.g. black race)
Focusing program services for African-
Americans throughout the CT 5,000 660,000 757.58 5,500 660,000 833.33
Finding cases of adult lead poisoning
based on sociodemographic
characteristics (e.g. Hispanic) Finding more cases of hispanic origin 22 874,500 2.52 30 874,500 3.43
Composite reach 5,365 1,609,750 333.28 5,880 1,609,750 365.27
Used these rows as a worksheet for the health outcomes on next slide
Did not include data in ROI calculations (unchecked)
Outcome/Output Measures: Health Outcomes Savings
6/10/2013 58
Include data in ROI calculations (checked)
Measures of Health-Related
Outcomes Baseline Post Period -
Year I
Post Period -
Year 2 Post Period - Year 3
TRUE Description Total Total Total
Total
Est. Cost per
Unit
Outcome**
Preventing cancer deaths
(based on better follow-up)
people who self-identify as
other than male or female
Can prevent 5% of cancer
deaths if invasive cancer
found and reported 17 18 18 19 -$50,000.00
Preventing readmissions for
reportable diseases in blacks
Can reduce readmissions
by 10% 500 550 600 660 -$9,500.00
Preventing further cases of
adult lead poisoning in
Hispanics in
neighborhoods/communities
For every one person
exposed and
poisoned,there are 2
others exposed and
potentially poisoned 44 60 70 80 -$1,000.00
Composite outcomes 561 628 688 759
Total value -$5,644,000 -$6,185,000 -$6,670,000 -$7,300,000
Return on Investment Analysis
6/10/2013 59
II. Analysis INCLUDING Output and Outcome Measures (Production Time, Reach, and Health-Related Outcome
Factors)
Implementation Period Total
Baseline 1 2 3 (All Periods)
Investment in Project
Amortized Pre-Implementation Costs $17,815.61
Ongoing Implementation Costs $0.00 $14,766.42 $22,158.71 $20,256.01
Total Annual Implementation Costs $17,815.61 $14,766.42 $22,158.71 $20,256.01
x Present Value Factors (see below) $1.00 $0.97 $0.94 $0.92
Total Discounted Annual Implementation Costs $17,815.61 $14,336.33 $20,886.71 $18,537.11 $71,575.76
Financial Gain/Loss Attributable to Project
Estimated Changes in Operating Costs $ (21,330.43) $ 146,265.80 $ 129,506.18
Estimated Changes in Revenue $ - $ - $ -
Total Net Financial Gain/Loss $ 21,330.43 $ (146,265.80) $ (129,506.18)
x Present Value Factors (see below) $ 0.97 $ 0.94 $ 0.92
Total Discounted Annual Net Gain/Loss $ 20,709.15 $ (137,869.54) $ (118,516.50) $ (235,676.89)
Non-Financial Gains/Losses Attributable to Project
Estimated Production Time Gain/Loss 4.86 (28.85) (20.40)
Monetary Value of Time Gain/Loss $ 2,443.55 $ (16,117.17) $ (11,396.54)
x Present Value Factors (see below) $ 0.97 $ 0.94 $ 0.92
Total Discounted Value of Time Gain/Loss $ 2,372.38 $ (15,191.98) $ (10,429.45) $ (23,249.05)
Cumulative ROI including Time Cost $ 0.72 $ (2.45) $ (3.62) $ (3.62)
Estimated Changes in Health-Related Outcomes 67.00 127.00 198.00
Estimated Cost per 1 Unit Improvement in Outcomes $ (95.12) $ 1,250.05 $ 692.19
Monetary Value of Outcome Improvement/Loss $ 541,000.00 $ 485,000.00 $ 630,000.00
x Present Value Factors (see below) $ 0.97 $ 0.94 $ 0.92
Total Discounted Value of Outcome Improvement/Loss $ 525,242.72 $ 457,159.02 $ 576,539.25 $ 1,558,940.98
Cumulative ROI including Value of Health-Related Outcomes $ 16.98 $ 16.31 $ 18.49 $ 18.49
Cumulative ROI Including All Non-Financial Gains/Losses $ 17.05 $ 16.07 $ 18.16 $ 18.16
Used prospectively in this case
Seemed daunting at first, but once data is collected, calculations are done for you
Used customer satisfaction survey to gather data on time spent on planning and making modifications to databases
Customized to fit our needs
Highlights of Using the ROI Tool for a Connecticut DPH QI Project
6/10/2013 60
Can be applied for many different types of QI projects
Data quality projects may not have a large return, but it is the “right thing to do”
Projects that improve efficiency and reduce process steps/time will see a greater return on investment
Future Use of the ROI Tool:
ASTHO National Demonstration Project (3/1/12 – 10/31/13)
Make data accessible to local health departments – reduce time for state DPH staff to handle data requests
Save time cleaning up data
Applications of the ROI Tool
6/10/2013 61
Thank you
6/19/2013 62