Measuring and Improving System OutcomesMeasuring and Improving System Outcomes • NAEH Conference,...

Measuring and Improving

System Outcomes

• NAEH Conference, Washington DC

• July 17, 2012

• Katharine Gale Consulting

• Berkeley, CA

• (510) 710-9176,

• kgaleconsulting@sbcglobal.net

• www.focusstrategies.net

1

Overview

• What are we measuring?

• What does the data look like?

• How can we use the information for system

improvement?

• at the program level

• at the sector/program type level

• at the system level

Measuring System Performance

Program

Program

Program

Program

Program

Program

Evaluation

Measuring Program Performance

Program

Program

Program

Program

Program

Program

Evaluation

From the new CoC Regs

• "The Continuum of Care must...

• Consult with recipients and subrecipients to

establish performance targets appropriate for

population and program type, monitor recipient

and subrecipient performance, evaluate

outcomes, and take action against poor

performers

• Evaluate outcomes of projects funded under the

Emergency Solutions Grants program and the

Continuum of Care program, and report to HUD"

Key Elements of

Performance

• How many household's secure permanent

housing?

• How quickly?

• Do they gain jobs and/or income?

• Do they experience homelessness again?

Also, consider cost per outcome

Setting your Benchmarks

• How is your system performing new?

• How widely variant are outcomes from program to

program?

• What is aspirational but achievable?

9

Benchmarking Example: Exits to Permanent Housing from Transitional

Housing Programs including average, median and 75th percentile*

*Example based on performance range in 08-09, not actual programs’ performance

Benchmarks by program type

Alameda County EveryOne Home "Measuring

Success"

www.everyonehome.org

Samples ways to

look at these key

outcomes

Source: Sacramento HMIS and HPRP data, 2010, chart prepared by Focus Strategies

Average Rate of Exits to Permanent Housing - Families

Source: Data from 14 Continuums in seven states that prepared Evaluators for NAEH Performance

Improvement Clinics in 2011-2012

Average Rate of Exits to Permanent Housing - Singles

Source: Data from 14 Continuums in seven states that prepared Evaluators for NAEH Performance

Improvement Clinics in 2011- 2012

Averages are only

part of the story.

Outcome vary

across programs!

Average lengths of stay by outcome

- system wide and by program type

Source: Montana HMIS, 2010 data, Chart prepared by Focus Strategies

Snapshots are only part of

the story. Outcomes vary

across time!

Source: Montana HMIS data, Chart prepared by Focus Strategies

Source: Montana HMIS data, Chart prepared by Focus Strategies

Rates of Return within 12 months after

gaining permanent housing

Source: Data averaged from seven Continuums in four states that prepared Homeless System

Evaluators for NAEH Performance Improvement Clinics in 2011-2012

Returns to Homelessness -

system wide

Source: Montana HMIS data, Chart prepared by Focus Strategies

What about cost?

Investments by Program Type

Families

Source: CA county, Data from provider budgets (only partial participation)

Average Cost Per Exit and Per PH Exit

Source: Data from 14 Continuums in seven states that prepared Homeless System Evaluators for NAEH

Performance Improvement Clinics in 2011-2012

Caveat: costs, like

outcomes, vary

across programs!

What to do with this information

• How can we use the information for system

improvement?

• at the program level

• at the sector/program type level

• at the system level

At the program level

• Programs need to know their performance

• Have the data...

• Know how to read the data..

• Trust the data!

• Opportunities to learn/change/improve

• Training

• Technical assistance

• Time

Sample HMIS Report:

Emergency Shelters EMERGENCY

SHELTERS Jan 1, 2010 - Dec 31, 2010 Dec 2010 Nov 2010

Jan 1, 2009 - Dec 31, 2009 GOAL

Outcome Measures

People obtaining permanent housing 24.2% 26.2% 21.5% 21.3% {30%}

Exiting to streets or shelter 15.3% 18.5% 23.4% --- {<30%}

Exit with employment income 16.1% 17.9% 21.7% 15.6% {20%}

Adults entering with no income who exit with an

income 4.4% 3.7% 3.6% 5.7% {15%}

Efficiency/Process Measures

Exit to Known Destination 67.7% 74.7% 71.8% 57.8% {85%}

% of those who obtain

permanent housing who do so within 60 days 58.1% 17.4% 0.00% 46.8%

{50% <60 days}

9 Source: Alameda County HCD InHOUSE HMIS, 2/8/11

At the program level

• Incentives for change

• resources (increases, new funds, awards)

• recognition

• getting to keep grants

• Opportunities to try something new (and

latitude for experimentation.)

* OK if some programs decide to move on...

At the program-type/sector level

• Understanding of expectations/models

• Opportunities to collaborate, or specialize

• Peer learning exchanges

• Training and technical assistance

• Time...

At the system level

• Clarity about what buying and expecting

• Information for allocation decisions, across

programs and program-types

• Better outcomes!!

• A story to tell!