It’s Not Just Numbers: Implementing Point-in-Time Counts, Using HMIS, and Ensuring Data Accuracy Erin Wilson, Abt Associates Inc. Julie Eberbach, Iowa

It’s Not Just Numbers: Implementing Point-in-Time

Counts, Using HMIS, and Ensuring Data Accuracy

Erin Wilson, Abt Associates Inc.

Julie Eberbach, Iowa Institute for Community Alliances

September 23, 2008

2008 HMIS Training: Setting the Standard - U.S. Department of Housing and Urban Development2

Overview

• Learning Objectives

– Review basics of Point-in-Time (PIT) counts

– Learn how to incorporate HMIS into PIT counts

– Understand how to prepare HMIS for use in a PIT count

– Learn simple data checks for population and subpopulation information reported to HUD


What Does It Mean to “Count” Homeless People?

• A “count” = collecting information about the sheltered and unsheltered homeless population in your community

• It is possible just to do a count, but preferable to gather descriptive information too

Data on the number of homeless

people

+Descriptive

information on those counted

-Demographic

-Service Use

-Needs

=Point-in-time

count of homeless

people


Why Count Homeless People?

• For planning and program development

– Understand characteristics and needs

– Develop programs based on need

– Access resources for services and housing

• To measure progress in eliminating homelessness and to ensure accountability

• To raise public awareness


HUD Guidelines and Reporting Requirements

• According to HUD, PIT counts must be:– Actual counts or statistically reliable estimates of

homeless persons in sheltered and unsheltered locations on one night

– Conducted at least every other year during the last ten days of January (22-31) in odd calendar years

– 2009 is a required year!

• Designate one night for the count • CoCs must count:

– Sheltered and unsheltered adults, children and unaccompanied youth

– Number of households with and without dependent children


HUD Guidelines and Reporting Requirements

• At a minimum, count homeless persons according to HUD definitions and guidelines– Unsheltered, emergency, transitional

• CoCs may also want to collect additional information for local program planning – Homeless persons according to local definitions – In-depth information about a specific

subpopulation– Information about people at risk of homelessness

or precariously housed

• Must be able to report data according to HUD definitions and requirements!


Homeless Population Data


Homeless Subpopulation Data


How does HUD Use PIT Data?

• Reporting– The Annual Homeless Assessment Report

(AHAR) – Reporting the number of homeless persons in

each CoC on HUD’s website

• To make funding decisions Exhibit 1– HUD scores PIT counts based on quality of

methods and reliability of data• It’s about accurate counts, not large numbers!

– CoCs that conduct an annual PIT may receive additional credit


Counting Sheltered Homeless Persons

• Count people in emergency shelters and transitional housing programs– All programs listed on the e-HIC– Temporary emergency hotel/motel/apartment

vouchers for homeless people– Faith-based providers and VA homeless programs

• Collect data using:– Homeless Management Information System

• Requires high provider coverage + excellent data quality• All providers must enter/update information on all clients

in a bed on the night designated for the count

– Provider report or survey– Client-level survey using standardized instrument


Counting Unsheltered Homeless Persons

• Methods for counting unsheltered individuals:

– Street count using observation/interviews

• Complete coverage (block-by-block) and/or known locations

• Probability sampling

– Service-based count

• Non-shelter services (i.e. soup kitchens, healthcare centers)

• Can include outreach teams


Using HMIS for a PIT Count?

• Provides two components:– Count– Subpopulation information

• Typically used in combination with other data collection techniques to offset issues with low coverage

HMIS


Benefits and Limitations of Using HMIS for PIT Count

• Benefits of HMIS– Requires fewer resources over time than a non-

HMIS PIT count– Can reduce duplicate counting– Provides in-depth subpopulation data on persons

who are counted without intrusive or repetitive interviews

– Reinforces the value of the HMIS and contributes to year-round HMIS participation and data quality

• Limitations of HMIS– Requires high level of data quality and coverage


Key Considerations for Using HMIS

• Data quality

• Make sure information is gathered or extrapolated for each provider

• Start planning early

– 4 to 6 months prior to count

– Leave adequate time to assess data quality, improve data quality, and decide whether and how to use HMIS


Key Considerations for Using HMIS (cont.)

• Uses universal and program-specific HMIS data elements, as established by HUD

– HMIS Data and Technical Standards Final Notice (July 2004) FR 4848-N-02

– Revised HMIS Data and Technical Standards currently in public comment

• HMIS is a tool for gathering information about people who use services

– More useful for sheltered count


Using HMIS Data for an Unsheltered Count

• Most communities will need to continue street and service-based counts even if outreach workers are using HMIS

• Potential uses for unsheltered counts– Provides an opportunity to populate the HMIS with

information on unsheltered people– Helpful with de-duplicating street or service-based

counts


HMIS Data Elements for the Population Chart

Step 1: Determine if

the individual was in the

program on the night of the count…

Step 2: Classify whether

the individual is in an ES

or TH program…

2.13 Program ID2.10 Program Entry / 2.11 Program Exit

To count number of people and number of households…


HMIS Data Elements for the Population Chart (cont.)

Step 4: Determine whether the individuals should be counted as

a single or as part of a household with children…

Step 3: Determine whether the

individual is an adult or child…

2.3 Date of Birth

2.14 Household ID


HMIS Data Elements for the Subpopulation Chart

3.6 Mental Health

3.7 Substance Abuse

Severely Mentally Ill

Chronic Substance Abuse

2.6 Veteran Status

3.5 HIV/AIDS

Veterans

Persons with HIV/AIDS

3.8 Domestic ViolenceVictims of Domestic Violence

2.3 DOB + 2.14 Household ID

Unaccompanied Youth


HMIS Data Elements for Determining Chronic Homelessness

A chronically homeless person is defined as: • An unaccompanied homeless individual • with a disabling condition • who has either been continuously homeless for a year or

more OR has had at least four episodes of homelessness in the past three years.

2.3 Date of Birth

2.14 Household ID 2.13 Program ID

Determine if the individual is an

adult

Determine if the individual is

unaccompanied

Determine where the individual is

living (ES)

Step 1: Determine who is an unaccompanied homeless individual.


HMIS Data Elements for Determining Chronic Homelessness (cont.)

Step 2: Determine disability.

2.7 Disability

3.6 Mental Health

3.7 Substance Abuse

3.5 HIV/AIDS

3.3 Physical Disability

3.4 Developmental Disability

-OR-

Dis

abil

ity


HMIS Data Elements for Determining Chronic Homelessness (cont.)

Step 3: Determine chronicity.

2.10 Program Entry

2.11 Exit Across ES Programs and Stays

2.8 Residence Prior to Program Entry

Also consider if the individual was unsheltered prior to entering the program.


Examples of Using HMIS for a Count

HMIS

Option 1:

only – for count + subpopulation information

• 100% bed coverage required

HMIS

Option 2:for participating providers + extrapolation for non-participating providers

• Need 75% coverage rate for emergency shelter + transitional housing, both individuals + families• Non-participating providers need to be similar to participating providers (program type and clients served)


Examples of Using HMIS for a Count (cont.)

HMIS

Option 3:for participating providers

Option 4:

Paper/ manual survey

for non-participating providers

HMIS

for count Paper/ manual survey

for subpopulation information

Option 5:

HMIS

Participating providers complete paper/manual surveys using HMIS-generated reports Paper/

manual survey


Major Steps in Using HMIS for a PIT Count

Planning Steps and Suggested Timeline

6 months before – Assess HMIS Data Quality

3-4 months before – Finalize data collection approach and request provider participation

2 weeks before – Remind providers

3-5 days before – final data quality check

1 week after – Assess results and follow-up

In months after – Assemble findings

In months after – Evaluate the process

THE PIT COUNT

January

22nd – 31st

August OctoberSeptember November December January MarchFebruary


6 Months – Assess HMIS Data Quality

• As a first step, assess data quality at the system-level– Provider participation– HMIS data coverage

• If you think you may be able to use HMIS:– Begin in-depth data quality assessment and correct

data accuracy• Assess data quality at program level • Address data quality issues with providers

Number of people staying in shelter on a single night with data

reported in HMIS÷

Number of homeless people

served on that night overall

X 100


Ongoing – Assess and Improve Data Quality

• Develop reports to assess data quality – Client roster report: Verifies current population and

exit dates– Program and system-level occupancy reports:

Ratio of clients to beds– Missing (null) value report: Missing data fields– Program and system-level report to check for

duplicate records

• Develop a program and system-level Population and Subpopulation report

• Train program managers on using the reports


Ongoing – Assess and Improve Data Quality (cont.)

• Assess how programs are using the HMIS and adjust PIT reports as needed – Do some programs use a bed management system that

would provide more accurate PIT occupancy information than the program entry/exit dates?

– Are some programs exiting clients on the same day as entry?

– Do some programs do all data entry only once per week?

• Work with program managers to implement steps to improve data quality and clean up data errors

• Continue to run data quality reports to ensure that managers follow through as planned


3 to 4 Months – Finalize the Data Collection Approach

• Make a final decision about how/if you will use HMIS for the count

– For residential providers that are using HMIS:

• Are staff entering the necessary data fields on all clients for count and subpopulation information?

• Is complete data available for all clients? (e.g., are exit dates being entered regularly?)

• Is using HMIS for the count and subpopulation information a feasible strategy for all participating residential providers?


3 to 4 Months – Finalize the Data Collection Approach (cont.)

• Make a final decision about how/if you will use HMIS for the count (cont.)– For residential providers that are not using HMIS:

• Can you extrapolate from your existing HMIS data to cover these programs?

• Do you need to provide manual/paper surveys for non-participating providers to collect the required data?

• Establish a regular count meeting where the HMIS lead can coordinate with the point-in-time planning team


3 to 4 Months – Finalize the Data Collection Approach (cont.)

• Communicate with and secure the participation of providers– Work with the planning group to help ensure that

every provider is reporting information via HMIS or manual/paper survey

• Develop procedures for integrating the HMIS sheltered data with other sheltered and street survey data– For example, residential providers that complete

manual/paper surveys


2 Weeks – Remind Providers

• Remind program managers that the HMIS data is being used for the point-in-time count.

• Providers should:– Conduct a final data quality check during the days

before the count– Enter information about each required data

element for each client, including subpopulation information

– Enter data on a daily basis, even if they don’t routinely do so

• Give providers a date by which they must enter the data – typically two to three days after the count


A Few Days Before – Final Data Quality Check

• 2 to 3 days before the count:– Ask providers to review their active client rosters

one final time to ensure that all residents are reflected

– Run a system-level occupancy report to conduct your own estimate of problems and follow-up with providers as needed

• 1 day before the count:– Run the missing values report to see if critical data

elements are missing on current clients

– Ask providers to collect and enter the information while clients are still accessible


The Week After – Assess Results and Follow-up

• Evaluate the completeness/accuracy of data

• De-duplicate transitional/shelter programs

• Produce draft system-wide HMIS count reports for discussion– System-occupancy report showing list of programs

represented in the chart, along with client count and bed inventory for that night

– Overall counts and subpopulation estimates– Missing value report on count fields and

subpopulation fields to inform discussion on percent error


The Week After – Assess Results and Follow-up (cont.)

• Produce draft reports on the HMIS count for each program:– The count and subpopulation numbers– The client to bed ratio– The missing value % for each field

• Share these program reports with providers; work with providers to see if they can fill in any missing data

• Seeing the results may help providers realizethe benefit of HMIS and may motivate themto work on improving their data quality


The Month(s) Following the Count – Assemble Findings

• Develop final HMIS count report– Population and Subpopulations Chart– Document the process / methodology– Document limitations/external influences (i.e.

weather; barriers to achieving 100% accuracy)

• Merge HMIS sheltered count data with other data sources– Survey or extrapolated data from non-participating

providers– Survey data on client characteristics not captured

by HMIS (if HMIS used for count only)– Unsheltered count data

• Must de-duplicate with sheltered data


Evaluate the Process

• Should HMIS be used for future PITs? – For count data? For subpopulation data?

• Develop written recommendations on year-round and pre-count activities to improve the HMIS count for next year– Focus on year-round data quality– Identify critical providers to bring into the HMIS– Improve pre-count communication strategies– Define additional count-related report needs– Improve integration of data from other sources– Address other issues / concerns


Iowa Balance of State PIT Count

• One statewide HMIS implementation– The I-COUNT Network– Primarily rural, with some urban areas

• Strong data support structure– Iowa Council on Homelessness

• Research and Analysis Standing Committee plans PIT strategy

• Point in Time (Sheltered and Unsheltered)– Combined use of automated HMIS data and

manual survey instruments


Iowa BoS PIT Count Execution

• Research/Analysis Committee develops Point In Time strategy and timeline (sets date)

• HMIS staff execute the plan – Design of HMIS network reports– Development of hard copy collection instruments– Provides state-wide trainings for both sheltered

and unsheltered counts– Receives verified HMIS reports from participating

agencies and tabulates “hard copy” reports– Produces final Point In Time report for each

continua and the “state-wide” report


First Iowa Statewide Unsheltered Count

• Utilized “known locations” methodology

• Survey instruments were cooperatively developed

• State wide trainings provided for rural local Homeless Coordinating Boards


First Iowa Statewide Unsheltered Count (cont.)

• Iowa Department of Public Safety– Organized street count participation by law

enforcement (state, county and city)

• Iowa Department of Education– Facilitated participation of school district homeless

liaisons

• HMIS staff received survey summary documents and tabulated results for PIT inclusion– Staff also provided follow-up to “call-in” missing data

and also investigate any “anomalies”


Example of Homeless Street Count Survey (Part 1)


Example of Homeless Street Count Survey (Part 2)


Sample Data Collection Form


Example Data Collection Form


Overview of PIT Data Checks

• HUD conducts several data checks on all PIT data submitted by CoCs

• CoCs can check PIT data to ensure accuracy before submitting it to HUD

• Simple checks for:– Households with dependent children

– Households without dependent children

– Ratio of unsheltered to sheltered households

– Chronically homeless subpopulation numbers


PIT Data Checks – Households with Dependent Children

# of Households # of Persons in Householdsis less than

Average # of Persons per Household is typically

greater than 2, but less than 6

• For households with dependent children make sure:

# of Persons in Households

# of Households

If = 0, then = 0


If = 0= 0, then # of Households


PIT Data Checks – Households without Dependent Children

• For households without dependent children make sure:

Average # of Persons per Household is typically less than 2


is less than or equal to

# of Households


PIT Data Checks – Unsheltered to Sheltered Ratios

• Calculate ratio for households with dependent children and households without dependent children

• If either ratio is greater than 5, review the data.

EXAMPLECoC reports 205 unsheltered and 20 sheltered households

without dependent childrenRatio: 205 / 20 = 10.25

10.25 > 5 Check the data

=# of UnshelteredHouseholds ÷ # of Sheltered

HouseholdsRatio


PIT Data Checks – Chronically Homeless

• To check the number of chronically homeless make sure:

# of Individuals in

Emergency Shelter

Is less than or equal to

# Sheltered Chronically

Homeless Persons

• Remember to follow the definition of chronically homeless!

# of Unsheltered Persons

Is less than or equal to

# Unsheltered Chronically

Homeless Persons


Additional Resources

• Revised HUD documents– A Guide to Counting Unsheltered Homeless

People• http://www.hudhre.info/documents/counting_unsheltered.pdf

– A Guide to Counting Sheltered Homeless People• http://www.hudhre.info/documents/counting_sheltered.pdf

– Annual Homeless Assessment Report• http://www.hudhre.info/documents/3rdHomelessAssessmentReport.pdf

• www.hmis.info • www.hudhre.info• PIT Technical Assistance

– [email protected] – 1-877-789-2427

http://www.hudhre.info/documents/counting_unsheltered.pdf

http://www.hudhre.info/documents/counting_sheltered.pdf

http://www.hudhre.info/documents/3rdHomelessAssessmentReport.pdf

http://www.hmis.info/

http://www.hudhre.info/

mailto:[email protected]

Documents

It’s Not Just Numbers: Implementing Point-in-Time Counts, Using HMIS, and Ensuring Data Accuracy Erin Wilson, Abt Associates Inc. Julie Eberbach, Iowa