Evaluation of EHDI Programs

Terry FoustKaren Muñoz

Kathleen WattsNCHAM Technical Assistance

Why Evaluate?

Why Evaluate?

Acountability Demonstrate program improvement

over time Demonstrate evidence based practice Demonstrate attainment of program

goals (1.3.6) Required by grants, funding agencies

Regular and continuous evaluationIdentifies areas to work on for improvement. Example – Utah, Montana

Identify Star Performers

Levels of Evaluation

ScreeningDiagnosticEarly InterventionState ProgramFinancing the program

Screening LevelStatistics

Number of live birthsNumber and percent screenedNumber and percent missedNumber and percent referred

Information needed to identify possible solutions

Equipment at each hospitalScreening protocolScreenersScreener trainingDesignated program coordinatorNotification of parent/physician/stateTracking protocol

Diagnostic LevelStatistics

Number and percent obtaining outpatient testingNumber and percent identified with a hearing lossAverage time lag between screening and diagnostic testingAverage age of identificationNumber and percent lost to follow-up

Information needed to identify possible solutions

Number of pediatric test sites with appropriate equipmentLocation of test sitesAudiologist compliance with reporting proceduresProtocol for linkage with EI

Early Intervention LevelStatistics

Number and % enrolled in EINumber and percent lost to follow-upAverage age enrolled in EIAverage age fit with amplificationAverage time lag between diagnosis and intervention

Information needed to identify possible solutions

Timeliness of referrals to EI following diagnosisTimeliness of intervention following enrollmentTraining/knowledge of EI case managers on issues related to hearing lossReporting protocol from EI to State EHDI coordinator

State Program LevelSupport

Parent-to-parent family supportDissemination of resources for familiesAvailability of medical home for all newbornsPerformance feedback from state to hospitals

Tracking and Quality Assurance

Program improvement over timeIdentification of expected number of infants with hearing lossAttainment of program goals (1.3.6)Existence of periodic and ongoing evaluation, action plan, and monitoringIdentification & implementation of educational needs

Information needed to identify possible solutions

Existence of financial support for diagnostic testing, and amplificationExistence of a hearing aid loaner programMechanism for:

Informing families Informing practitioners Dissemination of materials

How to Evaluate

Hospital surveysPhysician surveysParent surveysPatient focus groupsGetting patients back for follow up Data AnalysisHospital site visits

Hospital Survey

Physician Survey

Parent Surveys

Parent Focus Groups

Getting Patients Back for Follow Up

How are programs getting patients back for follow up?Audiologists are key

Data Analysis

Importance of tracking software Is it flexible enough to generate a wide variety of program statistics?Is it compatible with a variety of programs and equipment?Examples

Hospital Site VisitsEVALUATION CRITERIA

INPATIENT SCREENING ___ >98% of infants born are screened ___ >90% of infants screened pass initial screening ___ All births recorded into tracking system (Hi*Track)

OUTPATIENT RESCREENING ___ >90 of outpatients are rescreened prior to two months of age ___ 100% of infants needing outpatient screening are scheduled prior to discharge ___ PCP notified of need for follow up rescreening

DIAGNOSTIC FOLLOW UP ___ Diagnostic evaluations are completed by 3 months of age ___ All infants identified with hearing loss will receive appropriate medical, audiologic and

educational intervention services before 6 months of age ___ Results of diagnostic evaluations are reported back to the State ___ Physicians are notified of positive identifications – all infants with hearing loss will have

a medical home

FAMILY SUPPORT ___ Families of children positivley identified are referred to early intervention ___ All families receive culturally competent family support

REPORTING ___ Hospital participates in States EHDI tracking and surveillance System to minimize loss to

follow up. ___ Completes data transfer by required date as per state (ie. 10th of each month) ___ Reports transferred babies ___ Provides timely data updates on individual babies as requested

TRAINING ___ Program has competency guidelines for screeners ___ Trainers are experienced and competent according to program standards for screeners ___ Coordinators attend coordinator meetings References: T. Foust, MCHB, CDC and JCIH components/standards for evelaution, IHC evaluation standard

Montana Universal Newborn Hearing Screening

Program Evaluation

Hospital: _____________________________________ Date of Evaluation: ________________________________ Rating: Excellent = Successfully completes 100% of criteria during evaluation period Good = Successfully completes 75% of criteria during evaluation period Fair = successfully completed 50% of criteria during evaluation period Unsatisfactory – Successfully completed <50% of criteria during evaluation period

Inpatient Screening ____ Excellent ____Good ____ Fair ____ Unsatisfactory Comments:

Outpatient Screening ____ Excellent ____Good ____ Fair ____ Unsatisfactory Comments:

Diagnostic Follow-up ____ Excellent ____Good ____ Fair ____ Unsatisfactory Comments:

Reporting ____ Excellent ____Good ____ Fair ____ Unsatisfactory Comments:

Training ____ Excellent ____Good ____ Fair ____ Unsatisfactory Comments:

Program Summary ____ Excellent ____Good ____ Fair ____ Unsatisfactory Comments:

Evaluation Forms

www.infanthearing.org/programevaluation/

Lessons Learned From Site Visits

Level of commitment to program is evidentChallenges are brought to life Importance of onsite problem solvingEquipment and data management processes are clearStaff input reflects program health

Summary

Evaluation takes time and effortMust be well planned

Go Ahead and “Jump” In!