PHAB's Approach to Internal and External Evaluation Jessica Kronstadt | Director of Research and...
If you can't read please download the document
PHAB's Approach to Internal and External Evaluation Jessica Kronstadt | Director of Research and Evaluation | November 18, 2014 APHA 2014 Annual Meeting
PHAB's Approach to Internal and External Evaluation Jessica
Kronstadt | Director of Research and Evaluation | November 18, 2014
APHA 2014 Annual Meeting
Slide 2
Improved community health indicators / reduced health
disparities Organizational structure Board, committees and work
groups Staffing and expertise Principles for standards, measures,
and assessment process Site visitors Interest, buy-in and
commitment to seek accreditation Appropriate stability, resources,
and readiness to apply Previous quality improvement and assessment
experience Increased visibility and credibility of public health
agencies Ultimate Outcomes Improved responsiveness to community
priorities Public Health Agency Accreditation System Approved
December 2013 Enhanced internal and external collaboration Legend
Accrediting Agency Individual Public Health Agencies Stakeholders
and Partners Public Health Field Funders Partners at national,
state, regional, and local levels Funding Incentives Technical
Assistance Researchers and research networks Improved conditions in
which people can be healthy Improved identification and use of
evidence- based practices and policies Market program Implement the
7 steps of accreditation Train agencies and site visitors Develop
e-PHAB Evaluate program and improve quality Promote research
Promote national accreditation Encourage agencies to meet national
standards and seek accreditation Support agencies through TA
before, during, and after process Conduct and disseminate research
Participate in training and TA Assess readiness Submit application
and documentation Host site visit Review and share findings Develop
and implement improvement plan Implement QI Mentor other agencies
Participate in reaccreditation process Accreditation program:
marketed, implemented, evaluated, and improved e-PHAB developed and
data captured National consensus standards for public health
agencies Communication efforts delivered Technical assistance,
trainings, and QI tools provided Research conducted and
disseminated Agencies are accredited Report received and acted on
QI efforts are in place Agencies are mentored Plans for
reaccreditation underway Increased science base for public health
practice Increased support for accreditation Increased knowledge of
organizational strengths and weaknesses Increased consistency in
practice Increased use of benchmarks for evaluating performance
Increased organizational accountability Increased capacity for
optimal investment in public health Increased public recognition of
public health role and value Intermediate Outcomes Proximate
Outcomes OutputsStrategiesInputs Increased use of proven QI methods
and tools resulting in improvements in practice Increased
inter-agency and inter-sectoral collaboration Public health
agencies more effectively and efficiently use resources
Strengthened organizational capacity and workforce Strong, credible
and sustainable accreditation program in place Increased awareness
of importance of QI and a supportive culture Improved communication
about public health Strengthened public health agencies and systems
Standards adopted as performance measures Standards drive public
health transformation
Slide 3
Overview Internal evaluation External evaluation Program data
Annual Reports
Slide 4
Internal Evaluation Overview Primarily focused on process
Allows PHAB to make informed decisions about improving the
accreditation process Data Collection Health Department Surveys (n
= 63) Site Visitor Surveys Training evaluations PHAB Accreditation
Specialist Surveys
Slide 5
Health Department Experience
Slide 6
External Evaluation Overview Initial 3-year contract Focus on
process and short-term outcomes Data collection from HDs Survey 1:
After HDs submit their Statement of Intent (n=122) Survey 2: After
HDs are accredited (n=28) Survey 3: One year after HDs are
accredited (n=17) Interviews with 18 HD staff/stakeholders
Slide 7
Perceived Benefits Data reported by applicant HDs prior to
participating in PHAB training Accreditation will stimulate quality
and performance improvement opportunities (100% Strongly Agree or
Agree) Accreditation will allow HD to better identify strengths and
weaknesses (98% Strongly Agree or Agree) Accreditation will improve
management processes used by HD leadership team (98% Strongly Agree
or Agree)
Slide 8
Benefits and Outcomes One Year Post Accreditation Survey of
health departments accredited one year, n=17
Slide 9
Quality Improvement Quality Improvement Outcomes % Strongly
Agreed or Agreed Shortly after Accredited (n=28) 1 Year After
Accredited (n=17) Documentation selection and submission process
helped identify areas for performance and quality improvement
initiatives. 100%N/A Because of accreditation, we have implemented
or plan to implement new strategies to monitor and evaluate
effectiveness and quality. 100%N/A As a result of accreditation, we
have implemented or plan to implement new strategies for quality
improvement. 89%100% As a result of accreditation, we have used or
plan to use information from our QI processes to inform decisions.
100%94% As a result of the accreditation process, our health
department has a strong culture of quality improvement. N/A88%
Slide 10
The way we document things has changed. Since we have been
through the accreditation process, we know it is very important to
have proof that you did do something. Our Board of Health has been
supportive, and we give them updates on where we are in the
process. [Accreditation] has helped our agency in a lot of ways.
Applicant health department, on early outcomes experienced prior to
the PHAB Site Visit
Slide 11
Annual Reports Overview Required for accredited HDs Purpose
Continue communication with PHAB concerning conformity with
standards and measures Support health department in sustaining and
advancing its quality improvement culture Support health department
in being prepared for reaccreditation Includes descriptions of:
Improvement activities CHA, CHIP, strategic plan, QI plan
Slide 12
14 Annual Reports describe 29 Improvement Activities Getting
into compliance with mandated frequencies of inspections Improving
a program that works with schools to implement environmental/policy
changes Improving communications with governing entity Procuring an
EMR system to get better data for evaluation & performance
management Improving new employee orientation Streamlining &
strengthening process for responding to grant RFPs Almost all
indicated progress towards goals We also feel that [HDs] PHAB
accreditation status demonstrated our commitment and value to our
community and policy makers. IMPROVEMENT ACTIVITIES
Slide 13
8 of 14 HDs reported improvements in health indicators in CHIP
Examples of activities related to prerequisites and QI plan
Incorporating Essential Public Health Services and PHAB domains
into all job descriptions and annual employee evaluations
Coordinating with a local hospital for their IRS requirements for
the CHA Engaging all divisions in the HD in at least one QI project
in the past fiscal year Providing data to community partners as
part of efforts to address social determinants of health (e.g.,
high school graduation rates) Expanding opportunities for community
involvement in CHA and partner engagement in CHIP CONTINUING
PROCESSES