Upload
christine-hodge
View
217
Download
0
Embed Size (px)
DESCRIPTION
Outline of Presentation Overview of National PRC Program Evaluation Purpose Indicators Qualitative studies Activities supplementing evaluation Lessons Learned
Citation preview
1
National Indicators and National Indicators and Qualitative Studies to Measure the Qualitative Studies to Measure the Activities and Outcomes of CDC’s Activities and Outcomes of CDC’s
PRC ProgramPRC ProgramDemia L. Sundra, MPH
Prevention Research Centers Program
AEA 2006, Portland, OR
The findings of this presentation are the authors and do not necessarily represent the views of the Centers for Disease Control and Prevention
PRC National Evaluation Contributors PRC National Evaluation Contributors 2004-Present2004-Present
Collaborative Evaluation Design Team (CEDT)Geri Dino (co-chair), West Virginia University
Cynthia Boddie-Willis, MA Dept. of HealthJosefina Dhungana, UCLA community representative
Katie Barnes, UNC Chapel Hill community representativeKenneth McLeroy, Texas A & M
Thomas Reischl, Univ. of MichiganDelores Pluto, Univ. of S. CarolinaMichelle Kegler, Emory University
David Collins, Morehouse School of Medicine
PRC Program Office at CDCEduardo Simoes (co-chair), Demia Sundra, Jo Anne Grunbaum,
Margaret Kaniewski
ORC MacroNicola Dawkins, Doryn Chervin, Stephanie Kamin, Amee Bhalakia,
Gayle Payne and Jesse Gerwig-Moore
Outline of PresentationOutline of Presentation
Overview of National PRC Program Evaluation Purpose Indicators Qualitative studies Activities supplementing evaluation
Lessons Learned
Levels of PRC EvaluationLevels of PRC Evaluation
Evaluation of PRC researchVs.
Local evaluation of single research center Vs.
Evaluation of national program/network of 33 centers
Project DEFINE Project DEFINE 2004-Present2004-Present
DDelivering elivering EEvidence and vidence and FFindings: indings: IImplementing the mplementing the NNational ational EEvaluationvaluation
Developing a systematic approach that will allow the PRC program to describe and assess program inputs, activities, outputs and outcomes over time
Guided by CDC Framework for Program Evaluation in Public Health1
1Centers for Disease Control and Prevention. Framework for Program Evaluation in Public Health. MMWR 1999;48(No. RR-11). 03/05 from: http://www.cdc.gov/eval/framework.htm
Purposes of the PRC National Purposes of the PRC National EvaluationEvaluation
The national evaluation will accomplish the following purposes:
Primary purpose: To demonstrate national program accountability
Secondary purpose: To facilitate program improvement
Evaluation Approach Evaluation Approach
Quantitative Indicators33 PRCs Qualitative Studies
Document Review – 33 PRCsInterviews – Sample of PRCs
NATIONAL EVALUATION ACTIVITIES
Supplements Helps Identify
Case Studies & Stories from single PRCs
COMMUNICATION/DISSEMINATION ACTIVITIES
Program IndicatorsProgram Indicators
Quantitative Indicators33 PRCs
NATIONAL EVALUATION ACTIVITIES
Program IndicatorsProgram Indicators Addressing national accountability
Looking for the overarching critical components of PRCs that are feasible and meaningful to count and summarize
Balancing burden and highlighting variety
Collecting data through PRC Information System
Development of IndicatorsDevelopment of Indicators CEDT rated indicators: Useful, feasible,
meaningful, relevant Ongoing work with CEDT to refine
Developed corresponding questions for PRC Information System
Sent out for feedback from all PRCs Finalizing changes/beginning Information
System development
Current Status of IndicatorsCurrent Status of Indicators
23 program indicators cover inputs, activities, outputs, and outcomes of a national prevention research program
Research Training Partnerships Dissemination
Reflect variety of program perspectives
Information System is in development
Examples of Research Examples of Research Indicator ConceptsIndicator Concepts
ACTIVITIES Research
projects
INPUTS Project funding Academic institution
support
OUTCOMES Distribution of products Dissemination and
adoption of interventions Policy and environmental
changes
OUTPUTS Effectiveness of
interventions Publications
Sample Reporting – Fictitious Examples of Sample Reporting – Fictitious Examples of Research Indicator DataResearch Indicator Data
Across 33 PRCs, there are 287 projects, totaling $147 million in funding
Funding: 32% CDC, 15% NIH, 52% Other 82% of projects are research
In the past 5 years of research at 33 PRCs, 29 interventions were funded out of core PRC Program dollars:
10 interventions found promising 4 found effective 1 found adoptable
Examples of TrainingExamples of TrainingIndicator ConceptsIndicator Concepts
INPUTS Project funding
OUTPUTS Number of people trained Number of students
working with PRCs
ACTIVITIES Trainings, by
topic, audience, format, duration
Sample Reporting – Fictitious Examples of Sample Reporting – Fictitious Examples of Training Indicator DataTraining Indicator Data
Nationally, 1,484 individuals attended training programs sponsored by PRCs in 2006. The numbers and types of individuals trained include the following:
105 Policy Makers 541 Students 287 Public Health and Health Care Practitioners 311 Community Members 151 Researchers 89 Project Staff
Evaluation Approach Evaluation Approach
Qualitative StudiesDocument Review – 33 PRCsInterviews – Sample of PRCs
NATIONAL EVALUATION ACTIVITIES
Qualitative Study DevelopmentQualitative Study Development Used for concepts
critical to program priority for some stakeholders suitable for qualitative work
Reserved resources from beginning
Four Descriptive Studies Four Descriptive Studies Interaction between PRC researchers and
their communities around core prevention research project
PRCs’ characteristics related to organizational and partnership structures
Variety, goals, and contextual factors of PRCs’ core prevention research
Diversity of PRC training, technical assistance, and mentoring with communities and partners
Data Collection MethodsData Collection Methods Document Review – example topics
Staffing counts and structures Community Committee Guidelines
Interviews – example topics Participation of communities in research Benefits of being in the PRC network
Supplementing Evaluation Data Supplementing Evaluation Data
Supplements Helps Identify
Case Studies & Stories from single PRCs
COMMUNICATION/DISSEMINATION ACTIVITIES
Case Studies and StoriesCase Studies and Storieswww.cdc.gov/prcwww.cdc.gov/prc
Case Study – WVU/Not-On-TobaccoCase Study – WVU/Not-On-Tobacco
Story Story – WVU/Not-On-Tobacco– WVU/Not-On-Tobacco
Evaluation Approach Evaluation Approach
Quantitative Indicators33 PRCs Qualitative Studies
Document Review – 33 PRCsInterviews – Sample of PRCs
NATIONAL EVALUATION ACTIVITIES
Supplements Helps Identify
Case Studies & Stories from single PRCs
COMMUNICATION/DISSEMINATION ACTIVITIES
Lessons LearnedLessons Learned
Expect to use mixed methods
Local feedback was critically important to national evaluation strategy
Advisory group (CEDT) feedback Broader PRC Program feedback
Keep your eye on the prize Accountability of national program helps everyone
www.cdc.gov/prc
Demia L. Sundra, MPHHealth Education Specialist, PRC Program,
Division of Adult and Community Health,National Center for Chronic Disease Prevention and Health Promotion