View
27
Download
1
Category
Preview:
DESCRIPTION
Evaluation 101. Prevention Education Meeting May 29, 2013. What is Evaluation?. Definitions of Evaluation. Examine and judge carefully. Determine value or worth. - PowerPoint PPT Presentation
Citation preview
Definitions of Evaluation
Examine and judge carefully.
Determine value or worth.
Process that attempts to determine, systematically and objectively, the relevance, effectiveness, and impact of activities in light of their objectives.
Purpose of Evaluation
Use evidence to prove that a program has delivered intended services.
Show that program has made an impact on the health of a certain population.
Windsor, Clark, Boyd, & Goodman. Evaluation of Health Promotion, Health Education, & Disease Prevention Programs
Purpose of Evaluation
To determine achievement of objectives related to improved health status
To improve program implementationTo provide accountability to funding
agencies, community, and other stakeholders
McKenzie, Neiger, & Smeltzer. Planning, Implementing, & Evaluating Health Promotion Programs
Outputs
The amount of product or service that the program intends to provide Type of activity, product or service
and target audienceParticipants ClientsAgencies Decision makersCustomers PublicHealth care providers
Type of evaluation process evaluation Can also include satisfaction with activity/product/service
Short Term Outcomes
Immediate effects of activities including changes in
Awareness KnowledgeAttitudes BeliefsValues OpinionsAspirations MotivationSkills (knowledge)
Type of evaluation impact evaluation
Intermediate Outcomes
Changes in behavior
Risk factor reduction NormsSelf management PoliciesSocial action PracticeProcedures Decision-
makingDemonstration of skills
Type of evaluation impact evaluation
Long Term Outcomes
Changes in burden of disease, health status, or quality of life prevalence or incidence mortality also social, economic, civic, environmental factors
Type of evaluation outcome evaluation
Evaluation Terminology
Process evaluation measurement of activities during implementation
Impact evaluation measurement of immediate observable effects of a
program including awareness, knowledge, attitudes, skills, and behaviors
Outcome evaluation measurement of the ultimate goal of a program
Process Evaluation Data
Quantitative Counts
# meetings held, # training sessions conducted, # of people who attended
Descriptive Statistics mean satisfaction rating for event, percent of people who
responded they liked the eventQualitative
Descriptive information description of event, date of event, location of event,
description of who attended, what information was included during event
Process Evaluation Data
Monthly Progress Report Example:
# professional training sessions/workshops conducted # professionals trained # informational materials/brochures distributed
# primary prevention education sessions conducted for students # students attended primary prevention education sessions # informational materials/brochures distributed
# sessions conducted with faith-based organizations descriptions of topics covered during sessions
Impact Evaluation Data
Short term outcomes on a logic model awareness, knowledge, attitude change data
Quantitative data collected from surveys/questionnaires, written
tests
Qualitative data collected from surveys/questionnaires,
interviews, focus groups
Impact Evaluation Data
Intermediate term outcomes on a logic model behavior change data
Quantitative BRFSS data
Qualitative interviews, focus groups, observations
Impact Evaluation Data
Example: The results of the pre/post knowledge tests indicted that
participants’ knowledge about healthy relationships increased by 500%.
The results indicated that students learned to use negotiation techniques as assessed by post-test.
Responses on the modeling questionnaire indicate that 85% of students would intervene when other students act disrespectfully.
School administrators at 10 schools strengthened their anti-bullying policies.
Outcome Evaluation Data
Long term outcomes Changes in health statusQuantitative
YRBS data BRFSS data Vital data Hospital Discharge data ER or ED data
Qualitative Interviews, focus groups
Outcome Evaluation Data
Example: 2007 YRBS
29.9% of high school students had been in a physical fight at least once in the past year
11.8% of high school students had been hit by their boy/girlfriend in the past year
9.5% of high school students have been forced to have sexual intercourse against their will
2008 BRFSS 12.5% of adult females have been victims of forcible rape 16.7% of adult females have either been raped or have
experienced an attempted rape 4.7% of adult males have either been raped or have
experienced an attempted rape
ABCDE Method of Writing Outcomes
A – Audience (Who will change?)B – Behavior (What will change?)C – Condition (By When?)D – Degree (How Much?)E – Evidence (How will the change be
measured?)
General Evaluation Guides
The Program Managers Guide to Evaluation http://www.acf.hhs.gov/programs/opre/other_resrch/pm_guide_eva
l/reports/pmguide/pmguide_toc.html
Kellogg Foundation Logic Model Development Guide http://www.wkkf.org/Pubs/Tools/Evaluation/Pub3669.pdf
Developing a Logic Model: Teaching and Training Guide http://www.uwex.edu/ces/pdande/evaluation/pdf/lmguidecomplete.
Understanding Evaluation: The Way to Better Prevention Programs http://www.ed.gov/PDFDocs/handbook.pdf
Kellogg Foundation Evaluation Handbook http://www.wkkf.org/Pubs/Tools/Evaluation/Pub770.pdf
Topical Evaluation Guides
Core Performance Measures for Steps to a HealthierUS http://www.cdc.gov/eval/steps/Steps%20Core%20Performance%20Me
asures%2007-2007.pdf
Introduction to Program Evaluation for Comprehensive Tobacco Control Programs http://www.cdc.gov/tobacco/tobacco_control_programs/surveillance_e
valuation/evaluation_manual/00_pdfs/Evaluation.pdf
Physical Activity Evaluation Handbook http://www.cdc.gov/nccdphp/dnpa/physical/health_professionals/interv
entions/handbook.pdf
CDC DASH Evaluation Briefs http://www.cdc.gov/HealthyYouth/evaluation/resources.htm#4
Strategic Planning Kit for School Health Programs http://www.cdc.gov/HealthyYouth/evaluation/pdf/sp_kit/sp_toolkit.pdf
SMART Cards http://www.cdc.gov/HealthyYouth/evaluation/pdf/SMARTcards.pdf
Violence Prevention Evaluation Guides
Sexual and Intimate Partner Violence Prevention Programs Evaluation Guide Book by CDC, 2007, can be ordered from http://wwwn.cdc.gov/pubs/ncipc.aspx
Measuring Violence-Related Attitudes, Behaviors, and Influences Among Youths: A Compendium of Assessment Tools, Second Edition http://www.cdc.gov/ncipc/pub-res/pdf/YV/YV_Compendium.pdf
Evaluation Websites
CDC’s Evaluation Framework http://www.cdc.gov/eval/framework.htm
Community Tool Box http://ctb.ku.edu/en/
American Evaluation Association http://www.eval.org/
Coalitions Work http://coalitionswork.com/
Writing Good Goals and Objectives http://apps.nccd.cdc.gov/dashoet/writing_good_goals/menu.html
Basic Guide to Program Evaluation http://www.managementhelp.org/evaluatn/fnl_eval.htm
University of Wisconsin Extension – Program Development and Evaluation http://www.uwex.edu/ces/pdande/index.html
Kellogg Foundation http://www.wkkf.org/default.aspx?
tabid=75&CID=281&NID=61&LanguageID=0
Recommended