View
252
Download
1
Category
Tags:
Preview:
Citation preview
PROGRAM EVALUATION:KEEPING SCORE TO DEMONSTRATE
IMPACT
Molly Chamberlin, Ph.D. Indiana Youth Institute
9-28-2011
WHY IS EVALUATION IMPORTANT?
Accountability Progress and results More effective/efficient service Communication opportunities Focuses staff effort Engages employees throughout
organization Funding/resources
GOOD MEASURES PROVIDE:
Means to see if strategic plan is working
Focus attention on what matters most Common language within agency Measurement of outcomes, not just
outputs Systems for verifying results Support for funding requests
IMPACT CYCLE Mission >
Goals > Inputs >
Activities > Outputs >
Outcomes
PROGRAM LOGIC MODEL
KEYS TO SUCCESS
Evaluation / assessment is NOT a one-time event
Evaluation starts at the beginning of the project (not at the end)
Assignment of responsibility for process, particularly data analysis
Involve all levels of staff within your organization
ARE YOU READY?
Indicator Is there support from organizational leadership? Are there clearly written goals and objectives? Are the activities of the program clearly outlined? Is there a person or a group of people who are responsible
for leading evaluation efforts? Have financial and/or human resources been allocated for
evaluation efforts? Do all staff members agree upon the written goals and
objectives for the program? Do all staff members consider evaluation activities part of
their job requirements? Do other stakeholders (board, service recipients,
volunteers) value evaluation?
STEPS IN DESIGNING YOUR PLAN
Assign responsibility
Create a timetable
Communicate with staff
Use any and all available resources
REMEMBER AS YOU PLAN TO:
Be clear Be concise Remember that it all hasn’t happened
yet – adjustments to the process can (and should) be made as you learn from results
PROGRAM LOGIC MODEL AS PLAN FOR PROGRAM EVALUATION
Resources (Inputs)
Activities Outputs
Outcomes(immediate, short, long-
term)
WHAT IS YOUR PROGRAM GOAL?
GOAL: A big picture statement of what the program wants to accomplishGoal: Parents and caregivers possess the knowledge and skills to ensure safe and healthy development for infants and children
Inputs
Activities
Outputs
Outcomes
Success Indicators
WHAT ARE YOUR PROGRAM OUTCOMES?
OUTCOMES: What does your program want to CHANGE?
Inputs
Activities
Outputs
Outcomes: Expected observable and
measurable results; usually a change in behavior, knowledge, attitude, or skill
Increased parent/caregiver knowledge in specific areas Increased parent/caregiver practical skills in specific areas
Success Indicators
WHAT ARE YOUR ACTIVITIES?
ACTIVITIES: What will the program DO in order to achieve the desired outcomes?
Inputs
Activities: Actions or strategies employed to achieve desired
outcomes (if we do this, then the outcome)
Outputs
Outcomes: Expected observable and measurable results;
usually a change in behavior, knowledge, attitude, or skill
Survey parents and local providers to identify specific areas of need/knowledge and skill gaps
Provide online and local in-person training for parents and child care providers in specific areas of need Develop and administer grants for provider training (“train the trainer” models) in specific areas of need
Increased parent/caregiver knowledge in specific areas Increased parent/caregiver practical skills in specific areas
Success Indicators
WHAT ARE YOUR OUTPUTS?
OUTPUTS: What will the activities produce or create? (What will our clients do as a result of our activities?
Inputs
Activities: Actions or strategies employed to achieve desired outcomes (if we do this, then
the outcome)
Outputs: what will clients do as a result of our activities?
Outcomes: Expected observable
and measurable results; usually a change in behavior, knowledge, attitude, or skill
Survey parents and local providers to identify specific areas of need/knowledge and skill gaps
Provide online and local in-person training for parents and child care providers in specific areas of need Develop and administer grants for provider training (“train the trainer” models) in specific areas of need
Parents/caregivers participate in surveyParents/caregivers participate in online trainingsParents/caregivers participate in on-site trainingsProviders apply for training grantsProviders receive training grantsParents/providers are satisfied with training opportunities
Increased parent/caregiver knowledge in specific areas Increased parent/caregiver practical skills in specific areas
Success Indicators
WHAT ARE YOUR RESOURCES (INPUTS)?
INPUTS: What resources will you use to conduct your activities?
Inputs: Resources (usually people/funds)
used in program development and implementation
Activities: Actions or strategies employed to
achieve desired outcomes (if we do this, then the outcome)
Outputs: what activities will produce or create (quantity and/or quality of activities)
Outcomes: Expected
observable and measurable results;
usually a change in behavior, knowledge, attitude, or skill
2 FTE Training Staff
.5 FTE Training Coordinator
1 FTE Grant Coordinator
Funding for trainings
Funding for training staff
Funding for grants
Survey parents and local providers to identify specific areas of need/knowledge and skill gaps
Provide online and local in-person training for parents and child care providers in specific areas of need Develop and administer grants for provider training (“train the trainer” models) in specific areas of need
Parents/caregivers participate in surveyParents/caregivers participate in online trainingsParents/caregivers participate in on-site trainingsProviders apply for training grantsProviders receive training grantsParents/providers are satisfied with training opportunities
Increased parent/caregiver knowledge in specific areas Increased parent/caregiver practical skills in specific areas
Success Indicators
HOW DO YOU QUANTIFY/MEASURE YOUR OUTCOMES?
SUCCESS INDICATORS: How do we know if we’re on track to achieve or achieved our outcomes?
Inputs: Resources (usually people/funds) used in
program development and implementation
Activities: Actions or strategies
employed to achieve desired outcomes (if we do this, then the
outcome)
Outputs: what activities will produce or create (quantity and/or quality of activities)
Outcomes: Expected observable
and measurable results; usually a change in behavior, knowledge, attitude, or skill
2 FTE Training Staff
.5 FTE Training Coordinator
1 FTE Grant Coordinator
Funding for trainings
Funding for training staff
Funding for grants
Survey parents and local providers to identify specific areas of need/knowledge and skill gaps
Provide online and local in-person training for parents and child care providers in specific areas of need Develop and administer grants for provider training (“train the trainer” models) in specific areas of need
Parents/caregivers participate in surveyParents/caregivers participate in online trainingsParents/caregivers participate in on-site trainingsProviders apply for training grantsProviders receive training grantsParents/providers are satisfied with training opportunities
Increased parent/caregiver knowledge in specific areas Increased parent/caregiver practical skills in specific areas
By 12/14, obtain at least $50,000 for trainings
By 12/14, obtain at least $10,000 for grants
By 10/14, hire or identify training staff and training coordinator
By 10/14, hire or identify grant coordinator
n/aBy 3/30, 40% of served parents and providers will participate in the survey (survey records)By 5/15, 20 online/5 onsite trainings will be offered (program records)By 5/15, 50% of served parents and local providers will participate in trainings (training logs)By 5/15, 20 applications will be submitted (program records)By 6/15, 17 applications will be approved (program records)By 6/15, 90% of participants will express satisfaction with trainings (paarticipant surveys)
By 4/30, 100% of trainings will meet established quality standards (expert content review) By 7/31, 85% of participants will pass a knowledge assessment in specified areas (knowledge assessments) By 7/31, 90% of participants will report increased knowledge and practical skills in specified areas (participant surveys)
WHAT’S NEXT?
Implement Repeat Communicate
Staff Board Funders Volunteers Community
MIS-STEPS TO AVOID
Collecting data before you have set goals and identified outcomes
Collecting too much data Using data collection tools or measurements that don’t
match the goals and outcome Attempting without leadership support Assigning to staff member(s) regardless of time and skills Using only goals required by funders Identifying outcomes that are not feasible or not connected
to activities and organizational/program mission Overcomplicating the evaluation
EVALUATION RESOURCES
On-line Resources Management Library http://managementhelp.org Kellogg Foundation http://www.wkkf.org University of Wisconsin Extension www.uwex.edu Children Youth and Families Education and
Research Network www.cyfernet.org Innovation Network www.innonet.org Capacity Building Resource Library
www.strengtheningnonprofits.org IYI Resources
IYI Kids Count Data Report www.iyi.org IYI Virginia Beall Ball Library www.iyi.org IYI Consulting Services and links www.iyi.org
Recommended