Embed Size (px)
Citation preview
It is my business to locate trouble…Making Evaluation Work
Using Resultsin many ways with many people who need to know
Who
needs
to know?
Using Resultsin many ways with many people who need to know
What do they
need to know?
Using Resultsin many ways with many people who need to know
Using Resultsto communicate with decision-making stakeholders
Demonstration
Advocacy
Accountability
Using Results to communicate with broader stakeholders
Marketing
Informing
Recruiting
Using Resultsto make improvements …with the help of participants
Program
Improvement
Participant
Engagement
Using Resultsto celebrate and encourage staff, volunteers, and participants
Participant
Recognition
Using Resultsto build capacity for staff and organizations
Training and Supervision
Organizational Improvement
Using Resultsto advance practice
Publications
Proposals
Using Resultsin multiple ways
Using Resultsto write a report
Accountability
Report Elementscomponents of a formal report
Accountability
Opportunity
Program Response
Impact
Problem/NeedResearch Base
-Local Needs
-National Trends
-Prior Research
Report Elementscomponents of a formal report
Accountability
Opportunity
Program Response
Impact
Problem/Need
Program
Details
Research Base
-Local Needs
-National Trends
-Prior Research
Best Practice Strategy
-Audience
-Setting
-Activities
-
Report Elementscomponents of a formal report
AccountabilityEvidence of
Merit and
Worth
Evidence of
Merit and
Worth
Opportunity
Program Response
Impact
Problem/Need
Program
Details
Research Base
-Local Needs
-National Trends
-Prior Research
Best Practice Strategy
-Audience
-Setting
-Activities
Changes in Key Indicators
relative to context and
expectations
Report Elementscomponents of a formal report
AccountabilityEvidence of
Merit and
Worth
Evidence of
Merit and
Worth
Opportunity
Program Response
Impact
Problem/Need
Program
Details
Research Base
-Local Needs
-National Trends
-Prior Research
Best Practice Strategy
-Audience
-Setting
-Activities
Changes in Key Indicators
relative to context and
expectations
Why?
Who
What
When
Where
How
What’s so?
So What?
Report Elementscomponents of a formal report
AccountabilityEvidence of
Merit and
Worth
Evidence of
Merit and
Worth
Opportunity
Program Response
Impact
Problem/Need
Program
Details
Research Base90% of jobs require STEM skills
50% of local teachers cited need for
STEM
90% of jobs require STEM skills
50% of local teachers cited need for
STEM
Report Elementscomponents of a formal report
AccountabilityEvidence of
Merit and
Worth
Evidence of
Merit and
Worth
Opportunity
Program Response
Impact
Problem/Need
Program
Details
Best Practice Strategy
500 youth…50 events…
60 volunteers
Partnering with schools,
businesses, civic clubs…
500 youth…50 events…
60 volunteers
Partnering with schools,
businesses, civic clubs…
Research Base90% of jobs require STEM skills
50% of local teachers cited need for
STEM
90% of jobs require STEM skills
50% of local teachers cited need for
STEM
Report Elementscomponents of a formal report
AccountabilityEvidence of
Merit and
Worth
Evidence of
Merit and
Worth
Opportunity
Program Response
Impact
Problem/Need
Program
Details
Best Practice Strategy
Changes in Key Indicators
500 youth…50 events…
60 volunteers
Partnering with schools,
businesses, civic clubs…
500 youth…50 events…
60 volunteers
Partnering with schools,
businesses, civic clubs…
Research Base90% of jobs require STEM skills
50% of local teachers cited need for
STEM
90% of jobs require STEM skills
50% of local teachers cited need for
STEM
Survey: 82% of 345 youth…
Obs: 12 teams Excellent on Tech
Interview…
Records: Service, Training..
Survey: 82% of 345 youth…
Obs: 12 teams Excellent on Tech
Interview…
Records: Service, Training..
Report Elementscomponents of a formal report
Try to describe
your own program
Evidence of
Merit and
Worth
Evidence of
Merit and
Worth
Opportunity
Program Response
Impact
Problem/Need
Program
Details
Best Practice Strategy
Changes in Key Indicators
500 youth…50 events…
60 volunteers
Partnering with schools,
businesses, civic clubs…
500 youth…50 events…
60 volunteers
Partnering with schools,
businesses, civic clubs…
Research Base90% of jobs require STEM skills
50% of local teachers cited need for
STEM
90% of jobs require STEM skills
50% of local teachers cited need for
STEM
Survey: 82% of 345 youth…
Obs: 12 teams Excellent on Tech
Interview…
Records: Service, Training..
Survey: 82% of 345 youth…
Obs: 12 teams Excellent on Tech
Interview…
Records: Service, Training..
Report Elementscomponents of a formal report
Accountability
Opportunity
Program Response
Impact
Challenges
New Opportunities
Enhancements
Appendices
Problems,
Details,
Evidence
Obstacles,
Adjustments
Difficult Experiences30% had no transportation…
When the budget was cut…\
30% had no transportation…
When the budget was cut…\
Conclusions
Support
Materials
Program
Details
Report Elementscomponents of a formal evaluation
Accountability
Opportunity
Program Response
Impact
Challenges
New Opportunities
Enhancements
Appendices
Problems,
Details,
Evidence
Obstacles,
Adjustments Implications
Advanced training is needed…
Organization policy change…
Advanced training is needed…
Organization policy change…
Difficult Experiences30% had no transportation…
When the budget was cut…\
30% had no transportation…
When the budget was cut…\
Conclusions
Support
Materials
Program
Details
Report Elementscomponents of a formal evaluation
Accountability
Opportunity
Program Response
Impact
Challenges
New Opportunities
Enhancements
Appendices
Problems,
Details,
Evidence
Obstacles,
Adjustments Implications
Illustrations
Advanced training is needed…
Organization policy change…
Advanced training is needed…
Organization policy change…
Difficult Experiences30% had no transportation…
When the budget was cut…\
30% had no transportation…
When the budget was cut…\
As shown in the Table …
One participant said…
Pictured in the photo are…
As shown in the Table …
One participant said…
Pictured in the photo are…
Conclusions
Support
Materials
Program
Details
Report Elementscomponents of a formal evaluation
Accountability
Opportunity
Program Response
Impact
Challenges
New Opportunities
Enhancements
Appendices
Problems,
Details,
Evidence
Obstacles,
Adjustments Implications
Illustrations
Advanced training is needed…
Organization policy change…
Advanced training is needed…
Organization policy change…
Difficult Experiences30% had no transportation…
When the budget was cut…\
30% had no transportation…
When the budget was cut…\
As shown in the Table …
One participant said…
Pictured in the photo are…
As shown in the Table …
One participant said…
Pictured in the photo are…
Conclusions
Support
Materials
Program
Details
Additional Materials Program History… Organizational
Chart…
Detailed Budget…
Program History… Organizational
Chart…
Detailed Budget…
Report Elementscomponents of an detailed evaluation
So what would you
add to your report?
Opportunity
Program Response
Impact
Challenges
New Opportunities
Enhancements
Appendices
Problems,
Details,
Evidence
Problems,
Adjustments Implications
Illustrations
Advanced training is needed…
Organization policy change…
Advanced training is needed…
Organization policy change…
Difficult Experiences30% had no transportation…
When the budget was cut…\
30% had no transportation…
When the budget was cut…\
As shown in the Table …
One participant said…
Pictured in the photo are…
As shown in the Table …
One participant said…
Pictured in the photo are…
Conclusions
Support
Materials
Program
Details
Additional Materials Program History… Organizational
Chart…
Detailed Budget…
Program History… Organizational
Chart…
Detailed Budget…
Evaluation and Researchpaws for thought
Research
Discovery of Knowledge
Evaluation
Evidence for Effectiveness
Evaluabilityis this the right time to ask?
Missing the Mark…or the Data
Focusingwhat difference was intended?
IndicatorsLevelPriorityPrecision
Big P, little Pwhat did you expect?
Big picturePurpose-drivenCommunity-drivenEvidence-drivenDriven to completion
Mapping Evaluationenjoying the journey
Program Plan of WorkLogic ModelQuality and OutcomesPurposes of Evaluation
Mapping Evaluationenjoying the journey
Program Plan of WorkLogic ModelQuality and OutcomesPurposes of Evaluation
The End
Bye
Reportingwhat do you have to say for yourself?
Annual Report: Program Impact
