Program Evaluation for Nonprofit Professionals

Unit 2:Creating an Evaluation Plan

Your evaluation plan is your road map

http://farm8.staticflickr.com/7213/7278697432_a6137fd1a8_c.jpg

Resources: what you put

in

Activities: What you

will do

Outputs: Immediate

tangible results

Outcomes: The changes

you anticipate

Logic Model

Activities: What you will do

Outputs: Immediate

tangible results

Data Collection Method, Effort &

Schedule

Outcomes: The changes you

anticipate

Indicators: What those changes will look like

Data Collection Method, Effort

& Schedule

Process Evaluation Plan

Outcomes Evaluation Plan

Adapted from Innovation Network www.innonet.org

The logic model informs your plans

Two Part Evaluation Plan

Process Evaluation PlanWhat did we do? How well did we do it?

Outcomes Evaluation PlanWhat difference did it make?

Process Evaluation PlanningActivities

(from logic model)

Outputs

(from logic model)

Data Collection

(How will you get the info?)

Schedule

(When will you get the

Info?)

Level of Effort

(low, med, high)

Output:Question:Output:Question:Output:Question:Output:Question:Output:Question:

Adapted from Innovation Network www.innonet.org

Process Evaluation Planning

Activities

(from logic model)

Outputs

(from logic model)

Data Collection

(How will you get the info?)

Schedule

(When will you get the

Info?)

Level of Effort

(low, med, high)

Conduct dairy worker safety education sessions at 55 participating dairies.

Outputs:# of sessions conducted# of workers that attended sessions

Questions:Was session attendance what we anticipated?

Were workers satisfied with the information presented?

Attendance records

Session feedback form

After each session

After each session

Low

Low

Adapted from Innovation Network www.innonet.org

Outcomes Evaluation Plan

Resources: what you put

in

Activities: What you will

do

Outputs: Immediate

tangible results

Outcomes: The changes

you anticipate

Outcomes: The changes you

anticipate

Indicators: What those changes will look like

Data Collection Method, Effort &

Schedule

Logic Model

Outcomes Evaluation Plan

Outcomes Evaluation Planning

Step 1: Identify your priority outcomes

Could be:• Outcomes at the core of your mission• Outcomes you were specifically funded for• Outcomes vital to decision-making in the

future

Outcomes Evaluation Planning

Step 2: Draft indicators for each priority outcome.

Indicators are the evidence an outcome has occurred. They answer the questions:

• What will it look like when it happens?• How will you know change has occurred?• What does success look like?

Outcome Indicator Example

Outcome

Increase dairy worker’s knowledge of occupational risks.

IndicatorsDairy workers demonstrate increased knowledge in training pre/post test scores.

75% of dairy workers say “yes” when asked if their knowledge has increased in participant focus groups.

Outcomes Evaluation Planning

Guidelines for Creating Indicators:

1. Make them specific.2. Make them measureable.3. They should directly relate back to the

outcome.4. No more than three. Collecting data on more

than thee indicators per outcome is unrealistic.

Weak v. Strong IndicatorsExample Outcome: Improve worker health and safety practices

Weak• Workers report feeling safer

on the job in participant focus groups.

Strong• Workers report using one or

more practices taught in the training within 3 months during focus groups.

• Top 3 most important improvements are observed in monitoring visits.

• Dairy producers report making at least one change aimed at improving worker health and safety (within 6 months).

Example Outcome:Improved dairy worker health and safety practices

Indicator(s) Data Collection Method

Schedule Effort

- Decrease in documented workplace hazards within 6 months

Document review

Y1Y2Y3

Low

- Workers report using one or more practices aimed at improving health and safety during focus groups

Worker focus groups

Y1Y3

High

- Top 3 most important improvements are observed during monitoring visits.

Observation checklist

Y2Y3

Med

- Dairy producers report making at least one change in practices aimed at improved health & safety within 6 mo.

Dairy producer focus groups

Y1Y3

High

Real World Consideration: Time

• When do you need this information?

• What is realistic given your grant cycle or other time limitations?

• What amount of time is realistic for your staff to dedicate?

http://www.flickr.com/photos/kim_carpenter_nj/5462877477/sizes/z/in/photostream/

Real World Consideration: Staffing

• Who will be executing this evaluation plan, current staff or an external consultant?

• Does your staff have the knowledge necessary?

• Can your staff get help when necessary?

http://www.flickr.com/photos/53558245@N02/4978362207/sizes/z/in/photostream/

Real World Consideration: Budget

• What evaluation efforts are realistic given your budget?

• What parts of the evaluation plan can be dropped or changed if the budget is reduced unexpectedly?

http://www.flickr.com/photos/epsos/5394616925/sizes/z/in/photostream/

Real World Consideration: Context

• What kind of environment will you be conducting this evaluation in?

• What challenges do you anticipate?

• What will help you along?

Tips for Planning Your Evaluation

• Keep good notes on the thinking behind the plan, you will likely revisit this in the future

• Put all the ideas out there, then establish priorities based on what is realistic

• What is feasible given staff capacity?• What parts of this plan may be the most challenging?• Which parts depend on factors outside your control?• Where can cuts be made if the budget changes?

• Include others in the planning process• Program staff doing the work• Those who will be responsible for collecting the data• Executive team member or Board member

Give it a try!

• Review the additional information in the readings section

• Draft your evaluation plans (add them to the logic model template, use additional templates provided or simply write them out on a sheet of paper)

• Bring your questions to the discussion board!