Unit 7: Using your evaluation – Communicating, reporting, improving

University of Wisconsin - Extension, Cooperative Extension, Program Development and EvaluationUniversity of Wisconsin - Extension, Cooperative Extension, Program Development and Evaluation

Unit 7: Unit 7: Using your evaluation – Using your evaluation –

Communicating, reporting, improvingCommunicating, reporting, improving

1. Who2. What (Content)3. How (Format)4. When


“Think like a wise man, but communicate in the language of the people.”

− William Butler Yeats


Why communicate? Why communicate?

The proper function of evaluation is to speed up the learning process by communicating what might otherwise be overlooked or wrongly perceived…Success is to be judged by… success in communication…Payoff comes from the insight that the evaluator’s work generates in others.

- L. J. Cronbach

Cronbach, L.J. (1982). Designing evaluations of educational and social programs. San Francisco: Jossey-Bass, p. 8.


Plan with the end in mind.


Types of communications - reportsTypes of communications - reports

• Written report - long, short, summary

• Success story• Impact statement;

spotlight• Elevator story• Press release• Media appearance

• Public meeting• Memo, email, fax,

postcard• Newsletter• Personal discussion• Bulletin, brochure• Display/exhibit• Audio/video

presentation


Communication plan Communication plan

WHO?WHAT?HOW?WHEN?


WHO - Potential UsersWHO - Potential Users• County board/City council (elected and appointed officials)• Funders• Program Committee• Participants• Team / Staff• Volunteers• Collaborating agencies; partners• Businesses; business groups• Police – law enforcement• Schools boards, parent-teacher organizations• Community• Church organizations• State legislators• Professional organizations• You


WHO is the Audience?WHO is the Audience?

1.

2.

3.

4.

INTERNAL

1.

2.

3.

4.

EXTERNAL

Primary audience or secondary audience??


WHAT – WHAT – will you say, report, communicate?? will you say, report, communicate??

• What does the audience care about?• What do you want the audience to have?• What type of information resonates with the

audience: numbers, quotes, stories?


HOW will you communicate – HOW will you communicate – what formats will you use?what formats will you use?

• Report• Impact statement• Executive summary• Personal discussion• Oral presentation• Photo display• Press release• Newsletter, bulletin• Poster• ??????

Overview

Cooperative Extension- 72 county offices- Family Living Programs- 4-H Youth Development- Community, Natural

Resources andEconomic Development

- Agriculture and NaturalResources Education

- Geologic and NaturalHistory Survey

- 1.4 million educationalcontacts per year

Continuing Education- Partnerships with 26

UW System campuses- UW-Learning

Innovations- School for Workers- 162,000 enrollments per

year

Broadcasting & MediaInnovations- Wisconsin Public

Television- Wisconsin Public Radio- Instructional

CommunicationsSystems

- Nearly 1 million viewersand listeners per week

Small BusinessDevelopment Centers- 13 campus-based

centers- Wisconsin Innovation

Service Center- 15,000 hours of

counseling per year

[email protected](608) 262-3786

University of Wisconsin-Extension programs serving the98th Assembly District

Representative Marc Duff

Waukesha County Extension — The Waukesha County UW-ExtensionCooperative Extension office has developed partnerships with numerousorganizations to deliver educational programs addressing local needs. The Waukesha County Extension office developed a Hunger in Waukesha

County Nutrition Survey for area communities. These communities alsoreceive nutrition education services through the Waukesha County NutritionEducation Program at sites like the WIC clinics, elderly nutrition sites andfood pantries.

For the past two years, budgeting, nutrition and food safety have been taughtthrough Waukesha County Extension’s Nutrition Education Program at theWaukesha County Jail. The program also links inmates to resources that willbe available to them once they are released.The Nutritional Education Program also works with individuals with drug andalcohol abuse problems who are living at the Waukesha County HuberFacility. The program focuses on healthy eating and the effects of drugs andalcohol on the body, food safety and budgeting.

The Parenting the First Year newsletter is distributed through CommunityMemorial Hospital in Menomonee Falls with funding from the local Kiwanisclub. The Parenting the Second and Third Year newsletter is distributedthrough Community Memorial under a grant from the Child AbusePrevention Fund (CAP). The project currently is being evaluated in order tomeasure the value to parents. As a result of this partnership, parentingeducation programs such as Home Alone, Parenting Your Teen andDiscipline vs. Punishment are also offered.

The Waukesha County Extension office provided leadership to a collaborationof 14 agencies/organizations serving children and families to conduct a two-to-three-year program to disseminate research and conduct comprehensivetraining about early learning and brain development of infants and children.

Waukesha County Extension offers a Grant Opportunities Newsletter, whichidentifies grants available to the community from a variety of state and federalagencies as well as local foundations. The list includes federal agencies, suchas the Department of Health and Human Services, Department of Housing andUrban Development, Department of Labor and Employment and TrainingAdministration, and state entities including the Department of WorkforceDevelopment, Department of Administration - Division of Housing, TEACH,Department of Transportation, Department of Natural Resources and theWisconsin Advanced Telecommunications Foundation.

The Waukesha County Extension-Urban Initiative Program in Sussex willaddress neighborhood revitalization, building a sense of community, crimeand public safety. In 2001 the program received a $10,000 CommunityDevelopment Block Grant which will allow expansion of the partnership andimplementation of an action plan in at-risk, troubled neighborhoods.


HOW – what format, style will you use?? HOW – what format, style will you use??

• Format depends on purpose and audience

• Written, oral• Short, long• How does the audience prefer receiving

information? How does the audience best learn?

• What do you have the resources to do?


WHEN will you communicate?WHEN will you communicate?

• Quarterly?• Opportunistically?• When requested?• ???


Reporting outline follows your logic modelReporting outline follows your logic model


Formal evaluation reports typically include…Formal evaluation reports typically include…• Abstract/executive summary• Introduction

− Purpose of the evaluation; key questions− Program background, description

• Methods/procedures− Data sources− Data collection procedures− Sampling− Limitations

• Results• Discussion• Conclusions/recommendations• References• Appendices


Effective communicationsEffective communications• Tailor message to issue and audience• Avoid jargon and technical terms• Be clear, concise• Use active voice• Eliminate wordiness• Check writing, grammar• Be accurate, balanced, impartial• Be timely• Use graphics, quotes, photos, real stories• Consult a communications specialist• Write-rewrite-rewrite


MythsMyths• One report is enough.• People read written reports.• Complex analysis and big words impress people.• Oral reports have the same effect as written

reports.• Describing limitations weakens report.• Everything should be reported.• The audience knows why they are getting the

report.


Ingredients of a Good Success StoryIngredients of a Good Success Story

• Involves human interest: catches your attention • Demonstrates behavior change; action taken

− change in practice that results in benefits to individual, community, environment...

• Uses words of/meaning for people involved • Indicates profit to the individual, if appropriate• Based on reliable, credible information • Includes numbers and narrative• Presents balanced, fair assessment


Make sure the evaluation isn’t ignoredMake sure the evaluation isn’t ignored

• Engage stakeholders in doing the evaluation: in the design, data collection, in analysis, interpretation of results…

• Get the information to the right people• Address issues that people think are

important• Keep it in front of people• Present it in time to be useful


Discuss limitationsDiscuss limitationsWritten reports:

• Be explicit about your limitationsOral reports:

• Be prepared to discuss limitations• Be honest about limitations• Know the claims you cannot make

− Do not claim causation without a true experimental design

− Do not generalize to the population without random sample and quality administration (e.g., <60% response rate on a survey)


Reporting results to the mediaReporting results to the media

All Media: • Avoid using too many statistics. • Focus on the key points.• For quotes, speak more globally about

the issue.• Always give the source and timeliness

of your stats. It’s the “news peg.”Steve Busalacchi

Director, News & InformationWisconsin Medical Society


Reporting results to the mediaReporting results to the media

Radio and TV:• Do not offer exact statistics

– ear cannot track. “73.6% of respondents”vs.“Nearly three quarters of those surveyed”

• Don’t go into great detail. Have backup info ready.

Steve BusalacchiDirector, News & Information

Wisconsin Medical Society


Using graphicsUsing graphicsWhat do I want my audience to

learn – remember –

from my graph or table?


Using graphicsUsing graphics• Title • Clear units of measure • Date(s) data collected• Simple, straightforward design without

“clutter”• Font size 10 point or larger• Explicit data source(s)• Sample size, if applicable for the

audience


GraphsGraphs

• Bar graphs: show comparisons− Stacked bar− Multiple bars− Vertical, horizontal bars

• Pie charts: show parts of a whole• Line graphs: show progress over time


Bar graphBar graph

• Compare groups of client outcomes• Compare points in time

Each group you compare gets a “bar”


Percentage of worksites implementing "Healthy Lifestyles" program

60%

74%

0102030405060708090

100

Williams CountyWorksites

Wisconsin Worksites

Perc

enta

ge o

f wor

ksite

s


Horizontal Graph

Example Bar Graph

0 10 20 30 40 50 60

1

2

3

4

Series2

Series1


Vertical Graph

Example Bar Graph

0

10

20

30

40

50

60

1 2 3 4

Series1Series2


Column Chart

0%

20%

40%

60%

80%

100%

1

Column Chart


Pie charts: parts of a wholePie charts: parts of a whole

• Useful for displaying proportions− Percentage of clients achieving the outcome− Percentage of participants who are boys− Percentage of sites that are rural

• Ensure individual parts add to 100%


Pie ChartPie Chart

Example Pie Chart

123

4


Line graphs: time linesLine graphs: time lines

• X axis conveys the time • Y axis is the variable of interest• Multiple lines or multiple line graphs

show performance of different groups


Line GraphLine Graph

Example Line Graph

0

10

20

30

40

50

60

1 2 3 4

Series1Series2


Area GraphArea Graph

Example Area Graph

0

10

20

30

40

50

60

70

1 2 3 4

Series2Series1


Pictogram Pictogram End of

November Total!


Graphing using Microsoft ExcelGraphing using Microsoft Excel• Click on graph icon on toolbar, or select

“insert” and “chart”• Proceed through chart wizard

− Select the type of graph you want− Highlight the range of data to include− Set up titles, axis labels, scales, legend− Finish

• Edit the chart using drop down menu for charts

− Click on chart to highlight it− Use the top toolbar selection “chart” to edit

parts of the chart

Documents

Unit 7: Using your evaluation – Communicating, reporting, improving