Getting the word out: A comparison of methods for disseminating research findings Ryan C. Burke

Nancy B. MuellerJenine K. Harris

Tanya P. Montgomery

Report Series and WorkshopReport Series and WebsiteReport Series

Report Series, Website, and Workshop

Information flow of Project LEaP results between dissemination modes

Rationale

Methods: Background

Participants completed both a web-based baseline and post survey. Workshop participants also completed pre/post surveys at the time of the workshops. Visitors to the websites were tracked with StatCounter, a web tracking service.

To evaluate and compare the effectiveness of the dissemination methods, five major outcome measures were examined using a 5-point scale:

l Overall usefulness of LEaP results to the individual participant l Usefulness of LEaP participation for state's tobacco control program l Usefulness of LEaP participation for individual's agency l Usefulness of LEaP participation for individual's own tobacco control work l Participation in LEaP was worth individual's time

Methods: Data Collection

Results: Main Outcomes

Conclusion

Results: Are More Modes Better?

Participants receiving all three modes of dissemination (report series, website, and workshop) were more likely to:

l Find the Project LEaP results useful;l Be satisfied with their agency's participation in the project; andl Have shared Project LEaP results with others.

Based on these findings, the use of the traditional reports is not enough. Evaluators and researchers need to implement a multi-mode approach to disseminate findings.

Map of Project LEaP states

109Workshop Invitations Sent

167Reports Distributed

69Attended Workshop

43Shared Workshop Info

88Read Reports

64Shared Reports

92Website Links Sent

43Accessed Website

11Shared Website

l A 26-page report series was developed covering five areas: m Introduction and Executive Summary m Program Environment m Program Resources m Program Capacity m Program Sustainability

at the Saint Louis University School of Public Health

l The website contained five sections and included all the report series content. l Other features of the website included polling questions, a feedback form, a site search engine, and a tell-a-friend form.

l The overall response rate for the post survey was 91%. l The reports were distributed to all 167 survey participants. l 93% (n=155) of participants reported being aware of the Project LEaP results. l Of the 167 reports distributed, 53% (n=88) reported reading the reports.l Of the 109 invited to a workshop, 62% (n=69) reported attending a workshop.l Of the 92 website links sent, 47% (n=43) reported accessing the website.

Results: Comparison of Dissemination Modes l Overall, 62% of participants shared the Project LEaP results with their colleagues.

l Over 80% of those receiving all three modes reported sharing the project results with their colleagues. l There was an increasing trend in the percent of participants sharing the evaluation results from those receiving reports only to those receiving all three modes (chi-square=7.4; p=0.06).

l There was some evidence of usefulness for all five outcome variables. However, usefulness was more apparent when the report series was combined with the website, workshop, or both. l Planned comparisons revealed that participants engaging in all three modes found the project results significantly more useful for their own work or their agency than participants engaging in one or two modes of dissemination (p<0.01).

During the workshop, you will:

l Learn about the Project LEaP results from the 2004 evaluation of Oregon's tobacco control program.

l Examine the relationships between tobacco control partners and their importance to the program.

l Learn about Oregon's ability to sustain its program and the tools used to assess program sustainability.

l Determine how to use the results to capitalize on your program's strengths and overcome its challenges.

There will be lots of opportunities to talk with your colleagues and participate in discussions, activities, and planning! Your participation is important to

determine how to utilize the evaluation results to strengthen Oregon's tobacco control efforts!

Breakfast and lunch will be provided. Limited lodging assistance is available.

For more information or to register, contact Angela Recktenwald

at (314) 977-8109 or at reckteaj@slu.edu.

at Saint Louis University

Thursday, August 25, 2005 9 am to 3:30 pm

McMenamins Kennedy School5736 N.E. 33rd Avenue, Portland(503) 249-3983 or (888) 249-3983

http://www.kennedyschool.com

D issemination Workshop

You are invited to attend the

REGISTRATION DEADLINE IS August 12 .th

l The one-day workshop was designed to be an interactive learning experience to help practitioners use the evaluation results for program planning.l The workshop was organized into four sections: Project LEaP introduction, partner relationships, program sustainability, and program planning. l Participants developed short-term action plans addressing 2-3 challenges for their programs.

Extent of secondary dissemination by mode

0

20

40

60

80

100

Dissemination Mode

Per

cen

t w

ho

shar

ed r

esu

lts

ReportSeriesOnly

Report Series and

Web

Report Series andWorkshop

ReportSeries, Web,

and Workshop

l Using planned comparison analysis, a significant difference was seen when comparing usefulness of project participation to individual's agency between those receiving all three dissemination modes and those receiving the report series only. l Those engaging in all three dissemination modes found participation in the project and the project results the most useful compared to those receiving fewer dissemination modes. Those engaging in two types of dissemination found it less useful, and those only receiving the report series found it the least useful.

Mean response to outcome measures across modes

In 2005 the Center for Tobacco Policy Research (CTPR) conducted Project LEaP (Linking Evaluation and Practice), an evaluation of eight state tobacco control programs. The evaluation results were disseminated to the tobacco control partners in each state via a print report series, website, and/or interactive workshop. The states and their assigned dissemination modes are shown in the map below:

Although evaluation of state tobacco control programs has become more prevalent over the past decade, there has been little focus on effective techniques for disseminating evaluation results to state tobacco control professionals.

Disseminating evaluation results to program stakeholders is an essential part of any program evaluation. There is limited evidence about the most effective way to disseminate information to state program partners so that the results can be utilized in their program planning efforts. Therefore, the main research question for this project was:

What is the most effective dissemination strategy (e.g., print, website, workshop) for disseminating evaluation results to state program partners?

Secondary Dissemination

Mean response to outcome measures across collapsed modes1 Mode 2 Modes 3 Modes

Outcome Measure Reports only

(n=74)

Reports & web OR

reports & workshop

(n=74)

Workshop, web, & reports (n=19)

p-value

Overall usefulness of LEaP results to the individual participant

2.77 2.85 3.32 .11

Usefulness of LEaP participation for state’s tobacco control program

3.07 3.38 3.82 .07

Usefulness of LEaP participation for individual’s agency

2.70 3.07 3.72 .01

Usefulness of LEaP participation for individual’s own tobacco control work

2.60 2.98 3.47 .03

Participation in LEaP was worth individual’s time 3.90 3.89 4.16 .52

Mean response to outcome measures across collapsed modes

1 Mode 2 Modes

3 Modes

Outcome Measure Report only

(n=74)

Reports & web

(n=24)

Reports &

workshop (n=50)

Reports, web, &

workshop (n=19)

p-value

Overall usefulness of LEaP results to the individual participant

2.77 2.82 2.86 3.32 .22

Usefulness of LEaP participation for state’s tobacco control program

3.07 3.44 3.36 3.82 .15

Usefulness of LEaP participation for individual’s agency

2.70 3.19 3.02 3.72 .02

Usefulness of LEaP participation for individual’s own tobacco control work

2.60 2.94 3.00 3.47 .07

Participation in LEaP was worth individual’s time

3.90 4.00 3.85 4.16 .64