TRICKY TERMINOLOGYCE Conference, October 20, 2010

1. Think of ONE program that you do. Write that program down.

2. What outcome do you want for this program?

What do you think?Wow. I’m sure this will

work. It has strong, scientific studies to

back it up. Sounds like an interesting idea but how do you

know it will work?

Hmmm. This sounds

promising and there’s some

good evaluation but the

evidence is limited.

Evidence-basedResearch-basedEvidence-informed

Purpose of session Have a better understanding of the terms

research-based, evidence-based and evidence-informed, and examples of FLP, WNEP and 4-H resources corresponding to these terms

Explore how various settings and learning environments determine the level of research and evidence used in programming efforts

Apply principles of effective programs to current efforts

History of ‘Evidence-based’ Medical model – What is the body of evidence

for prescribing a drug or recommending a treatment? How rigorously has this drug/treatment been

tested? Have results been confirmed through multiple

studies? Are there conflicting studies? What ‘risk to benefit’ needs to be considered?

Adopted by education and programs

Evidence-based programs Developed and grounded in a clear

theoretical foundation Carefully implemented and evaluated (in

rigorous experimental or quasi-experimental studies)

Evaluation findings document expected positive outcomes

Critical peer review of findings Program has been replicated

Research-based programs Draw on published research about

the causes and consequences of an issue

Vary considerably in scientific rigor With rigorous evaluation, some

research-based programs have potential to become evidence-based programs

Moving towards more evidence

Research-Based Programs

Evidence-Based Programs

Reality

No evidence

Evidence-based

Research-based

Evidence-Informed Evidence Informed Program

Improvement (EIPI) • A process for improving program quality

and impact (in new or existing programs)

• Draws on the principles of effective prevention programs

• Helps existing programs become more aligned with these principles

Principles of Effective ProgramsI. Program Design and Content Theory-driven and research-based Sufficient dosage and intensity Comprehensive Limit use of fear and punishment Active learning techniques

Principles of Effective ProgramsII. Program Relevance Developmentally appropriate Reach participants when ready to change Socio-culturally relevant

III. Program Delivery Foster good relationships Well-trained and committed staff

Principles of Effective ProgramsIV. Program Assessment and Quality

Assurance Well-documented Evaluation and refinement

Fidelity and adaptation How might removing a portion of this

lesson affect its effectiveness? Will adding something to this lesson

affect its effectiveness? How could shortening a series of

lessons affect the impact of the program?

What adaptations might be needed for this program to be more effective?

Tricky TerminologyNo research or evidence base

Research-based Evidence-based

Parent Education

The Mozart Effect (Linking intelligence to listening to classical music)

How to Talk So Kids Will Listen and Listen So Kids Will Talk

Raising a Thinking Child

Strengthening Families Program for Youth 10-14

Parenting Wisely

Childhood Obesity

Power Panther word search

Loving Your Family, Feeding Their Future

Color Me Healthy

We Can! (Ways to Enhance Children’s Activity & Nutrition)

Out of School Time

Recreation at the YMCA

4-H after-school programs

Quantum opportunities program

Why are evidence-based programs important? Increases the chances that the program will be

effective Greater efficiency Increased accountability

Funders and policy makers recommending & sometimes requiring

Cost-effectiveness Availability of good science Marketability Ethics

ResourcesWhat Works Wisconsin: Effective

Prevention Programs for Children, Youth and Families

http://whatworks.uwex.edu/index.html Evidence-Based Programs: An Overviewhttp://whatworks.uwex.edu/attachment/whatworks_06.pdf Evidence-Based Program Registrieshttp://whatworks.uwex.edu/attachment/EBRegistriesAug2009.pdf

Evidence-Based Parenting Programshttp://whatworks.uwex.edu/attachment/Directoryofeb.pdf What Works for Health: Programs and

Policies to Improve Wisconsin’s Healthhttp://whatworksforhealth.wisc.edu/intro.asp

Wisconsin Clearinghouse for Prevention Resources

http://wch.uhs.wisc.edu/01-Prevention/01-Prev-EvidenceBased-rating.html

Social Programs that Workhttp://evidencebasedprograms.org

Guide to Community Prevention Serviceshttp://www.thecommunityguide.org/index.html

Centers for Disease Control and Prevention (CDC), Recommended Community Strategies and Measurements to Prevent Obesity in the United States

http://www.cdc.gov/mmwr/preview/mmwrhtml/rr5807a1.htm

Comprehensive site listing evaluated, effective programs for parenting, early childhood, school-aged children, and teens

http://www.cyfernet.org

"Out-of-school time program research & evaluation database" at the Harvard Family Research Projecthttp://www.hfrp.org/out-of-school-time/ost-database-bibliography/database

Consider your program Where would you put it on the

continuum? Why?

What would be needed to move it further along the continuum to evidence-based?

No evidence Evidence-basedResearch-based