Upload
gloria-summers
View
214
Download
1
Tags:
Embed Size (px)
Citation preview
Why Use Evidence-based Programs?
• Mandate - Funding may require EBP
• Efficiency - More effective use of scarce resources
• Accountability - More confidence that what we do will make a difference
• Why reinvent the wheel?
• Take advantage of existing expertise (prevention science)
• Programs that aren’t evidence-based may actually be harmful
Summary of Benefits and Costs (2003 Dollars)Summary of Benefits and Costs (2003 Dollars)Dollars Per Youth (PV lifecycle) BenefitsBenefits CostsCosts B B -- C CEarly Childhood Education $17,202 $7,301 $9,901
Nurse Family Partnership $26,298 $9,118 $17,180
Functional Family Therapy $16,455 $2,140 $14,315
Life Skills Training $746 $29 $717
Seattle Soc. Dev. Project $14,246 $4,590 $9,837
Guiding Good Choices $7,605 $687 $6,918
Multi-D Treat. Foster Care $26,748 $2,459 $24,290
Intensive Juv. Supervision $0 $1,482 -$1,482
Selected Findings Selected Findings ((www.wa.gov/wsipp)
Aggression Repl. Trng. $9,564 $759 $8,805
S. Aos
Programs can be placed along a continuum of
“proof” of effectiveness
Not much
confidence
• Best Practices“We’ve done it and we like it”
• Promising Approaches
“We really think this will work…but we need time to prove it”
• Research-based“This program is based on sound theory informed by research”
• Evidence-based“This program has been rigorously evaluated and shown to work”
Very much confidence
Evidence-based ProgramsTheoretically sound interventions that
have been evaluated using a well-designed study (randomized controlled trial or quasi-experimental design) and have demonstrated significant improvements in the targeted outcome(s).
Evidence is strengthened by independent replication and sustained benefits.
Federal Working Group Standard for EB Certification*
Experimental Design/Randomized Control Trial
Effect sustained for at least 1 year post- intervention
At least 1 independent replication with RCTRCT’s adequately address threats to
internal validityNo known health-compromising side effects
*Adapted from Hierarchical Classification Framework for Program Effectiveness, Working Group for the Federal Collaboration on What Works, 2004.
EBP work, but ..EBP still represent the minority of prevention programs in
useResearch has shown that many (most?) aren’t being
implemented with sufficient quality or fidelityThere is tension between advocates of strict fidelity and
those who encourage local adaptationVery few programs measure or monitor implementation
fidelity and quality Policy work is needed to capture & redistribute dollars
saved through preventionPrograms are often “pull-out” and operate singularlySustainability remains a challenge – no permanent
infrastructure
Evidence-based programs are most effective when they are implemented with fidelity
Fidelity = the practitioners use all the core intervention components skillfully
IMPLEMENTATION IS IMPORTANT
Dissemination of information by itself does not lead to successful implementation.
Training alone, no matter how well done, does not lead to successful implementation. It must contain on-going oversight
Improving fidelity locallyWhat gets measured mattersImprove practitioner knowledge of
prevention scienceUse adaptation discussion as a tool for
training on the logic model of an intervention
Build a sustainable infrastructure for monitoring implementation fidelity and quality
Build internal capacity AND desire
Adaptation happens…Between 23% and 81% of program activities
may be omitted during implementation. (Durlak, 1998)
Only 19% of schools implement research-based curricula with fidelity. (Hallfors & Godette, 2002)
Only about 75% of the students received 60% or more of the Life Skills Training Program. (Botvin, et al., 1995)
Is adaptation Inevitable/necessary?Research shows that a high degree of
fidelity is attainable (Project TND, PROSPER, Blueprints)
There is little empirical support for cultural adaptation of EVPsMost have shown similar effects across
gender, ethnicity/race, SESStudies of prospective cultural adaptations
have failed to yield positive outcomes
Many Sources for EBPBlueprints for Violence Prevention *
www.colorado.edu/cspv/blueprints
SAMSHAwww.modelprograms.samsha.gov
OJJDP Model Programs Guidewww.dsgonline.com/mpg2.5/mpg_index.htm
The International Campbell Collaboration*http://www.campbellcollaboration.org/Fralibrary.html
Safe and Sound: An Educational Leader’s Guide to Evidence-Based Social and Emotional Learning Programs http://www.CASEL.org
What Works Clearinghouse (U.S. Department of EducationInstitute of Education Sciences)http://ies.ed.gov/ncee/wwc/index.asp
Top Tier Evidence Initiative www.toptierevidence.org
Center for Disease Control Effective Programshttp://www.cdc.gov?healthyYouth.partners/registries.htm
Child Trends What Works *(www.childtrends.org )
Guide to Community Preventive Services www.thecommunityguide.org
Promising Practices Networkwww.promisingpractices.net
Social Programs that Work www.evidencebasedprograms.org
Best Evidence Encyclopedia (K-12 Education) www.bestevidence.org
A Google search on “Evidence Based Programs” returned over 63 million hits
Professionals who will select the EBP, train or be trained, and/or implement the program
Considerations:Knowledge of programsAwarenessAttitudeMotivation to change or adopt something
newBehavior routines- Can the existing
structure be changed?
Barriers – What do you think?
MoneyResistance to
changeBuy-inRed tape TimeEgosOrganization’s
traditions
Insufficient professional development
ContractsUnions Cost or availability
of training for a program
Sustainability
Have you heard this one before?
I already have too much to do.I’ve heard it doesn’t work with all
audiences.I am not giving up what I believe works to
try something new that might not work.Most of the program is OK but there are
parts that I can make better.Not possible to get individuals/families to
participate in a program like this. It takes too much time, too many sessions.
Overcoming the Barriers Summary of Barriers Strategies for Success Difficult to find the right
programFear of changePeople are invested in the
status quoDevalues professional
expertiseOrganizational barriers:
personnel rules, staff turnover
Limits to the flexibility of the system
Cultural competency
Administrative supportTeacher/educator/
stakeholder supportSecure financial resourcesProvide high-quality training
to ensure program fidelityAlign intervention with
school/community goals, policies and programs
Make program outcomes visible
Develop strategy for staff turnover
Questions to Ask…How strong is the evidence of each program?Do we need it?What are the other options?Does the evidence apply to our population?Is it worth the investment, and can we afford
it?Can we assemble the necessary resources and
stakeholders?Will our community find it acceptable?How broad might the impact be?Can the developer support our site?Do we know others who have used it?What will it take to sustain it?
ResourcesAddressing Barriers to Learning. (2007). Evidence-based practices
in schools: Concerns about fit and implementation. UCLA Center, 12(3).
Dunifon, R., Duttweiler, M., Pillemer, K., Tobias, D., & Trochim, W.M.K. (2004). Evidence-based extension. Journal of Extension, 42(2).
Forman, S.F., Olin, S.S., Hoagwood, K.E., Crowe, M., & Saka, N. (2008). Evidence-based interventions in schools: Developers’ Views of implementation barriers and facilitators. School Mental Health, 1.
Henderson, M.L., Mathias-Humphrey, A., & McDermott, M.J. (2008). Barriers to effective program implementation: Rural school-based probation. Federal Probation, 72(1).
Identifying barriers to evidence-based uptake. (2006). National Institute of Clinical Studies, Melbourne VIC.
Research Development Associates. Evidence-based Practices. Power Point retrieved from http://www.resourcedevelopment.net/projects/workshops.html