Upload
buinhan
View
216
Download
3
Embed Size (px)
Citation preview
Adoption of Evidence-Based Primary Prevention Programs: Identifying
Characteristics of Ready, Willing, and Able Schools
Paul D. Flaspohler, Ph.D., Cricket Meehan, Ph.D.,
& Vanessa Watts, M.A.
Miami University
Kathryn E. Keller, MPA
Health Foundation of Greater Cincinnati
Learning Objectives
• To identify key features in determining a school's readiness and capacity to implement and sustain an evidence-based prevention program (EBP) targeting school violence
• To understand how to assess a school's readiness and capacity to implement and sustain an EBP
• To describe the promise of EBPs and related services and to describe the challenges inherent in dissemination of EBPs and related services
Working with the “Ready”
Matching Intervention to
Need
Mainstreaming Evaluation and Accountability
Building a Community of
Practice
6678 KEY STAKEHOLDERS Evidence of Need
Children (N=6678) in 15 different elementary and middle schools
0
200
400
600
800
1000
1200
1400
7 8 9 10 11 12 13 14
Age
0
200
400
600
800
1000
1200
1400
3 4 5 6 7 8
Grade
Ethnicity by School
0
100
200
300
400
500
600
A B C D E
Asian/Indian/PI
African American
Hispanic
White
SOME CHALLENGES Evidence of need
Barriers to Successful Development
I feel safe in my school 17 % strongly disagree/disagree
I worry a lot 59.7 % certainly/somewhat true
I have many fears 43.4 % certainly/somewhat true
I get mad easily 28.4 % most, all of the time
I fight a lot 28.4 % certainly/somewhat true
I like myself 22.7 % never/some of the time
I feel lonely 19.7 % most/all of the time
Other children pick on me 33.4 % somewhat/certainly true
SOME FORM OF UNIVERSAL PREVENTION COULD MEET NEEDS OF MORE THAN 3000 “AT RISK” CHILDREN IN SCHOOLS
WHAT DOES THE RESEARCH TELL US ABOUT MEETING THAT NEED?
The Research to Practice Gap Practice
• Community Systems
• Schools
• Health Agencies
• Community Coalitions
Science
• Basic research
• Efficacy
• Effectiveness
• Services Research
RESEARCH TO PRACTICE AND COMMUNITY CENTERED MODELS
Bridging the Gap
Research to Practice models
• Begin with basic research questions
• Based on generation of generalizable knowledge
• View “best practice” in terms of discreet programs and strategies to be adopted as intended in the original
Problems with IOM Model
• Mismatch between Innovation design and community capacity
• Mismatch between community/organization ideology and values that underlie a Innovation
• Pro-Innovation Bias
• Simplistic models of decision making
– (Miller & Shinn, 2008)
Community centered models
• Start with the community
• Focus attention on local needs (contextual knowledge)
• See “best practice” as a process rather than magic bullet programs
• Emphasize that practitioners, clients, and communities should have control as well as accountability
Problems with CC Models
• Rarely evaluated
• When evaluated, many prove not to work. Some prove worse than nothing at all.
• Often re-invent the wheel
CAPACITY In both research to practice models and community centered models
B. Distilling the Information—Prevention Research System
C. Supporting the Work—Prevention Support System
A. Putting It Into Practice—Prevention Delivery System
Synthesis
General Capacity
Building
Innovation-Specific
Capacity Building
General Capacity
Use
Innovation-Specific
Capacity Use
Policy Climate
Funding
Existing Research and Theory
Translation
Effective Dissemination/ Implementation requires attending to:
Dissemination/ Implementation
Characteristics of the
Innovation
Process of Transitioning Innovation to
Setting
Characteristics of the Setting
Practice
• Practice-Based Evidence
Best Practice Process
• Local control
• Self-evaluation
• Tailoring process and technology
Research
• Evidence –Based Practices
Bridging the gap
The SWEBP Process
Pre-planning
Planning
Implementation
Evaluation
Level 1 Grant
Level 2 Grant
Working with the “Ready”
Matching Intervention to
Need
Mainstreaming Evaluation and Accountability
Building a Community of
Practice
Working with the “Ready”
Core Team RFP Workshop
Capacity/ Readiness
Assessment
Planning Grant Proposal
Core Team – The Right People Administration
Principal
Superintendent
Education
Teacher
Health/ Mental Health
Counselor
School Psych
School Social Worker
Nurse
Other
Other
Parent
Community Member
RFP Workshop – Initial Commitment
• Core team must attend
• Access to Capacity/Readiness Assessment
• Access to Proposal
Capacity/Readiness Assessment
Can you use data?
• Readiness Assessment includes: – Collective Efficacy
– School Characteristics
– Strength-Based Practices
– School Readiness
– Life Satisfaction
Spreading the word…
• Staff complete assessment online
• Data is collected, aggregated and distributed by TA team
• Schools receive aggregate data to use in planning grant
Readiness Indicator: Staff Participation
0
10
20
30
40
50
60
70
80
90
100
Yr 2006 Yr 2007
Yr 2008
% Staff Response (funded)
% Staff Response (not funded)
Application Review Criteria • Eligibility (y/n)
• Commitment to EBP (20)
• Strength of Argument (10)
• Staff participation % (10)
• Survey Results (20)
• Behavioral Health Problems (10)
• SED Students (10)
• Poverty – F/R Lunch (5)
• Addresses Disparity (5)
• Budget (10)
Reducing applicant pool from 300+ Schools
0
2
4
6
8
10
12
14
16
18
20
2006 2007 2008
The SWEBP Process
Pre-planning
Planning
Implementation
Evaluation
Level 1 Grant
Level 2 Grant
Planning Phase
Needs and Resource
Assessment
Problem Identification
Program Selection/ Adaptation
Implementation Grant Proposal
Needs and Resource Assessment
• Informants – Staff Survey
– Student Survey • 3-5 grade; 6-8 grade
– Parent Survey
• Content (CASEL+) – Classroom behaviors
– Socio-emotional functioning
– School Climate
– Bullying and Violence
– Others
• Technical Assistance
– Consent/Parent Notification
– Packaging
– Administration (limited)
– Data entry • Remark
• Prezza
• TurningPoint
– Data analysis
Problem Identification
• Data informed
• Combine N/R data with other school data sources
NO MAGIC BULLETS
Students: Strengths and Difficulties
0.00
0.10
0.20
0.30
0.40
0.50
0.60
0.70
0.80
0.90
Rural Elementary
Rural Middle Urban Elementary
Urban Middle
emotion
conduct
Hyper
peer prob
Prosocial
Staff: Disruptive Behavior Problems
0
10
20
30
40
50
60
A B C D E
Staff: School Orientation to Innovation
0.0
10.0
20.0
30.0
40.0
50.0
60.0
70.0
80.0
A B C D E
Program Selection/Adaptation
• Must be EBP
– Blueprints
– NREPP
– Whatworks.gov
• Consider
– Need
– Readiness
– Feasibility
• Plan for adaptation to local needs
Rea
dine
ss
Fe
asib
ility
Evidence of Need
EBPs
Olweus Bullying Prevention Program
PATHS
Life Skills Training
Caring School Communities
Project Achieve
Peacebuilders
The SWEBP Process
Pre-planning
Planning
Implementation
Evaluation
Level 1 Grant
Level 2 Grant
Implementation Phase
Program training
Ongoing technical
assistance
Monitoring process
Monitoring outcomes
Program Training
• “Innovation Specific” Capacity building – Training provided by certified program experts
• Coordinated where possible to reduce costs
• TA team serves as buffer between trainers and schools
Ongoing Technical Assistance
• Proactive and Reactive strategies to promote general capacity
– Using Data
– Parent Engagement
– Culturally Competent Adaptation
– Sustainability
• Quarterly meetings of programs in implementation phases to celebrate success and problem solve
Local evaluation
• Process monitoring
• Fidelity monitoring
• Outcomes monitoring As planned
With Quality
Outcomes
Outcome Evaluation
• Innovation Specific (program) Outcomes
• Common Outcomes
– Quality of Life
– Strengths and Difficulties
– Organizational Capacity
HOW DO YOU KNOW YOUR EFFORTS ARE WORKING, PAUL?
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
PATHS PATHS PATHS Olweus Olweus LifeSkills LifeSkills CSC CSC
First Year Fidelity
Two year fidelity
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
PATHS Olweus Olweus LifeSkills
Year 1
Year 2
The SWEBP Process
Pre-planning
Planning
Implementation
Evaluation
Level 1 Grant
Level 2 Grant
Evaluation Phase
Continuous Improvement
Sustainability
LESSONS LEARNED... Strengths:
• 100% process and outcomes monitoring!
• can help to raise awareness of the complex nature of planned change processes
• engagement in the process builds general capacity to participate in data-informed decision making
Challenges
• Are we being told what we want to hear?
• Working “the Cream”?
• Unsolvable systems problems?
• Sustainability of our efforts?
Next steps…
Agricultural Extension Model Professional Development Model
Best Practice Process involves
• Developing a cadre of skilled change agents to drive the work
– “the more I see, the less I know”
• Embracing the paradox – dynamic equilibrium
• Resisting “umbrage”
"Good judgment
comes from
experience
...and a lot of that
comes from bad
judgment.“ Will Rogers
THE HEALTH FOUNDATION OF GREATER CINCINNATI GRANTEES
Ready, Willing, and Able Schools
The Need Identified From Our Grantees’ Applications…
• “The socioeconomic status [of our families] often leaves students in a home environment that is deficient in stability. As a result, children are not taught social and emotional skills needed to succeed in school and social settings.”
• “Teachers are constantly correcting inappropriate behavior and modeling good social behaviors, which interferes with academic teaching time.”
• “Students come to school tired, hungry, and unkempt.”
The Need Identified From Our Grantees’ Applications…
• “[Our school] is making cuts in basic programs, staff and services in order to balance the budget deficit of $629,599.”
• “Mental health services in [our county] are few and fragmented in their accessibility and availability.”
• “[Our school] is located in a small isolated town. No public transportation exists in the town and resources are difficult to access and not within walking distance.”
WE HAVE THE NEED, BUT ARE WE READY?
The Health Foundation of Greater Cincinnati Grantee Schools
The Right People…
• “The staff at [our school] is dedicated to helping their students achieve both academic and social success. They are supportive of mental health programming within the school.”
• “The staff have expressed a desire to help students with issues such as social skills, anger management, peer socialization, respect for authority, peers, and themselves.”
• “The administration was very positive regarding implementing a school wide behavior program.”
Ability to Implement…
• “The planning team members are completely dedicated and invested in finding a program that will aid in preventing problems in the future as well as assisting student achievement in academics and with life.”
• “Once a program is identified as being a viable option, the planning team will visit schools that have implemented the program.”
• “The program, once selected, will have a special kick-off to jump start and encourage enthusiasm for the program.”
RESULTS FROM OUR GRANTEES Evidence of Ready, Willing, and Able Schools
Funded Schools, “Ready” for EBP
• Funding status (funded vs. not funded) was associated with a significant effect on the readiness variables, using Wilk’s Lambda, F(12, 984) = 1.974, p = .024
Believe EBP will be beneficial
Believe EBP will make their work easier
Believe now is good timing for EBP
Funded Schools, “Willing” to Work Together
• Funding status (funded vs. not funded) was associated with a significant effect on the individual teacher collective efficacy variables, using Wilk’s Lambda, F(12, 434) = 2.797, p = .001
Students come to school ready to learn
Teachers have the skills needed to produce meaningful learning and discipline effectively
Teachers have the ability to get through to difficult students
Home life contributes to students’ learning
Funded Schools, “Able” to Succeed
• Funding status (funded vs. not funded) was associated with a significant effect on the capacity variables, using Wilk’s Lambda, F(17, 424) = 2.501, p = .001
Openness
Friendliness
Stability
Safety
Valuing relationships
Caring about families
Valuing parents ideas
Well-regarded
Proactiveness
Coordination
Clearness of mission & goals
Focus on improvement
Helping students succeed
Performance
QUESTIONS?
Colleagues
• Abe Wandersman (USC) – Matt Chinman
– Pam Imm
– Dana Keener
– Kelly Lewis
– David Fetterman
• Funders – Kay Reitz (ODMH)
– Kate Keller (Health Foundation of Greater Cincinnati)
– Janet Saul (CDC)
– Kathy Lesesne (CDC)
• MHEDIC – Mark Weist
– Seth Bernstein
– Karen Weston
– Steve Evans
• OMHNSS-EPIC – Julie Owens (OU)
– Dave Julian (OSU)
– David Hussey (KSU)
– Betsy Tracey (CWRU)
– Eric Dubow (BGSU)
• CSBMHP • Carl Paternite
• Bob Burke
• Dawn Anderson-Butcher
• Amy Wilms
• Cricket Meehan
Graduate Students
• Melissa Maras
• Chris Reiger
• Rochelle Fritz
• Angie Ledgerwood
• Jennifer Elfstrom
• Raven Cuellar
• Holli Sink
• Dana E. Crawford
• Julie Platten
• Anna Hung
• Joel Gaffney
• Liz Brown
• Jim Mosher
• Kathy Conaway
• Gillian Finocan
• Karin Vanderzee
• D’Arcy Reynolds
• Tommy Koopman