Upload
alec-workman
View
19
Download
0
Embed Size (px)
DESCRIPTION
Peering into the black box between pre and post: A theory of change related to interactive/ experiential delivery methods. Susan Carter Evaluation Coordinator National Indian Youth Leadership Project [email protected] with J. Fred Springer, EMT Associates [email protected]. - PowerPoint PPT Presentation
Citation preview
Peering into the black box between pre and post: A theory of change related to interactive/ experiential delivery methods
Susan CarterEvaluation Coordinator
National Indian Youth Leadership [email protected]
with J. Fred Springer, EMT Associates
A presentation for the 6th Annual Research and Evaluation in Adventure Programming Symposium
Gaithersburg, MDMarch 17-19, 2010
The Multi-Site High Risk Youth Study funded by SAMHSAFred Springer, EMT, Co-PI
48 site quasi-experimental design10,500 youth4 point longitudinal studyCommon instrumentComprehensive process dataHierarchical statistical designAmerican Evaluation Association Study of the
Year award, 2000Largest direct service ATOD prevention
evaluation ever implemented
4
Common Factors Reducing 30-day Drug Use
Behavioral Life-Skills FocusedConnection Building
Program CoherenceIntrospective Learning
Intense Contact
0
0.045
0.09
0.135
0.18
Effe
ct S
ize
.18
.135
.09
.045
Life-skillsFocused
ConnectionBuilding
Coherent IntrospectiveOrientation
IntenseContact
HRY Common Factor Findings
Promoting introspection / connectedness are critical common factors
Out of school more effective Manualized programs less effective Intensity countsPromoting behavioral skills more effective
than focus on knowledge and attitude change
Application: Project Venture
Project Venture identified as one of eight best programs in HRY study◦PV included all of the common factors
Incorporates connectedness building and introspection
Uses interactive experiential program delivery strategy
Essential Elements of Project Venture
Population◦Middle school aged American Indian youth◦High school aged AI service staff◦Mix of risk levels (at risk + high risk)
Program Components◦Experiential education◦Connecting to the natural world◦Physical and social-emotional challenge◦Service learning◦Focus on developing positive relationships◦Positive Youth Development approach
Project Venture: Overview
Experiential, adventure-based positive youth development program for American Indian youth
Recognized by NREPP as evidence-basedThe only American Indian ATOD prevention
program currently in NREPPWidely disseminated throughout the US and
CanadaTwenty year implementation and evaluation
history
Project Venture: Strategies
Outdoor, adventure activities:ropes course, rappelling, canoeing, backpacking, camping, mountain biking, etc.
Service LearningMeaningful projects that are youth centered and designed, address cultural, environmental and other community needs.
In a research-supported, year long sequential program.
Project Venture: Evaluation Findings
Positive substance abuse prevention findings (prevent, reduce, delay onset) especially related to alcohol abuse
Increased internal assetsIncreased external assets, especially
community and peer domains
Purpose of proposed study
We propose to advance knowledge concerning how adventure based strategies and activities can be strengthened, promoted, and replicated.
Hypothesis
Interactive /experiential activities that clearly incorporate reflective learning skills and promote connectedness to positive people and environments will promote intended behavioral change more effectively than interactive/experiential activities that have less focused emphasis on these interim outcomes.
Method notes
Quasi-experimentBaseline, exit, six and twelve month follow
upAssess similar participants in Project
Venture programs:◦6 programs using an enhanced version of PV◦6 “regular” PV programs◦(perhaps 6 groups with no PV)
Measures
Behavioral outcomes (ATOD, etc.)Connectedness (e.g., Michael Karcher’s
Hemmingway scales)Reflective learning (perhaps adapted from
education literature)
Analysis notes
Rigorous development and testing program to develop the measures
Rigorous statistical methodology including propensity scores for improving comparison equivalence
Hierarchical modeling to account for nesting effects, treatment interactions, treatment settings
Strong fidelity/coherence measurements
High Risk Youth Study, SAMHSA mapped onto Project Venture
Characteristics of Most Successful Prevention Programs
How Project Venture Incorporates these Characteristics through Interactive Experiential Delivery Strategies
Engagement Cultural content, leadership, values / experiential learning / connecting to the natural world / physical and emotional challenge
Intensity 150 + hours year (3 + hours per week)68 sessions / In & after school, weekends, summer
Reflective Orientation Experiential learning cycle, debriefing; Outdoor adventure: problem solving; Service learning: student led, problem solving; Camp, treks: journaling, values, norms, natural world, spiritual awareness; Native cultural values: spiritual traditions; In school: Problem solving, skill building, socialization
Connectedness Building Outdoor adventure: team building, trust building; Service learning: youth led, meaningful, relationship emphasis; service ethic; Camp, treks: build caring, cooperative community; Native cultural values; family events, cultural traditions
Life-skills Focused Outdoor adventure: cooperation, leadership, efficacy ; Service learning: problem solving, leadership, cooperation; Camp, treks: leadership, efficacy; In school: problem solving, skill building
Positive Environment Cultural content; adult participation, interaction, trust, support;
Coherence Clear purpose and process; training; collaboration; implementation project
Community Support Guided by Native Elders; Native cultural values; positive youth development approach; collaboration.
Other study ideas
Secondary data analysis of 5 years of pre/post survey data from Project Venture 7th grade participants (600) and matched comparison group (400) collected between 2005-2010
ANDLongitudinal follow-up study of PV program
participants who were in 7th grade in 2006 – including original program survey (adapted for older youth cohort) AND life history interviews