Keeping Fidelity in an Evidence-Based Program: A University-Community Agency Research Partnership Susan B. Stern Factor-Inwentash Faculty of Social Work.

  • Published on
    26-Mar-2015

  • View
    212

  • Download
    0

Embed Size (px)

Transcript

<ul><li>Slide 1</li></ul><p>Keeping Fidelity in an Evidence-Based Program: A University-Community Agency Research Partnership Susan B. Stern Factor-Inwentash Faculty of Social Work Factor-Inwentash Faculty of Social Work University of Toronto Katina Watson Katina Watson YOUTHLINK YOUTHLINK Child Development Institute Child Development Institute Funded by SSHRC Institutional Grant, Royal Bank Fellowship Slide 2 Overview Context Context Evidence-Based Parenting Program Evidence-Based Parenting Program IY Early Learning Centres in Toronto IY Early Learning Centres in Toronto Treatment fidelity - transporting evidence-based interventions to community practice Treatment fidelity - transporting evidence-based interventions to community practice Understand barriers/facilitators Understand barriers/facilitators Study implementation process as it unfolds Study implementation process as it unfolds Slide 3 Why is Fidelity Important? Slide 4 WHY EFFECTIVE PARENTING PROGRAMS ARE IMPORTANT In the absence of treatment, child conduct problems typically intensify In the absence of treatment, child conduct problems typically intensify The majority of chronic behaviour problems in school age children had an early onset The majority of chronic behaviour problems in school age children had an early onset If the difficulties continue after entry into school, the risk of peer rejection, later substance abuse and delinquency increases. If the difficulties continue after entry into school, the risk of peer rejection, later substance abuse and delinquency increases. Parent-child interaction key risk factor for early onset conduct problems (esp. harsh/inconsistent discipline, low nurturance). Parent-child interaction key risk factor for early onset conduct problems (esp. harsh/inconsistent discipline, low nurturance). Parent training is the most empirically supported intervention strategy for decreasing problem behaviours. Parent training is the most empirically supported intervention strategy for decreasing problem behaviours. Slide 5 Incredible Years Model Emphasizes building positive parent-child interactions and breaking coercive cycles Emphasizes building positive parent-child interactions and breaking coercive cycles Increase parent self-efficacy/coping, and problem-solving skills Increase parent self-efficacy/coping, and problem-solving skills Increase parents positive communication skills and reduce anger and the use of criticism and unnecessary commands. Increase parents positive communication skills and reduce anger and the use of criticism and unnecessary commands. Strengths-based and goal focused Strengths-based and goal focused Learning through modeling and practice Learning through modeling and practice Action-oriented with home activities Action-oriented with home activities Creates support networks and opportunities to share a range of responses and experiences Creates support networks and opportunities to share a range of responses and experiences Flexible and collaborative, tailored to each childs temperament and family situation. Flexible and collaborative, tailored to each childs temperament and family situation. Focus on key principles behind strategies Focus on key principles behind strategies Led by skilled and knowledgeable leaders Led by skilled and knowledgeable leaders Slide 6 Expert Model Collaborative Model Leader in charge, dispenses advice &gt;Parents are active and invested Leader in charge, dispenses advice &gt;Parents are active and invested Parents passive recipients of service &gt;Highlight strength and change Parents passive recipients of service &gt;Highlight strength and change Discusses problems and deficits &gt; Reciprocal working relationship Discusses problems and deficits &gt; Reciprocal working relationship Can increase guilt, self blame &gt;Non-blaming, non-hierarchical Can increase guilt, self blame &gt;Non-blaming, non-hierarchical Diagnoses problems, attempts &gt; Mutual problem-solving, Diagnoses problems, attempts &gt; Mutual problem-solving, to impose a general solution increases self-efficacy to impose a general solution increases self-efficacy &gt;Can increase resistance &gt; Climate of trust and openness Slide 7 Incredible Years 6 randomized control group studies -Webster-Stratton 6 randomized control group studies -Webster-Stratton 5 independent replications 5 independent replications SAMHSA Model Strengthening Families Program SAMHSA Model Strengthening Families Program OJJDP Blueprints for Violence Prevention OJJDP Blueprints for Violence Prevention UK Home Office recommended EBP for antisocial children UK Home Office recommended EBP for antisocial children Clinic studies Clinic studies Clinically significant improvement in positive parenting/child behavior Clinically significant improvement in positive parenting/child behavior Head Start - universal prevention with disadvantaged high risk families Head Start - universal prevention with disadvantaged high risk families Significant differences in parenting practices and child behavior/affect Significant differences in parenting practices and child behavior/affect Maintenance at one year follow ups high Maintenance at one year follow ups high BASIC Preschool and School Age, 3-10 years Slide 8 Research (contd) Parent Training group shown to be as effective as individual (family) training with high risk parents whose children had conduct problems. 70% maintained improvements at one year. Parent Training group shown to be as effective as individual (family) training with high risk parents whose children had conduct problems. 70% maintained improvements at one year. Many families stated preference for group over individual family treatment. Many families stated preference for group over individual family treatment. Children with high levels of baseline conduct problems and mothers with high levels of critical parenting benefited the most. Children with high levels of baseline conduct problems and mothers with high levels of critical parenting benefited the most. Participation and engagement rates high as many practical barriers are reduced Participation and engagement rates high as many practical barriers are reduced. Slide 9 IY Key Dissemination Strategies Three day intensive training workshop for group leaders Three day intensive training workshop for group leaders Clear, detailed manual with session by session checklists outlining key principles Clear, detailed manual with session by session checklists outlining key principles Series of video taped vignettes illustrating parent-child interactions Series of video taped vignettes illustrating parent-child interactions Weekly home activities and summary lists of key concepts for parents Weekly home activities and summary lists of key concepts for parents Comprehensive book Comprehensive book Series of integrity checklists and leader evaluations Series of integrity checklists and leader evaluations Slide 10 Research Foci Questions Barriers to adherence Barriers to adherence Reasons for departures Reasons for departures What facilitates adherence to IY protocol What facilitates adherence to IY protocol Slide 11 Program Context 3 year project 3 year project Year 1-3 data Year 1-3 data 29 groups 29 groups 18 Group leaders 18 Group leaders MSW or Post-secondary education MSW or Post-secondary education Range of 4-20 years experience (M=10 yrs.) Range of 4-20 years experience (M=10 yrs.) Slide 12 Parent Demographics Round 3 and 7 93% mothers, 7% both parents (n=44) 93% mothers, 7% both parents (n=44) Child age range, 6 mo - 15 yrs (n=43)* Child age range, 6 mo - 15 yrs (n=43)* 48% Parent Demographics (cont.) Income (n=41)* Income (n=41)* 34.1% &lt; $15,000 34.1% &lt; $15,000 18.2% $15,000 $24,999 18.2% $15,000 $24,999 40.9% &gt;= $25,000 40.9% &gt;= $25,000 50 % on social assistance (n=43) 50 % on social assistance (n=43) 71% high school completed/GED or higher (n=44) 71% high school completed/GED or higher (n=44) Slide 14 IY Fidelity Measures Leader Session x Session Checklist Leader Session x Session Checklist Group Leaders Written Narratives Group Leaders Written Narratives Leader Rating Scale (Program Elements &amp; Collaborative Process) Leader Rating Scale (Program Elements &amp; Collaborative Process) Independent Supervisor Written Observations Independent Supervisor Written Observations Audiotapes of group leaders joint meeting Audiotapes of group leaders joint meeting Slide 15 Group Leaders Focus Group To understand program implementation and adherence issues over the course of multiple IYPP groups To understand program implementation and adherence issues over the course of multiple IYPP groups Semi-structured by interview guide Semi-structured by interview guide Two focus group facilitators Two focus group facilitators Trained by one of the lead researchers Trained by one of the lead researchers Slide 16 Focus Group Questions What barriers did you as group leaders face or perceive the parents to experience in implementing the IYPP. What barriers did you as group leaders face or perceive the parents to experience in implementing the IYPP. What facilitated implementing the program and/or overcoming barriers? What facilitated implementing the program and/or overcoming barriers? Given your perceptions of the parents, what else would have helped in the implementation of the program? Given your perceptions of the parents, what else would have helped in the implementation of the program? What do you think were the most effective intervention components and processes? What do you think were the most effective intervention components and processes? If IY was to be implemented here or in another organization, what modifications do you think the program needs, or parents have suggested, to improve the fit with a diverse and universal prevention population? If IY was to be implemented here or in another organization, what modifications do you think the program needs, or parents have suggested, to improve the fit with a diverse and universal prevention population? What organizational factors or other supports do you think would be necessary for program sustainability? What organizational factors or other supports do you think would be necessary for program sustainability? Slide 17 Trustworthiness of the Data Dependability of the data through audio-taping and transcriptions Dependability of the data through audio-taping and transcriptions Open and axial coding for thematic analysis Open and axial coding for thematic analysis Credibility through persistent observation with participants (e.g. group meetings, observations of sessions) Credibility through persistent observation with participants (e.g. group meetings, observations of sessions) Prolonged engagement Prolonged engagement Negative case analysis Negative case analysis Multiple independent coders Multiple independent coders Member checking Member checking Confirmability through quotes Confirmability through quotes Slide 18 IY Protocol Adherence All Rounds Session x Session Checklist Session x Session Checklist Overall adherence DID I Grand mean 83.88 % Overall adherence DID I Grand mean 83.88 % Role play Grand mean 59.94 % Role play Grand mean 59.94 % Buddy calls Grand mean 52.77 % Buddy calls Grand mean 52.77 % Vignette Grand mean 63.51% Vignette Grand mean 63.51% Group Leader Rating Scale Group Leader Rating Scale Overall adherence across sites 4.48 Overall adherence across sites 4.48 Leader group process skills 4.69 Leader group process skills 4.69 Showing vignettes 4.45 Showing vignettes 4.45 Practice and role plays 3.8 Practice and role plays 3.8 4 th session observation 3.83 4 th session observation 3.83 8 th session observation 3.71 8 th session observation 3.71 Slide 19 IY Protocol Adherence Contd Vignettes: Vignettes: Videotape modeling with core vignettes occurred in at least 6 (55%) and often all 11 (100%) sessions that called for vignettes (M= 9.7 or 88%). Role Plays: Role Plays: Role Plays occurred in as few as two (20%) but sometimes all ten (100%) sessions that called for at least one role play (M = 6.34 or 63%) Slide 20 Negative Cases: Program Adherence Buddy calls Buddy calls Vignettes Vignettes Role plays Role plays Slide 21 Balancing Parent Preferences with Structured Program Components Collaborative vs. Expert role Collaborative vs. Expert role Parent discussion preference Parent discussion preference Desire to keep parental involvement high Desire to keep parental involvement high Discomfort with new, unfamiliar (strange) activities &amp; skills (parents &amp; leaders) Discomfort with new, unfamiliar (strange) activities &amp; skills (parents &amp; leaders) Cultural preference for learning style e.g., lecture, education Cultural preference for learning style e.g., lecture, education Slide 22 Balancing parent preferences with structured program components It is very hard to cover the agenda because of the large group and there is a lot of discussion. Often that gets into a pretty large story. Evaluations show that parents enjoyed group discussions and did not find role-play as helpful. Less vignettes were shown to make time for more discussion. Slide 23 Balancing Parent Needs within the Groups Various educational levels Various educational levels Mixed socioeconomic levels Mixed socioeconomic levels Various learning styles (lecture, culture) Various learning styles (lecture, culture) Range of child ages Range of child ages Different perceived parent needs re: relevance of IY strategies (e.g., -need for play, time-out) Different perceived parent needs re: relevance of IY strategies (e.g., -need for play, time-out) Slide 24 Parent Needs There were people with different levels of education and sometimes with some people it was pretty basic and others...so it was sometimes difficult to maintain a level where they dont get bored because you are repeating. There were people with different levels of education and sometimes with some people it was pretty basic and others...so it was sometimes difficult to maintain a level where they dont get bored because you are repeating. Having too much of a wide spectrum of parents with children from literally 6 months old to twelve years old in a group that was specifically designed for a specific age group was a bit challenging for me to do. Having too much of a wide spectrum of parents with children from literally 6 months old to twelve years old in a group that was specifically designed for a specific age group was a bit challenging for me to do. Slide 25 Time Constraints Leader workload/preparation time Leader workload/preparation time Amount of content Amount of content Balance content/structured activity time with parent talk time Balance content/structured activity time with parent talk time Balancing individual parent needs with collective needs Balancing individual parent needs with collective needs Time for reviewing &amp; setting up homework Time for reviewing &amp; setting up homework Parent tardiness (time constraints in their own lives to get to gro...</p>

Recommended

View more >