Keeping Fidelity in an Evidence-Keeping Fidelity in an Evidence-Based Program: A University-Based Program: A University-Community Agency Research Community Agency Research

PartnershipPartnership

Susan B. SternSusan B. Stern Factor-Inwentash Faculty of Social WorkFactor-Inwentash Faculty of Social Work

University of TorontoUniversity of Toronto

Katina WatsonKatina Watson YOUTHLINKYOUTHLINK

Child Development InstituteChild Development Institute

Funded by SSHRC Institutional Grant, Royal Bank FellowshipFunded by SSHRC Institutional Grant, Royal Bank Fellowship

OverviewOverview

ContextContext Evidence-Based Parenting ProgramEvidence-Based Parenting Program IY Early Learning Centres in TorontoIY Early Learning Centres in Toronto Treatment fidelity - transporting Treatment fidelity - transporting

evidence-based interventions to evidence-based interventions to community practicecommunity practice

Understand barriers/facilitatorsUnderstand barriers/facilitators Study implementation process as it Study implementation process as it

unfoldsunfolds

Why is Fidelity Why is Fidelity Important?Important?

WHY EFFECTIVE PARENTING WHY EFFECTIVE PARENTING PROGRAMS ARE IMPORTANTPROGRAMS ARE IMPORTANT

In the absence of treatment, child conduct In the absence of treatment, child conduct problems typically problems typically intensifyintensify

The majority of chronic behaviour problems The majority of chronic behaviour problems in school age children had an in school age children had an early onsetearly onset

If the difficulties continue after entry into If the difficulties continue after entry into school, the school, the riskrisk of peer rejection, later of peer rejection, later substance abuse and delinquency substance abuse and delinquency increasesincreases..

Parent-child interaction Parent-child interaction key risk factorkey risk factor for for early onset conduct problems (esp. early onset conduct problems (esp. harsh/inconsistent discipline, low harsh/inconsistent discipline, low nurturance).nurturance).

Parent training is the most empirically Parent training is the most empirically supported intervention strategy for supported intervention strategy for decreasing decreasing problem behaviours.problem behaviours.

Incredible Years ModelIncredible Years Model Emphasizes building Emphasizes building positivepositive parent-child parent-child

interactions and interactions and breaking coercive cyclesbreaking coercive cycles Increase parent Increase parent self-efficacyself-efficacy//coping, coping, and and

problem-solvingproblem-solving skills skills Increase parents’ Increase parents’ positive communication skillspositive communication skills

and reduce and reduce angeranger and the use of and the use of criticism and criticism and unnecessary commands.unnecessary commands.

Strengths-basedStrengths-based and and goal focusedgoal focused Learning through Learning through modelingmodeling and and practicepractice Action-orientedAction-oriented with home activities with home activities Creates Creates support networkssupport networks and opportunities to and opportunities to

share a range of responses and experiencesshare a range of responses and experiences Flexible and collaborativeFlexible and collaborative, tailored to each , tailored to each

child’s temperament and family situation.child’s temperament and family situation. Focus on Focus on key principleskey principles behind strategies behind strategies Led by Led by skilledskilled and and knowledgeableknowledgeable leaders leaders

Expert Model Expert Model Collaborative ModelCollaborative Model

Leader in charge, dispenses adviceLeader in charge, dispenses advice >>Parents are active and Parents are active and investedinvested

Parents passive recipients of service Parents passive recipients of service >>Highlight strength Highlight strength and changeand change

Discusses problems and deficitsDiscusses problems and deficits >> Reciprocal working Reciprocal working relationshiprelationship

Can increase guilt, self blame Can increase guilt, self blame > >Non-blaming, non-Non-blaming, non-hierarchicalhierarchical

Diagnoses problems, attempts Diagnoses problems, attempts >> Mutual problem- Mutual problem-solving,solving,

to impose a general solution to impose a general solution increases self-increases self-efficacyefficacy

>> Can increase “resistance”Can increase “resistance” > > Climate of trust and Climate of trust and openness openness

Incredible YearsIncredible Years

6 randomized control group studies -Webster-Stratton6 randomized control group studies -Webster-Stratton 5 independent replications5 independent replications SAMHSA Model Strengthening Families ProgramSAMHSA Model Strengthening Families Program OJJDP Blueprints for Violence PreventionOJJDP Blueprints for Violence Prevention UK Home Office recommended EBP for antisocial UK Home Office recommended EBP for antisocial

children children Clinic studies Clinic studies

Clinically significantClinically significant improvement in positive parenting/child behavior improvement in positive parenting/child behavior Head StartHead Start - - universal preventionuniversal prevention with with disadvantaged high riskdisadvantaged high risk

families families Significant differencesSignificant differences in parenting practices and child behavior/affect in parenting practices and child behavior/affect

MaintenanceMaintenance at one year follow ups high at one year follow ups high

BASIC Preschool and School Age, 3-10 yearsBASIC Preschool and School Age, 3-10 years

Research (cont’d)Research (cont’d) Parent Training group shown to be as Parent Training group shown to be as

effective as individual (family) trainingeffective as individual (family) training with high risk parents whose children had with high risk parents whose children had conduct problems. 70% maintained conduct problems. 70% maintained improvements at one year. improvements at one year.

Many families stated Many families stated preference for grouppreference for group over individual family treatment.over individual family treatment.

Children with high levels of baseline conduct Children with high levels of baseline conduct problems and mothers with high levels of problems and mothers with high levels of critical parenting critical parenting benefited the most.benefited the most.

Participation and engagement rates high Participation and engagement rates high as many practical barriers are reducedas many practical barriers are reduced

.

IY Key Dissemination StrategiesIY Key Dissemination Strategies Three day intensive Three day intensive trainingtraining workshop for workshop for

group leaders group leaders Clear, detailed Clear, detailed manualmanual with session by with session by

session checklists outlining key principlessession checklists outlining key principles Series of Series of videovideo taped vignettes illustrating taped vignettes illustrating

parent-child interactionsparent-child interactions Weekly Weekly home activitieshome activities and summary lists and summary lists

of key concepts for parentsof key concepts for parents Comprehensive Comprehensive bookbook Series of integrity Series of integrity checklistschecklists and leader and leader

evaluationsevaluations

Research Foci QuestionsResearch Foci Questions

Barriers to adherenceBarriers to adherence Reasons for departuresReasons for departures What facilitates adherence to IY What facilitates adherence to IY

protocolprotocol

Program ContextProgram Context

3 year project3 year project Year 1-3 dataYear 1-3 data 29 groups29 groups 18 Group leaders 18 Group leaders

MSW or Post-secondary educationMSW or Post-secondary education Range of 4-20 years experience (Range of 4-20 years experience (MM=10 =10

yrs.)yrs.)

Parent DemographicsParent DemographicsRound 3 and 7Round 3 and 7

93% mothers, 7% both parents (n=44)93% mothers, 7% both parents (n=44) Child age range, 6 mo - 15 yrs (n=43)*Child age range, 6 mo - 15 yrs (n=43)*

48% <=3yrs. 48% <=3yrs. 39% 4-7 yrs39% 4-7 yrs 7% 8-11 yrs. 7% 8-11 yrs.

Ethnicity (n=41) *Ethnicity (n=41) * 41% White41% White 18% Black18% Black 18% South Asian18% South Asian 4.5 % Aboriginal, 4.5% Filipino, 2.3% Latin 4.5 % Aboriginal, 4.5% Filipino, 2.3% Latin

American, 4.5% OtherAmerican, 4.5% Other 59% English as first language (n=43)59% English as first language (n=43)

Parent Demographics Parent Demographics (cont.)(cont.)

Income (n=41)*Income (n=41)* 34.1% 34.1% < $15,000< $15,000 18.2% 18.2% $15,000 – $24,999$15,000 – $24,999 40.9% 40.9% >= $25,000 >= $25,000

50 % on social assistance (n=43)50 % on social assistance (n=43)

71% high school completed/GED or 71% high school completed/GED or higher (n=44)higher (n=44)

IY Fidelity Measures IY Fidelity Measures

Leader Session x Session Checklist Leader Session x Session Checklist Group Leaders Written NarrativesGroup Leaders Written Narratives Leader Rating Scale (Program Leader Rating Scale (Program

Elements & Collaborative Process) Elements & Collaborative Process) Independent Supervisor Written Independent Supervisor Written

ObservationsObservations Audiotapes of group leaders joint Audiotapes of group leaders joint

meetingmeeting

Group Leaders Focus Group Leaders Focus GroupGroup

To understand program To understand program implementation and adherence implementation and adherence issues over the course of multiple issues over the course of multiple IYPP groupsIYPP groups

Semi-structured by interview guide Semi-structured by interview guide Two focus group facilitatorsTwo focus group facilitators Trained by one of the lead Trained by one of the lead

researchersresearchers

Focus Group QuestionsFocus Group Questions What barriers did you as group leaders face or What barriers did you as group leaders face or

perceive the parents to experience in implementing perceive the parents to experience in implementing the IYPP.the IYPP.

What facilitated implementing the program and/or What facilitated implementing the program and/or overcoming barriers? overcoming barriers?

Given your perceptions of the parents, what else Given your perceptions of the parents, what else would have helped in the implementation of the would have helped in the implementation of the program? program?

What do you think were the most effective What do you think were the most effective intervention components and processes? intervention components and processes?

If IY was to be implemented here or in another If IY was to be implemented here or in another organization, what modifications do you think the organization, what modifications do you think the program needs, or parents have suggested, to program needs, or parents have suggested, to improve the fit with a diverse and universal improve the fit with a diverse and universal prevention population?prevention population?

What organizational factors or other supports do you What organizational factors or other supports do you think would be necessary for program sustainability?think would be necessary for program sustainability?

Trustworthiness of the DataTrustworthiness of the Data

Dependability of the data through audio-taping and Dependability of the data through audio-taping and transcriptions transcriptions

Open and axial coding for thematic analysisOpen and axial coding for thematic analysis Credibility through persistent observation with Credibility through persistent observation with

participants (e.g. group meetings, observations of participants (e.g. group meetings, observations of sessions)sessions)

Prolonged engagementProlonged engagement Negative case analysisNegative case analysis Multiple independent codersMultiple independent coders Member checkingMember checking Confirmability through quotesConfirmability through quotes

IY Protocol Adherence All IY Protocol Adherence All RoundsRounds

Session x Session ChecklistSession x Session Checklist Overall adherence “DID I” Grand mean Overall adherence “DID I” Grand mean 83.88 83.88

%% Role play Grand mean Role play Grand mean 59.94 % 59.94 % Buddy calls Grand mean Buddy calls Grand mean 52.77 52.77

%% Vignette Grand mean Vignette Grand mean 63.51% 63.51%

Group Leader Rating Scale Group Leader Rating Scale Overall adherence across sites 4.48Overall adherence across sites 4.48 Leader group process skills 4.69Leader group process skills 4.69 Showing vignettes 4.45Showing vignettes 4.45 Practice and role plays 3.8Practice and role plays 3.8

44thth session observation 3.83 session observation 3.83 88thth session observation 3.71 session observation 3.71

IY Protocol Adherence IY Protocol Adherence Cont’dCont’d

Vignettes:Vignettes:Videotape modeling with core vignettes occurred in Videotape modeling with core vignettes occurred in at least 6 (55%) and often all 11 (100%) sessions at least 6 (55%) and often all 11 (100%) sessions that called for vignettes (M= 9.7 or 88%).that called for vignettes (M= 9.7 or 88%).

Role Plays:Role Plays:Role Plays occurred in as few as two Role Plays occurred in as few as two (20%) but sometimes all ten (100%) (20%) but sometimes all ten (100%) sessions that called for at least one sessions that called for at least one role play (M = 6.34 or 63%)role play (M = 6.34 or 63%)

Negative Cases:Negative Cases:Program AdherenceProgram Adherence

Buddy calls Buddy calls VignettesVignettes Role playsRole plays

Balancing Parent Balancing Parent Preferences with Preferences with

Structured Program Structured Program ComponentsComponents

Collaborative vs. Expert roleCollaborative vs. Expert role Parent discussion preference Parent discussion preference Desire to keep parental involvement highDesire to keep parental involvement high Discomfort with new, unfamiliar Discomfort with new, unfamiliar

(“strange”) activities & skills (parents & (“strange”) activities & skills (parents & leaders)leaders)

Cultural preference for learning style e.g., Cultural preference for learning style e.g., lecture, “education”lecture, “education”

Balancing parent Balancing parent preferences with structured preferences with structured

program componentsprogram components““It is very hard to cover the agenda because of It is very hard to cover the agenda because of the large group and there is a lot of the large group and there is a lot of discussion.”discussion.”

““Often that gets into a pretty large story.”Often that gets into a pretty large story.”

““Evaluations show that parents enjoyed group Evaluations show that parents enjoyed group discussions and did not find role-play as discussions and did not find role-play as helpful.”helpful.”

““Less vignettes were shown to make time for Less vignettes were shown to make time for more discussion.”more discussion.”

Balancing Parent Needs Balancing Parent Needs within the Groupswithin the Groups

Various educational levelsVarious educational levels Mixed socioeconomic levelsMixed socioeconomic levels Various learning styles (lecture, Various learning styles (lecture,

culture)culture) Range of child agesRange of child ages Different perceived parent needs re: Different perceived parent needs re:

relevance of IY strategies (e.g., -need relevance of IY strategies (e.g., -need for play, time-out) for play, time-out)

Parent NeedsParent Needs

““There were people with different levels of There were people with different levels of education and sometimes with some people it education and sometimes with some people it was pretty basic and others. . .so it was was pretty basic and others. . .so it was sometimes difficult to maintain a level where sometimes difficult to maintain a level where they don’t get bored because you are they don’t get bored because you are repeating.”repeating.”

““Having too much of a wide spectrum of Having too much of a wide spectrum of parents with children from literally 6 months parents with children from literally 6 months old to twelve years old in a group that was old to twelve years old in a group that was specifically designed for a specific age group specifically designed for a specific age group was a bit challenging for me to do.”was a bit challenging for me to do.”

Time ConstraintsTime Constraints

Leader workload/preparation timeLeader workload/preparation time Amount of content Amount of content Balance content/structured activity Balance content/structured activity

time with parent talk time time with parent talk time Balancing individual parent needs with Balancing individual parent needs with

collective needscollective needs Time for reviewing & setting up Time for reviewing & setting up

homework homework Parent tardiness (time constraints in Parent tardiness (time constraints in

their own lives to get to group)their own lives to get to group)

Language IssuesLanguage Issues

Language (ESL)Language (ESL) Time to express self in 2Time to express self in 2ndnd language language Translation timeTranslation time Difficulties in understanding vignettes, Difficulties in understanding vignettes,

written materialswritten materials LingoLingo Homework difficulties – reading/writingHomework difficulties – reading/writing

ESL issues took precedence over ESL issues took precedence over cultural differences. cultural differences.

Language (ESL)Language (ESL)

““Sometimes we have to be patient, as people are Sometimes we have to be patient, as people are struggling with language and having trouble struggling with language and having trouble explaining themselves.” (relates to issue of not explaining themselves.” (relates to issue of not enough time)enough time)

““We were worried that we have so much content to We were worried that we have so much content to begin with this is going to take forever with two begin with this is going to take forever with two interpreters” interpreters”

““Some parents had difficulties understanding the Some parents had difficulties understanding the video-tapes because of language.” video-tapes because of language.”

““Parents who had difficulty with English could not Parents who had difficulty with English could not do the written assignments.”do the written assignments.”

CultureCulture

““Spanking discussion was excellent Spanking discussion was excellent because it became a discussion about because it became a discussion about cultural values, how people were raised.”cultural values, how people were raised.”

““In many cultures its different, and I In many cultures its different, and I think the tendency is to get to the role of think the tendency is to get to the role of education”education”

““I find people from traditional cultures I find people from traditional cultures learn and push us to be more learn and push us to be more educational”educational”

RelevanceRelevance Buddy callsBuddy calls VignettesVignettes

OutdatedOutdated Most focus on one child without siblingsMost focus on one child without siblings Few examples of single parents, father Few examples of single parents, father

only in some topicsonly in some topics Many parents had younger children-Many parents had younger children-

need to address attachment, need to address attachment, developmental issuesdevelopmental issues

Did not cover some common concerns, Did not cover some common concerns, e.g., bullying, sibling conflicte.g., bullying, sibling conflict

RelevanceRelevance

““The vignettes do not match the teaching points of The vignettes do not match the teaching points of the topics. For example it [time out] should be the topics. For example it [time out] should be used as a last resort yet vignettes show minor used as a last resort yet vignettes show minor things to be timed out (e.g. not hanging up coat).”things to be timed out (e.g. not hanging up coat).”

““Vignettes were an issue this week as in some Vignettes were an issue this week as in some sections there were not mother-child examples sections there were not mother-child examples and some mothers had more trouble identifying and some mothers had more trouble identifying with the father-child vignettes . . . Other sections with the father-child vignettes . . . Other sections were difficult because they show two-parent were difficult because they show two-parent families.”families.”

““Some seem redundant, same topic. Although Some seem redundant, same topic. Although parents comment they enjoy them but they are parents comment they enjoy them but they are dated and some seem staged.”dated and some seem staged.”

Role Play ReluctanceRole Play Reluctance

AnxietyAnxiety Self-consciousnessSelf-consciousness Parents did not think of or Parents did not think of or

experience them as usefulexperience them as useful ChildishChildish Prefer discussionPrefer discussion Lack of timeLack of time Leaders’ own discomfortLeaders’ own discomfort

Role Play Role Play Discomfort/ReluctanceDiscomfort/Reluctance

“ “ . . . We weren’t so great at role plays and . . . We weren’t so great at role plays and neither were the parents . . . They neither were the parents . . . They preferred to have discussion.”preferred to have discussion.”

““Because of discussion time didn’t have Because of discussion time didn’t have time for role plays.”time for role plays.”

““We have more trouble fitting in the role We have more trouble fitting in the role plays because they are time consuming.”plays because they are time consuming.”

Concerns with Time outConcerns with Time out Confusion between home use and child Confusion between home use and child

care policycare policy Does not workDoes not work Values and cultures Values and cultures Safety issuesSafety issues Inappropriate, not relevant for their Inappropriate, not relevant for their

familyfamily Child temperamentChild temperament Parents own styleParents own style Concerns about abandonment, “letting Concerns about abandonment, “letting

kids scream”kids scream”

Structure of IY materials, Structure of IY materials, manualmanual

Hard to follow at times – not user Hard to follow at times – not user friendlyfriendly

MismatchMismatch Manual instructions to use Time out for Manual instructions to use Time out for

serious behaviors/ vignettes show minor serious behaviors/ vignettes show minor behavior (e.g., not hanging up coat)behavior (e.g., not hanging up coat)

One session had 2 homework assignmentsOne session had 2 homework assignments Mixing 2 topics in one sessionMixing 2 topics in one session

Disconnect Disconnect

Between IY procedures for selected/ indicated Between IY procedures for selected/ indicated population population and universal preventive oneand universal preventive one

Parent reactions to specific IYPP strategies (e.g., Parent reactions to specific IYPP strategies (e.g., ddepth of teaching time-out, buddy calls)epth of teaching time-out, buddy calls)

Group leader discomfort/boundaries teaching vs. Group leader discomfort/boundaries teaching vs. clinician role (e.g., follow up calls) clinician role (e.g., follow up calls)

““I think because the program was developed for I think because the program was developed for high risk situations and they did put in all of high risk situations and they did put in all of these things to help and maybe it isn’t these things to help and maybe it isn’t appropriate for a parenting group”appropriate for a parenting group”

Organizational BarriersOrganizational Barriers

CommunicationCommunication Commitment to the programCommitment to the program Outreach to parentsOutreach to parents Clear expectations between child Clear expectations between child

care centres and group leaderscare centres and group leaders

Organizational BarriersOrganizational Barriers““It’s a three-way relationship, you have us, the It’s a three-way relationship, you have us, the

parents, and the childcare centre and all parts parents, and the childcare centre and all parts of it have to be strong in order for it to work.”of it have to be strong in order for it to work.”

““I think that if there are expectations and I think that if there are expectations and boundaries about what the childcare centre is boundaries about what the childcare centre is supposed to do and what we are supposed to supposed to do and what we are supposed to do, we need that, for some groups that wasn’t do, we need that, for some groups that wasn’t very clear.” very clear.”

““It’s challenging and goes back to the buy-in It’s challenging and goes back to the buy-in from the centre and I think we have to screen from the centre and I think we have to screen very carefully who we’re going to be doing very carefully who we’re going to be doing [the program] with.”[the program] with.”

Barriers to Protocol Barriers to Protocol AdherenceAdherence

Balancing Parent Preferences with Structured Balancing Parent Preferences with Structured Program Components Program Components

Time Constraints Time Constraints Language (ESL)Language (ESL) RelevanceRelevance Role Play ReluctanceRole Play Reluctance Confusion/Concerns with Time Out Confusion/Concerns with Time Out Structure of the Materials and Manual Structure of the Materials and Manual Disconnect between High Risk/Clinical and Universal Disconnect between High Risk/Clinical and Universal

Preventive IY GroupsPreventive IY Groups Lack of Control Lack of Control Cultural Cultural Balancing Parent NeedsBalancing Parent Needs OrganizationalOrganizational

Facilitators/Strategies to Facilitators/Strategies to Overcome BarriersOvercome Barriers

Highlighting and reinforcing fidelityHighlighting and reinforcing fidelity ValidatingValidating Provides rationales [with group leaders & Provides rationales [with group leaders &

parents] parents] CollaborationCollaboration Problem solving barriersProblem solving barriers Focusing on goalFocusing on goal

Highlighting and Highlighting and Reinforcing FidelityReinforcing Fidelity

“It is (role plays) for many intimidating. I remember in training lots of us hesitated. But we also found out we needed them because some skills needed practicing and it was hard to remember everything.”

“Reiterating the importance of vignettes in Reiterating the importance of vignettes in general - (research) shows they are general - (research) shows they are helpful even if parents don’t like them.” helpful even if parents don’t like them.”

Facilitators of AdherenceFacilitators of Adherence Supervisor responsiveness, facilitator of Supervisor responsiveness, facilitator of

organizational support, concrete resourcesorganizational support, concrete resources 11stst session preparation session preparation Elicits ongoing feedbackElicits ongoing feedback Follow throughFollow through Mediator between IY group leader and the CC Mediator between IY group leader and the CC

centrecentre

““Definitely [supervisor] makes it easier for us”Definitely [supervisor] makes it easier for us”

““What are your thoughts and ideas about what would be What are your thoughts and ideas about what would be helpful right now?”helpful right now?”

““Is there anything more I could be doing?”Is there anything more I could be doing?”

IY structure, materials (manual, vignettes, IY structure, materials (manual, vignettes, book)book)

Monitoring Fidelity and Monitoring Fidelity and Problem Solving BarriersProblem Solving Barriers

Accountability for monitoring/commitmentAccountability for monitoring/commitment Understanding reasons for departuresUnderstanding reasons for departures Ensuring program’s responsiveness to Ensuring program’s responsiveness to

parent needs parent needs Tenor of problem solving collaborativelyTenor of problem solving collaboratively

““Weekly evaluations [parent] and follow up calls give Weekly evaluations [parent] and follow up calls give us a way to make sure we are being responsive to us a way to make sure we are being responsive to parents, and our own leader checklists will give us parents, and our own leader checklists will give us feedback about what we are and aren’t doing.” feedback about what we are and aren’t doing.”

““Please note which [vignettes] you used on leader Please note which [vignettes] you used on leader checklist and why if skipped or added vignettes.” checklist and why if skipped or added vignettes.”

““Let’s make notes of barriers to doing home activities Let’s make notes of barriers to doing home activities which are core to the group.” which are core to the group.”

Monitoring Fidelity/PS Monitoring Fidelity/PS BarriersBarriers

““When I observe groups, it will be in the same spirit, to When I observe groups, it will be in the same spirit, to learn and understand the needs of each group and how learn and understand the needs of each group and how and why the group leaders have made any modifications and why the group leaders have made any modifications to meet the needs of their particular parents” to meet the needs of their particular parents”

““Our motto always was, that any departure was OK, they Our motto always was, that any departure was OK, they could trust their judgment or suggest something, as could trust their judgment or suggest something, as long as they had a rationale or a clinical reason for it.”long as they had a rationale or a clinical reason for it.”

Attention to adherence & procedures embedded Attention to adherence & procedures embedded in agency, feasibility for adherence monitoring in agency, feasibility for adherence monitoring in social work community settingsin social work community settings

Group Leader Fit & Belief in Group Leader Fit & Belief in IYPP Core Components & IYPP Core Components &

Collaborative ProcessCollaborative Process

““You might find that people self select, cause You might find that people self select, cause I agree with you, when I came to it, I really I agree with you, when I came to it, I really liked the material, and agreed with most of liked the material, and agreed with most of it – with the foundation, so I choose to it – with the foundation, so I choose to continue doing it, where others might not continue doing it, where others might not like it , find it to rigid, so you find people like it , find it to rigid, so you find people that will adhere to it because they like it.”that will adhere to it because they like it.”

“ “I think you do things that you believe in, andI think you do things that you believe in, andmost of us are philosophically similar.”most of us are philosophically similar.”

Key Contributors to Key Contributors to AdherenceAdherence

Overall, adherence was good with the exception of Overall, adherence was good with the exception of role plays, # of vignettes shown, & buddy calls role plays, # of vignettes shown, & buddy calls

Therapists’ belief in the importance of the program’s Therapists’ belief in the importance of the program’s core components and collaborative process core components and collaborative process

Emphasis on accountability and ongoing session Emphasis on accountability and ongoing session monitoring monitoring

Therapists’ sense of ownership and responsibility for Therapists’ sense of ownership and responsibility for quality controlquality control

Supervisor understanding of IY and underlying Supervisor understanding of IY and underlying research, commitment to high quality implementationresearch, commitment to high quality implementation

Supervisor commitment to supporting group leaders Supervisor commitment to supporting group leaders and follow-throughand follow-through

Training/structure of program and materialsTraining/structure of program and materials Concrete resourcesConcrete resources

LimitationsLimitations

Fidelity data either self report or supervisor Fidelity data either self report or supervisor observation – possibility of systematic biasobservation – possibility of systematic bias

Supervisor’s dual role – potential conflict of Supervisor’s dual role – potential conflict of interestinterest

Validation of quantitative adherence measuresValidation of quantitative adherence measures

Triangulation of findings with clear and Triangulation of findings with clear and consistent patterns across different methods & consistent patterns across different methods & data sources, and stability over time of seven data sources, and stability over time of seven IYPP cycles, increases confidence in the results.IYPP cycles, increases confidence in the results.

Next Steps Next Steps

Upcoming ArticleUpcoming Article

Stern, S.B., Alaggia, R., Watson, K. & Stern, S.B., Alaggia, R., Watson, K. & Morton, T.R. (in press). Implementing an Morton, T.R. (in press). Implementing an Evidence-Based Parenting Program With Evidence-Based Parenting Program With Adherence in the Real World of Adherence in the Real World of Community Practice. Community Practice. Research on Social Research on Social Work Practice.Work Practice.