PowerPoint Presentation

Kansas Intensive Permanency ProjectScaling Up an Evidence-Based Intervention: An Implementation DiscussionOctober 29, 20131An Initiative of the Childrens Bureau1Todays Presentation2Brief backgroundImplementation FrameworksKIPPs Use of Implementation FrameworksSupports and Challenges of ImplementationDiscussion

An Initiative of the Childrens Bureau2Brief Background3

Kansas Context4PII Project: Kansas Intensive Permanency Project (KIPP)Convened by: University of Kansas School of Social WelfareKey partnersState public child welfare agency (Kansas DCF)Kansas foster care providers KVC Behavioral HealthcareSt. Francis Community and Family ServicesPrivatized foster care since 1997Long history of public-private-university partnershipAn Initiative of the Childrens BureauMap of Kansas Counties by Population Density5

An Initiative of the Childrens BureauKIPPs Service Model6Proximal OutcomesIncrease in positive parenting behaviorsDecrease in coercive parenting practicesIncrease in use of community resources and social supportsIncreased readiness for reunificationImprovements in parental mental health and substance useDecrease in child problematic behaviorIncrease in child functioningDistal OutcomesIncrease reunification ratesDecrease long-term foster-care ratesIncrease in stable permanency ratesEvidence Based InterventionOregon Model of Parent Management Training (PMTO)Tailor PMTO for Parents of Children with SED in Kansas Foster CareEarly intervention & engagementIn-home, intensiveLow caseloadAccessible & responsiveTrauma-informedComprehensive family assessmentRobust referralsService coordinationEmphasis on parent/child visitsClinical & team supervisionTarget population: Children with serious emotional disturbance (SED)6Questions for ParticipantsHave you implemented an evidence-based intervention (EBI)?Have you utilized implementation frameworks?Were your experiences with implementing an EBI largely positive? Did you work with a purveyor to implement an EBI?What has been your greatest challenge in implementing an EBI?

7An Initiative of the Childrens Bureau7Implementation FrameworksSource: National Implementation Research Network (NIRN)8Rationale for Implementation FrameworksMany factors influence implementationProvide structure and organization for complex processesIncrease awareness & knowledgeEffective planning for upcoming tasks, stages, necessary supportsUsed to assess/evaluate the process and resultsDid we implement what we intended to implement?Did we make a difference in outcomes?Promote long-term sustainability

9An Initiative of the Childrens Bureau9NIRNS Four Frameworks of Implementation ScienceImplementation TeamsImplementation StagesImplementation DriversImprovement Cycles10An Initiative of the Childrens Bureau10Implementation TeamsMake it happen vs let it happenIncrease buy-in and readinessInstall and sustain implementation driversProblem-solveAssess fidelity and outcomesEstablish and maintain linkages with external systemsBuild infrastructure and resources for sustainability

11Source: National Implementation Research NetworkAn Initiative of the Childrens BureauTeamwork and collaboration are at the heart of KIPPs workPeace, Love, KIPP12

An Initiative of the Childrens BureauImplementation Teams (1 of 2)13Include most critical stakeholders Establish at project inception; needed at every stageBuild relationships within team (in-person time)Flexibility is keyTeams focus changes with new stageSupportsChallengesDefine roles as they evolveNew stages = new teams = new communication protocolsTurnover of key stakeholdersUnpredictable challenges require agile teams13Implementation Teams (2 of 2)14Check-in on terms of referenceContinuity of team members Communication up, down, sideways, all-aroundPurposeful agendas, structured meetings, defined roles, linkagesSupportsChallengesRelationship building, cross-cultural learning between purveyor and implementer14Implementation Stages15Source: National Implementation Research NetworkAn Initiative of the Childrens BureauImplementation Stages - Exploration16Exploration builds foundationSustainability from the start; begin with the end in mindSupportsChallengesTimehurry up to implementationTime for selection of EBI; included interviews of purveyors, implementers, and researchers (200+ hours)Iterative, not linear, process16Implementation Stages - Installation17Installation deserves timeEstablish hospitable environment within agencyLeadership plays critical role (vision & communication)SupportsChallengesDeveloping and using new & shared selection protocols for hiringHiring in rural areasCoordinating and timing selection with purveyors requirements17Implementation Stages Initial Implementation18Initial implementation coupled with usability testing lays a solid foundationCreate environment of excitement; also safe for learningMultiple meetings with external stakeholdersRapid cycle problem solving moment by moment Q&A

SupportsChallengesStaged model was a shift for agencies used to quick implementation timelines; learn to go slowly and revise as neededPurveyor mandate required cultural shift (videos)Vast geographic area18Implementation Drivers19

An Initiative of the Childrens BureauImplementation Drivers Staff Selection20Selection protocols tied to fidelity (e.g., require role play, use of feedback)Feed-forward and feedback loops improve staff selection, training, coachingTiminghiring around graduationSome people successfully weed themselves outSupportsChallengesSelection protocols that capture the intangibles and unteachablesTurnoverShort hiring periodsOverhiringHiring qualified staff, esp rural areas

Implementation Drivers Training21Use active teach and role playingHire with EBI fit in mind (behavioral, observational, social learning theory)Create safe learning cultureUse cohorts & peer supportSupportsChallengesEfficient use of training vs sequencing supervisors before practitionersTraining must be accompanied by appropriate caseload & opportunity to apply skills (pacing)Clear, organized, usable materials/manual

2122

23High quality coaching with broad and deep feedbackDirect observation by coachesCongruency between model and coachingSupportsChallengesCoaching capacity must be adequate for number of practitionersLocal coaching = local expertise and responsivenessPractitioners desire for direct, constructive feedback; models focus on positive feedback with little emphasis on weaknesses

Implementation Drivers Coaching24Supervisor is critical for supporting new practiceSupervisors need practice knowledge/experienceSupportsChallengesLoss of supervisor presents unique challengesBig do over: Train supervisors before other practitioners

Implementation Drivers Supervision2425Gold standard for fidelity assessment: observationalRich feedback on clinical skillsSupportsChallengesCapacity must be adequate for number of practitionersFrequency of fidelity assessment for timely feedback loop for trainers and supervisorsSharing fidelity data with practitioners in a supportive manner

Implementation Drivers Fidelity Assessment26Balance an internal focus on implementation and an external focus on gaining support from community stakeholders

SupportsChallengesOther systems that overlap but may not understand or support the EBI (courts, mental health, psych facility)Randomized studyImplementation Drivers Systems Intervention27Engage other key child welfare staff; build their trust and buy-in so youre working as a team, not against one anotherConstant communication and building buy-in with case managers due to turnover in CWBuy-in of randomization and eligibility criteriaSupportsChallengesBe careful & selective in creating more work for case managersLearning new practice is hard work; normalize the struggles Too many bossesAdding or layering a practice is different than changing a practice

Implementation Drivers Facilitative Administration28Every stage requires dataData systems are needed for ongoing progress monitoringBoth quantitative and qualitative data are helpfulValue rigorous researchSupportsChallengesFunding positions for data supportCommunication and feedback loops are important; making the data meaningfulImplementation Drivers Decision Support Data Systems29Leaders knowing the model well shows commitment Have a vision for building evidence for your agency and beyondCelebrate successes at every step and levelSupportsChallengesValues-fit with purveyor; leaders emerge from training process vs leaders based on agency position and tenureFlying the plane as we build itFunding cycles may be too short for full transfer from purveyor to local leadersImplementation Drivers Leadership

29Improvement Cycles30An Initiative of the Childrens Bureau30

3132From the start, rely on data to define problems and measure success of problem-solvingCreate safe learning environment

SupportsChallengesSlow enough for testing but fast enough to utilize staff effectively and efficiently

Implementation Cycles 32Discussion33

Questions for ParticipantsWhat questions do you have?What has been your experience implementing EBIs?How were your experiences similar or different than the supports and challenges we mentioned?What advice do you have for implementers?34An Initiative of the Childrens BureauContacts35KIPP Lead/Applicant Agency:KU School of Social Welfare Becci Akin, beccia@ku.edu Tom McDonald, t-mcdonald@ku.edu

KIPP State Partner:Kansas Department for Children and Families Patricia Long

KIPP Agency Partners:KVC Behavioral Healthcare, Inc. Linda Bass (llbass@kvc.org)St. Francis Community and Family Services Vickie McArthur (vickie.mcarthur@st-francis.org)

Implementation Frameworks Technical Assistance: PII Training/Technical Assistance Project and National Implementation Research Network Karen Blase (karen.blase@unc.edu)

An Initiative of the Childrens BureauOther InfoPII information: http://www.acf.hhs.gov/programs/cb/resource/pii-project-resourcesNIRN Website: http://nirn.fpg.unc.edu/KIPP Publications:Akin, B., Bryson, S., McDonald, T., & Walker, S. (2012). Defining a target population at high-risk of long-term foster care: Barriers to permanency for families of children with severe emotional disturbances. Child Welfare, 91(6), 79-101.Bryson, S., Akin, B., Blas, K., & McDonald, T. (in press). Selecting an evidence-based intervention for families of children with severe emotional disturbances. Journal of Evidence-Based Social Work. Akin, B., Bryson, S., Testa, M., Blase, K., McDonald, T. & Melz, H. (2013). Usability testing, initial implementation and formative evaluation of an evidence-based intervention: Lessons from a demonstration project to reduce long-term foster care. Evaluation and Program Planning, 41, 19-30.

36An Initiative of the Childrens Bureau