Allison Metz, PhD Karen Blase, PhD Melissa Van Dyke, LCSW Frank Porter Graham Child Development Institute University of North Carolina at Chapel Hill Installing

Allison Metz, PhDKaren Blase, PhD

Melissa Van Dyke, LCSWFrank Porter Graham Child Development Institute

University of North Carolina at Chapel Hill

Installing Alternative Response in OhioUsing Implementation

Drivers for Systems Change

Ohio Alternative Response Summit

Workshop on Implementation Drivers

May 14, 2010

Copyright © Dean L. Fixsen and Karen A. Blase, 2008

Common Goal

The Ohio Department of Job and Family Services [in collaboration with several entities] will develop a statewide child protection system with two pathways: an investigation pathway and an alternative response pathway – that assesses and responds to unique safety concerns, risks, and protective capacities of each family who is subject of an accepted report of child maltreatment.


Linking the “What” AND “How”

To successfully implement and sustain a new intervention, you need to understand:

The WHAT - What is the intervention (e.g. Evidence-based practice, “best practice” initiative, broad system change initiative)

AND

The HOW - Effective implementation and sustainability frameworks (e.g. strategies to change and maintain behavior of adults)


Defining your Intervention

From what current state to what future state?

The “intervention” must be operationalized whether it is:

An evidence-based practice or program

A best practice Initiative

A broad systems change initiative

OperationalizePart of Speech: verb Definition: to define a

concept or variable so that it can be measured or expressed quantitativelyWebster's New Millennium™ Dictionary of English, Preview Edition (v 0.9.7)Copyright © 2003-2008 Lexico Publishing Group, LLC


OH Alternative Response Model

Principles

All child protective services (CPS) interventions

Alternative response specific

Core Elements

E.g., Two or more discrete pathways; screening decisions; assessments are child-centered and family focused


OH Alternative Response Model

Foundational Tenets of the Practice Model

Alternative response, families, services

Workers’ Skill Sets

Characteristics, competencies, values

Selected Indicators of Quality Practices with Families

Behaviors indicative of fidelity to the Alternative Response model


Reflection Work – Operationalizing. . .

Take a moment to consider. . . .When the Alternative Response Model has been

developed and implemented as intended –

What will child welfare staff be saying and doing more often? . . . Less often?

What will child welfare supervisors, managers, and leaders be saying and doing more often? . . . Less often?

What will partner agency personnel be saying and doing more often? . . . Less often?

Which child welfare activities and processes will be affected?


Putting “It” Into Action

Who will do what differently?

Who will help change occur? AND How will change happen? At all levels….

Practice

Program

Organization

System


Work of Implementation“Systems don’t change - people do.”

– Jim Wotring, Georgetown University

Changing the thinking and behavior of:Adult human service professionals (frontline practitioners) so that positive outcomes are more frequently created for children and families.

Administrators in organizations (Supervisor, Managers, Directors) so that structures, cultures, and climates change to support new ways of work

System directors, policy makers, and funders (State Agency, Family Courts) so that implementation is supported and outcomes can be achieved


The Work of Implementation

Changing the behavior of adult human service professionals is hard work

It requires a systematic approach to support behavior change of practitioners, volunteers, administrators, and policy makers

It requires the application of Implementation Drivers

© Karen A. Blase & Dean L. Fixsen, 2004


What are the Implementation Drivers?

Competency Drivers are mechanisms that help to develop, improve, and sustain one’s ability to implement an intervention with fidelity and benefits to consumers. Competency Drivers include: Selection, Training, Coaching, and Performance Assessment

Organization Drivers are mechanisms to create and sustain hospitable organizational and systems environments for effective services. Organization Drivers include: Decision Support Data System, Facilitative Administration, and Systems Intervention

Leadership Drivers are methods to manage Technical problems where there is high levels of agreement about problems and high levels of certainty about solutions and to constructively deal with Adaptive challenges where problems are not clear and solutions are elusive

© Fixsen & Blase, 2008

Performance Assessment

Coaching

Training

Selection

Systems Intervention

Facilitative Administration

Decision Support Data System

Adaptive

Technical

Integrated & Compensatory

Com

pete

ncy

Driv

ers

Com

pete

ncy

Driv

ers O

rganization Drivers

Organization D

rivers

LeadershipLeadership

Improved outcomes for children and families

Graphics by Steve Goodman,2009

Implementation Drivers


Why These Drivers Focus First on Practitioner’s Behavior?

In child welfare, the Practitioner IS THE INTERVENTION

Build supports in relationship to what the Practitioner needs to be competent

Create the conditions under which practitioners can do the right thing for the right reason at the right time to maximize positive results

Wide ranging inputs (individuals with all their past history and current realities)



Coaching

Training

Selection

Com

pete

ncy

Driv

ers

Com

pete

ncy

Driv

ers




Implementation Lens


Recruitment and Selection

Purposes:

Select for the “unteachables”

Screen for pre-requisites

Make expectations explicit

Allow for mutual selection through interview process

Improve likelihood of retention after “investment”

Improve likelihood that training, coaching and supervision will result in implementation


Recruitment and Selection

Implementation Best Practices:Job or role description clarity about accountability and expectations

Sampling of skills and experience is related to “new practices” and expectations

Interactive Interview Process

Skilled interviewers

Using Data for Integration and Compensatory Features

Feed interview information forward to trainers, coaches, administrators


Recruitment and Selection Driver in Early Stages of Implementation

Identify selection criteria for different levels of staff and roles

Select and recruit participants for Implementation Work Groups

Assess capacity to recruit and select appropriate staff to implement planned systems reforms

Assess staff readiness and commitment

Identify possible data sources and processes that will tell you how well the Selection Driver is currently functioning in your State or County agency.


Pre-Service and In-Service Training

Purposes:

Knowledge acquisition

Basic Skill Development

“Buy-in”

Implementation Best Practices:

Theory grounded (adult learning)

Skill-based

Behavior Rehearsals vs. Role Plays

Knowledgeable Feedback Providers

Practice to Criteria

Feedback to Selection and Feed Forward to Supervision

Data-based (pre and post testing)


Training Driver in the Early Stages of Implementation

Identify your populations of concern

Whose capacity are you building?...to do what?

Whose knowledge skills and abilities are you attempting to impact?

Identify and assess training requirements and needed resources

Assess capacity to train staff effectively at multiple levels of the system to implement and sustain change initiative


Training Driver in the Early Stages of Implementation

Identify training requirements and available training resources

Assess how current child welfare training practices in your state or county currently reflect implementation best practices?

Identify possible data sources and processes that will tell you how well the Training Driver is currently functioning in your State or County agency.


Supervision and Coaching

Purposes:

Ensure implementation

Develop good judgment

Ensure fidelity

Provide feedback to selection and training processes


Coaching Driver

OUTCOMES% of Participants who Demonstrate Knowledge, Demonstrate New Skills in a Training Setting,

and Use new Skills in the Classroom

TRAINING

COMPONENTS

Knowledge Skill

Demonstration

Use in the

Classroom

Theory and Discussion

10% 5% 0%

..+Demonstration in Training 30% 20% 0%

…+ Practice & Feedback in

Training

60% 60% 5%

…+ Coaching in Classroom

95% 95% 95%


Coaching


Design a Coaching Service Delivery Plan

Develop accountability structures for Coaching – Coach the Coach!

Regular satisfaction feedback from employees and volunteers

Regular review of adherence to Coaching Service Delivery Plan

Look at data – Fidelity, Staff Satisfaction with Support, Skill Acquisition


Coaching Driver in Early Stages of Implementation

Ensure that Alternative Response has been fully operationalized and practice profiles completed

Assess levels of buy-in for executing coaching plan in agency

Assess coaching requirements and identify available resources

Assess capacity to coach staff effectively at multiple levels of the system to implement change initiative

Identify possible data sources and processes that will tell you how well the Coaching Driver is currently functioning in your State or County agency.

Copyright © Dean L. Fixsen and Karen A. Blase, 2008 © Fixsen & Blase, 2008


Coaching

Training

Selection

Com

pete

ncy

Driv

ers

Com

pete

ncy

Driv

ers




Implementation Lens



Purposes:

Ensure implementation

Reinforce staff and build on strengths

Develop skills and abilities

Measure fidelity

Interpret Outcome Data

Feedback to Agency, County, State Implementation Teams on functioning of

Recruitment and Selection Practices

Training Programs (pre and in-service)

Supervision and Coaching Systems




Transparent Processes – Orientation

What, When, How, Why

Use of Multiple Data Sources

Content

Competency

Context

Tied to positive recognition – not used ‘punitively’


Definition of Fidelity

The degree to which the program or practice is implemented ‘as intended’ as defined by the program developers/researchers or experts

Adherence

Integrity

Delivered in a “comparable” manner

SO THAT it is more likely that comparable outcomes will be more consistently achieved

Competency Implementation Drivers Analysis and Discussion Template

Competency Implementation Drivers

Does Driver exist?(yes/no)

Responsibility for Driver? (who, what dept/entity)

How does the Driver currently support implementation?

How well integrated is the Driver with the other Drivers?

Is there a measure of Driver effectiveness? How might you know the Driver is effective?

How can the Driver be used more purposefully to support fidelity, outcomes, and/or sustainability?

Staff Selection

Staff Training

Staff Coaching

Staff PerformanceEvaluation (Fidelity)

Copyright © Dean L. Fixsen and Karen A. Blase, 2008© Dean Fixsen, Karen Blase, Robert Horner, George Sugai, 2008

Sobering Observations

"All organizations [and systems] are designed, intentionally or unwittingly, to achieve precisely the results they get." R. Spencer Darling

Business Expert

“The tyranny of the status quo.” Fritz Oser Educator



Coaching

Training

Selection



Decision Support Data SystemC

ompe

tenc

y D

river

s

Com

pete

ncy

Driv

ers

Organization D

rivers

Organization D

rivers





Decision Support Data Systems

Purposes:

Provide information to assess effectiveness of Alternative Response Model

To guide further program and practice development

Celebrate success

Engage in continuous quality improvement

Be accountable for quality infrastructure (are Drivers ‘working’) and for outcomes




Includes intermediate and longer term outcome measures

Includes process measures (fidelity)

Measures are “socially important”

Useful data are:Reliable (standardized protocols, trained data gatherers)

Reported frequently (e.g. weekly, quarterly)

Reported at relevant and “actionable” levels (e.g. child, family, agency)

Widely shared

Practical to collect

Useful for and used for making decisions (PDSA)


Decision Support Data Systems in Early Stages of Implementation

Assess data collection requirements including technology and training needs

Assess technology capacity and integration issues

Identify resources for assessing fidelity

Identify possible data sources and processes that will tell you how well the Decision Support Data Systems Driver is currently functioning in your State or County agency.

Make plans for incorporating data into program improvement cycles


Facilitative Administrative

No such thing as a purely administrative decision!!....They are all decisions about

Quality Practice with Children and Families!!

Purposes:Facilitates moving through implementation stagesEnsures effective use of Competency DriversTakes the lead on Systems InterventionsUtilizes data for improvementLooks for ways to make work of practitioners easier and more effective!!


Facilitative Administrative


Forms and supports multi-level, functional implementation teams (e.g. service area, cross-agency, senior management)

Uses feedback to make changes in Implementation Drivers

Revises policies and procedures to support the new way of work

Solicits and uses feedback from child welfare staff

Reduces administrative barriers


Facilitative Administration in Early Stages of Implementation

Assess policies and procedures needed to support new way of work

Provide leadership and support

Identify champions

Create communication plans

Support teaming approach; share decision-making

Create buy-in for business NOT as usual - Take risks!

Identify possible data sources and processes that will tell you how well the Facilitative Administration Driver is currently functioning in your State or County agency.



Purposes:

Identify barriers and facilitators for the new way of work

Create a “hospitable” environment for the new way of work

Contribute to cumulative learning in multi-site projects.



Implementation Best Practices

Match leadership level needed to intervene

Engage and grow “champions” and “opinion leaders”

Objectively document barriers

Establishes formal PEP – PIP cycles

Uses Transformation Zones to

Identify Systems Issues

Create time-limited, barrier busting processes

Make constructive recommendations and assist in implementing and evaluating them (PDSA)


Systems Interventions Driver in Early Stages of Implementation

Identify systems barriers and facilitators

Create and strengthen necessary external partnerships

Set up policy to practice to policy to practice (4P) feedback loops

Identify transformation zones

Engage and grow systems champions

Identify possible data sources and processes that will tell you how well the Systems Intervention Driver is currently functioning in your State or County agency.

Organizational Implementation Drivers Analysis and Discussion Template

OrganizationalImplementation Drivers

Does Driver exist?(yes/no)

Responsibility for Driver? (who, what dept/entity)

How does the Driver currently support implementation?

How well integrated is the Driver with the other Drivers?

Is there a measure of Driver effectiveness? How might you know the Driver is effective?

How can the Driver be used more purposefully to support fidelity, outcomes, and/or sustainability?

Systems Interventions





Coaching

Training

Selection




Adaptive

TechnicalCom

pete

ncy

Driv

ers

Com

pete

ncy

Driv

ers O

rganization Drivers

Organization D

rivers


Prevention and Intervention Benefits for Individuals and Community




Types of Leadership Needed

Different strategies for different challenges

Technical Leadership

Adaptive Leadership

According to Ron Heifitz and his colleagues, one of the biggest mistakes “leaders” make is to incorrectly identify the type of challenge they are facing

Using technical approaches for adaptive issues


Technical Work

Perspectives are aligned (views, values)

Definition of the problem is clear

Solution and implementation of the solution is clear

Primary locus of responsibility for organizing the work is the leader


Adaptive Work

Legitimate, yet competing, perspectives emerge

Definition of the problem is unclear

Solution and implementation is unclear and requires learning

Primary locus of responsibility is not the leader


The Adaptive Work of the Leader

Get on the balcony

Identify the adaptive challenge

Regulate distress

Maintain disciplined attention

Give the work back to the people

Protect all voices» R. Heifetz and D. Laurie: The Work of

Leadership. Harvard Business Review, 1998.



Coaching

Training

Selection




Adaptive

Technical

Integrated & Compensatory

Com

pete

ncy

Driv

ers

Com

pete

ncy

Driv

ers O

rganization Drivers

Organization D

rivers






Integrated and Compensatory

Integrated

Consistency in philosophy, goals, knowledge and skills across these processes

Compensatory

Assessment of weaknesses and strengths in Driver functioning

Installation of Drivers at multiple levels of the system (practitioner, agency, county, state)


Are you ready to get a bit more complicated? Applying the Drivers to a Driver

Example: Creating a “system” for competent trainers for Alternative Reponse Model in OH

How to Select them?

How to train them?

How to coach them?

How and what to measure to see if fidelity to the training model, content and processes over time?

What training outcomes should we measure?

What internal and external systems changes will be needed for trainers to be supported and effective?


Innovation Fluency X Drivers

How much do the experts know about what it takes to implement Alternative Response in typical settings?

What will they contribute related to Drivers and their “innovation”?

Knowledge?Theory?Data?Recommendations based on implementation in typical settingsTools, training, monitoring systems?


Applications

Implementation Drivers INTRODUCING DRIVERS TO SUPPORT YOUR CHANGE INITIATIVE: Discuss how you will introduce Drivers to your county workgroups or implementation teams. What would you need to do to prepare for such a conversation?

DRIVERS AND DATA: Consider Alternative Response model. List the data sources for assessing each driver and discuss a yearly process for evaluating the quality and integration of the Drivers.

BEST PRACTICES FOR DRIVERS: Consider an innovation that your state is currently implementing. Using the Drivers Checklist, assess the current implementation practice against the best practices listed on the handout for one “key position” (e.g. family group conferencing facilitator).

Given your assessment, what are your next steps.

If you don’t have information, what are your sources of information?


For More InformationFor More InformationAllison Metz, Ph.D.

919-218- 7540

[email protected]

At the Frank Porter Graham Child Development Institute

University of North Carolina

Chapel Hill, NC

www.scalingup.org http://nirn.fpg.unc.edu/

http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/


For More Information

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).

Download all or part of the monograph at:

http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/

To order the monograph go to:

https://fmhi.pro-copy.com/

Documents

Allison Metz, PhD Karen Blase, PhD Melissa Van Dyke, LCSW Frank Porter Graham Child Development Institute University of North Carolina at Chapel Hill Installing