Voice and Choice: The Consumer / Family Role in Transformation

Dean L. Fixsen, Ph.D.

Karen A. Blase, Ph.D.National Implementation Research Network

Louis de la Parte Florida Mental Health Institute

Voice and Choice:The Consumer / Family Role in Transformation

Kentucky Meeting, 2006

Copyright © Dean L. Fixsen and Karen A. Blase, 2006

Transformation Agenda

Effective services

Implementation of effective services

Consumer, family, advocacy role in transformation

Discussion


Transformation GoalsMental Health Care Is Consumer

and Family Driven

Disparities in Mental Health Services Are Eliminated

Early Mental Health Screening, Assessment, and Referral to Services Are Common Practice

Excellent Mental Health Care Is Delivered

New Freedom Commission on Mental Health, 2003


Evidence-Based Programs

The integration of best practice evidence with clinical expertise and patient value.

Institute of Medicine, 2001


Evidence-Based Programs

Leading edge of MH System Change

Restructure therapeutic, administrative, and financial environments to enable MH systems that are:

•More dynamic and adaptable•More accountable•More effective


Use of “Evidence-Based” (Hoagwood & Johnson, 2003)

1900-1990 0 1990-1995 86 1995-1999 5,525

Google (10/2004): 2,320,000Google (01/2006): 16,600,000


Consumer & Family Voice

Several EBPs have included the consumer & family voice to help guide the development of their programs

ACT, Supported Employment, Teaching-Family Model, Multisystemic Therapy, Functional Family Therapy, and others

X

Cor

rect

ing

Prob

lem

s

Coo

pera

tion

Serv

ing

Com

mun

ity N

eeds

Cor

rect

ing

Prob

lem

s

Coo

pera

tion

Serv

ing

Com

mun

ity N

eeds

Hom

e En

viro

nmen

t

Cor

rect

ing

Prob

lem

s

Coo

pera

tion

Follo

w G

uide

lines

Com

mun

ity C

omm

ents

Coo

pera

tion

Cor

rect

ing

Prob

lem

s

Com

mun

icat

ion

Coo

pera

tion

Effe

ctiv

e In

Hel

ping

Chi

ld

Com

mun

icat

ion

Fairn

ess

of T

-Ps

Con

cern

of T

-Ps

Effe

ctiv

enes

s of

T-P

s In

Cor

rect

ing

Prob

lem

s

Plea

sant

ness

of T

-Ps

Hel

pful

ness

of T

-Ps

Goo

d Tr

eatm

ent P

rogr

am

Soci

al S

kills

of Y

outh

s

Teac

hing

Ski

lls

Self-

gove

rnm

ent

Mot

ivat

ion

Syst

em

Fam

ily-S

tyle

Liv

ing

Con

ditio

n of

Hom

e (T

-P R

es.)

Rec

ord

Kee

ping

Con

ditio

n of

Hom

e (B

oard

Res

.)

7

6

5

4

3

2

1

Court Social Service ParentsSchoolsBoard Professional EvaluationYouths

T-Ps ______________________

Address: __________________

Evaluation Type: ___________

7=Completely Satisfied6=Satisfied5=Slightly Satisfied4= Neither Satisfied nor Dissatisfied3=Slightly Dissatisfied2=Dissatisfied1=Completely Dissatisfied

Shaded area representsacceptable rangeof performance

Current License Yes No

Certified Teaching-Parents

XX

X XX XX

XXX

X XX X

XXX

XX

XXXX

XXXX

XX

XX


Effective Programs

Nurse-Family Partnership

Parent Management Training

Functional Family Therapy

Positive Behavior Support

Multi-Systemic Therapy

Multidimensional TFC

Teaching-Family Model


Effective Programs

Medication algorithms

Family psychoeducation

Illness management

Integrated dual disorders treatment

Supported employment

Assertive Community Treatment

Fountain House Lodges


What does NOT Work?

Non-behavioral interventions for disruptive behavior disorders and/or ADHD (Weisz et al., 1995; Pelham et al., 1998)

Group, peer-based interventions for disruptive disorders (GGI, PPC)

Instead may facilitate “deviancy training”: increases in rates of substance abuse, delinquent behaviors, and violence post-intervention (Dishion et al., 1999).


DARE (5th and 6th grade curriculum)

Gun Buyback programs

Boot Camps

Peer counseling programs

Summer job programs (at risk youth)

Home detention with electronic monitoring

Wilderness / challenge programs

Casework / counseling

What does NOT Work?


What produces negative (iatrogenic) outcomes?Waivers to adult (criminal courts)

Scared Straight

Shock Probation / Parole

What is HARMFUL?


A Scale of UsefulnessEvidence-based programs have demonstrated

effectiveness through research

Promising practices have some evidence for benefits to consumers (Jim Wotring & Kay Hodges in Michigan).

Common practices have no data to support their use but over many years they have been built into a series of laws, regulations, funding mechanisms, professional and organizational accreditation standards, etc. that sustain them.

Harmful practices have evidence indicating harm to clients but often still are supported as a common practice


Implementation

The act of accomplishing some aim or executing some order

To put into practical effect; carry out

Pursue to a conclusion– Dictionary.com

A lot of hard work!


Implementation

= Access to effective services

= Benefits to you and your family

Consumer & Family Choice


Craft knowledgeEBP purveyors (program developers)

EBP implementation site managers

Implementation researchers

Survey of EBP program developers

Scientific informationCRUSK, Follow Through, Lodges, et al.

Program development and replication data

Synthesis of the implementation evaluation and research literature

The NIRN


Implementation Review

Human service prevention and treatment programs (e.g. substance abuse, adult / children’s MH, justice, health, education)

Advanced manufacturing technologies

Research-based clinical guidelines

Engineering: bridge maintenance

Hotel service management

National franchise operations

Cancer prevention & treatment


Implementation Attempts

Homebuilders national implementation grants: $500 million, 5 years

Program / system change grants plus TA for systems of care, state transformation, school reform, medical clinical guidelines, etc:

$?? billion a year


Implementation Attempts In business, change initiatives that are heavily dependent on people (reengineering, TQM, culture change) fail 80-90% of the time

About 10% of what is taught in training gets transferred to the job "Up to 70% of the failures in

business are not due to poor strategy or a lack of good ideas, but to flawed execution."

R. W. Rogers, 2002


Lisbeth Schorr, 1993

Author of Within Our Reach

Implementation Attempts

Successful programs do not contain the seeds of their own replication.


Implementation SuccessPMTO program in Norway: National

implementation of parent management program (NIDA funded research)

DOTS program in India: National implementation of TB treatment protocol, 1 million patients in 4 years

MST program in USA: National implementation of homebased interventions for juvenile offenders


Common to SuccessGood science and technology

Baseline / decision support data

Expanding commitment & flexibility

Purposeful workforce development

Organizational change & supports

Alignment of system functioning

Knowledgeable purveyor, constant monitoring, feedback, intervention

Khatri & Frieden, 2002


Good News

Knowledge of successful implementation strategies can benefit all human services


Consumer Outcomes

Effective intervention practices

+

Effective implementation practices

=

Good outcomes for consumers


Consumer Outcomes

Effective NOT Effective

Effective

NOT Effective

IMPLEMENTATION

INT

ER

VE

NT

ION Performance

ImplementationPaper / Procedure Implementation


Paper Implementation Review of TQM adoptions in 5,492 hospitals

"If organizations can minimize evaluation and inspection of their internal operations by external constituents through adoption alone, they may neglect implementation altogether, decoupling operational routines from formally adopted programs."

Westphal, Gulati, & Shortell (1997)


Consumer Outcomes

Effective NOT Effective

EBP

NOT Effective

IMPLEMENTATION

INT

ER

VE

NT

ION

High Fidelity Low Fidelity


Bedlington, et al., 1979

0%

10%

20%

30%

40%

50%

Homes

1.0

1.2

1.5

1.7

1.9

2.0

1.1

1.4

1.6

1.8

1.3

Delinquency

Teaching

rs = – .94

Per

cent

Par

enta

l-Tea

chin

g W

ith Y

outh

s

Mean S

elf Reported

Delinquency R

atings

1 2 3 4 5 6

Teaching-Family Model


What you need to know

An evidence-based program is one thing

Implementation of an evidence-based program is a very different thing


Stages of Implementation

Implementation is not an event

A mission-oriented process involving multiple decisions, actions, and corrections


Stages of Implementation

Exploration

Installation

Initial Implementation

Full Implementation

Innovation

Sustainability

Implementation occurs in stages:

Fixsen, Naoom, Blase, Friedman, & Wallace, 2005

2 – 4 Years


Installation

Initial Implementation

Implementation Stages:Non-Linear Processes

Practitioner

Evidence-based Practices

Purveyor

FidelityMeasure

Implementation Framework

Organizational Structures/Culture



EBPs: The Source

The usability of a program has little to do with the quality or weight of the evidence regarding that program

Evidence on intervention effectiveness for specific populations helps us choose what to implement

Evidence on the effectiveness of the intervention does not help implement the program or practice successfully


EBPs: The Source

Core intervention components

Clearly described (what/how)

Practical measure of fidelity

Fully operationalized (do/say)

Field tested (iterative revision)

Contextualized (org./systems fit)


Implementation Site: The Destination

Defined Need

Identified Resources

Fully informed consent re: innovation and implementation

Top level commitment to change and agreement re: access


Implementation Site: The Destination

Practitioners impact consumers

It is the job of directors, managers, and funders to align policies and structures to facilitate effective practitioner practices

There is no such thing as an “administrative decision” – they are all treatment decisions



Be a part of the process of choosing what to implement (ask about core intervention components, match with strengths and needs, costs to consumers & families)

Advocate with MH directors and legislators (what is the plural form of anecdote?)

Be informed (ACMH Guide for Families, Malisa Pearson & Tiffany Leischner; www.acmh-mi.org )


Implementation of EBPs

Changing the behavior of adult human service professionals

Changing organizational structures, cultures, and climates

Changing the thinking of system directors and policy makers

Leading edge of MH System Change

New Freedom Commission


Human Service Systems

“Systems trump programs.”

– Patrick McCarthy,

Annie E. Casey Foundation“

Advocate at the systems levels to enable longer lasting local impacts.


PurveyorExcellent experimental evidence for

what does not work

Dissemination of information by itself does not lead to successful implementation (research literature, mailings, promulgation of practice guidelines)

Having information and knowledge is helpful to decision making, but it is NOT implementation


PurveyorExcellent experimental evidence for

what does not workTraining alone, no matter how well

done, does not lead to successful implementation

Information dissemination and training by themselves repeatedly have been shown to be ineffective in human services, education, health, business, and manufacturing


Active Purveyor Role

Successful implementation on a useful scale requires a purveyor

An individual or group of individuals representing a program or practice who actively work to implement that practice or program with fidelity and good effect

Purveyors accumulate data & experiential knowledge -- more effective and efficient over time (information economics, K. Arrow)



Change the behavior of adult human service professionals

Change organizational structures, cultures, and climates

Change the thinking of system directors and policy makers

Successful and sustainable implementation of evidence-based practices and programs always requires organizational change.



Organization

Management (leadership, policy)

Administration (HR, structure)

Supervision (nature, content)

Practitioner

State policies

System of care

Pu

rvey

or

Simultaneous, Multi-Level Interventions


INTEGRATED & COMPENSATORY

CONSULTATION & COACHING

CONSULTATION & COACHING

STAFF PERFORMANCE EVALUATION

STAFF PERFORMANCE EVALUATION

FACILITATIVE ADMINISTRATIVE

SUPPORTS

FACILITATIVE ADMINISTRATIVE

SUPPORTS

RECRUITMENT AND SELECTION

RECRUITMENT AND SELECTION

PRESERVICE TRAINING

PRESERVICE TRAINING

SYSTEMSINTERVENTIONS

SYSTEMSINTERVENTIONS

Keys to Implementation

DECISION SUPPORT DATA SYSTEMS

DECISION SUPPORT DATA SYSTEMS

OUTCOMES(% of Participants who Demonstrate Knowledge, Demonstrate

new Skills in a Training Setting, and Use new Skills in the Classroom)

TRAININGCOMPONENTS

KnowledgeSkill

DemonstrationUse in the Classroom

Theory and Discussion

10%

5% 0%

..+Demonstration in Training

30%20%

0%

…+ Practice & Feedback in Training

60% 60% 5%

…+ Coaching in Classroom

95% 95% 95%

Joyce and Showers, 2002



An evidence-based program is one thing

Implementation of an evidence-based program is a very different thing

Good outcomes occur ONLY when evidence-based practices and programs are implemented well.

As a consumer of evidence-based practices and programs, you need to know how to assess the quality of implementation.



Fidelity is important.

Fidelity means actually using the treatment procedures as they were tested and found to be effective.

Implementation with fidelity is required to achieve advertised outcomes

Practitioner

Evidence-based Practices

Purveyor

FidelityMeasure

Implementation Framework

Organizational Structures/Culture



Bedlington, et al.

0%

10%

20%

30%

40%

50%

Homes

1.0

1.2

1.5

1.7

1.9

2.0

1.1

1.4

1.6

1.8

1.3

Delinquency

Teaching

rs = – .94

Per

cent

Par

enta

l-Tea

chin

g W

ith Y

outh

s

Mean S

elf Reported

Delinquency R

atings

1 2 3 4 5 6

Implementation & Delinquency


Questions you can ask

What evidence do you have that the recommended treatment is effective for me / a child like mine?

Have you been specifically trained to carry out this treatment? By someone approved by the program developers?

Have you been coached in the use of this treatment? By someone approved by the program developers?


Questions you can ask

How is your use of the treatment evaluated? Do you typically achieve high fidelity scores?

In the past year, how many people have you treated using this program? What results have you achieved for those people?

Are consumers, family members, and others involved in carrying out and evaluating the treatment? How much time will that require?


SummaryConsumer and family involvement is key

to initiating and sustaining transformation efforts

Opportunities, transportation, child care, expense reimbursement

Presenting information, confident discussion, persistence

Networking and coalition building

Willing / able to take on important roles (e.g. selection, training, evaluation; community support; legislation and funding)


SummaryVoice and choice are important parts of the

development and evaluation of many evidence-based practices and programs.

Consumer/family voice is an important part of the selection and initiation of evidence-based practices and programs in a community.

Consumer/family choice can be enhanced by evidence-based practices and programs that are implemented and used with fidelity.

EBPs are no panacea. They are a lot of hard work to implement and maintain, but they can be helpful to you and your family.


Thank YouWe thank the following for their support

Annie E. Casey Foundation (EBPs and cultural competence)

William T. Grant Foundation (implementation literature review)

Substance Abuse and Mental Health Services Administration

(ORC Macro sub-contract for SOC analyses of implementation; implementation strategies grants)

Centers for Disease Control (implementation research contract)

National Institute of Mental Health (research and training grants)

Juvenile Justice and Delinquency Prevention (program development and evaluation

grants)


For More Information

Dean L. Fixsen

813-974-4446

[email protected]

Karen A. Blase

813-974-4463

[email protected]

National Implementation Research Network

At the Louis de la Parte Florida Mental Health Institute

University of South Florida

http://nirn.fmhi.usf.edu


For More Information

Fixsen, D. L., Naoom, S. F., Blase, K. A., Friedman, R. M. & Wallace, F. (2005). Implementation Research: A Synthesis of the Literature. Tampa, FL: University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network (FMHI Publication #231).

E-mail us for a free copy or download all or part of the monograph at:

http://nirn.fmhi.usf.edu/resources/publications/Monograph/index.cfm

Documents

Voice and Choice: The Consumer / Family Role in Transformation