Implementation Implementation Research for Children Research for Children and Families in a State and Families in a State

Policy Context Policy Context

Kimberly Eaton Hoagwood Kimberly Eaton Hoagwood Columbia University Columbia University

December 6, 2010December 6, 2010

OutlineOutline

DefinitionsDefinitions State context characteristicsState context characteristics TheoriesTheories Fitting effective practices to the Fitting effective practices to the

state context: Which context?state context: Which context? State D-I research examplesState D-I research examples Concluding thoughtsConcluding thoughts

Definitions of D-I Definitions of D-I ConstructsConstructs

(McHugh & Barlow, 2010)(McHugh & Barlow, 2010)

ADOPTION: The decision by a provider or system to learn and ADOPTION: The decision by a provider or system to learn and implement an intervention. implement an intervention.

IMPLEMENTATION—TIMPLEMENTATION—The process of transferring an intervention he process of transferring an intervention to a setting; it includes training approaches, consultation, and to a setting; it includes training approaches, consultation, and decision supports. decision supports. The pThe process of introducing or changing practice in a specific local rocess of introducing or changing practice in a specific local setting.setting.

DISSEMINATION—TDISSEMINATION—The effort to facilitate initial adoption through he effort to facilitate initial adoption through ttargeted distribution of a well-defined set of informationargeted distribution of a well-defined set of information

Focuses on how information is created, packaged, transmitted, and Focuses on how information is created, packaged, transmitted, and interpreted among various stakeholder groups.interpreted among various stakeholder groups.

DIFFUSION--The DIFFUSION--The intended or unintended spread of information or intended or unintended spread of information or treatmentstreatments

Effectiveness vs. Effectiveness vs. ImplementationImplementation

How well an intervention works, and How well an intervention works, and how well it is implemented, are two how well it is implemented, are two different things (Schoenwald, 2006)different things (Schoenwald, 2006)

Effective treatments can be Effective treatments can be implemented poorly; and, ineffective implemented poorly; and, ineffective treatments can be implemented treatments can be implemented well. well. ((Fixsen, Naoom, Blasé, Friedman, & Wallace, Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005)2005)

State Initiatives to Improve the State Initiatives to Improve the Quality of Services for Children Quality of Services for Children

and Familiesand Families 20+ State Consortium—EBP-CA (Bruns et 20+ State Consortium—EBP-CA (Bruns et

al, 2008): CA, CO, CT, DE, HA, MD, MI, al, 2008): CA, CO, CT, DE, HA, MD, MI, MN, NM, NY, NJ, OH, OK, PA, WA, MN, NM, NY, NJ, OH, OK, PA, WA, CanadaCanada

Most common therapiesMost common therapies Multi-systemic therapies (MST) Multi-systemic therapies (MST) Functional family therapy (FFT)Functional family therapy (FFT) Treatment foster care (TFC)Treatment foster care (TFC) Cognitive-Behavioral Therapies for TraumaCognitive-Behavioral Therapies for Trauma CBT for depression CBT for depression Range of behavior management programsRange of behavior management programs

State Mental Health Budgets: State Mental Health Budgets: Overall Picture (NASMHPD 2009)Overall Picture (NASMHPD 2009)

According to the Center on Budget and Policy According to the Center on Budget and Policy PrioritiesPriorities

(June 29, 2008):(June 29, 2008): Cumulative state budget gaps are expected to exceed Cumulative state budget gaps are expected to exceed

$350 billion for the period of FY2009 – FY2011.$350 billion for the period of FY2009 – FY2011. 48 States are facing shortfalls in FY2010 totaling 48 States are facing shortfalls in FY2010 totaling

$166 billion.$166 billion. At least 29 states expect total gaps of $38 billion At least 29 states expect total gaps of $38 billion

alone in 2011.alone in 2011. The current recession has a greater impact on state The current recession has a greater impact on state

budgets than the previous recession due to high budgets than the previous recession due to high unemployment and lack of revenue from property and unemployment and lack of revenue from property and income taxes.income taxes.

Thus an interest in improving quality and containing Thus an interest in improving quality and containing costscosts

# of States with Anticipated # of States with Anticipated Deficits (FY 09-11) Deficits (FY 09-11)

NASMHPD Research Institute, Inc. NASMHPD Research Institute, Inc.

2009 2009 2010 2010 20112011 No No deficitdeficit

As of 12/08 As of 12/08 32 32 32 32 13 13 1010 As of 6/09 As of 6/09 32 32 40 40 22 22 55

Some Special Challenges Some Special Challenges

Policies to promote uptake of Evidence-Based Practices Policies to promote uptake of Evidence-Based Practices move faster than the knowledge base: creates jagged move faster than the knowledge base: creates jagged interfaceinterface

Pressures for egalitarianism may preclude experimental Pressures for egalitarianism may preclude experimental designsdesigns

Accountability standards and billable time in conflict with Accountability standards and billable time in conflict with need for ongoing consultationneed for ongoing consultation

Outcomes of interest usually functional and pragmatic: Outcomes of interest usually functional and pragmatic: feasible measures for assessing change hard to find feasible measures for assessing change hard to find

Clinical service workforce is largely unprepared: Clinical service workforce is largely unprepared: Retraining has to be intensive Retraining has to be intensive

Technological supports for clinical decision-making and Technological supports for clinical decision-making and tracking outcomes are absent (few Electronic Medical tracking outcomes are absent (few Electronic Medical Records)Records)

Fundamental difference in epistemologies between Fundamental difference in epistemologies between science and policy. Need for people who can translate science and policy. Need for people who can translate between the 2 worldsbetween the 2 worlds

Different epistemologiesDifferent epistemologies Science Science

Cumulative, builds on Cumulative, builds on past knowledgepast knowledge

Generates more Generates more questions than answersquestions than answers

Proceeds slowly, Proceeds slowly, incrementallyincrementally

Self-correctingSelf-correcting Scientific community—Scientific community—

not locally bound not locally bound although may be although may be discipline-bound and discipline-bound and specializedspecialized

Policies Policies Driven by immediate Driven by immediate

political needspolitical needs Can move quickly, Can move quickly,

errantly, erraticallyerrantly, erratically Pragmatic, action-Pragmatic, action-

orientedoriented 1 decision can affect 1 decision can affect

millions of lives for millions of lives for better or worsebetter or worse

Locally driven Locally driven (counties, states, (counties, states, countries)countries)

Three Relevant Theories for State Three Relevant Theories for State Implementation: Can they be Implementation: Can they be

integrated?integrated? Unified Theory of Behavior Change (Jaccard et al, 2002) Unified Theory of Behavior Change (Jaccard et al, 2002)

drawn from basic behavioral sciences to understand the drawn from basic behavioral sciences to understand the triggers for behavior change within individuals. triggers for behavior change within individuals. Incorporates principles from the Theory of Reasoned Incorporates principles from the Theory of Reasoned Action (Fishbein & Ajzen, 1975; Ajzen & Fishbein, 1981) Action (Fishbein & Ajzen, 1975; Ajzen & Fishbein, 1981) and Theory of Planned Behavior (Ajzen & Madden, 1986; and Theory of Planned Behavior (Ajzen & Madden, 1986; Ajzen, 1991), and Self-Efficacy (Bandura)Ajzen, 1991), and Self-Efficacy (Bandura)

Organizational social context theory (Glisson 2002) to Organizational social context theory (Glisson 2002) to understand the organizational and team processes understand the organizational and team processes (culture, climate, structure) that affect agency adoption (culture, climate, structure) that affect agency adoption of new practices. of new practices.

Participatory action research based on Habermas’ Participatory action research based on Habermas’ (1990) theory of communicative action, to support (1990) theory of communicative action, to support shared decision-making among diverse constituents. shared decision-making among diverse constituents.

Our working model in New York State integrates these Our working model in New York State integrates these theories with the goal of improving the practical utility theories with the goal of improving the practical utility of the findings. of the findings.

System & Policy Context of the State methods of reimbursement, fiscal incentives, linkages to other

healthcare systems,

certification, accreditation, licensing standards

Organizational Social Context of AgenciesCulture, Climate, Structure

Clinical Care Improvement

Training on EBP’s, supervision, consultation and support,

outcome monitoring, measurement feedbk

Family Engagement Family Empowerment

Attitudes, Beliefs & Expectancies of Families & Youth

Improved Child & Family Outcomes

Attitudes, Beliefs & Expectancies of Clinicians and Supervisors

Improved Implementation Efficiency & Effectiveness

Prior to implementation: Prior to implementation: Consider the Fit of the Consider the Fit of the

Practice for the ContextPractice for the Context Appropriateness for system: Appropriateness for system:

population, setting, fiscal support, population, setting, fiscal support, policiespolicies

Appropriateness for Appropriateness for setting/organization: Social-setting/organization: Social-organizational context including organizational context including provider attitudes, beliefs, expectationsprovider attitudes, beliefs, expectations

Appropriateness for family context and Appropriateness for family context and perspectivesperspectives

Implementation Resource Guide (Toolkit) for Implementation Resource Guide (Toolkit) for Disruptive Behavior Disorders (BJ Burns, 2007)Disruptive Behavior Disorders (BJ Burns, 2007)

Hawaii EB Services Report (Chorpita & Dalaiden, Hawaii EB Services Report (Chorpita & Dalaiden, 2007): Strength of evidence, levels, and practice 2007): Strength of evidence, levels, and practice elementselements

PracticeWise (Dalaiden & Chorpita)PracticeWise (Dalaiden & Chorpita) Contextualized Feedback Systems (Bickman & Contextualized Feedback Systems (Bickman &

Kelley, 2010)Kelley, 2010)

Examples of Practical Fit Examples of Practical Fit

Positive Parenting Program – Level 4 GOOD 0 to13 B and G W A

Incredible Years GOOD 2 to 9 B and G W B H A

Helping the Noncompliant Child GOOD 3 to 9 B and G W B

Parent-Child Interaction Therapy (PCIT) GOOD 3 to 7 B and G W B

Parent Management Training – Oregon GOOD 4 to 12 B and G W B

Brief Strategic Family Therapy GOOD 6 to 18 B and G W B H

Problem-Solving Skills Training GOOD 7 to 14 Mostly B W B

Coping Power GOOD 9 to 12 B and G W B

Mentoring GOOD 6 to 18 B and G W B H N

Multisystemic Therapy GOOD 10 to 18 B and G W B

Functional Family Therapy GOOD 11 to 18 B and G W B H N

Adolescent Transitions Program GOOD 11 to 15 B and G W B H N

Multidimensional Treatment Foster Care GOOD 12 to 18 B and G W B

INTERVENTION Level of Evidence

AgeRange

Race/EthnicityWhite – Black –HispanicNative Am. - Asian Am.

GenderBoys or Girls

Is Study Population Comparable to Yours?

Positive Parenting Program – Level 4 Yes On-site andRegional

2-3 days with repeat in 4-6 weeks

$20,000 – 20 traineesShort-term

Incredible Years Yes On-site, Off-site and Regional 2-3 days

$300-400 off-site$1,500/day on-site

Yes

Helping the Noncompliant Child Yes On-site 2 days minimum $3,000 Yes

Parent-Child Interaction Therapy (PCIT) Yes Off-site 5 days $3,000 No

Parent Management Training – Oregon Yes On-site 18 workshop days $25,000 per trainee Yes

Brief Strategic Family Therapy Yes On-site 4 (3-day) workshops $58,000

(includes coaching)Yes

Problem-Solving Skills Training No ---- ---- ---- ----

Coping Power Yes On-site 3 days $5,000 Yes

Mentoring No ---- ---- ---- ----

Multisystemic Therapy Yes On-site andRegional

5 days for staff2 days for supervisors

$8,000 (on-site)$750 (per person regional)

Yes

Functional Family Therapy Yes On-site and Off-site

3 days with2 follow-ups

Approximately $35,000For 3-8 therapists

Yes

Adolescent Transitions Program Yes On-site and Off-site 4-5 days

$4,000 (up to 20 trainees)

Yes

Multidimensional Treatment Foster Care Yes On-site and Off-site

5 days for staff2 days for parents

$40-50,000 per site Yes

INTERVENTION

Fit with Agency Resources ? Training and Coaching/Consultation

Is Training Provided by Developer?

Where is Training Provided?

Is Follow-Up Coaching Available?

What is Length of Training?

How Much Does Training Cost?

Implementation Research Implementation Research in New York State for in New York State for Children and Families Children and Families

Focus on clinic referred youthFocus on clinic referred youth Youth 0-17 across New York State who Youth 0-17 across New York State who

are served in outpatient mental health are served in outpatient mental health clinicsclinics

• 21,737 youth served weekly21,737 youth served weekly• 87,539 youth served annually with a 95% 87,539 youth served annually with a 95%

confidence interval of (77588, 99469)confidence interval of (77588, 99469)• 239 agencies (357 licensed clinics across 239 agencies (357 licensed clinics across

942 sites)942 sites)

NYS Child and Family InitiativesNYS Child and Family Initiatives• Clinic Plus: Clinic Plus: early detection, partnership, public early detection, partnership, public

healthhealth• 85 clinics 85 clinics

• Evidence-Based Treatment Dissemination Evidence-Based Treatment Dissemination Center Center • 195 clinics/72 also in Clinic Plus195 clinics/72 also in Clinic Plus

• Family Support Services:Family Support Services:• Multi-family Groups for Conduct Disorders—13 clinicsMulti-family Groups for Conduct Disorders—13 clinics• Family peer advocates and parent empowerment:Family peer advocates and parent empowerment:

• 400 FPAs serving approx 20,000 parents/yr400 FPAs serving approx 20,000 parents/yr

• Clinical decision supports/QIClinical decision supports/QI• Psychiatric Services and Clinical Knowledge Psychiatric Services and Clinical Knowledge

Enhancement System (PSYCKES) for Enhancement System (PSYCKES) for polypharmacy (322 clinics)polypharmacy (322 clinics)

• Contextualized feedback systems (4 clinics)Contextualized feedback systems (4 clinics)

OMH Data Tracking System for OMH Data Tracking System for Clinic Plus: 2010Clinic Plus: 2010

Next StepNext Step

Clinic fiscal restructuringClinic fiscal restructuring Emphasis on business practices and Emphasis on business practices and

organizational structuresorganizational structures Learning collaborative structure to Learning collaborative structure to

support teamssupport teams

Evidence-based Evidence-based Treatment Dissemination Treatment Dissemination

CenterCenter Funded by OMH to support training Funded by OMH to support training

and one full year of expert and one full year of expert consultation on CBT skills for consultation on CBT skills for specific disordersspecific disorders

1212 clinicians/supervisors trained 1212 clinicians/supervisors trained in CBT for trauma, depression, and in CBT for trauma, depression, and behavior therapy for disruptive behavior therapy for disruptive behaviorsbehaviors

Workforce Issues: Basic CBT Workforce Issues: Basic CBT Training in Mental Health Training in Mental Health ProfessionsProfessions

0%10%20%30%40%50%60%70%80%90%

100%

MD PhD PsyD MSW

OfferedRequired

Weissman et al., 2006, Archives of General Psychiatry;Weissman et al., 2006, Archives of General Psychiatry;

Ns: MD=73, PhD=62; PsyD=21; MSW=62Ns: MD=73, PhD=62; PsyD=21; MSW=62

% T

rain

ing

Prog

ram

s

EBTDC Consultation EBTDC Consultation CallsCallsYear 1 Year 2

Length 90 mins 60 mins

Supervisor Specific Calls

None Avg of 11 (range 8-16 peo.)

Average # of participants on calls

12 (range 6-18 peo.) 8 (range 5-13 peo.)

# of call groups 35 42 clinician

8 supervisor

Total Calls Held 731 1007 Clinician

80 Supervisor

Consultation Call Consultation Call FindingsFindings

Year 1 Year 2

Attendance-Clinicians

84.6% 83.4%

Attendance -Supervisor

39.0% 70.0%

Drop outs 28%-C, 23%-S 30.5%-C,17.3%-S

Call groups completed

35/36 42/42

-Attendance did not vary significantly over the course of the year

Assessment 30%

Specific CBT 29%

Administrative Issues 17%

Parent/Family 16%

Engagement 4%

Off-Task 1%General CBT 1%

Case ID 2%

Crisis 1%

What happens on the calls?

Small but statistically significant relationship between consultants and therapists discussing specific CBT techniques, r = .33, p<.05

Attendance – Consultant Attendance – Consultant DifferencesDifferences

Year 1Year 1ConsultantConsultant ClinicianClinician (%) (%) Supervisor (%) Supervisor (%) 1 88.51 88.5 60.0 60.0

2 84.8 38.5 2 84.8 38.5 3 84.0 49.13 84.0 49.1 4 74.2* 25.2* 4 74.2* 25.2*

Overall 83.3 39.6 Overall 83.3 39.6

Certificate Completion: Certificate Completion: Consultant DifferencesConsultant Differences

ConsultantConsultant ClinicianClinician (%) (%) 1 94.2*1 94.2* 2 68.42 68.4 3 82.53 82.5 4 68.6 4 68.6 Overall 79.6Overall 79.6

Next stepsNext steps

Trauma-focused CBT for residential Trauma-focused CBT for residential treatment and hospitalstreatment and hospitals

To include milieu-based supportsTo include milieu-based supports Follow back on continuation of skillsFollow back on continuation of skills Simple measures of adherenceSimple measures of adherence Supervisor calls: Content of Supervisor calls: Content of

consultationconsultation

Family Support Studies: Family Support Studies: Multifamily group (McKay & Multifamily group (McKay &

Hoagwood)Hoagwood) NIMH-funded R01: RCT at clinic level. 13 NIMH-funded R01: RCT at clinic level. 13

clinics get MFG/13 get SAUclinics get MFG/13 get SAU Focus on urban, low-income children of color.Focus on urban, low-income children of color.

Youth 7-11 and their familiesYouth 7-11 and their families ODD or CDODD or CD New clinic referralsNew clinic referrals Primarily low-income African American and Primarily low-income African American and

Latino familiesLatino families Designed in collaboration with parents & Designed in collaboration with parents &

providersproviders

MFG Service DeliveryMFG Service Delivery

Clinician and parent advocate co-facilitationClinician and parent advocate co-facilitation

Clinicians provide professional expertiseClinicians provide professional expertise

Parent advocates provide mutual supportParent advocates provide mutual support

Parent consumers helped to develop treatment Parent consumers helped to develop treatment manual (e.g., helped bring stress to the forefront)manual (e.g., helped bring stress to the forefront)

MFG Attendance MFG Attendance (in comparison to rates on retention in (in comparison to rates on retention in outpatient urban individualized mental outpatient urban individualized mental

health services)health services)

AnalysesAnalyses Random coefficient modeling to examine Random coefficient modeling to examine

change over time and differences between change over time and differences between MFG and SAUMFG and SAU

Time modeled as months from baseline Time modeled as months from baseline using measurements from 4 time points: using measurements from 4 time points: baseline, mid-way through intervention, baseline, mid-way through intervention, post-test , 6 month follow-up.post-test , 6 month follow-up.

Preliminary results Pre/Post (N=322)Preliminary results Pre/Post (N=322)

MFG MFG ControlControl Baseline Baseline Post Post Baseline Baseline

Post Post Mean (SD)Mean (SD) Mean (SD)Mean (SD) Mean (SD)Mean (SD) Mean (SD)Mean (SD)

FFbb

ODDODD 2.84 2.84 2.45*** 2.45***aa 2.54 2.54 2.86 2.86 14.73*** 14.73***(Iowa Conners)(Iowa Conners) (0.68) (0.91) (0.68) (0.91) (0.77) (0.94) (0.77) (0.94)

Inattention 2.83 2.40***Inattention 2.83 2.40***aa 2.73 2.73 2.73 2.73 6.43* 6.43*(Iowa Conners)(Iowa Conners) (0.75) (0.75) (0.70) (0.70) (0.84) (0.71) (0.84) (0.71)

Social SkillsSocial Skills 76.8476.84 83.02*** 83.02***aa 81.95 81.95 78.05 78.05 14.84***14.84***

(SSRS)(SSRS) (10.62)(10.62) (10.48) (10.48) (13.64) (11.24) (13.64) (11.24)aa Paired sample t test from baseline to post test.Paired sample t test from baseline to post test.bb F tests between condition interaction for repeated measures ANOVA. F tests between condition interaction for repeated measures ANOVA.*p < .05, ***p < .001*p < .05, ***p < .001

Family Support: Skill Training for Family Support: Skill Training for family peer advocates to improve parent family peer advocates to improve parent

empowerment (Olin et al., 2010) empowerment (Olin et al., 2010) NIMH-funded R34 on Parent EmpowermentNIMH-funded R34 on Parent Empowerment N=32 family advisors and 124 parents in New York City (85% low N=32 family advisors and 124 parents in New York City (85% low

income, minority) N=18 trained FA; N=14 comparisonincome, minority) N=18 trained FA; N=14 comparison Impact on Impact on

FPA’s knowledge of MH services, collaborative skills, and self-FPA’s knowledge of MH services, collaborative skills, and self-efficacyefficacy

Parents’ working alliance, self-efficacy, empowerment, strainParents’ working alliance, self-efficacy, empowerment, strain Significant improvements pre/post in knowledge (p<.001), skills Significant improvements pre/post in knowledge (p<.001), skills

(p<.003) and service self-efficacy (p<.02) among FA(p<.003) and service self-efficacy (p<.02) among FA Significant difference pre/post among parents working with PEP-Significant difference pre/post among parents working with PEP-

trained advisors in working alliance at 6 months (p<.05) but not trained advisors in working alliance at 6 months (p<.05) but not among parents in comparison groupamong parents in comparison group

No differences in parents’ service self-efficacy, empowerment, or No differences in parents’ service self-efficacy, empowerment, or strainstrain

Strongest predictor of parents’ working alliance: working with Strongest predictor of parents’ working alliance: working with advisor who provided home/school visits (R2=.61; F=.0001)advisor who provided home/school visits (R2=.61; F=.0001)

High levels of depressive symptoms among parents (CES-D High levels of depressive symptoms among parents (CES-D average 22.6 (cut off is 16); 2/3 above clinical cut-off)average 22.6 (cut off is 16); 2/3 above clinical cut-off)

Heterogeneity of agency’s social-organizational contexts and Heterogeneity of agency’s social-organizational contexts and undervalued roles of family advisors undervalued roles of family advisors

Social Norms/Pressure

Attitude toward Mental Health

Self-Efficacy

Behavioral Intentions

Behavior

PARENT INTERVENTION

Skills/Knowledge,Habits, EnvironmentalObstacles, Priorities

Possible Barriers

Intervention Target

Model of Behavior Change: Model of Behavior Change: (adapted from (adapted from Jaccard et al., 2002)Jaccard et al., 2002)

Expected Value

What do important others think/do? How motivated am I?

How do I feel about mental health/illness? (Stigma, Past experience with MH system or providers)

Do I believe I know how to navigate the system and overcome obstacles?

Behavioral Intentions

Active involvement in child MH services

PARENT INTERVENTION

Skills/Knowledge,Habits, EnvironmentalObstacles, Priorities

Possible Barriers

Intervention Target

Model of Behavior Model of Behavior ChangeChange

What do I get out of this?

Next StepsNext Steps

MFG implemented state-wide: Will results MFG implemented state-wide: Will results hold uphold up

Engagement training implemented state-wide: Engagement training implemented state-wide: Will results hold up Will results hold up

FPA and PEP: New NIMH-funded R01 on FPA and PEP: New NIMH-funded R01 on content of FPA services (using Walkthrough content of FPA services (using Walkthrough Methods) and improving integration of FPAs Methods) and improving integration of FPAs into workplace: into workplace:

Focus on social organizational context, Focus on social organizational context, enhancing working alliances, improving enhancing working alliances, improving family and youth outcomesfamily and youth outcomes

37

Dimensions of Organizational Dimensions of Organizational Social Context (OSC) (Glisson et Social Context (OSC) (Glisson et

al., 2002, 2005, 2008, 2009, al., 2002, 2005, 2008, 2009, 2010)2010)

Organizational CultureOrganizational Culture System norms and valuesSystem norms and values ““The way things are done”The way things are done” What is expected and rewardedWhat is expected and rewarded

Organizational ClimateOrganizational Climate Work environment’s psychological impact on employeesWork environment’s psychological impact on employees Employees’ affective response to their work Employees’ affective response to their work

environmentenvironment Employees’ sense of “psychological safety”Employees’ sense of “psychological safety”

38

Organizational Culture

1. Proficiency – expectation that service providers will be competent, have up-to-date knowledge, and place the well-being of clients first

2. Rigidity – expectation that service providers will have limited discretion and flexibility, and closely follow extensive bureaucratic rules and regulations

3. Resistance – expectation that service providers will show no interest in change or new ways of providing services

OSC Measurement System (Glisson et al)

39

Five Studies to Date Link Five Studies to Date Link Organizational Culture to:Organizational Culture to:

Individualized care Individualized care by 408 service providers in by 408 service providers in 30 social service organizations (Glisson, 1978)30 social service organizations (Glisson, 1978)

Family-centered careFamily-centered care by 131 service providers by 131 service providers in four emergency rooms (Hemmelgarn, Glisson in four emergency rooms (Hemmelgarn, Glisson & Dukes, 2001) & Dukes, 2001)

Turnover, work attitudes,Turnover, work attitudes, and and service service qualityquality among 283 caseworkers in 33 child among 283 caseworkers in 33 child welfare and juvenile justice case management welfare and juvenile justice case management teams (Glisson & James, 2002)teams (Glisson & James, 2002)

Service qualityService quality in 15 child welfare teams in 15 child welfare teams serving 21 urban and rural counties (Glisson & serving 21 urban and rural counties (Glisson & Green, 2006)Green, 2006)

New program sustainabilityNew program sustainability in nationwide in nationwide sample of 100 mental health clinics (Glisson, sample of 100 mental health clinics (Glisson, Schoenwald, Kelleher et al., 2008)Schoenwald, Kelleher et al., 2008)

40

OSC Measurement System

Organizational Climate

1. Engagement – service provider perceptions of personal accomplishment, involvement and concern for clients

2. Functionality – service provider perceptions that they receive the needed cooperation and support to do their jobs

3. Stress – service provider perceptions that they are emotionally exhausted and overloaded in their work

41

Five Studies to Date Link Five Studies to Date Link Organizational Climate to:Organizational Climate to:

Job satisfactionJob satisfaction and and commitmentcommitment among 319 among 319 service providers in 22 human service service providers in 22 human service organizations (Glisson & Durick, 1988)organizations (Glisson & Durick, 1988)

Service qualityService quality and and outcomes outcomes among 250 among 250 children served by 32 children’s services offices children served by 32 children’s services offices (Glisson & Hemmelgarn, 1998)(Glisson & Hemmelgarn, 1998)

Work attitudesWork attitudes and and service qualityservice quality in 33 child in 33 child welfare and juvenile justice service teams (Glisson welfare and juvenile justice service teams (Glisson & James, 2002)& James, 2002)

Service outcomesService outcomes for 1,678 children in 88 for 1,678 children in 88 county child welfare offices in nationwide NSCAW county child welfare offices in nationwide NSCAW sample (Glisson, 2007)sample (Glisson, 2007)

Therapist turnoverTherapist turnover in national sample of 100 in national sample of 100 mental health clinics (Glisson, Schoenwald, mental health clinics (Glisson, Schoenwald, Kelleher et al., 2008)Kelleher et al., 2008)

- 3.0

0

3.0

engaged func stressfulClimate

z score

- 3.0

0

3.0

engaged func stressfulClimate

z score

- 3.0

0

3.0

engaged func stressfulClimate

z score

- 3.0

0

3.0

Culture

z score

resistant rigid proficient- 3.0

0

3.0

Culture

z score

resistant rigid proficient

- 3.0

0

3.0

Culture

z score

resistant rigid proficient

Examples of Clinic Profileswith z scores based on National Norms (Glisson et al 2008)

Closing ThoughtsClosing Thoughts States offer unique laboratories for States offer unique laboratories for

implementation research to enhance public implementation research to enhance public health impacthealth impact broad population base broad population base increasing public health emphasis—cross systemsincreasing public health emphasis—cross systems commitment to quality improvementcommitment to quality improvement commitment to efficiency, especially in current fiscal commitment to efficiency, especially in current fiscal

climateclimate Requires Requires

Constant negotiation and revisiting of goalsConstant negotiation and revisiting of goals Flexible methods and use of a wider array of methods Flexible methods and use of a wider array of methods

(mixed)(mixed) Flexible personalities among research teamFlexible personalities among research team PatiencePatience

ConclusionConclusion Improving implementation of EBPs:Improving implementation of EBPs:

Attend to fit between the innovation and Attend to fit between the innovation and contextcontext

Identify, measure and track context Identify, measure and track context variables (policy, fiscal, social-variables (policy, fiscal, social-organizational, family/peer)organizational, family/peer)

Be judicious in selection of primary context Be judicious in selection of primary context for investigation. Can’t do them all wellfor investigation. Can’t do them all well

Identify practical measuresIdentify practical measures Use mixed methods to squeeze the most Use mixed methods to squeeze the most

knowledge out of each studyknowledge out of each study Look for opportunities to piggy-back data Look for opportunities to piggy-back data

collection onto electronic measurement collection onto electronic measurement systems already in placesystems already in place