Building Professional Networks to Support Implementation of Evidence-Based Mental Health Services

Funding: NIMH (R25 MH080916-01A2, T32 MH019117; F31 MH098478), VA (QUERI)

Alicia BungerOhio State University

Byron PowellWashington University in St. Louis

Rochelle HansonMedical University of South Carolina

Nathan DooganOhio State University

Yiwen CaoOhio State University

Jerry DunnUniversity of Missouri-St. Louis

Purpose

Examine change in professional advice-seeking patterns among mental health clinicians participating in a learning collaborative for implementation.

Learning Collaborative Models• IHI’s Breakthrough Series

Learning Collaborative• Quality Improvement• Teams from multiple agencies

• Emphasizes shared learning• Stimulating interactions• Within & Across organizational

teams

• Are they effective? How?• Mixed evidence (Schouten et al, 2008)

• “Black Box” (Mittman, 2004)

• Expert Panel• Commitment

Preparatory Work

• In-Person Learning Sessions (3)• Plan-Do-Study-Act (PDSA) cycles

Active Learning

• Team Calls• Web Support• Quality Improvement Techniques

Supports

(IHI, 2003; Nadeem, et al, 2013)

Social Networks and Implementation• LCs may support implementation by building social

networks within and across participating agency teams.

• Networks are conduits for technical information and social support

3 Ways LCs May Build Networks:

Content Experts

Peers at Home

Agency

LC Peers at other

agencies

Clinician

Intra-Organizational Support

Technical info – knowledge/skill

Inter-organizational support, New ideas, Referrals

Opportunities for Interaction

• Learning Sessions

• Consultation• Calls

• Learning Sessions

• PDSAs

• Learning Sessions

• Group Calls

Do learning collaboratives “rewire” social networks in a way that supports implementation?

AIMS:

1. Assess change in the composition of clinicians’ professional advice networks over the duration of a learning collaborative.

2. Examine how changes in clinician advice seeking patterns alter the structure of the regional network.

Study Setting• $2 million regional initiative to

implement TF-CBT funded through a county-based tax levy

• 32 Children’s behavioral health agencies

• Community-based trainers, certified by NCTSN as TF-CBT therapists

• Expert Panel• Commitment

Preparatory Work

• In-Person Learning Sessions (3)• Plan-Do-Study-Act (PDSA) cycles

Active Learning

• Team Calls• Web Support• Quality Improvement Techniques

Supports

Study Setting• $2 million regional initiative to

implement TF-CBT funded through a county-based tax levy

• 32 Children’s behavioral health agencies

• Community-based trainers, certified by NCTSN as TF-CBT therapists

• Enhanced Learning Collaborative Model

• Expert Panel• Commitment

Preparatory Work

• In-Person Learning Sessions (3)• Plan-Do-Study-Act (PDSA) cycles

Active Learning

• Team Calls• Web Support• Quality Improvement Techniques

Supports

• Coaching Calls• On-Site Visits• Local Trainers• Rostering

Enhanced Features

Method

Sample• 132 participants from 32 agencies (with pre & post data)• 90% of Learning Collaborative completers

Data Collection• Surveys administered in-person during 1st & 3rd learning

sessions (est. 10 months apart)

Discipline Role Experience

Social Work 53% Sr. Leader 10% GT 5 yrs in field 65%

Counseling 28% Supervisor 22% LT 1 yr in job 43%

Psychology 12% Clinician 68%

MeasuresRank 

Column A – Names

Column B - Communication

Who do you turn to for professional advice about youth with trauma histories? Please list their name and organization in order you would contact them.

In the past 6 months, how frequently have you communicated or been in contact with this person via in-person contact, telephone, or email? (Circle the most accurate number from the answer scale below for each person.) 

Not Once1-2

times

About once/ month

About every 2 weeks

About once/ week

About daily

Many times daily

1.

Name:  Organization:

1 2 3 4 5 6 7

……

• Nominate up to 5 sources of professional advice• 422 Unique individuals nominated across both waves of data collection

Analysis

1. Compare Composition of Professional Advice Networks• Clinician Ego-Network at LS1 and LS3• Calculate and compare Exposure (% of Ego-net) using

paired samples t-test in Stata 13 (Valente, 2010)

2. Compare Network Structure• Visualize • Network Descriptives (R - sna, igraph)

Content Experts

LC Peers at other

agencies

Private Practice

OtherPeers at Home

Agency

Ego-Net: Size of Professional Advice Networks

LS1 LS30%

20%

40%

60%

80%

100%

3.9 3.6

Ego-Net Size*

Nu

mb

er o

f N

om

inat

ion

s

5

4

3

2

1

0

*t(131)=2.06, p<.05

LS1 LS30%

20%

40%

60%

80%

100%

0.05

0.72

0.030.06

0.15

Other

Private Practice

Peers-Other Agencies

Peers-Home

Experts

Exp

osu

re

Ego-Net: Composition of Professional Advice Networks

LS1 LS30%

20%

40%

60%

80%

100%

0.05

0.20

0.72

0.66

0.03

0.050.06

0.030.150.07

Other

Private Practice

Peers-Other Agencies

Peers-Home

Experts

Exp

osu

re

Ego-Net: Composition of Professional Advice Networks

**Experts:T(131)=6.60, p<.001

**Private Practice:T(131)=-3.24, p<.001

**Other:T(131)=-3.41, p<.001

Whole Network Structure – LS 1

Node = PersonN=422

Diamond = Faculty ExpertN=5

Line = Nomination/TieN=2487

Isolate = Person w/no tiesN=74

Components

Compare Network StructureLearning Session 1 Learning Session 3

Compare Network Structure

Learning Session 1 Learning Session 3

N 422 422Isolates 74 177Density 0.014 0.013Centralization (in-degree) 0.097 .182Clustering (weighted) 0.293 .356

Compare Network Structure

Learning Session 1 Learning Session 3

N 422 422Isolates 74 177Density 0.014 0.013Centralization (in-degree) 0.097 .182Clustering (weighted) 0.293 .356Reciprocity 0.164 0.227Weighted Reciprocity 0.188 0.246

Compare Network Structure

Learning Session 1 Learning Session 3

N 422 422Isolates 74 177Density 0.014 0.013Centralization (in-degree) 0.097 .182Clustering (weighted) 0.293 .356Reciprocity 0.164 0.227Weighted Reciprocity 0.188 0.246Agency Homophily 89.08 77.72

Limitations• Generalizeability

• 1 region

• No comparison/control• Was the LC responsible for making net change?• What elements of the LC??

• Measurement validity• Self-report measures

• Drop-Out/Missing data• Some participated in only one wave of data collection

• Drop-Out• Opt-Out• Snow-Out (winter weather during one LS)

Summary of Findings

Clinician-Level• Clinicians rely on colleagues at their home agency• Exposure to faculty experts increased • Slight reduction in exposure to external sources of advice

(perhaps because of coaching+consultation)

Whole Network• Centralization around Faculty Experts• Reciprocity

Implications• For Learning Collaborative Organizers

• Provide additional opportunities for participants to network across organizational boundaries.

• For Policy Makers and Administrators• Benefits of local experts/knowledge leaders for scale-up initiatives.

• Potential for sustainment?

• Integration of local service delivery system (in terms of advice sharing)

• Small changes at the individual clinician-level can translate to big changes at the systems-level.

Future Research Questions:• Why do professional advice ties change?

• LC Components: LS? Coaching? LS + Coaching? • Network dynamics? Readiness for implementation? Supportive

climate?

• Do professional advice networks have a role in implementation success?• What is the relationship between ego-net composition, position in

the network, etc. with implementation fidelity? Treatment outcomes?

• Why do some clinicians/organizations remain disconnected?• Initial Readiness?• Innovation-values fit?

Contact information

Alicia Bunger

Bunger.5@osu.edu

ReferencesAarons, GA, Hurlburt, M, & Horwitz, SM. (2011). Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors. Administration and policy in mental health, 38(1), 4–23.

Damschroder, LJ, Aron, DC, Keith, RE., …(2009). Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implementation science, 4, 50.

IHI (2003). The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. Cambridge, MA. Retrieved from http://www.ihi.org/IHI/Results/WhitePapers/

Mittman, BS. (2004). Creating the Evidence Base for Quality Improvement Collaboratives. Ann Intern Med, 140(11), 897–901.

Nadeem, E, Olin, SS, Hill, LC, Hoagwood, KE, & Horwitz, SM. (2013). Understanding the components of quality improvement collaboratives: a systematic literature review. The Milbank quarterly, 91(2), 354–94.

Powell, BJ, McMillen, JC, Proctor, EK … (2011). A Compilation of Strategies for Implementing Clinical Innovations in Health and Mental Health. Medical care research and review, 69(2), 123–157.

Schouten, LMT, Hulscher, MEJ, van Everdingen, JJE, …(2008). Evidence for the impact of quality improvement collaboratives: systematic review. BMJ (Clinical research ed.), 336(7659), 1491–4.

Valente, TW (2010). Social networks and health: models, methods, and applications . Oxford University Press.

Acknowledgements• Missouri Academy of Child Trauma Studies (MoACTS) at

the Child Advocacy Center of Greater St. Louis (UMSL).

• NIMH • Postdoctoral Traineeship (T32 MH019117) sponsored by UNC-CH & Duke

(Bunger)• Predoctoral traineeship (F31 MH098478) (Powell)

• NIMH/VA• Implementation Research Institute (R25 MH080916-01A2) (WUSTL)

(Bunger & Hanson)

• Doris Duke Charitable Foundation• Fellowship for the Promotion of Child Well-Being (Powell)