Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Applying Implementation Science to
Improve Care in Community Settings
Gregory A. Aarons, Ph.D.
University of California, San Diego
Department of Psychiatry
Child & Adolescent Services Research Center
Presented at: Be There San Diego: University of Best Practices 03 April, 2017
Implementation Research
Implementation research:– Scientific study of the use of strategies to adopt and
integrate evidence-based health interventions into clinical and community settings in order to improve patient outcomes and benefit population health.
Dissemination research:– Scientific study of targeted distribution of information
and intervention materials to a specific public health or clinical practice audience. The intent is to understand how best to spread and sustain knowledge and the associated evidence-based interventions.
R01: https://grants.nih.gov/grants/guide/pa-files/PAR-16-238.html
R21: https://grants.nih.gov/grants/guide/pa-files/PAR-16-236.html
R03: https://grants.nih.gov/grants/guide/pa-files/PAR-16-237.html
Implementation
Outcomes
Feasibility
Fidelity
Penetration
Acceptability
Sustainability
Uptake
Costs
*IOM Standards of Care
What?
QIs
ESTs
How?
Implementation
Strategies
Implementation Research Methods
Service
Outcomes*
Efficiency
Safety
Effectiveness
Equity
Patient-
centeredness
Timeliness
Patient
Outcomes
Clinical/health
status
Symptoms
Function
Satisfaction
CONTEXT
CONTEXT
CO
NT
EX
TC
ON
TE
XT
Conceptual Model of Implementation Research
Proctor et al. (2009)
Implementation
Outcomes
Feasibility
Fidelity
Penetration
Acceptability
Sustainability
Uptake
Costs
*IOM Standards of Care
What?
QIs
ESTs
How?
Implementation
Strategies
Implementation Research Methods
Service
Outcomes*
Efficiency
Safety
Effectiveness
Equity
Patient-
centeredness
Timeliness
Patient
Outcomes
Clinical/health
status
Symptoms
Function
Satisfaction
CONTEXT
CONTEXT
CO
NT
EX
TC
ON
TE
XT
The UsualThe Core of
Implementation
Science
Conceptual Model of Implementation Research
Proctor et al. (2009)
Review of Models (Tabak, et al., 2012)
Reviewed 61 models – Models (aka “theories” or “frameworks”)
– Frameworks evaluated on:
Construct flexibility– Broad highly operationalized
Focus on dissemination vs. implementation– D-only D=I I-only
Socioecologic framework level– Individual Community System
Source: Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012). Bridging research and
practice: models for dissemination and implementation research. American journal of preventive medicine,
43(3), 337-350.
Most frameworks also are adapted or modified in practice
Source: Tabak, R. G., Khoong, E. C., Chambers, D. A., & Brownson, R. C. (2012). Bridging research and practice: models
for dissemination and implementation research. American journal of preventive medicine, 43(3), 337-350.
Common Elements of Frameworks
Multiple Levels
– Implementation occurs in complex systems
– Need to identify concerns at different levels
Process of Implementation– Implementation occurs over time
– There may be relatively discrete phases or stages
CFIR Matrix
Source: Damschroder et al., 2009
Consolidated Framework for Implementation
Research
Damschroder, L.J., et al., Fostering implementation of health services research findings into practice: a consolidated
framework for advancing implementation science. Implement Sci, 2009. 4: p. 50.
Damschroder, L. Consolidated Framework for Implementation Research (CFIR) Wiki. 2010 November 8, 2010 [cited 2011
October 15]; Available from: http://wiki.cfirwiki.net.
Consolidated Framework for Implementation Research (CFIR)
Intervention characteristics– Intervention source
– Evidence strength/quality
– Relative advantage
– Adaptability
– Trialability
– Complexity
– Design/packaging
– Cost
Inner setting– Structural Characteristics
– Implementation climateTension for change
Compatibility
Relative priority
Organizational incentives/rewards
Goals/feedback
Learning climate
– Readiness for implementationLeadership engagement
Available resources
Access to information and knowledge
Damschroder, L. J., Aron, D. C., Keith, R. E., Kirsh, S. R., Alexander, J. A., & Lowery, J. C. (2009). Fostering implementation of
health services research findings into practice: a consolidated framework for advancing implementation science. Implement Science
4(1), 50.
Outer setting- Patient needs and resources- Cosmopolitanism (org networks)- Implementation climate- Peer pressure- External Policies/incentives
Characteristics of the individuals involved– Knowledge/beliefs/ (attitudes?)
– Self-efficacy
– Individual stage of change
– Identification with organization
Process of implementation– Planning
– Engaging
– Executing
– Reflecting and evaluating
Exploration, Preparation, Implementation,
Sustainment (EPIS) Model
Key phases of the implementation process
Multilevel
Frames implementation factors across levels within each phase
Enumerates common and unique factors across levels and across phases
Source: Aarons, G. A., Hurlburt, M., & Horwitz, S. M. (2011). Advancing a conceptual model of evidence-based practice implementation in
public service sectors. Administration and Policy in Mental Health and Mental Health Services Research, 38(1), 4-23.
Service Environment
Leadership
Legislation
Policies
Resources
Service Contracts
Inter-organizational Environment
Relationship of service
system with CBOs
Relationships between
provider organizations
Collaboration
Competition
Co-opetition
Outer Context
EBP Characteristics
EBP Developers
Purveyors / Implementation Support
EBP
System
Fit
EBP Organization
Fit
Individual Adopter Characteristics
Attitudes to EBP
Commitment to EBP
Citizenship
Org. Commitment
Job Satisfaction
Turnover Intentions
Turnover
Intra-Organizational Characteristics
Leadership
Implementation Leadership
Policies
Structure
MIS/EHR
Data monitoring/feedback
Inner Context
EBP Provider
Fit
Patients/Consumers
Need
Advocacy
Interconnections
Interactions-Linkages-Relationships
Procurement-Contracting
EBP
Patient
/ Client
Fit
Aarons et al. (2014)
Sustainment Outcomes
QUALITATIVE
Level of Institutionalization
Continued use of EBP
Adaptations
QUANTITATIVE
Provider Reach/Penetration
Client Reach/Penetration
Org. Sustainment Climate
Fidelity
Determinant & Process Framework Example: EPIS
Implementation & Sustainment Model
Service Environment
Leadership
Legislation
Policies
Resources
Service Contracts
Inter-organizational Environment
Relationship of service
system with CBOs
Relationships between
provider organizations
Collaboration
Competition
Co-opetition
Outer Context
Patients/Consumers
Need
Advocacy
Determinant & Process Framework Example: EPIS
Outer Context
Aarons et al. (2014)
Individual Adopter Characteristics
Attitudes to EBP
Commitment to EBP
Citizenship
Org. Commitment
Job Satisfaction
Turnover Intentions
Turnover
Intra-Organizational Characteristics
Leadership
Implementation Leadership
Policies
Structure
MIS/EHR
Data monitoring/feedback
Inner Context
Aarons et al. (2014)
Determinant & Process Framework Example: EPIS
Inner Context
EBP Characteristics
EBP Developers
Purveyors / Implementation Support
EBP
System
Fit
EBP Organization
Fit
EBP Provider
Fit
EBP
Patient
/ Client
Fit
Aarons et al. (2014)
Determinant & Process Framework Example: EPIS
EBP Fit – Values-Innovation Fit
Outer Context
EBP Characteristics
EBP Developers
Purveyors / Implementation Support
Inner Context
Interconnections
Interactions-Linkages-Relationships
Procurement-Contracting
Aarons et al. (2014)
Determinant & Process Framework Example: EPIS
Networks – Linkages - Collaborations
Why Consider Multiple Phases?
Characterizes process of implementation
Develops a way to think about what supports are needed during the implementation process
Helps in providing a “long-term view”
Helps in planning and conducting implementation
Aarons, G.A., Hurlburt, M. & Horwitz, S.M. (2011). Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors. Administration and Policy in Mental Health and Mental Health Services Research.38, 4-23.
EXPLORATION
OUTER CONTEXT
Sociopolitical Context
Legislation
Policies
Monitoring and review
Funding
Service grants
Research grants
Foundation grants
Continuity of funding
Client Advocacy
Consumer organizations
Interorganizational networks
Direct networking
Indirect networking
Professional organizations
Clearinghouses
Technical assistance centers
INNER CONTEXT
Organizational characteristics
Absorptive capacity
Knowledge/skills
Readiness for change
Receptive context
Culture
Climate
Leadership
Individual adopter characteristics
Values
Goals
Social Networks
Perceived need for change
PREPARATION
OUTER CONTEXT
Sociopolitical
Federal legislation
Local enactment
Definitions of “evidence”
Funding
Support tied to federal and
state policies
Client advocacy
National advocacy
Class action lawsuits
Interorganizational networks
Organizational linkages
Leadership ties
Information transmission
Formal
Informal
INNER CONTEXT
Organizational characteristics
Size
Role specialization
Knowledge/skills/expertise
Values
Leadership
Culture embedding
Championing adoption
IMPLEMENTATION
OUTER CONTEXT
Sociopolitical
Legislative priorities
Administrative costs
Funding
Training
Sustained fiscal support
Contracting arrangements
Community based organizations.
Interorganizational networks
Professional associations
Cross-sector
Contractor associations
Information sharing
Cross discipline translation
Intervention developers
Engagement in implementation
Leadership
Cross level congruence
Effective leadership practices
INNER CONTEXT
Organizational Characteristics
Structure
Priorities/goals
Readiness for change
Receptive context
Culture/climate
Innovation-values fit
EBP structural fit
EBP ideological fit
Individual adopter characteristics
Demographics
Adaptability
Attitudes toward EBP
SUSTAINMENT
OUTER CONTEXT
Sociopolitical
Leadership
Policies
Federal initiatives
State initiatives
Local service system
Consent decrees
Funding
Fit with existing service funds
Cost absorptive capacity
Workforce stability impacts
Public-academic collaboration
Ongoing positive relationships
Valuing multiple perspectives
INNER CONTEXT
Organizational characteristics
Leadership
Embedded EBP culture
Critical mass of EBP provision
Social network support
Fidelity monitoring/support
EBP Role clarity
Fidelity support system
Supportive coaching
Staffing
Staff selection criteria
Validated selection procedures
Aarons, G.A., Hurlburt, M. & Horwitz, S.M. (2011). Advancing a Conceptual Model of Evidence-Based Practice Implementation in
Public Service Sectors. Administration and Policy in Mental Health and Mental Health Services Research.38, 4-23.
EPIS Model Adapted for Specific Context
(Child/Adolescent Mental Health) Based on Systematic Review
Novins, D.K., Green, A.E., Legha, R.K., & Aarons, G.A. (2013). Dissemination and Implementation of Evidence-Based Practices for Child and Adolescent Mental Health: A Systematic Review. Journal of the American Academy of Child and Adolescent Psychiatry. 52(10), 1009-1025
Implementation Strategies
Address specific factors identified in implementation frameworks
Discrete implementation strategies – Clinical reminders, training only
Multifaceted implementation strategies– Training + reminders
– Training + fidelity monitoring + coaching
Blended implementation strategies (comprehensive)– Community Development Team strategy (CDT)
– Interagency Collaborative Team strategy (ICT)
– Dynamic Adaptation Process strategy (DAP)
– Leadership and Organizational Change for
Implementation (LOCI)
Source: Powell , McMillen, Proctor et al (2011). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review, 69(2) 123-157.
Domains of StrategiesType of Strategy Description Context Level N
Planning Info gathering, leadership, relationships Outer/Inner n=17
Education Training, materials, influence
stakeholders
Inner/Outer n=16
Financing Incentives, financial support Inner/Outer n=9
Restructuring Change roles, create teams, alter record
systems, create relationships
Inner/Outer n=7
Quality
Management
MIS + feedback, clinical reminders,
decision support, PDSA cycles
Inner/Outer n=16
Policy Change Licensure, accreditation, certification,
mandates
Outer/Inner n=3
Source: Powell , McMillen, Proctor et al (2011). A compilation of strategies for implementing clinical innovations in health and mental health. Medical Care Research and Review, 69(2) 123-157.
Aligning leadership
22
Leadership for implementation
– Full range leadership
– Implementation Leadership
Climate embedding mechanisms
– Primary
– Secondary
Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning leadership across systems and organizations to develop a strategic climate for evidence-based practice implementation. Annual Review of Public Health, 35, 255-274.
Change Oriented
Leadership
Transformational Leadership
23
Change Oriented
Leadership
Transactional Leadership
24
Non-Leadership
Passive-Avoidant Leadership
25
Full Range Leadership Model
Act i
ve
Pass
I
ve
Less Effective
More Effective
Passive-Avoidant
Transformational
Transactional
26
Implementation Focused
Implementation Leadership
27
Aarons, G.A., Ehrhart, M.G., & Farahnak, L.R. (2014). The Implementation Leadership Scale (ILS): Development of a Brief Measure of Unit Level Implementation Leadership. Implementation Science, 9(1) 45.
Implementation Leadership
The Implementation Leadership Scale– Identifies specific behaviors that leaders may enact to support and engage
followers in the larger goal of successful EBP implementation
– Focused on leader behaviors related to climate embedding mechanisms to promote strategic climates
– 1. Proactive Leadership“Has a plan to address implementation of EBP”
– 2. Knowledgeable Leadership“Is able to answer staff questions about EBP”
– 3. Supportive Leadership“Recognizes and appreciates employee efforts toward successful implementation of EBP
– 4. Perseverant Leadership“Carries on through the challenges of implementing EBP”
– Implementation Leadership Total Scale Total α = .97, 12 total items
Aarons, G.A., Ehrhart, M.G., & Farahnak, L.R. (2014). The Implementation Leadership Scale (ILS): Development of a Brief Measure of Unit Level Implementation Leadership. Implementation Science, 9(1) 45.
28
29
Mean ILS Discrepancy
3.01
2.82
1.751.9
2.82.87
0
1
2
3
4
33% 35% 38%
Me
an
Im
ple
me
nta
tio
n L
ea
de
rsh
ip S
ca
le S
co
re
Percent of Sample in Each Category
Supervisor Clinican
Leader N=80; Clinician N=513; Standardized scores that differ > .5 SD considered discrepant values
Humble Leader
Aarons, G. A., Ehrhart, M. G., Torres, E. M., Finn, N. K., & Beidas, R. S. (2017). The Humble Leader: Association of Discrepancies in Leader and Follower Ratings of Implementation Leadership With Organizational Climate in Mental Health. Psychiatric Services, 68(2) 115-122.
30
ILS Discrepancy and Organizational Climate - Involvement
The curve along the line of incongruence (dotted line) is significant. Discrepancy between staff and supervisor ratings of ILS does influence Involvement. As scores become more discrepant, Involvement increases (a4 = 0.42, t = 3.468, p = 0.001).
Involvement climate is impacted by discrepancy differently depending on who is rating ILS more favorably (direction of discrepancy matters). Involvement is highest at the left corner, where staff rate ILS high and supervisors rate themselves low (a3 = -0.30, t = -3.154, p = 0.002)
Humble leadership predicts better climate for involvement-
Aarons, G. A., Ehrhart, M. G., Torres, E. M., Finn, N. K., & Beidas, R. S. (2017). The Humble Leader: Association of Discrepancies in Leader and Follower Ratings of Implementation Leadership With Organizational Climate in Mental Health. Psychiatric Services, 68(2) 115-122. 31
ILS Discrepancy and Organizational Climate - Performance
Feedback
The curve along the line of incongruence (dotted line) is significant, meaning discrepancy between staff and supervisor ratings of ILS does influence Performance Feedback. As scores become more discrepant, Performance Feedback increases (a4 = 0.47, t = 6.289, p = 0.000)
Performance Feedback is impacted by discrepancy differently depending on who is rating ILS more favorably (direction of discrepancy matters). Performance Feedback is highest when staff rate ILS high and supervisors rate themselves low (a3 = -0.37, t = -5.147, p = 0.000)
Humble leadership predicts better climate for performance feedback-
Aarons, G. A., Ehrhart, M. G., Torres, E. M., Finn, N. K., & Beidas, R. S. (2017). The Humble Leader: Association of Discrepancies in Leader and Follower Ratings of Implementation Leadership With Organizational Climate in Mental Health. Psychiatric Services, 68(2) 115-122. 32
O2 SUSTAINMENT PROJECT(NIMH R01MH072961)
Mixed-methods examination of EBP sustainment factors in two states representing 87 counties:
– OK Children’s Services System
– 10 California Counties
ICT Project (NIMH)
SKCP/ADAPTS project– (ACF, CDC, HRSA) 33
Outer Context System Leadership Predicts EBI
Sustainment in 11 Service Systems
3.15 3.2
1.37
0
0.5
1
1.5
2
2.5
3
3.5
4
Leadership Competence
Outer Context
Full Sustainment Partial Sustainment Non-Sustainment
*
Aarons, G. A., Green, A. E., Trott, E., Willging, C. E., Torres, E. M., Ehrhart, M. G., & Roesch, S. C. (2016). The roles of system and organizational leadership in system-wide evidence-based intervention sustainment: A mixed-method study. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 991-1008.
34
Inner Context Team Leadership Predicts EBI
Sustainment in 11 Service Systems
2.73
2
0.57
2.7
2.28
0.58
2.242.11 2.18
0
0.5
1
1.5
2
2.5
3
3.5
4
Transformational Leadership Transactional Leadership Passive-Avoidant Leadership
Inner Context
Full Sustainment Partial Sustainment Non-Sustainment
**
Aarons, G. A., Green, A. E., Trott, E., Willging, C. E., Torres, E. M., Ehrhart, M. G., & Roesch, S. C. (2016). The roles of system and organizational leadership in system-wide evidence-based intervention sustainment: A mixed-method study. Administration and Policy in Mental Health and Mental Health Services Research, 43(6), 991-1008.
35
Leadership and Organizational Change for
Implementation (LOCI)
Pilot Study National Institute of Mental Health
NIMH R21MH082731
– Developed and tested in mental health programs
– Small scale (12 programs – 6 LOCI vs. 6 Webinar)
National Institute on Drug Abuse
NIDA R01DA038466
– Large scale
– 60 programs (30 LOCI – 30 Webinar)
36
LOCI Pilot Study OutcomesLOCI (n=5) Compared to Control Condition (n=6)
Feasibility– Engagement in leadership training (t=2.39, p<.05)
– Increased leadership knowledge (t=6.43, p<.001)
Acceptability– Applied what was learned (t=4.80, p<.001)
– Improved leadership behaviors (t=4.88, p<.001)
– Ability to manage change (t=4.54, p<.01)
– Change in behavioral routines (t=4.94, p<.01)
– Changed leadership behaviors (t=4.97, p<.01)
– Increased emphasis on EBP with subordinates (t=4.11, p<.01)
Perceived Utility– Greater overall utility (t=7.20,p<.001)
– Higher utility in managing organizational change (t=5.72, p<.001)
– Greater utility implementing/using EBPs in team (t=4.84, p<.001)
Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Hurlburt, M. S. (2015). Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science, 10(1), 11. 37
0
1
2
3
4
Baseline 3 Month 6 Month
LOCI (n=41)
Webinar (n=59)
Subord
inate
Rate
d L
eader
Support
for
Score
Assessment Time
LOCI Pilot Study
Improved Supportive Leadership
Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Hurlburt, M. S. (2015). Leadership and organizational change for implementation (LOCI): a randomized mixed method pilot study of a leadership and organization development intervention for evidence-based practice implementation. Implementation Science, 10(1), 11.
38
Leadership and Organizational Change for Implementation
R01DA038466
Developing leaders for EBP implementation and sustainment– http://implementationleadership.com/index.html
First level alone not sufficient– Strategically involve other levels in a way that minimizes
organizational stress and increases implementation climate development
Intervention needs consideration of timing of:
– Leadership development activitiesDidactic TrainingOngoing very brief coachingCollaboration
– Implementation climate development activities
– Linking organizational and multilevel leader buy in strategy with leader development
Aarons, G. A., Ehrhart, M. G., Moullin, J. C., Torres, E. M., & Green, A. E. (2017). Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: a cluster randomized trial study protocol. Implementation Science, 12(1), 29. 39
Overview of LOCI
Developing leaders for EBP implementation takes time and ongoing support– Two conditions
LOCI leader training + coaching + org strategy
Webinar leader training
Intervention needs consideration of timing of:– Leadership development activities
– Implementation climate development activities
– Linking organizational and multilevel leader buy in
strategy with leader development
Strategic alignment
across all levels
On Target =
Implementation Success
ExecutivesCounselors Supervisors
40
Organizational Context for Implementation
Enhance general leadership and “implementation
leadership”
Align leadership to develop strategic climate for
implementation
Support congruence of leadership across
organizational levels
Schein’s (2010) culture/climate “embedding
mechanisms”Aarons, G.A., Ehrhart, M.G., & Farahnak, L.R. (2014). The Implementation Leadership Scale (ILS): Development of a Brief Measure
of Unit Level Implementation Leadership. Implementation Science, 9:45.
Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning Leadership Across Systems and Organizations to
Develop a Strategic Climate for Evidence-Based Practice Implementation. Annual review of public health, 35, 255-274.
Ehrhart, M.G., Aarons, G.A., & Farahnak, L.R. (2014). Assessing the Organizational Context for EBP Implementation: The
Development and Validity Testing of the Implementation Climate Scale (ICS). 9:157
Jacobs, S. R., Weiner, B. J., & Bunger, A. C. (2014). Context matters: measuring implementation climate among individuals and
groups. Implementation Science, 9(1), 46.
Schein, E. H. (2010). Organizational culture and leadership (4th Ed). John Wiley & Sons. 41
Primary Climate Embedding Mechanisms
What leaders can do that can be applied at the system,
organization, team or workgroup level
– What leaders pay attention to, measure, control
– How leaders react to critical incidents, crises
– How leaders allocate resources
– Role modeling, teaching, coaching
– How leaders allocate rewards and status
– How leaders recruit, select, promote, excommunicate
Aarons, G. A., Ehrhart, M. G., Farahnak, L. R., & Sklar, M. (2014). Aligning Leadership Across Systems and Organizations to Develop a Strategic Climate for Evidence-Based Practice Implementation. Annual review of public health, 35, 255-274.
Schein, E. H. (2010). Organizational culture and leadership (4th Ed). John Wiley & Sons. 42
LOCI Leadership Program
LOCI Condition
(12-months)
Didactic Leadership
Training
Leadership Coaching
Data Driven 360-degree
Assessments
Organizational Strategy
4 Org. Strategy Meetings (OSMs)
Monthly OSM Check-in calls
Agency wrap-up to conclude project @
16-months
Strategies in place for sustained EBP
2-day in-person LOCI leadership training
Two 1-day in-person leadership booster trainings
Graduation
Weekly one-on-one coaching calls with LOCI trainer
Monthly group collaborative call with other LOCI leaders
5 web-based surveys assessing climate & leadership
Completed by providers, supervisors, & executives)
Occurs at baseline, 4-, 8-, 12-, & 16-months
Aarons, G. A., Ehrhart, M. G., Moullin, J. C., Torres, E. M., & Green, A. E. (2017). Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: a cluster randomized trial study protocol. Implementation Science, 12(1), 29.
43
Where to from Here?
Develop implementation strategies tailored to specific
contexts
Consider at what levels (system, organization, provider,
client) strategies should be aimed and coordinated
Consider phases (e.g., readiness) to tailor D or I
strategies
Requires empathy and understanding of challenges and
motivations of systems, organizations, providers,
patients
Identify metrics to determine success (or not) of
dissemination/implementation efforts
Contact
Gregory Aarons, Ph.D.
Department of Psychiatry
University of California, San Diego
9500 Gilman Drive (0812)
La Jolla, CA 92093-0812
e-mail:
Web:
http://psychiatry.ucsd.edu/faculty/gaarons.html
http://ImplementationLeadership.com