- -

Report prepared by Jane Collins & Kristy Hoyak, BC Mental Health & Addiction Services

May 2013

1

Final Evaluation Report

Table of Contents Section 1: Introduction ...................................................................................................................... 3

Purpose of the Evaluation, Period Covered ................................................................................... 3

Who conducted the Evaluation (i.e. internal or third party) ......................................................... 3

Who was involved (e.g. P/Ts, NGOs, stakeholders) ....................................................................... 3

Section 2: Project Description ............................................................................................................ 3

Project Goal .................................................................................................................................... 3

Project Objectives .......................................................................................................................... 4

Target Population .......................................................................................................................... 5

Activities ......................................................................................................................................... 5

Outputs .......................................................................................................................................... 6

Expected Outcomes (including links to DTFP Program Outcomes) ............................................... 6

Operating Context (i.e. contextual or project delivery factors relevant to implementation)....... 7

Project Management/Governance Structure & Administration ................................................... 8

Project Stakeholder Relationships ................................................................................................. 9

Project Logic Model: BC DTFP Strengthening the Substance Abuse Treatment System ............ 10

Section 3: Evaluation Scope and Methods ...................................................................................... 11

Evaluation Scope .......................................................................................................................... 11

Evaluation Issues, Questions and Indicators ............................................................................... 11

Data Collection Procedures (Append tools, description of data sources) ................................... 11

Analytical Procedures for both Qualitative and Quantitative Information ................................. 12

Process Used to Arrive at Conclusions and Recommendations (who was involved and how) ... 12

Methodological Limitations ......................................................................................................... 12

Section 4: Interim Evaluation Findings ............................................................................................ 13

Was the project implemented as intended? ............................................................................... 13

Were the expected activities undertaken and outputs delivered? ............................................. 13

Activities ....................................................................................................................................... 14

Contributions ............................................................................................................................... 14

What course corrections were made, why, and what is the impact on the project’s ability to reach its expected outcomes? ..................................................................................................... 15

What challenges were encountered and how were they addressed? ........................................ 15

What worked particularly well? ................................................................................................... 16

What could have been improved? ............................................................................................... 18

What are the key lessons from this project? ............................................................................... 18

To what extent was progress made toward the project’s expected outcomes? ........................ 18

What changes are recommended to help ensure the project reaches its expected outcomes? 18

Section 5: Final Evaluation Findings ................................................................................................. 20

Success Story: Cultivating Communities of Practice .................................................................... 23

Success Story: DTFP Emergent Outcomes – DTFP as a catalyst for collaboration / protocol / service development .................................................................................................................... 27

Success Story: Enhanced KE Links to Aboriginal Communities.................................................... 29

2

Success Story: Workforce Development – Core Addictions Practice Series ............................... 33

Success Story: KE to Support the Development and Implementation of Residential Standards 35

Success Story: Expanding Evaluative Capacity to Address Stigma through Compassion, Inclusion and Engagement .......................................................................................................................... 37

Success Story: Evaluation of KE Network Infrastructure ............................................................. 40

Success Story: Whole Systems Approach .................................................................................... 42

Success Story: Enhancing Links to Primary Care .......................................................................... 53

Success Story: Investment in Champions .................................................................................... 54

Section 6: Project Lessons ................................................................................................................ 56

What are the key lessons from this project? ............................................................................... 56

Section 7: Conclusion and Recommendations ................................................................................ 59

Conclusion Statement of Project Findings and how these findings have contributed to DTFP Outcomes ..................................................................................................................................... 59

Recommendations for the Project Sponsor ................................................................................ 60

Health Canada .............................................................................................................................. 60

Ministry of Health - British Columbia .......................................................................................... 62

Recommendations for similar projects ........................................................................................ 62

Recommendations for the DTFP .................................................................................................. 63

Section 8: List of References ............................................................................................................ 64

Section 9: Appendices ...................................................................................................................... 64

3

Purpose of the Evaluation, Period Covered

This evaluation has been conducted to support:

Health Canada funding reporting requirements

Reflection and shared learning in BC context

Celebration and communication of significant contributions

Ongoing planning

Time period covered in this evaluation: Final Evaluation interviews and focus group data were gathered January – March 2013. Data synthesized in this final report includes ongoing evaluation data collected from April 2009 to March 2013 inclusive.

Who conducted the Evaluation (i.e. internal or third party)

This evaluation has been conducted by an internal evaluation team led by Jane Collins and Kristy Hoyak from BC Mental Health and Addiction Services with external evaluation consultation provided by Michael Quinn Patton and Terry Smutylo.

Who was involved (e.g. P/Ts, NGOs, stakeholders)

This evaluation includes input from the following stakeholder groups:

BC’s DTFP Addiction Knowledge Exchange Team (including health authority Co-Leads)

BC Substance Use Network

Health Authority Leadership

Ministry of Health Leadership

Service providers, managers, and leadership from the following sectors: substance use services, mental health services, Aboriginal services, primary care, family services, not-for-profit and non-governmental organizations, academic research, external consultants, justice/corrections/RCMP, primary care, public health, acute care, education

Clients and families

Community of Practice members

Project Goal

BC DTFP Strengthening the Substance Abuse Treatment System Goal: To build an addiction KE infrastructure that facilitates the implementation and support of evidence-informed practice and KE activities, and fosters linkages across the system of substance use services and supports in BC.

4

Project Objectives

The objectives of the initiative are to (1) facilitate the implementation and support of evidence-informed practice, and (2) foster linkages and exchange and support KE activities.

Key Concepts and Definitions

BC DTFP Strengthening Systems Initiative is informed conceptually by an evolving recognition, respect and support for:

Appreciative approach with a sustained focus on cross sector relationship-building

Contribution rather than attribution lens

A whole systems approach using open, transparent, collaborative processes

Non-linear processes that support holistic complexity and reduce tendency to focus on predetermined and tightly defined methods and models

The importance of the interplay between KE mechanisms, culture and context

Dynamical systems and complexity theory

Utilization-focused evaluative approach using adaptive, intentional cycles of planning, doing, reflecting, learning, and evaluating that are reciprocal, and not necessarily sequential in nature

Figure 1 illustrates the knowledge continuum. In this initiative, the KE approaches used provide the space and skilled facilitation to promote shared meaning making of evidence-informed practice.

The focus of our work is to support connections, meaning making and understanding.

Figure 1: Knowledge Continuum (Achterbergh, Jan & Vriens, Dirk (May-June 2002). Managing viable knowledge. Systems Research and Behavioral Science, 19(3), p223)

Knowledge exchange (KE) is generally understood as the co-creation of situations, conditions, and ecologies that support a culture of shared learning through knowledge creation, translation, dissemination, uptake, and evaluation.

5

KE infrastructure is the tangible, tactile, tools, structures, scaffolding, etc. It is the space to do the work, and the people to move things forward.

KE mechanisms are the conceptual approaches guiding the work, which includes collaborations, frameworks, and approaches that are developed to inform decisions and processes.

Target Population

The scope of this initiative is to develop an Addiction KE infrastructure to support ongoing system improvements to the substance abuse treatment system in BC for different populations (youth, families, adults, etc.). This systems initiative’s initial primary target group was substance use system providers and decision makers across BC. This target group expanded to reflect a whole systems approach and engage service providers, decision makers across sectors, settings and tiers.

Activities

Key Project Activities

Development of core components of KE infrastructure to support evidence-informed practice

Development of KE mechanisms to support facilitation of linkage and exchange

The DTFP supported activities are intentional and interconnected with learning shared across contexts to foster connections across substance use services and supports, tiers, settings and sectors.

This initiative is conceptually guided using two interconnected KE streams. One stream focuses on workforce development capacity; the other stream focuses on system level standards, frameworks and protocol development. The streams support and strengthen the system of substance use services and supports across six priority focus areas. The priority areas are linked to strategic objectives across British Columbia’s Health Authorities and linked to BC’s ten-year plan1.

BC's DTFP Addiction Knowledge Exchange Priority Areas

KE to support evidence informed practice (e.g., Core Addiction Practice, Motivational Dialogue/Interviewing, Feedback Informed Treatment, Trauma Informed Practice)

KE to support National Treatment Strategy alignment

KE to address stigma: building compassionate, inclusive, welcoming systems of care

KE to enhance capacity to engage and support youth, families, and caregivers of youth with substance use and mental health concerns

KE to support links to primary care

KE to support development and implementation of youth and adult residential substance use service standards

1 Healthy Minds, Healthy People – A ten year plan to address mental health and substance use in BC Ministry of Health

Services, Ministry of Children and Family Development, November 1, 2010

6

Outputs

Knowledge exchange and change management tools, materials, and plans to enhance the KE infrastructure across the system of services to support collaborative planning, shared learning, evaluation, and the uptake of evidence-informed practice including:

Tools and frameworks for the development, monitoring and evaluation of strategic change initiatives

Tools to support linkage and exchange with Aboriginal services

Surveys, scans, briefings, reviews and proposals

Workshops and knowledge exchange forums build workforce capacity infrastructure for practice- and system-level change across the priority areas of focus. The workshops and KE forums foster renewal and regeneration, and respect context, local knowledge and wisdom. The workshops and KE forums are often a port of entry and catalyst for cross sector engagement supporting a common knowledge base and creating space for reflections, interpretation, and evaluation of practice.

Curricula and toolkits support practice- and system-level change and are designed to enhance availability, understanding and uptake of evidence-informed practice and facilitate linkages across sectors and regions.

KE mechanisms including Communities of Practice, collaborative protocols, service frameworks and standards, and structured system consultation and planning sessions support knowledge sharing and the development of collective knowledge for those providing services at the point of care, and for those involved in system management and planning. KE mechanisms foster adaptive responses and the spread of innovation.

Expected Outcomes (including links to DTFP Program Outcomes)

Immediate Outcomes

HC DTFP Project IMMEDIATE Outcome: Enhanced commitment to affect system change in DTFP treatment system investment areas

BC Project IMMEDIATE Outcomes

Enhanced KE infrastructure, and tools to facilitate/support evidence-informed practice

Enhanced KE mechanisms to facilitate/support evidence-informed practice to enhance stakeholder commitment and engagement

HC DTFP Project IMMEDIATE Outcome: Enhanced collaboration on responses to DTFP treatment systems’ issues within and among jurisdictions and stakeholders

BC Project IMMEDIATE Outcomes

Enhanced collaboration and understanding on how to implement and sustain evidence-informed practice using KE processes and structures

Intermediate Outcomes

HC DTFP Project INTERMEDIATE Outcome: Increased access to evidence informed practice information

7

BC DTFP Project INTERMEDIATE Outcome

Enhanced availability, understanding, and uptake of evidence-informed practice information

HC DTFP Project INTERMEDIATE Outcome: Increased Province/Territory (PT) capacity to evaluate substance abuse treatment systems performance

BC DTFP Project INTERMEDIATE Outcome

Increased evaluation capacity to plan, monitor and evaluate system change initiatives.

Long Term Outcomes

HC & BC DTFP Project Long Term Outcome: Strengthened evidence-informed substance abuse treatment systems

Operating Context (i.e. contextual or project delivery factors relevant to implementation)

In BC, we do not have one system of addiction services; we have a networked system, with specialized substance use services and supports located across a variety of structures regionally and provincially. These systems often cross organizational, regional, ministerial and jurisdictional boundaries. The majority of health services are provided by five regional and one provincial health authority (depicted in Figure 2)

Figure 2: BC's Health Authorities

The following contextual factors influenced the BC DTFP project plans and approaches:

System reorganizations involving substance use services occurred in five of the six health authorities over the course of the DTFP initiative. The reorganizations and funding cuts contributed to significant moral distress in the workforce. The reorganizations and moral distress in the workforce influenced project plans and approaches used. The initiative was launched mindful of local contexts, with an

8

appreciative lens to use KE approaches to support relationship-building and address moral distress. This approach was successful and increased the profile and legitimacy of substance use services within the re-organized structures. The reorganizations also contributed to shifting DTFP project staffing with associated requirements for additional orientation/training. In some regions the shift in project staffing opened possibilities.

Funding Uncertainty both within and across fiscal years has been the norm for this initiative and has contributed to BC’s Addiction KE team developing adaptive planning processes and approaches to proactively adjust plans in light of funding variability. These planning processes are not typical for BC’s Health Authorities where funding is set at the start of each fiscal period. The within-fiscal expanded funding in 2010/11 and 2011/12 contributed to accelerated project plans, however, delays in notification of the expanded funding put pressure on the treatment system with accelerated activities typically compressed into Q4 of each fiscal period.

Delay in 2013/14 DTFP project expansion contributed to losing project staff and reduced project plans and approaches particularly in Northern Health.

BC Teacher Federation Strike 2011/12 contributed to adjusted project plans to engage the education sector.

Project Management/Governance Structure & Administration

Leadership of BC’s DTFP Addiction KE Initiative

The depth and breadth of in-kind leadership support for BC’s DTFP initiative is remarkable, including administrative and support services (human resources, finance, contracting, clinical and administrative leadership), across treatment and service settings and sectors. BC Mental Health and Addiction Services, an agency of the Provincial Health Services Authority, is overseeing the provincial implementation of the DTFP through a letter of agreement between the Ministry of Health and BC Mental Health and Addiction Services.

BC Mental Health and Addiction Services developed a collective leadership team to oversee the implementation of the DTFP including:

Central Team: Shannon Griffin, Project Executive Lead; Program Manager, Jane Collins; Data Analyst, Kristy Hoyak; Aboriginal KE Leader, Kat Hinter; Administrative Assistant, Corinne Newell

Fraser Health Authority: Addiction KE Leader, Marika Sandrelli; Co-lead, Sherry Mumford

Vancouver Coastal Health Authority: Addiction KE Leader, Mary Marlow; Co-lead, Yasmin Jetha

Interior Health Authority: Addiction KE Leader, Tara Mochizuki; Co-leads, Cliff Cross, Rae Samson

Northern Health Authority: Addiction KE Leader, Tammy Marleau; Co-lead, Jim Campbell

Vancouver Island Health Authority: Addiction KE Leader, Paula Beltgens (Youth)/Chris Goble (Adult); Co-leads, Michelle Dartnall, Janet James, Jan Dorland

Ministry of Health: Ministry Lead on project, Emily Arthur

The health authority co-leads are mental health and substance use senior operational/strategic leaders from each of the Health Authorities. The co-management team (Addiction KE Co-Leads) are members

9

of the Provincial BC Substance Use Network (BC SUN) and the majority are also members of the Provincial Mental Health and Substance Use Planning Council. Co-management support is provided in kind by the Health Authorities. The health authority co-leads are involved in all aspects of the project, collaborative planning, professional capacity building, implementation, and ongoing support.

There is a matrix reporting structure for the Addiction KE Leaders who report to the designated health authority co-lead and the provincial Addiction KE Team Manager.

The DTFP team has developed a collaborative, open, transparent team culture and structures to support the work incorporating reflective processes to review and adapt structures and process to support functions.

Financial Controls to spend within 1% variance

The team adapted processes and structures for financial controls to ensure the initiative financial management is aligned with Health Canada 1% variance requirements. As with all team processes and structures the team engages in reflective practice to evaluate how the processes are working with the goal to improve and streamline processes. The adapted processes and structures have been established in collaboration across health authorities and the Ministry of Health, involving multiple financial systems with different financial calendars. The team incorporated financial reporting into weekly team check-ins, and team meetings. In addition, health authorities invoice monthly for DTFP-labour and non-labour costs.

Planning Cycle and Priority setting

As outlined in the proposal the initial planning cycle was to conduct one annual review and planning session in March with new priorities set annually. This original plan reflects the traditional annual planning cycles in place in health authorities that are tied to fiscal-year end and focused on outputs rather than longer term outcomes. This traditional planning cycle does not fit the reality of this DTFP systems initiative. The DTFP Addiction KE team, in consultation with Provincial BC SUN and the Provincial Mental Health and Substance Use Planning Council, agreed to priority areas of focus over multiple years and have developed an ongoing adaptive planning cycle with planning processes incorporated into team check-ins meetings and Provincial BC SUN meetings on an ongoing basis.

BC’s DTFP key priority areas of focus were linked to multi-year strategic objectives across British Columbia’s health authorities and linked to BC’s 10-year plan. Work under each priority area is contextualized to regional and local contexts with materials, resources and learning shared across health authorities.

Project Stakeholder Relationships

This systems initiative’s primary target group was substance use system providers and decision makers across BC. This target group expanded to reflect a whole systems approach and engage service providers, decision makers across sectors, settings and tiers. Developing and sustaining relationships is central to this work and KE appreciative approaches support trust, safety and open the possibility for collective action.

10

Project Logic Model: BC DTFP Strengthening the Substance Abuse Treatment System Legend: Project-level DTFP-level

Goal

Objectives

Target group

Activities

Project Immediate

Outcomes

Outputs

Project Intermediate

Outcomes

Long Term

Outcomes

Increased access to evidence-informed practice information

Implementation of Evidence-Informed Practice To facilitate the implementation and support of evidence-informed practice using a change

management framework

Enhanced collaboration on responses to DTFP treatment systems’ issues within and among

jurisdictions and stakeholders

Enhanced collaboration and understanding on how to implement and sustain evidence-informed practice using KE processes and structures

Strengthened evidence-informed substance abuse treatment systems

Development of core components of KE infrastructure to support evidence-informed practice

Facilitation of Linkage and Exchange To foster linkages and exchange and support KE activities using a change management

framework

Development of KE mechanisms to support facilitation of linkage and exchange

Increased understanding of effective treatment systems’ performance

Enhanced availability, understanding, and uptake of evidence-informed practice information

To build a knowledge exchange (KE) infrastructure that facilitates the implementation and support of evidence-informed practice and knowledge exchange activities, and fosters linkages across the substance abuse treatment system in BC

KE and change management tools, materials and plans Workshops CoPs Curricula, toolkits

Collaborative protocols System consultations and planning sessions

Standards / service frameworks

Enhanced provincial and territorial commitments to affect system change in DTFP treatment systems’ investment areas

Substance abuse treatment system providers and decision makers across BC

Enhanced KE mechanisms to facilitate/support evidence-informed practice to enhance stakeholder

commitment and engagement Enhanced KE infrastructure and tools to facilitate/support evidence-informed practice

11

Evaluation Scope

Final evaluation includes the following data sources:

Final Evaluation interviews and focus group data, gathered January – March 2013

Review and synthesis of evaluation data collected from April 2009 – March 2013 inclusive

DTFP Addiction KE team and BC Substance Use Network members’ reflections

Evaluation Issues, Questions and Indicators

Evaluation has been integrated into the day-to-day business of BC’s DTFP initiative with ongoing evaluative activities supporting intentional cycles of planning, doing, reflecting, learning, and evaluating that are reciprocal (and not necessarily sequential) in nature. The project has supported evaluative capacity development beyond individuals in traditional evaluator/decision support roles. The positive side of this approach is the expansion of evaluative systems thinking, the co-creation/adaptation of creative evaluative tools, mechanisms and processes with the multiple actors across multiple sectors and growth in distributed leadership. The data management and evaluation requirements are substantial. This approach would not be possible or recommended without a central core group of individuals on the ground in this initiative with substance use service content knowledge, appreciation for complexity, systems thinking and evaluation expertise, combined with periodic consultation with leading edge external evaluators.

One of the central evaluation issues in the DTFP initiative nationally has been the narrow conceptualization of KE and linear logic model inherent in Health Canada’s DTFP evaluation framework. The conceptualization of KE reflects the dominate view at the time of the DTFP call that equated information dissemination with KE, where change was facilitated by getting the right information to the right people at the right time. This conceptualization of KE is limited and not supported by the growing body of practice evidence. The linear logic model and attribution lens inherent in Health Canada’s DTFP evaluation framework is a mismatch with the reality and complexity inherent in system-level change initiatives that involve a confluence of actors and agents across multiple settings, tiers and sectors contributing to system change.

The BC DTFP team recognized these evaluation issues early and chose to incorporate utilization-focused evaluation methodologies, and to support system capacity to be adaptive and responsive. The evaluation activities and approaches used exceed Health Canada DTFP reporting requirements.

Data Collection Procedures (Append tools, description of data sources)

Data and analysis are drawn from mixed methods, reflecting the multifaceted scope of this initiative. The data set covering the period April 2009 – March 2013 includes (see Appendix A for samples):

Appreciative Inquiry with residential substance use service providers

KE forum/workshop participant surveys

12

Community of Practice surveys and reflections

Interviews and focus groups conducted across priority areas, regions, sectors, settings, and organizational levels to gather perspectives about how the DTFP has contributed to strengthening the system of care (e.g. Most Significant Change Interview schedule)

Interviews and focus groups conducted across priority areas, regions, sectors, settings and organizational levels to gather perspectives about lessons learned, key “a-ha” moments about the potential of KE, and recommendations for the province, the funder and other jurisdictions (e.g. Final Evaluation interview questions)

Focus groups with Motivational Dialogue practice champions and Core Addiction Practice (CAP) Facilitators to deepen our understanding of what it means to be a practice champion

Project document review (including ongoing project summaries, narratives, and change maps)

Analytical Procedures for both Qualitative and Quantitative Information

Microsoft Excel and/or SPSS programs were used to analyse quantitative data. Quantitative data collected via online survey instruments was exported from online platforms directly into Excel and/or SPSS for analysis. Quantitative data collected via paper-based instruments was manually entered into Microsoft Excel and/or SPSS for analysis.

Qualitative data collected throughout the initiative (interviews, focus groups, responses to open-ended survey questions) were transcribed, coded, and themed by members of the core evaluation group. Themed responses were reflected back to the broader DTFP team and stakeholders involved (interviewees, focus group members, and/or survey respondents) for reflection, confirmation and consolidation. Analysis of this rich and varied data set supports interpretation and communication of findings that illustrate the complexity of the context and circumstances of the initiative. Progress towards outcomes data are contextualized with narrative progress updates to describe the context of the contributions of BC’s DTFP initiative to strengthening the substance use treatment system.

Process Used to Arrive at Conclusions and Recommendations (who was involved and how)

Multiple perspectives were gathered to support conclusions and recommendations including interviews and focus groups with stakeholders across settings, sectors, tiers and organizational levels (first responders, direct service staff, management, and senior management). The BC Substance Use Network functioned as a system level consultation group for data interpretation and recommendations.

Methodological Limitations

Methodological limitations include:

Limitations to situational analysis (baseline): based on provincial surveys, scans and gap analyses reflect information available at the times the respective projects were conducted. The situational analysis base reports reflect parts of the addiction treatment system and must be considered within the broader continuum of addiction services in BC. While each of these separate planning

13

processes involved a range of stakeholders, there was not broad-based stakeholder engagement for the entire situational analysis.

Limitations to progress towards outcomes data interpretation: small cross-sector sample sizes for qualitative data; variability in health authority evaluative capacity; variability in evaluative tools (both a limitation and strength).

Limitations to sector reporting mechanisms: our primary target group for addiction KE is substance use service providers and decision makers across BC. With our increased understanding of how to support a whole systems approach to addiction KE, our sector engagement has grown to include clients, families and service providers across settings, sectors and tiers. The net result is both an under-report of sector involvement and an expansion of cross-sector engagement.

BC’s DTFP Strengthening Systems Initiative comprehensive Interim Process Evaluation can be found in the Appendix B. As directed by Health Canada this section highlights changes/updates from the interim evaluation findings.

Was the project implemented as intended?

Yes, with regional variation in focus areas and scope as intended, and responsive to regional context. Since the interim evaluation one change we made to the implementation was to the evaluation steering committee structure. We created this structure to support evaluation capacity across health authority specialty evaluators to support system level initiatives. However it was recognized quickly that this structure would not support the intended function as the specialized evaluators across the health authorities were a mix of contract and health authority staff with different levels of expertise and very limited experience with systems level utilization evaluation. From a cost and efficiency perspective it quickly became untenable. We adapted our evaluation steering committee for this initiative to include core members of the Addiction KE team who had deep dish evaluation experience and champions involved in DTFP-supported initiatives across the health authorities.

Were the expected activities undertaken and outputs delivered?

The DTFP supported activities and outputs have exceeded expectations. The whole systems approach with an intentional and sustained focus on cross-sectorial relationship building has contributed to emergent opportunities, activities and outputs well beyond expected. Table 1 summarizes the activities and tools that have been developed and/or adapted under the BC DTFP initiative. It represents a condensing of the outputs and outcomes data reported for the period October 2010 – March 2013. The first column outlines the types of activities, tools, and mechanisms; the second provides a high-level overview of the contributions they have made.

14

Table 1: Summary of BC DTFP Activities / Outputs / Contributions

Activities Contributions

182 KE tools, materials and plans across all regions of the province. These include:

Tools and frameworks for the development, monitoring and evaluation of strategic change initiatives

Tools to support linkage and exchange with Aboriginal services

Surveys, scans, briefings, reviews and proposals

These tools have been developed to enhance the KE infrastructure across the system of services to support collaborative planning, shared learning, evaluation, and the uptake of evidence-informed practice.

32 KE mechanisms developed or enhanced, with province-wide reach, including:

8 Communities of Practice

2 service frameworks, and associated sets of service standards

19 collaborative protocols and policies

4 regional oversight / change teams / committees

Collectively, the Communities of Practice have brought together over 381 members from 19 sectors.

Collaborative development of service frameworks, protocols and policies align with a whole systems approach to strengthening the systems of substance use service in BC.

The implementation of evidence-informed service frameworks and standards is designed to enhance the quality of services and outcomes for clients.

308 workshops and KE forums across all regions, focused on:

Evidence-informed clinical practice

Building capacity for evaluation and monitoring

Engaging youth with concurrent disorders

Primary care and medically assisted substance use treatment

Building compassionate, inclusive, welcoming systems of care

The workshops have reached over 8698 service providers across the province from 27 different sectors.

The workshops help to develop relationships and build a shared knowledge base across sectors to advance inter-sectorial and interdisciplinary client care.

32 curricula and toolkits developed or enhanced across all regions, including:

Historical perspectives on and approaches to substance use services

Clinical Improvement Steering Committee toolkit

Motivational Dialogue with Aboriginal partners

The curricula and toolkits support practice and system-level change, and are designed to enhance availability, understanding and uptake of evidence-informed practice and facilitate linkages across sectors and regions.

Over 881 system consultations across all sectors focusing on:

Engaging youth and family

Enhancing linkage and exchange with Aboriginal service providers

Enhancing primary care linkages and exchange

Building workforce and evaluation capacity

Addressing stigma

The consultations support the goals of enhancing collaboration and increasing evidence-informed practice.

Feedback from participants indicates consultations are supporting the sharing of knowledge, leveraging lessons learned, building trust, and facilitating planning.

15

What course corrections were made, why, and what is the impact on the project’s ability to reach its expected outcomes?

BC’s DTFP team does not use traditional linear project or program management methodologies; rather, adaptive processes are used to support planning in the context of complex systems. As such, course corrections are the expected norm to support complex systems change and adaptation to changing contexts has potentiated outcomes.

What challenges were encountered and how were they addressed?

The interim process evaluation outlined a number of challenges that are systematic, and approaches to addressing these ongoing challenges continue to be adapted to the context since the interim process report. This includes complexity and initiative management, implementing a systems level initiative during times of dynamic system change, success bandwagon and scope creep (see Appendix B for details). Since the interim process evaluation additional challenges include:

Tensions between historical practices and adaptive systems thinking

Central to BC’s DTFP initiative has been a focus on adaptation, appreciation of local context and culture, and encouragement and support for ongoing reflective practices at multiple decision-making levels informed by multiple evaluative practices. This is counter to business as usual in substance use and mental health system improvement initiatives, which have demonstrated a reliance on top-down, command and control processes, and linear change management that often leave people feeling “done to” and morally distressed; reflective processes are not the norm and multiple change initiatives are “rolled out” in succession; little time (if any) is permitted for formative evaluation, or to stay the course to explore efficacy and appreciate the contributions, shifts and growth of initiatives. Longer term planning is victim to the pace of healthcare “change-mania”, with short fiscal timelines and a focus on outputs, not longer-term outcomes. Strategy is often deficit-based, and decisions are made using simplified gap analysis at the individual service provider level, rather than building on successes and employing a whole systems perspective.

Funding uncertainty

The main additional challenge has been the significant funding uncertainty related to the next 5-year DTFP renewal and the 2013/14 DTFP program extension. The funding uncertainty was discussed in an open and transparent manner throughout 2012/13 with consistent messaging of hope in the face of uncertainty, and encouragement across the province to plan for bridge financing should the 2013/14 program extension application be unsuccessful. The 2012/13 year was significantly stressful for all involved and the uncertainty contributed to two team staff members leaving the project. The added stress of multiple contingency plans and unrelenting uncertainty damaged relationships. This period was experienced as a disrespectful potential close to the initiative.

Varying capacity to incorporate evaluation data to shift processes

There is variability in the understanding and use of evaluation data to inform planning and adaptive processes. There was early resistance across the team to incorporate evaluative thinking, from “evaluation is not my thing so I’m not going to do it”; to “evaluation is everyone’s business, so I’ve hired an external evaluator”. A stealth foreground/background dance with evaluation was taken over

16

an 18-month learning curve. A core group of internal and external people with deep conceptual understanding of KE, complexity, strong evaluative capacity, and good writing/communication skills supported the capacity of others on the team by using data to highlight successes, and engaging in generative dialogue and reflective processes. This shifted the team to actively evaluating to learn as we go, and shape what we were doing next. Evaluation is central to our KE approach.

We have come a long way, and there is still a long way to go. There continues to be variability in the use of evaluative data to inform ongoing planning. This is reflective of a larger issue of limited evaluative capacity across health authorities and ministries to interpret and use evaluation data.

Ideological differences in conceptualization and approaches to substance use services and supports

In healthcare, the default approach is to simplify procedures for complex situations. In many health service delivery areas there are known underlying pathogens, and known procedures that can successfully eradicate those pathogens. However, in the field of substance use there is no pathogen to eradicate, to treat, or to prevent by the accurate application of a ‘best practice’ and a standardized approach that can be replicated independent of context. There is a shift from substance use ‘treatment’ services, to substance use services and supports in recognition of the absence of a pathogen and the stigma associated with the treatment ‘fix’. Substance use clinical work is inherently culturally and contextually dependent, with heavy reliance on clinical judgement, reasoning, and discretion. Access to authentic and valuable sources of knowledge expands practice and builds resilience to buffer dynamical system changes. This understanding is in opposition to the concept of a ‘best practice’.

Prior to the DTFP, the focus of initiatives has been the specialized services sector. DTFP initiatives have actively engaged in silo-busting to expand our knowledge base by identifying and connecting with the broader community of service providers across organizations, sectors, tiers and settings. This has informed our growing knowledge base of principled informed practice across settings; principles that are congruent with a mission-driven workforce.

From scope creeps and success bandwagon, to butterfly effects

Introducing KE methodologies and non-linear change management methodologies ignites an excitement to learn more. When people are encouraged and given the opportunity to break free of the confines of traditional linear change management methodologies and try something new – and it works – they want more and need help to go further. We do not have the staffing capacity to respond to expanding requests for coaching and guidance to do this work in addition to supporting our existing (ambitiously opportunistic) work. This leaves our partners wanting, and creates an ethical dilemma in how to engage others in this work while respecting and supporting developing capacity.

What worked particularly well?

The interim process evaluation outlined a number of processes that were working well, and that continue to work well, including a strengthening, renewal, and regeneration focus, empowering service providers as leaders (unleashing creativity in what is essentially a cast of thousands), being open and responsive to emergent opportunities, and evolving adaptive structures to support initiatives (please see Appendix B for details). The following additional processes worked particularly well:

17

Approaching the work as a collective effort with intentional, sustained attention to relationship building

Within and across Health Authorities the DTFP team has incorporated a collective approach to the work, engaging stakeholders from across sectors in collective efforts with the shared goal of strengthening systems of care for people in BC with substance use concerns. Relationship building is often seen as a precursor or step required to get to the “real” work. Rather than a step in the process, the DTFP team has intentionally focused on developing, nurturing and sustaining relationships as both the means and the end to strengthen systems of care. This has contributed to engaging and leveraging the creative spirit of a cast of thousands of service providers.

Incremental adaptive planning

The team has developed adaptive incremental planning processes with ongoing attention to contextual/situational analysis and built-in feedback mechanisms to adapt plans based on learning. This is not typical in healthcare planning where project plans tend to be predefined and “rolled out” across contexts. This project affirmed that there are a number of different factors and influences at play in complex systems. More significantly, this makes it less likely that repeating a set of steps will produce the same results as the first time, especially in different contexts. The intentional approach to starting small, planning using reflective processes, identifying shared purpose and vision, adapting and expanding/shifting plans based ongoing learning has worked well.

Demystifying evaluation and distributed evaluation capacity building

BC’s DTFP approach to evaluation has been to include diverse stakeholders early and often in evaluation capacity building, to support expanding evaluative thinking, to use evaluation findings to inform ongoing planning and to communicate progress towards outcomes regularly to stakeholders across BC. This approach has helped to demystify evaluation, highlight the central role of evaluation in BC’s DTFP initiative, expand evaluative planning processes and garner support and enthusiasm for DTFP supported initiatives. This has also contributed to the use of utilization-focused evaluation methodologies and Outcome Mapping to help plan, monitor and evaluate system change initiatives beyond the scope of the DTFP.

Contribution lens and appreciation for complexity and systems thinking

DTFP evaluative approaches use a contribution lens rather than an attribution lens to plan, monitor and evaluate initiatives. DTFP-supported initiatives operate within a complex system of networked substance use services and supports with multiple interdependent and divergent components.

As they are currently applied, the concepts of “attribution” and “impact” can limit the potential of programs to learn from evaluations… Research results improve peoples’ lives via long, busy, discontinuous pathways. Tracing the connections is at best unreliable and at worst impossible… When donors and recipients try to be accountable for achieving impact, they are severely limiting their potential for understanding how and why impact occurs. The drive to claim credit interferes with the creation of knowledge. (pp. 5-6, Earl, Carden, & Smutylo, 2001)

2

A contribution lens opens space for possibilities: more openness to support adaptation, leaving space for the unexpected, grounding the work and providing a more realistic humble evaluation of

2 Earl, S., Carden, F., & Smutylo, T. (2001). Outcome Mapping. Ottawa, ON: International Development Research Centre.

http://www.outcomemapping.ca/

18

contributions of the DTFP-supported initiatives to strengthen the system of substance use services and supports.

Conceptual shift in our approach to stigma – from variable to be manipulated to a social process

Stigma has been conceptualized in the BC DTFP approach as a complex social process that is dependent on the relationship between the specific discrediting attribute (substance use) and the specific social context; it is a product of a social situation rather than any specific individual. In this conceptualization, stigma is treated as a sensitizing concept3 and space has been created to explore the meaning of compassion, inclusion and engagement within and across sectors as an intervention in and of itself, as shared understanding is emerging.

What could have been improved?

Comments in this section are limited to administrative areas that could have been improved based on our experience since the interim evaluation report (Appendix B).

Reporting Requirements

DTFP/Health Canada Logic model and evaluation frameworks do not fit the complexity of our initiative. It would be recommended that Health Canada explore consultations to modify the evaluation framework to reflect complexity to support future initiatives.

Pace of the work

The pace of the work has at times been unreasonable both for individual team members and the system. Historically, substance use services and supports have been underfunded and undervalued. The DTFP-funding opportunity created an enthusiasm to collectively make the most of the opportunity. This enthusiasm within a mission-driven field, an appreciative KE approach that fueled optimism, and the compressed timelines due to funding delays, overall contributed to an often frantic pace to the work. Leadership support is required across BC to monitor a pace that supports strategic learning during a projects life cycle.

What are the key lessons from this project?

See Section 6 in this report for synthesis of key lessons from this project.

To what extent was progress made toward the project’s expected outcomes?

Progress towards outcomes has exceeded expectations. See Section 5 in this report for update on progress towards outcomes and success stories.

What changes are recommended to help ensure the project reaches its expected outcomes?

The recommendations from this evaluation are summarized below and highlight the areas for continued focus to support the ongoing success of this initiative to reach expected outcomes.

3 Sensitizing Concepts Definition: http://tinyurl.com/celpf62

Also see: Patton, M. Q. (2010). Developmental Evaluation. New York, NY: Guilford Press (pp. 146-9)

19

Continuing to involve leaders in planning, monitoring and evaluation on an ongoing basis is critical to developing and strengthening leadership support.

Ongoing project investment to manage the tensions between expectations and emergence requires on-the-fly decision-making and continual refinement of feedback loops and communication channels.

Ongoing project investment to maintain and strengthen when necessary collective planning will strengthen ongoing reflective practices and strengthen alignment with strategic priorities and operational plans.

Continuing to refine open decision-making processes to evaluate and respond to emergent needs collectively to mitigate risks and increase possibilities within and across health authorities.

Continuing to enhance and refine knowledge management systems (including web-based) to support the growth of the initiative. The evolution of the narrative project documentation represents a cultural shift from busy work to documenting the emergent story of the initiative.

Continuing to respect and hold social spaces to promote practice. KE takes place in social interactions. Recognizing social spaces where clinicians thrive fosters meaning and sense-making in practice. Traditional knowledge dissemination in healthcare counters this approach with hierarchical, dominant procedural knowledge. Relying solely on a ‘best practice’ mentality undermines the role that sharing of tacit knowledge can play in strengthening care.

Shifting activity focus to convergence/divergence mapping and reflection to support sustainability and legacy will provide the opportunity to consolidate learning across regions, to harness the relationships, and to develop KE activities, infrastructure, tools and mechanisms that will continue to support shifts locally, regionally and provincially.

20

Personal and organizational commitment to affect system change

“Boy, I have so much more now, instead of cocooning in my office to take care of myself so that I have enough energy for my clients, now I have so much more to offer to my whole workspace… to my colleagues, and I can support them … share the kind of inspiration that I receive here with them in the workplace” ~ Practice Champion

“I notice a deepening of knowledge, more excitement for the work, or a renewed sense of hope perhaps – a greater understanding of ways to practice. There may be a similar knowledge base, but what’s changed is that through their experience of talking with someone from another community they’ve found another way to deliver an intervention that has worked for them in the past. What I see from the team here is that they are consistently working with individuals, so they are consistently busy, so clients are coming back” ~ Manager, Mental Health and Substance Use Services

“We have an amazing group of service providers …an amazing set of mental health substance use directors and executive director. And there was almost no resistance whatsoever to anything we brought forward. I mean it was fully supported. And for service providers, it was ‘Wow, all these things that we've been hoping for and training for over the years’… So we had a cast of thousands literally in terms of getting the support for many of the initiatives.” ~ Director, Mental Health & Substance Use Services

HC DTFP Project IMMEDIATE Outcome: Enhanced commitment to affect system change in DTFP treatment system investment areas

BC Project Outcome: Enhanced KE infrastructure, and tools to facilitate / support evidence-informed practice Indicators: #/type of KE tools used and developed; perceptions of enhanced KE infrastructure

BEFORE THE DTFP: The situational analysis revealed limited structures and supports for KE within and across health authorities in BC. These were characterized by a lack of dedicated resources, and limited commitment and understanding of how to affect system change in the DTFP treatment system investment areas.

PROGRESS TOWARDS OUTCOMES4:

One hundred and eighty-two KE tools, materials and plans have been developed, adapted and/or refined to enhance the KE infrastructure across the system of services to support collaborative planning, shared learning, evaluation, and the update of evidence-informed practice.

The 182 KE tools, materials and plans conceptually group into the following types:

107/KE tools, methods for the development, monitoring and evaluation of strategic change initiatives

18/KE tools to support linkage and exchange with Aboriginal services

57/KE scans, briefings, proposals and reviews

Figure 3: Example of KE tool to support planning, monitoring and evaluation: Change Management Map: Aboriginal Linkages, National Treatment Strategy Alignment Project

4 For brevity sake details of individual tools, workshops, curricula will not be relisted

in the present document. For details of project outputs to refer to submitted semi-annual project output reports.

21

Figures 4, 5: Examples of KE tools developed to support youth and families across service settings and sectors

Three hundred and eight KE workshops and forums have been developed, adapted and/or refined to enhance the KE infrastructure across BC. KE workshops and forums create space for reflection, interpretation, application, and evaluation of practice. Developing relationships supports dialogue across sectors, and fosters understanding and shared commitment. KE workshops and forums have been catalysts for cross-sector collaborations.

The 308 workshops/KE forums conceptually group into the following types:

28/Evidence-informed clinical practice workshops (205 workshops of this type were delivered). Evidence informed practice workshops supported: core addiction practice, motivational interviewing/dialogue, feedback-informed treatment/client-directed-outcome-informed treatment, mindfulness, narrative therapy, trauma informed practice

24/Facilitator and evaluation capacity-building workshops (37 workshops of this type were delivered)

14/Enhancing system capacity to engage youth with substance use and mental health concerns and their families (16 workshops of this type were delivered)

10/ Primary care and medically assisted services (35 workshops of this type were delivered)

2/Supporting implementation of Residential Standards (2 workshops/forums of this type were delivered)

8/Addressing Stigma and Discrimination (13 workshops/forums of this type were delivered)

Commitment from multiple sectors to collaboratively affect system change

“I think it's a wonderful way for clients and family members to get involved in providing feedback for the care that they receive, and also just to give us feedback on the services we're providing. Because I think… we tried different ways to get feedback from [the people who access our services], but we don't always manage to do that very well. So this is an awesome way to do it … and I think it empowers clients as well.” ~ Community Advisory Committee member

“We're engaging the community physicians more with what we're doing. In the past, I think it was, ‘Here, you take my patient. They have an addiction problem. I don't want to know about it.’ And now I think because they know that we sort of have their back if it goes badly, they're more willing to be involved, in the earlier stages anyway, and, you know, that they can call us. And because we have our journal clubs and other ways that we're connecting with each other face to face on a regular basis, it means that enhances this sense within the community that we really do have a versatile, effective way of treating mental health substance use issues.” ~ Psychiatrist

“I think it’s been collaborative in the truest sense, that there has always been an openness to come to any table to discuss how various sectors, individuals, disciplines, could be involved.” ~ Director, Mental Health & Substance Use Services

22

Figure 6: Reflections; Figure 7: Word Cloud – Insights and Innovations Conference

Thirty-two curricula and toolkits have been developed, adapted and/or refined to enhance the KE infrastructure across BC. Curricula and toolkits support practice and system-level change, and are designed to enhance availability, understanding and uptake of evidence-informed practice and facilitate linkages across sectors and regions.

#/type of curricula:

32/contextualized curricula were developed, enhanced

23

BC Project Outcome: Enhanced KE mechanisms to facilitate / support evidence-informed practice to enhance stakeholder commitment and engagement Indicators: #/type of KE mechanisms used and developed; perceptions of enhanced KE mechanisms

BEFORE THE DTFP: In BC, health authorities relied on “one-off” capacity-development initiatives that reflected specific short-term opportunities to address a perceived gap in service provider skills or knowledge (e.g., concurrent disorders). The “one-off” efforts were typically “rolled-out” over short time lines without guidance from an evidence-based theory of change (i.e., information dissemination changes practice).

PROGRESS TOWARDS OUTCOMES

KE mechanisms are the conceptual approaches guiding the work. The collaborations, frameworks, and approaches are developed to inform decisions and processes. KE mechanisms developed and refined through DTFP initiatives support knowledge sharing and the development of collective knowledge for those providing services at the point of care, and for those involved in system management and planning. KE mechanisms foster adaptive responses and the spread of innovation.

Thirty-two KE mechanisms have been developed, enhanced and supported including:

8 Communities of Practice to support practitioners’ linkage and exchange

o 6/regional; 2/provincial with participation from 381 members across 19 sectors

2 service frameworks and associated sets of service standards

19 collaborative protocols and policies engaging 21 sectors.

4 Regional Oversight /Change and Design teams/committees

Success Story: Cultivating Communities of Practice

Across BC, we are on the learning edge of how to support the development of Communities of Practice. CoPs are supported by relationships developed in the familiarity of regional service contexts, cultures, settings, shared client base and frequent face-to-face contact. In regions with vast geographies, we are exploring with CoP members and moderators ways to support a developing community. We are developing webspaces to support CoPs. It is important to note that a webspace is just one connection point for

Recognition of the positive role CoPs play in supporting service providers

“I think in terms of my own organization that we have seen more life in our building as a result of the energy we get from coming here, from being inspired, and then you're more passionate about your work, and you see that in even just having more clients served and more clients seeing outcomes because we are filled up. That's what makes this great, is it actually produces outcomes outside of these doors.” ~ Service Provider, Community of Practice member

“Having felt a little stagnant a couple years ago and just not feeling very grounded, I know that my attendance here has been a factor in feeling a little bit more grounded in what I do and excited just being able to kind of share even a few hours of a month with some people who you wouldn't have otherwise met. It's life-giving. I can't think of another word other than "inspired" – just to hear other people's experiences, what pumps them up about their work and their day and why they feel connected to the work that they do. It gets my own wheels turning through reflection and thinking about the future.” ~ Service Provider, Community of Practice member

“One of the strong by-products that have come out of many of the initiatives is the desire, the energy, and the curiosity across different sectors to develop communities of practice. Never did that before, they weren’t around before until we started down this road [before the DTFP]” ~ Director, Substance Use and Mental Health Services and Supports

24

members of a CoP, and that the webspace is not in itself the CoP. As noted in the literature there is a dominant misnomer that online webspaces are communities of practice. A webspace is no more the CoP, than a house is the inhabitants. Online spaces, web-based networking and exchange are a generational phenomenon. Younger practitioners who are newer to their careers are more likely to take advantage of the online environment, and careful moderation of online webspaces supporting CoPs may be helpful to bridge intergenerational KE. Cultivating a CoP requires time and careful, responsive attention and action to support a sense of belonging. Creating opportunities for members to connect that are responsive to membership preferences contributes to the sense of belonging.

Figure 8: Northern Youth Concurrent Disorders Community of Practice, 2012 Face-to-Face Forum

Face-to-face KE forums strengthen connections between service providers across tiers, settings, sectors, and geographical boundaries. Members of the Northern Youth Concurrent Disorders Community of Practice visually connected themselves to other forum attendees using a connection map:

Figure 9: Before Year 2 forum Figure 10: After Year 2 forum

There are many ways in which leadership across health authorities are supporting CoPs, including encouraging member attendance by making the space in work schedules for attendance, inquiring with curiosity about what was discussed, celebrating accomplishments, and supporting emergent opportunities. CoPs are also becoming incorporated into formal leadership planning structures. For example, the Youth Addiction Knowledge Exchange Community of Practice (YAKE CoP) has been recognized by health authority leadership as a group that represents youth substance use services and supports across organizational and sector boundaries.

The YAKE CoP’s first year of development was primarily relational and procedural, exploring group identity: Who are we? How do we fit together? What do we do when we’re together? How do we prioritize what we do together and make decisions? How do we operate? Figure 11 highlights the activities and developments of this CoP in the past six months.

25

Figure 11: Youth Addiction Knowledge Exchange CoP, Activities and Developments October 2012 - March 2013

The YAKE CoP has hit a rhythm and members report the CoP is contributing to improved coordination of services to youth at Creekside’s detox/stabilization bed program, and the development of the Valley Youth Partnership for Engagement and Respect (VYPER). VYPER involves long-time YAKE CoP participating organizations (and other partner organizations) in creating youth-directed, adult-facilitated activities, including team building, leadership development, instruction, and connection to community resources and services.

CoP-strengthening activities of YAKE CoP members include:

Members engage the entire YAKE CoP group in communications, indicating a reduction in the centralization of authority and initiative.

Funding/program development ideas are shared to collaboratively seek funding to develop programs that can take advantage of the regional connections that YAKE CoP has enabled. Ideas currently being developed include:

o Creating a region-wide calendar of existing youth events

YAKE Meetings

• Bi-monthly

• Participation of ~30

• Open floor practice concerns, troubleshooting, resource sharing

• Networking/lunch

Youth Stabilization

• YAKE members providing monthly programming at centralized detox/stabilization beds

• No cost expansion of services

Valley Youth Partnership

• YAKE-coordinated, including other partner organizations

• Youth-directed, adult-facilitated activities

• More activities, more youth participation, easily scalable, economies of scale

Web Space & Surveys

• Resource sharing

• Blogs

• Meeting follow-up and preparation

YAKE-Day

• Drawing on internal knowledge of YAKE members

• Common training to develop a common clinical language

Prevention

• Troubleshooting transition from prevention delivery to coordination/capacity building

• Growing participation from school district staff

Development and

Collaboration

• Suicide prevention, intervention and postvention

• Applications for funding for collaborative programs

Legend Unchanged/Continuing New Development Revised

26

o Engaging youth in planning, organizing and conducting activities

o Creating supportive networks among and around school staff to enable evidence-based, social constructivist drug education/prevention curriculum delivery

o Developing volunteer training curriculum to engage youth and adults in supporting youth activities

o Inventorying existing suicide prevention, intervention and postvention activities/initiatives in regional communities

Feedback Informed Transition Oversight Groups and sub-committees

Feedback Informed Treatment is not just a procedure (use of session rating scales and outcomes rating scales), but rather a process where professionals develop an “error-centric” service culture. The service team works collectively to be continuously learning about what can be improved at a practice level and then taking steps to put client-centred feedback into generative action. For many service settings, this represents a significant culture shift from an expert-centred to a client-centred service culture. There is commitment to introduce and support this practice shift and acknowledgement that these efforts demand “heavy front-end” loading and strong leadership to support practice change over the long term. The hope is that a genuine humility and curiosity amongst involved clinicians unfolds. KE mechanisms to support the implementation of Feedback Informed Treatment across health authorities include transition oversight committees, CoPs, supervision and consultation groups, and Outcome Mapping planning, monitoring and evaluation frameworks. The work across health authorities to support this shift in practice is in the “heavy front-end” stages and while there is enthusiasm and leadership commitment, there is also the recognition that this work will require ongoing long-term commitment to support this practice shift.

Vancouver Island Health Authority’s Adult Mental Health and Substance Use Services leadership has incorporated the implementation of Feedback Informed Treatment into their three year Strategic Operating plan for Adult Mental Health and Substance Use Services. Clinical champions have facilitated the co-creation of clinical video vignettes across service settings to support the implementation of Feedback Informed Treatment.

27

HC DTFP Project IMMEDIATE Outcome: Enhanced collaboration on responses to DTFP treatment systems’ issues within and among jurisdictions and stakeholders BC Project Outcome: Enhanced collaboration and understanding on how to implement and sustain evidence-informed practice using KE processes and structures Indicators: Nature/scope of collaborative activities; #/type of collaborative protocols and system consultations; Perceptions of enhanced collaboration

BEFORE THE DTFP: Health authorities operated in silos, and collaboration with other sectors or regions was not a strategic priority.

PROGRESS TOWARDS OUTCOMES

Collaborative protocols, structured system consultations and planning sessions are KE mechanisms to foster linkages and exchanges across the broad multi-sector system of care to support practice and system-level change. Eight hundred eighty-one system consultations have been held with participants from across 27 sectors:

236 community consultations to enhance system capacity to engage youth and families

62 community consultations to enhance Aboriginal linkages and exchanges

99 consultations to enhance primary care linkages and exchanges

267 consultations to enhance workforce and evaluation capacity

217 consultations addressing stigma

Across priority areas of focus, we have been ambitiously opportunistic. For example, we have engaged 27 sectors in a three-year exploration and development of what it means to support clients and families with substance use concerns. This engagement has contributed to the development of multifaceted, multileveled strategies to address stigma, engage primary care, enhance linkages to Aboriginal communities, engage youth and families, and build workforce and evaluation capacity.

Success Story: DTFP Emergent Outcomes – DTFP as a catalyst

for collaboration / protocol / service development

The DTFP KE forums have increased awareness, understanding, knowledge and collaboration across tiers, settings and sectors. Participation in DTFP Addiction KE workshops / forums has been a

Whole Systems Collaboration

“As a partner working with VIHA’s Youth and Family Substance Use Services, we applaud their investment in both content (such as the leadership they showed in developing the report “Keeping Youth Connected”) and in their commitment to working collaboratively with us and others as we work together to support school professionals and their partners in the uptake of evidence-informed practices related to promoting health and addressing problematic substance use. This partnership is a wonderful exemplar of how working together can achieve so much more.” ~ Helping Schools Program Consultant at CARBC

“I feel like we have physicians as colleagues now, which we didn't have earlier, whether that's around methadone services or just making referrals or following up or working as a team. I think there is a much stronger team approach.” ~ Manager, Mental Health and Substance Use Services

“Offering CAP and MI/MD training across sectors impacts other areas in a secondary manner. Bringing everyone into the room together, putting your ‘stuff’ on the table, everyone ends up collaborating and understanding each other better.” ~ Director, Mental Health and Substance Use Services

28

catalyst for the development of a number of collaborative protocols, policies and service developments including:

Students’ residency in addiction medicine expanded from 2-hours to 2-week rotation

Collaborative placement agreements for post-secondary professional facilities and community care centre

Community Municipal Alcohol Policy

Community Municipal Compassion Charter

Safe house operations policies and procedures

Development of new services (e.g. Street level service mobile van; Community Corrections Psychiatric Clinic)

The Vancouver Island Health Authority, in collaboration with the KE team from the Centre for Addiction Research of BC and the BC Healthy Schools initiative contributed to two webinars engaging school districts to ‘animate the evidence’ presented in the DTFP supported report Keeping Youth Connected, Healthy and Learning: Effective Responses to Substance Use in the School Setting5. The webinars focused on practical examples of how schools are supporting evidence-informed responses to substance use in the school setting with a focus on keeping youth connected to the school and community. The webinars hosted participants from 14 sectors: education, substance use services, mental health services, Aboriginal Services, social services, primary care, public health, acute care, justice/corrections/RCMP, families, academic research, not-for-profit/non-governmental organizations, provincial and municipal governments. The webinars have catalyzed linkages and exchanges across sectors. Public health, community and school districts have requested professional development to strengthen capacity of school counsellors and youth support workers to engage with youth on health promotion, fostering resilience and substance-related issues in the school system. Emergent opportunities include co-creation of a public health promotion and prevention curriculum for school settings, and engagement

with school trustees at BC’s School Trustees Annual Meeting in the summer of 2013.

The Vancouver Island Health Authority has demonstrated commitment to this work with the creation of a special project co-ordinator position and project support to promoting health, fostering resilience and reducing risks, including problematic substance use across Vancouver Island health care settings.

5 http://www.carbc.ca/Portals/0/school/keepingyouthconnected.pdf

29

Success Story: Enhanced KE Links to Aboriginal Communities

The DTFP has contributed to enhanced KE and links to Aboriginal communities that have expanded partnerships, and deepened the level of understanding of effective treatment systems. The depth and breadth of increased linkages to Aboriginal communities supported by the DTFP has been noteworthy.

Consultations and conversations with Aboriginal People – Chiefs, Tribal Leaders, Natural Helpers, Elders, Service Providers, Clients, and Families – have contributed to the co-creation with Aboriginal partners of content and design of Motivational Dialogue workshops that includes ongoing evaluation.

What started out as a two-day workshop with 25 participants from Aboriginal services and communities has grown to six two-day workshops with over 250 participants and a growing demand to expand this work across the province.

This ongoing partnership with Aboriginal people has deepened our understanding and appreciation for different ways of knowing. The work of Dr. Dan Pratt6 on pedagogical signatures has informed and inspired dialogue among champions. With our work with Aboriginal partners, we have discovered that:

Language is central to Indigenous Knowledge – use of words is critical. We need to be mindful of language, and take time to find the best possible word to share

Learning from place is a foundational aspect of learning – where one belongs is important and providing learning spaces in various settings in different nations is respectful

The drum is a unifying grounding element and was instrumental in starting workshops, returning from breaks, and when the group was struggling

Elders, knowledge seekers, and cultural workers are indispensable to our work – this goes beyond who we routinely involve in capacity-building (i.e., service providers)

A first principle of Aboriginal learning is a preference for experiential knowledge – we experienced more volunteers to role-play in these workshops than with other sectors/groups

Values are so deeply embedded within Indigenous Knowledge that it is difficult to distinguish its empirical content from any

6 Dan Pratt, Pedagogical Signatures in the Health Professions: Commonalities

and Differences, Interprofessional Education Network Day, January 2012

Reflections on Ways of Knowing

“I came with little or no expectations because I have found these kinds of trainings don’t really help me where I work. I work with people who are suffering, and no tools will help that. I really appreciate Mark talking about compassion and suffering. He spoke in such a way that is honest and respectful. This is probably the best workshop I have been at in about 10 years.” ~ Aboriginal Elder who attended a Motivational Dialogue workshop

On-going conversations about ‘ways of knowing’ and steering clear of any agenda that smells like ‘problem solving’ with Aboriginal Leaders, Elders and Natural Helpers has opened the door to more trusting, mutually-defined and longer term relationships

30

moral messaging – when Motivational Dialogue participants spoke about values, it was difficult to separate ‘values’ from what they do and who they are. When we moved to compassion as a value, the room opened up to a rich and honest discussion

It is not useful to compare and contrast Indigenous Knowledge (‘traditional helping’) to Eurocentric knowledge (‘scientific knowledge’) as it creates division and binary arguments. It is more helpful to co-create fresh vantage points to discover collective ‘ways of knowing’ and shared capacities that are more ‘both/and’, and less ‘either/or’ – a repeating concept in practicing Motivational Dialogue

Aboriginal National Treatment Strategy Alignment project

The Aboriginal National Treatment Strategy Alignment Project involved engaging Aboriginal communities located in BC’s interior in conversation to explore what is and is not working with respect to Interior Health Authority-funded substance use services. This initiative, led by the DTFP Aboriginal Addiction KE Leader, included 22 one-with-one conversations (10 women, 12 men) and 11 group conversations (with a total of 118 participants) in-person with First Nation community stakeholders. First Nations Health Authority staff and Hubs in the interior region were the conduit for connecting with communities. Hub coordinator, Health Director, Community Health Organization, and individual interviews showed representation from six out of seven Nations in the interior of BC (representing 50 Bands). All Hub coordinators that were consulted represented the bands within their Nation. The synthesis reports highlighted four themes that emerged from the Aboriginal community conversations (Relationship; Cultural Safety; Access and Capacity Building). The synthesis reports were shared with First Nations Community partners, and bands were unanimously in agreement with the themed findings. The synthesis reports were then shared with Interior Health Authority leadership and provincially with members of the BC Substance Use Network. One of the emergent outcomes has been the commitment of the members of the BC Substance Use Network to collectively work together to support this work across BC and have this work be a standing network agenda item.

31

If we’re going to make good on our promise to engage First Nations, Inuit and Métis people in capacity building, then we need to respect the process that’s meaningful to them. Some of the considerations for KE workshops/forums with Aboriginal people are:

Respectful relationship building – front end work MUST be done in order to establish a relationship and some level of familiarity and trust. This front end work would include getting to know the community / service providers and identifying what their protocol is around practices such as inviting an appropriate Elder to open with prayer. All Aboriginal communities have Elders; however, not all Elders are suitable for such roles.

Consider the intrinsic relationship with the content and provide some form of support for potential (and likely) ‘fall-out’. With our deeper understanding of the interplay between KE workshops/forums, context, and culture we’ve learned that substance use KE is intimately felt by our Aboriginal participants due to their unique history and the prevailing prevalence rates of addiction within community. We absolutely MUST take this in to consideration when inviting our Aboriginal partners to KE forums/workshops such as the Core Addiction Practice series.

A ‘learner centered’ rather than ‘information centered’ approach is imperative when working with the Indigenous population. Entire communities suffer with addictions as a whole, so to expect members of this population to receive and experience Core Addiction Practice series in the same manner as non-Aboriginal people is simply short-sighted and negligent.

Consider the ‘way’ in which KE forum is structured with particular attention to pacing, exchange, and dialogue. The speed with which the bureaucratic structure(s) and processes function are often foreign and uncomfortable for Aboriginal people, particularly when one considers the additional fact that they are often significantly outnumbered and/or underrepresented in the KE forum/workshop. Mainstream society seems to reward and/or value quick exchanges and the ability to perform verbal acrobatics as opposed to the more thoughtful and introverted style that is often seen amongst the Aboriginal population. As a result, it can be challenging for First Nations, Inuit and Métis participants to find opportunities to contribute during KE forums/workshops. It’s important to allow silences to occur and to create spaces for those less assertive to feel and experience true engagement and collaborative learning.

First Nations, Inuit and Métis people often struggle with their own internalized oppression as well, which compounds the intricacies of their participation, involvement and ‘take away’ from the KE forums/workshops. A culturally safe environment must be established by the KE workshop/forum facilitators.

It is customary in traditional Indigenous gatherings to make the space for introductions, so this should be acknowledged and respected when facilitating KE workshops/forums on traditional and non-traditional territories.

~ Aboriginal Addiction KE Lead

32

HC DTFP Project INTERMEDIATE OUTCOME: Increased access to evidence informed practice information BC Project Outcome: Enhanced availability, understanding, and uptake of evidence-informed practice information Indicators: #/type of evidence-informed practice information available, understood, used

BEFORE THE DTFP: The situational analysis reflected a narrow conceptualization of how to support evidence-informed practice that was focused on expert-centric information dissemination. The target population for expert-centric capacity building efforts were almost exclusively health authority specialized mental health and substance use direct service providers.

PROGRESS TOWARDS OUTCOMES

The DTFP has contributed to enhanced understanding and use of evidence-informed practice information. For example, KE workshops/forums have provided a mechanism for new cross-sectorial learning opportunities that have contributed to developing skill, building confidence, deepening knowledge and strengthening connections with other service providers.

To date, 8698 service providers across 27 sectors (substance use services, mental health and substance use services, mental health services, Aboriginal services, primary care, clients, families, family services, housing services, education and school districts, justice/corrections/RCMP, academic research, public health, acute care, chronic care/home health, emergency services/urgent care, private sector/business, non-governmental and not-for-profit organizations, shared administrative services, municipal and provincial government/ministries) participated in 308 DTFP-supported workshops and KE forums that are grouped conceptually into the following six types:

28/Evidence-informed clinical practice workshops (205 workshops of this type were delivered). Evidence informed practice workshops supported: Core Addiction Practice, Motivational Interviewing/Dialogue, Feedback Informed Treatment/Client-Directed-Outcome-Informed treatment, Mindfulness, Narrative Therapy, Trauma Informed Practice

At 3 month follow-up, Core Addiction Practice series participants report:

More advocacy behaviour

Less generalizing, i.e. curiosity and commitment to learning from client

More interest in learning beyond foundational aspects of CAP, requests for enhancement workshops, More valuing of evidence generating more dialogue among more sectors

Less anxious to work with clients who use substances, more confidence

Fewer ‘early discharges’ and deflection when substance use is revealed

More positive feedback from clients who use substances, i.e. ‘feeling heard’

Different relationships, share a common language, foundational knowledge – less misunderstandings

More colleagues are seeking them out to learn from them

More data to support sectors register at least 2 people from each site to avoid potentially alienating individuals when returning to that site or being burdened with the sole responsibility for all matters and clients indicating substance use issues

"I always relied on the fact that if I had the correct information and developed the right skill, I would be an effective officer. Addictions throw that thinking right out the window, and for most of us, this scares us and messes with our confidence. I appreciated the careful thought and way in which you delivered CAP to me and the rest of the group. I learned more about myself in those five days than in the last 5 years working with citizens who are under the influence. Now, I look forward to learning from the citizens as well." ~ RCMP Officer

33

24/Facilitator and evaluation capacity-building workshops (37 workshops of this type were delivered)

14/Enhancing system capacity to engage youth with substance use and mental health concerns and their families (16 workshops of this type were delivered)

10/ Supporting primary care and medically assisted services (35 workshops of this type were delivered)

2/Supporting implementation of Residential Standards (2 workshops/forums of this type were delivered)

8/Addressing Stigma and Discrimination (13 workshops/forums of this type were delivered)

The perceptual data from across addiction KE forums highlight the investment in people, practice, reflection, and connections to facilitate system change. This aligns with evidence-informed practices to support complex system change. Specifically, an investment in people, practices, feedback, and connections strengthens systems, promotes resiliency and sustains ongoing learning.

Success Story: Workforce Development – Core Addictions

Practice Series

The DTFP has contributed to a foundational substance use workforce development series, Core Addiction Practice (CAP). This series was originally designed and intended to be used by the specialized substance use service sector. At a system level, the DTFP team BC’s DTFP whole systems approach and ongoing utilization-focused evaluative activities have supported ongoing development of CAP and has expanded the content, facilitative infrastructure and usefulness.

The CAP series has evolved beyond a standardized content-centric course and is now a whole systems KE opportunity. CAP facilitates KE as participants acknowledge their own experiences and learn from others to deliver and expand evidence-informed services. CAP resource materials, curricula, and facilitative infrastructure have expanded as our knowledge base has expanded beyond the specialized substance use treatment service sector and settings.

Most Significant Change Story

A veteran nurse who has worked more than 20 years in the Emergency Room attended the four-day Core Addiction Practice for ER staff in February 2013. During the CAP series, she appeared withdrawn and disengaged. Five days after completing CAP, she called the Addiction KE Leader to share her experience:

“I was so angry with you and Mark for assuming that we didn’t know how to effectively engage with patients after I left that last day of CAP. I came back to work with a commitment to prove you both wrong. That first shift, I saw a patient come in who frequents our ER and has a long history of drug use. I looked around and noticed most of my colleagues were avoiding contact with him and some were rolling their eyes and whispering to each other. I decided to walk over to him, and before speaking, tried on of the attunement exercises, and then said to him ‘Hello, I am ___, and you are in the right place and the right time. How can I help you?’ I noticed his demeanour soften and he almost cried in front of me (he usually yells and swears at us). He was in and out within 30 minutes with no incident. When I returned to the line and nurses station, some colleagues commented, ‘Wow, ___ was on his best behaviour today!’ I turned around and said to them, ‘No, I was on my best behaviour today.’ They looked shocked at me, and walked away. It has been difficult here since my saying that to them, but I am committed to trying to find a way forward.” ~ Nurse, Emergency Department

34

DTFP supported CAP development include:

90 CAP series offered (2260 participants, 6 health authorities, 24 sectors)

CAP series content and design customized for a variety of service sectors and settings including: emergency room and high acuity acute care settings, and police/justice settings.7

Cross-regional CAP Facilitator Development series

CAP Facilitator forums to expand facilitators’ capacities beyond transmission/transactional facilitation methods to reflect on ‘ways of knowing’ and facilitation and learning ecologies

Table 2: Summary of how ongoing evaluation has been used to support the evolution of the CAP series

Evaluation Purpose of Evaluation Data Collection Mechanism

Developmental

(before and during CAP)

Adaptation and customization of CAP content and design to support cohort

Identify emergent possibilities for more responsiveness and effectiveness

Support facilitators in CAP preparation

Dialogue between key informants to explore need, expectations, contexts, learning culture

Document review

Learning ecology assessment

Formative

(during CAP)

Improve facilitation and course design

Indicate content areas that need more/less attention/resources

Address logistical issues/needs

Support learning ecology

Test, validate instruments and procedures for summative evaluation

Daily expectations discussed

Reflecting teams

Daily written questionnaire

Daily CAP facilitator de-briefing and review of feedback

Summative

(after CAP)

Assess CAP merit, worth, value, significance

Assess contributions of CAP learning experience in practice

Identify CAP participants’ growing edge in learning more

Inform future CAP content and design

End of course written survey

Follow up on line survey 3– 6 months after CAP completion

Follow up semi-structured interview to representative sample

Key informant interviews with CAP Facilitators and other sector partners involved in coordination

7 Emergency staff and acute care leadership are requesting more CAP opportunities based on feedback from the first CAP

series co-created for the emergency room setting. This is a significant shift as engaging emergency room staff in joint learning exchanges has been challenging. The demand for CAP outweighs capacity.

35

Figure 12 demonstrates the growth in CAP facilitation infrastructure from 2008 to 2013. CAP series are now offered across the province by a collective group of 97 facilitators, apprentices, and mentors supported by regional and provincial advisors and committees.

In 2008/9 the CAP facilitation infrastructure involved few sectors (substance use, mental health, primary care, clients/ families). In 2012/13 the facilitation infrastructure reflects a whole systems approach with involvement across sectors and

settings (substance use, mental health, home health, public health, emergency/urgent care, primary care, pharmacy, Aboriginal services, education, clients/families, youth and family services, housing, NGOs, justice / corrections / RCMP, municipal government, and academic research).

Success Story: KE to Support the Development and Implementation of Residential Standards

The appreciative KE approach to developing practice standards has been recognized by the senior stewards in the system as a new benchmark for developing provincial standards. The process of creating the standards, combining a systematic literature review with in-depth consultation from the community of practitioners, has had tremendous payoff.

The Ministry of Health continues to support provincial appreciative inquiry evaluation methodology, developed by the DTFP to support the implementation of the Residential Standards. An appreciative inquiry approach highlights innovations in service, and focuses on the sense of meaning and mission that motivates the work of a residential treatment centre. This inquiry has showcased how programs have created relationships, rituals, communication structures, planning tools, consultation networks, group processes, and decisional rubrics that support their work. Health authorities and residential treatment centres are in a cooperative relationship about the standards. Historically, standards have been brought forward as an accountability mechanism by authorities. The Residential Standards are also perceived and employed as supports for high quality innovative practice.

BC Minister of Health, Margaret MacDiarmid, providing opening remarks for Dialogue Days, January 2013

Figure 12: CAP Infrastructure Growth 2008 to 2013 (y-axis: # individuals)

36

Findings from the appreciative inquiry informed an interactive forum of residential substance use service providers across BC, mediated by highly skilled facilitators. These Dialogue Days supported relationship-building and the expression of the skills and knowledge of the community.

Residential substance use service providers, often from diverse professional disciplines, seem to derive professional identity from their programs and from the mission of their work, and not primarily from their professional discipline. With this understanding, the approach for the forum used KE amongst the participants as the guiding process, and not a hierarchy of external experts.

Amanda Seymour, former Ministry of Health DTFP Lead, received the BC Government People’s Choice Award for her leadership in the development of the standards. KE approaches (e.g., KE forums) are now being used to support planning and implementation of the BC Mental Health and Substance Use 10-year plan.

37

Our evaluative approach supports deeper understanding

“A strong clarity emerged: developmental evaluation is a way of responding to the work as “it is” and as “it is becoming” rather than insisting “it be” a particular way; a way that is within the command of our expectations or the constraints of our imagination / consciousness.” ~ Clinical Practice Specialist

“I think that we're paying attention to the subtleties and nuances in a way that we have not in the past. We would have said, "Well, anybody's welcome here," and we just open our doors and kind of ignore the fact that we know within Aboriginal communities, for example, what the prevalence rates are and what the level of utilization is. So we could go on saying everybody's welcome, but if we know it's not working, I think we're paying way more attention to that now where services are delivered, who's delivering them, what does the setting look like, which links nicely with the trauma-informed, right, in that those are examples, I think, of where we've inadvertently promoted stigma and exclusion, or sometimes maybe more overtly.” ~ Manager, Mental Health and Substance Use Services

HC DTFP Project INTERMEDIATE OUTCOME: Increased PT capacity to evaluate substance abuse treatment systems performance

HC Indicators: Proportion of jurisdictions which have developed and/or initiated performance measurement and evaluation frameworks, plans, tools or activities with DTFP support; Nature/scope of performance measurement and evaluation frameworks, plans or tools developed, Nature/Scope of performance measurement activities and evaluation studies initiated / implemented with DTFP support (e.g., strategic evaluations); level of understanding of effective treatment systems performance (e.g., evidence, strategies, best practices) among partners, stakeholders and communities of practice.

BEFORE THE DTFP: Evaluation capacity across substance use services and supports was limited. Evaluation was typically an isolated task required at the end of an initiative/project outsourced to evaluation specialists with limited understanding of the context and focused on accountability (i.e., “did it work?”).

PROGRESS TOWARDS OUTCOMES

BC’s DTFP Strengthening Systems initiative does not include the DTFP treatment systems performance evaluation and monitoring stream. Nonetheless, the developmental evaluation approach and activities have supported evaluation capacity-building due to its inherently participatory nature. The approach has informed the development of service evaluation frameworks, and evaluative methods have informed scaling of innovations across settings. This approach has supported the following evaluative components developed, adapted and/or refined, and used across regions:

107 KE tools, methods for the development, monitoring and evaluation of strategic change initiatives

37 evaluation capacity-building workshops and forums were held

881 consultations to support planning, monitoring and evaluation of system change initiatives

Success Story: Expanding Evaluative Capacity to Address

Stigma through Compassion, Inclusion and Engagement

This work started with recognition that there is a diverse understanding of how to address substance use-related stigma in the healthcare system. A systematic review was commissioned, and research recommendations indicated that there is a paucity of quality research performed in the area of interventions to address

38

substance use-related stigma; the review recommendations included that any interventions to address stigma be multifaceted and addressing the micro, meso, and macro structural levels, and be piloted and evaluated carefully.

Fraser Health Authority / DTFP team led the efforts to act on the evidence presented in the systematic review. The systematic review moderated the influence of opinion leaders and historical practices in decision making related to approaches used to address substance use related stigma. This contributed to the development of an evidence-informed multifaceted, multilevel stigma prevention and reduction strategy. This strategy evolved over the next six months, based on learning from dialogues with Aboriginal community members led by the Aboriginal Addiction KE Leader, in combination with consultations with clients, families and service providers, and ongoing consultations with Michael Quinn Patton (author of texts on Developmental Evaluation and Utilization-focused Evaluation), and the DTFP team. All of this contributed to a fundamental shift in BC’s DTFP approach to stigma.

Stigma is a complex social process that is dependent on the relationship between the specific discrediting attribute (substance use) and the specific social context; it is a product of a social situation rather than any specific individual. In this conceptualization, stigma is treated as a sensitizing concept8 and space has been created to explore the meaning of compassion, inclusion and engagement within and across sectors as an intervention in and of itself, as shared understanding is emerging. With this understanding, we shifted from a “stigma prevention and reduction” focus to a multifaceted, multilevel strategy to support compassionate, engaging, welcoming services. This shift attracted more enthusiasm and creative thinking.

Table 3: Approach to Stigma Strategy Map

Outcome mapping evaluation methodology has been used to plan, monitor, evaluate and communicate how strategies with eight boundary partners are contributing to compassionate, engaging, and welcoming services. The strategy map presented in Table 3 illustrates how this approach converges with existing DTFP-supported addiction KE.

The strategies implemented in the past year illustrate the confluence of activities,

agents and factors that are collectively contributing to supporting inclusive, engaging and welcoming

8 Sensitizing Concepts Definition: http://tinyurl.com/celpf62

Also see: Patton, M. Q. (2010). Developmental Evaluation. New York, NY: Guilford Press (pp. 146-9)

39

Observable changes in behaviours, relationships, activities and actions of partners and stakeholders in response to evidence on addressing stigma

Less reference to historical practices and splash campaigns by stakeholders and partners

Less ‘othering’ language when referring to clients and families and more awareness and intentional change in use of language by mental health and substance use staff

More leadership support for positive and aspirational approach; contributive lens, interrelated activities and relationships; monitoring domains across projects

Clients and families involved in compassion, inclusion and engagement activities talking about experience and perceptions of meaningful engagement

CAP Facilitators are reflecting more on their role/responsibilities as facilitators, and asking for more support to engage in transformative learning approaches which will contribute to the Compassion, Inclusion and Engagement project (CAP Transformative Learning Approach Resource Guide is being developed to support this request)

Emergency Room, RCMP, Mental Health staff are requesting capacity building for “attunement” and other mindfulness approaches that support compassion, inclusion and engagement

Municipalities (Mission, Langley, Chilliwack, Pitt Meadows, Abbotsford) are requesting information and presentations about developing compassionate city charters

services. An example of a strategy aimed at the macro level is the Welcoming Spaces Project. There is a strong evidence base linking client outcomes to service experience that extends to waiting areas and reception.9, 10, 11

.

The Welcoming Spaces project focuses on mental health and substance use waiting rooms and entrance areas to enhance service environments. “The social distance a service creates with clients is extended to the reception and waiting areas long before that person enters the counselling session... this first impression gets repeated with every visit... and is often overlooked in the service world” (summarized from Janulis, 2010, pp. 25-7).

In the past six months, a literature review and environmental scan of waiting rooms and service entrance areas were conducted, with findings presented to senior leadership. This project has garnered regional mental health and substance use leadership support for the development of a regional welcoming spaces project charter in partnership with existing regional Diversity Committee.

An example of a strategy aimed at the micro and meso levels (individual and group) is the Client Experience Project: a collaborative inquiry with clients in methadone maintenance therapy in residential substance use services. The collaborative inquiry report was completed and presented to regional mental health and substance use leadership. Plans are underway to respond to the report recommendations (April to June 2013). In addition, funding and contractors have been secured, and methods have been developed based on the findings to support a regional client follow-up inquiry. The inquiry will involve all services and supports across substance use service continuum (n=350 semi structured interviews) to inform mental health and substance use service planning: 1) workforce development areas, 2) evaluation approaches; and, 3) services and support planning – all from the client perspective.

The collective efforts are being supported in Fraser Health by an emerging regional Compassion, Inclusion and Engagement Network, with core members (clients, families, service providers, managers,

9 Routhieaux, R. L. & Tansik, D. A. (1997). The Benefits of Music in Hospital Waiting

Rooms. Health Care Supervisor, 16(2) 10

Janulis, P. F. (2010). Understanding addiction stigma: Examining desired social distance toward addicted individuals. DePaul University, Chicago, Illinois. Theses and Dissertations. Paper 16. http://via.library.depaul.edu/etd/16 11

Link, B. G. & Phelan, J. C. (2001). Conceptualizing Stigma. Annual Review of Sociology, 27 (363-85)

40

Youth involved in Photovoice are sharing disclosures of increased confidence, preferred identity and hope for future

“Finally people will listen and not jump to conclusions about ‘who I am’ and ‘who I should be’. Photovoice is not only giving me a ‘voice’ but staging a captive audience long enough for my voice to be heard.”

“Photovoice is the most liberating activity for me... much more effective than any drug because these images last forever.”

“I shared my pictures with my mom who actually smiled and hugged me. She meant it this time”

“I showed some of my images with my case worker who was surprised. He said that he learned more about me from my pictures and explanations that in the last year of meeting regularly. We estimated it would be about 50 hours of counselling verses 50 minutes of photovoice.”

“I really think these pictures and our stories will change peoples’ minds about who we are and what addiction means to us.”

Figure 13: Vancouver Island Trauma Informed Practice Initiative Boundary Partner Map

and directors) involved in leading complimentary large scale system initiatives in the Fraser region.

This approach is informing DTFP-supported approaches to stigma across the province including the Health Equity in Mental Health and Addictions services initiative in the Vancouver Coastal region. The Health Equity initiative aims to address stigma and discrimination through supporting the implementation of recovery oriented, trauma-informed and family relational practices.

The Health Equity project has recently expanded. Stakeholders involved in this project include leadership, staff, clients and families at two tertiary mental health and addiction sites and a hospital psychiatric emergency unit. The sites are located in Vancouver, Gibsons, Richmond and West Vancouver. Consent for change and support for this project was obtained at each site through consultation and dialogue with leaders, service providers, clients and families.

The Vancouver Island Health Authority Trauma Informed Practice

committee is using Outcome Mapping methodology to map out their vision of a whole systems approach to supporting trauma-informed practice across Vancouver Island Health Authority service portfolios. This committee has expanded its scope to include representation from youth and family substance use services, child, youth and family mental health services, adult mental health and substance use services, public health, acute care, Aboriginal health, seniors and spiritual health.

Success Story: Evaluation of KE Network Infrastructure

Funding from the DTFP has supported the operations and ongoing evaluation of the system of Provincial Specialized Mental Health

41

and Substance Use Networks. The networks are one component of KE infrastructure providing KE opportunities for a broad range of stakeholders in the substance use system of services and supports. The networks hold the space for dialogue, connectivity and collaboration regarding evidence-informed practice, system-related issues, and emerging trends within the substance use system of services and supports. DTFP-supported approaches including developmental evaluation and outcome mapping have been used to guide the further evolution of the system of networks. Through evaluation, network members have demonstrated increasing openness to KE and exploring KE mechanisms at networks. Some network members have shared that KE is now incorporated into the development and planning of their organizational initiatives. Systems-level conversations have also been ongoing across networks exploring system performance through data-related infrastructure, family engagement, service planning and cross-sectorial engagement. To build on existing relationships and connectivity, DTFP funding has supported the development and implementation of network webspaces and communities of practice to promote system-wide conversations.

42

HC DTFP Project LONG TERM OUTCOME: Strengthened evidence-informed substance abuse treatment systems HC indicators: Extent of uptake/integration of evidence-informed practices (e.g., changes in professional knowledge and service delivery); Perceptions regarding the extent to which the DTFP has contributed to and strengthened evidence-informed substance abuse treatment system; Extent DTFP funded activities are sustained in treatment systems BC indicators: #/breadth of participants overall, by health authority/sector; #/type of collaborative protocols, enhanced collaboration; Perceptions of enhanced initial and sustained uptake of practice change targets; Perceptions of KE to support evidence-informed practice; Perceptions of enhanced collaboration

BEFORE THE DTFP: The situational analysis revealed limited structures and supports for addiction KE within and across health authorities in BC. There was a lack of dedicated resources, limited commitment, communication, shared learning and understanding of how to affect system change in the DTFP treatment system investment areas. Health authorities operated in siloes, and collaboration with other sectors or regions was not a strategic priority. The National Treatment Strategy had just been developed. Senior Leadership across BC’s substance use services and supports recognized that translating the whole systems approach recommended in the National Treatment Strategy to a BC context would require shifts in approaches to support evidence-informed practice that extended beyond the specialized treatment sector.

PROGRESS TOWARDS OUTCOMES

Success Story: Whole Systems Approach

The depth and breadth of cross-sector involvement in this initiative has grown beyond the specialized addictions and mental health sectors; participation has expanded to a whole systems approach. Cross-sector participation is presented in Figures 14 to 21 for each of the last three years of the initiative.12

Cross-sector involvement, particularly of clients and families, is essential in the development of addiction KE mechanisms and infrastructure. Cumulatively, over the past three years (Figure 14, 15), the scale of the cross-sector engagement is extraordinary.

12

In addition, please see Appendices B to H for health authority-specific breakdowns. Graphs do not include 2009-10 as a full year of data was not available due to delays in first year funding and cascading delays in staffing.

KE and a Whole Systems Approach

“There's a fundamental reason why it's working well, and that is that everybody's keeping all the systemic lines of communication open and not confusing communication structures with authority structures” ~ Research Consultant

“It really opened the door for the province to talk about things that were going on in our own health authorities and to take on projects that we could then share provincially, and where the bumps in the road were for them or to help support them in getting some of these things going.” ~ Director, Mental Health and Substance Use Services

43

Looking back on the journey of the DTFP in BC, the first year of the initiative (Figure 16, 17) was a capacity-building year with limited cross-sector engagement. The second year brought a boom to our capacity to engage community partners due to Health Canada expanded funding, and clients and families were drawn into the work (Figure 18, 19). Finally, in the past year (Figure 20, 21), the potentiation of the relationships established from the first two years spurred significant growth in cross-sector engagement.

44

Figure 14: BC DTFP Addiction KE Cross Sector Engagement April 1, 2010 – March 31, 2013, axis # Addiction KE initiatives

0

50

100

150

200

250

Substance use servicesSubstance use & mental health services

Mental health services

Provincial Government / Ministries

Municipal Government

Clients

Families

Housing services

Employment services

Aboriginal services

Education / School Divisions

Justice / Corrections / RCMP

Academic researchPrimary care

Pharmacy

Public health

Acute care

Chronic care / Home health

Emergency services / Urgent care

Family services

Not-for profit

External consultants

Private sector, business

Parks and recreation

Tourism

Faith organizations

Workshops / KE Forums

Communities of Practice

System consultations

Standards, service protocols & frameworks

Curricula

45

Figure 15: BC DTFP Addiction KE Cross Sector Engagement April 1, 2010 – March 31, 2013, y-axis # Addiction KE initiatives

0

50

100

150

200

250

300

350

400

450

500 Curricula

Standards, service protocols &frameworks

System consultations

Communities of Practice

Workshops / KE Forums

46

Figure 16: Looking back on the journey – BC DTFP Addiction KE Cross-sector Engagement the first full year April 1, 2010 – March 31, 2011, axis # Addiction KE initiatives

0

50

100

150

200

250

Substance use servicesSubstance use and mental health

servicesMental health services

Provincial Government / Ministries

Municipal Government

Clients

Families

Housing services

Employment services

Aboriginal services

Education / School Divisions

Justice / Corrections / RCMP

Academic researchPrimary care

Pharmacy

Public health

Acute care

Chronic care / Home health

Emergency services / Urgent care

Family services

Not-for profit

External consultants

Private sector, business

Parks and recreation

Tourism

Faith organizations

Workshops / KE Forums

Communities of Practice

System consultations

Standards, service protocols & frameworks

Curricula

47

Figure 17: Looking back on the Journey – BC DTFP Addiction KE Cross-Sector Engagement the first full year April 1, 2010 – March 31, 2011, y- axis # Addiction KE initiatives

0

50

100

150

200

250

300

350

400

450

500 Curricula

Standards, service protocols &frameworksSystem consultations

Communities of Practice

Workshops / KE Forums

48

Figure 18: Looking back on the Journey – BC DTFP Addiction KE Cross-Sector Engagement the second full year April 1, 2011 – March 31, 2012, axis # Addiction KE initiatives

0

50

100

150

200

250

Substance use servicesSubstance use & mental health services

Mental health services

Provincial Government / Ministries

Municipal Government

Clients

Families

Housing services

Employment services

Aboriginal services

Education / School Divisions

Justice / Corrections / RCMP

Academic researchPrimary care

Pharmacy

Public health

Acute care

Chronic care / Home health

Emergency services / Urgent care

Family services

Not-for profit

External consultants

Private sector, business

Parks and recreation

Tourism

Faith organizations

Workshops / KE Forums

Communities of Practice

System consultations

Standards, service protocols & frameworks

Curricula

49

Figure 19: Looking back on the Journey – BC DTFP Addiction KE Cross Sector Engagement the second full year April 1, 2011 – March 31, 2012, y-axis # Addiction KE initiatives

0

50

100

150

200

250

300

350

400

450

500 Curricula

Standards, service protocols &frameworks

System consultations

Communities of Practice

Workshops / KE Forums

50

Figure 20: Looking back on the Journey – BC DTFP Addiction KE Cross Sector Engagement the past full year April 1, 2012 – March 31, 2013, axis # Addiction KE initiatives

0

50

100

150

200

250

Substance use servicesSubstance use & mental health services

Mental health services

Provincial Government / Ministries

Municipal Government

Clients

Families

Housing services

Employment services

Aboriginal services

Education / School Divisions

Justice / Corrections / RCMP

Academic researchPrimary care

Pharmacy

Public health

Acute care

Chronic care / Home health

Emergency services / Urgent care

Family services

Not-for profit

External consultants

Private sector, business

Parks and recreation

Tourism

Faith organizations

Workshops / KE Forums

Communities of Practice

System consultations

Standards, service protocols & frameworks

Curricula

51

Figure 21: Looking back on the Journey – BC DTFP Addiction KE Cross Sector Engagement the past full year April 1, 2012 – March 31, 2013, axis # Addiction KE initiatives

0

50

100

150

200

250

300

350

400

450

500 Curricula

Standards, service protocols &frameworks

System consultations

Communities of Practice

Workshops / KE Forums

52

BC’s DTFP has supported an intentional whole systems approach. The potentiation of this approach has been nuanced and contextualized in each health authority in BC supported by ongoing shared learning within and across health authorities. The priority areas of focus and contextual opportunities potentiated are evident in the health authority specific graphs (Appendices C to H).

Factors contributing to the potentiation of KE to strengthen the system within and across health authorities include:

Pre-existing networks of specialized substance use services and supports

Credible inclusive leadership responsive to evidence and supportive of emergence

Established track record of the benefits of KE approach

Conceptual understanding of KE and evaluative methodologies

Appreciative lens to celebrate practice excellence, unleash creative spirit and co-create knowledge with a mission driven workforce

Diversity of interconnected KE activities, not a singular focus on one KE activity (e.g., training workshops)

Contribution lens with an outcomes focus rather than an output focus

Commitment to supporting distributed leadership coupled with the number of service providers interested in participating, supporting, and leading KE initiatives: distributed leadership potentiates possibilities as work is not dependent on any one person or one source of funds.

Adapting and innovating to respond to regional context: this involves an authentic interest in potentiating emergent possibilities, supporting autonomy/loosening command and control egocentric leadership, and awareness of inter-relationships between roles and responsibilities within a system.

Substance use work is inherently culturally and contextually dependent with diversity across sectors and settings informed by work cultures/ roles/ norms/ approaches that inform practices supporting people with dependent/problematic substance use. Cross-sector forums provide the opportunity for participants to:

Be mindful of inclusion, compassion and engagement across settings, sectors, and tiers

Investments in a Whole Systems Approach

“I think our organizational culture has shifted…the acceptance of addictions care as a legitimate healthcare need, and that integration and embedding addictions practice into all sectors across community services. So pharmacy, disease, physician care, that kind of thing. And I feel like we have access to different areas in our service structure than we did before. We've built relationships, I think, as well, that will be sustainable post-DTFP, primarily with some… like the division of family practice in a number of areas where we've actually been sought out to educate physicians about our work, which is, again, quite a shift in the hierarchy.” ~ Manager, Mental Health and Substance Use Services

53

Address misattributions about the role of other sectors in providing substance use services and supports

Increase understanding of other workplace cultures, settings and service contexts

Explore and foster partnerships across sectors and silos

Expand knowledge of evidence-based approaches

Success Story: Enhancing Links to Primary Care

The Interior Health/DTFP Team has been the provincial lead in developing and strengthening links to primary care. Prior to the DTFP involvement, the integrated community health centre had no primary care services (including no methadone prescribing physicians). KE approaches have been used to engage primary care physicians to successfully co-create a community based integrated health service (King Street Clinic) for people with substance use and mental health concerns. Community stakeholders and clients provided essential contributions to the process. Intentionally co-designing the clinic to support physicians’ service needs has contributed to an abundance of interest from more physicians than required to support the clinic’s 24/7 client wrap/around services.

The King Street Clinic has become a professional and community hub that has provided a space for community service integration and support for innovation. The DTFP Addiction KE Lead in the Interior transitioned into a newly created core-funded position to lead the development of community co-created integrated mental health and substance use primary care clinics across nine sites in the Interior region.

An example of a “butterfly effect” service innovation that emerged from the King Street Clinic is the development a new Community Corrections Psychiatry Clinic (CCPC). The CCPC was created to reduce barriers and provide better pathways to care for people who are dually involved with both mental health and substance use services and the criminal justice system. The clinic has increased collaboration and responsiveness amongst service partners across settings (Probation and Parole, RCMP, mental health, substance use, primary care, street outreach in corrections settings, community care settings and the broader community).

Appreciation and Investment in the Potential of the Service Community

“King Street Centre, so it's an opportunity to see what it's like when you've got mental health substance use clients with quite complex issues actually have family doctors, and family doctors who enjoy collaborating with us… it's pioneering work that's taking place at that clinic.” ~ Psychiatrist

“CCPC [Community Corrections Psychiatry Clinic] exists because King Street exists” ~ Psychiatrist, CCPC co-founder.

54

Return on Investment in Champions – Renewal and Regeneration

“I was so surprised at 59 that I would be invited to something when clearly I was on the way out. So why would an organization invest in me? And so by that investment in me, I reinvented myself in terms of my investment to my work and the whole concept of ‘elder’ – which I wasn’t gravitating towards terribly at that point – I started to gravitate and own it, and I became an elder. And the concept of giving back, feeling like I had something actually to give back, rather than, ‘Move over, old lady,’ was so huge for me. And it ignited a spirit in me that is just like a wildfire, and it just grows and grows…My delight in getting a brand new grad… ‘You’re a gift to me and I’m a gift to you’.“ ~ Motivational Dialogue Practice Champion

“The community that is here I almost feel like I’m responsible for creating some community out there because of the investment that’s made in me here.” ~ Motivational Dialogue Practice Champion

“This time has contributed to enriching my family life…our family health and wellbeing has a significant impact on our ability to do our jobs. And having that enriched family life enriches my work life…they start feeding off of each other instead of feeding from each other, instead of taking away from each other they start enriching each other. So that’s been a glorious gift.” ~ Motivational Dialogue Practice Champion

“The burnout is a lot farther away, because it’s significantly easier to draw on the resources that I have, both in my own workplace and outside of my workplace as a result of the networks and capacity building that’s been built up here.” ~ Motivational Dialogue Practice Champion

Success Story: Investment in Champions

Peer support and clinical supervision in health care has been a casualty of numerous system reorganizations and funding cutbacks. At the inception of BC’s DTFP initiative, results of this loss had been observed in a workforce culture based on skills-deficit decision-making, low confidence, moral distress and staff burnout. Without peer support and clinical supervision, clinical staff report feeling isolated, working without appreciation for their skills and contributions and knowledge of local contexts, and disconnected from taking advantage of current developments in their field13.

Clinical supervision is a process-oriented professional relationship that supports the development of the supervisee’s competencies and meta-competencies. It involves reflective review, an opportunity for support and monitoring of practice, the processing of practice-related personal issues, and nurturing motivation (Grigg, 2011). As evidenced in the appreciative inquiries into clinical supervision practices across BC, with clinical supervision, “general training becomes competency, and that competency becomes integrated into actual performance in an accountable and documented way”.

Given the dynamical nature of change, and the service fallout from the degradation of clinical supervisory infrastructure, it was a priority of DTFP planning to develop mechanisms to support learning ecologies. The intentional focus on relationships and social capital has strengthened capacity within the system. The team uses KE strategies to identify, empower and link champions across all priority areas, levels of organizations, and the system. System change has been facilitated through relationships and social interactions; champions facilitate this as social currency and social capital are valuable resources. The investment in clinical champions demonstrates a movement from a more centralized leadership to a shared leadership structure that harnesses informal and formal networks. DTFP investment supports network creation among champions while expanding the number of champions to support a large service system that includes both contracted and direct service staff. The champions are invited to support evidence-informed practice initiatives, such as CAP, Feedback Informed Practice, and Motivational Dialogue/ Interviewing. These champions have become strong communication coordinators.

13

Grigg, G. (2011) A Preliminary Scan Into Clinical Supervision In Health Authorities in BC: An Inquiry About Possibilities

55

Reported shifts in practice from Motivational Dialogue Champions

Reducing Moral Distress

“Motivational Dialogue has changed everything about our agency- guiding principles, policies, procedures, staff orientation and education, staff meetings, client relations, etc. It has revolutionized our mandate and brought pride and unity among our team.”

Enhancing Client Engagement

“I am speaking less, and the client is speaking more.”

“I am feeling more connected to clients and learning from them about how I can be helpful.”

Increasing Client Retention

“I am noticing less no-shows and more clients consistently coming to appointments since practicing Motivational Dialogue.”

“I am struggling to keep up with non-clinical work, as more clients are showing up for appointments... practicing Motivational Dialogue has significantly increased my time with clients.”

“With clients early on in treatment, I am experiencing more clients keeping appointments which has been a huge shift for my practice since being involved in Motivational Dialogue.”

Reducing Adverse Events

“Clients are revealing more genuine disclosures... often his or her suffering… even suggesting risk for self-harm.”

“I am hearing more in-depth disclosures from clients and their living situations which help us design safety plans.”

The clinical champion role is a concept that has been widely promoted yet under-developed in health service literature. Questions emerge as to who these champions are, what roles they play in improving client outcomes, and what contexts serve to facilitate their efforts. Ongoing evaluation has expanded our understanding of the role of clinical champions. Champions can come into their roles in three ways: natural emergence, informal invitations and formal appointments by supervisors. It is observed that appointed champions who remain over a long term are uncommon, and naturally emergent champions tend to become the majority. Champions who are urged by colleagues or who come forward themselves report higher levels of satisfaction, visibility and influence in their roles. Champions report that their activities in support of uptake and integration of evidence-informed practice in their worksites and beyond include: building relationships, modelling practice, educating, advocating, and spanning boundaries across previously siloed worksites and sectors.

Champions continue to emerge from informal processes, include encouragements from colleagues and self-identification. These individuals volunteer to participate in KE events and activities, and demonstrate the intention outside formal organizational roles to work towards improving their practices and influencing others. Supporting champions provides an opportunity to support professional development and growth in a field that does not enjoy the rewards and recognition opportunities found in other sectors (e.g., meeting a sales target and receiving a financial bonus). What the DTFP activities have provided are opportunities for personal and professional renewal and regeneration through ritual and structure to support practice excellence. Rituals refer to a set of actions that have symbolic value and provide an opportunity to witness people bringing their whole selves into the open and making the commitment to deep and lasting change.

An example of ritual to support practice excellence is the annual practice summit in Fraser Health. Champions from across health authorities and practice sectors are invited to share their perceptions of what contributions Motivational Dialogue provided in his/her respective practice. A cohort of Champions reported the practice shifts they have experienced, which are displayed in the sidebar.

56

What are the key lessons from this project?

Appreciation for complexity and dynamical systems

A complexity and dynamical systems lens informed processes and approaches used to plan, monitor and evaluate. Prior to the DTFP initiative, it was business-as-usual in health care: working in siloes using ineffective non-adaptive linear tools, processes and mechanisms that are unresponsive to system dynamics, complexity and context.

A caution learned in the first year of this initiative is that traditional linear project structures/ frameworks/ processes can be detrimental. The premature adoption of formalized structures will result in frustration, and constrained innovation. Adaptive planning processes hold space to avoid premature focus and support for emergence. This involves a willingness to live with uncertainty, and acknowledge that plans regardless of how well laid out they are will shift in response to changing circumstances. In complex systems, traditional sources of evidence are not enough; complexity requires strategic learning and ongoing investment in learning from practice.

Creativity and freedom to plan, do, learn – no foul zone

The DTFP has provided credibility to innovate, be creative and experiment, and to gather the evidence from KE practice to back up the use of KE methodologies to support system change initiatives. The focus has been on learning, which involves learning from things that do not go so well. Reflective transformative KE approaches are generative through which participants actively build knowledge rather than passively consume information. Using a KE approach reorients the investment from a content-centric investment (the right information to the right people) to a focus on the process and design of the KE approaches used to support transformative learning. Attention to the process is a key characteristic of supporting non-linear learning.

Importance of ongoing review of adaptive structures

In health care planning, processes are typically fiscally dictated, with tight time frames and focused on outputs not outcomes. These planning processes are dominated by linear project management methodologies that do not support adaptive planning.

The early commitment to reflect, review, and use non-linear planning processes and adapt structures to support function was

A-ha moments of BC’s DTFP community

“It allowed us to work in a different way that's non-prescriptive, that involved a dialogue at all levels and an openness, a respectful environment in which we could do that and that no idea was a bad idea. And I think there was a freedom in that. And I think many people who maybe had felt tied in for quite some time were now able to spread their wings and start to say, "Okay. How do we work with this in a creative way? What are the needs of our communities and our families and our regions and so on?" And I think that without this, we wouldn't have got there.” ~ Director, Mental Health and Substance Use Services

57

based on trust and a collective leap of faith. The ability to develop, refine and use adaptive structures to support this initiative has been based on a solid foundation of mutual trust, respect, and shared decision making, accountability and responsibility. This early commitment has been strengthened by ongoing monitoring and evaluation data that has supported reflection, review, adaptation and emergent outcomes that have expanded possibilities.

Respect and recognition for strengths, regional variability, and shared opportunities

The respect and recognition for strengths, regional variability and shared opportunities are central to BC’s DTFP systems proposal. The system level priority areas have been endorsed at both the provincial and regional levels and are being addressed within regional and local contexts. One of the strengths of this approach is to link and leverage opportunities across health authorities by sharing materials, methods, staff resources, and lessons learned to plan and shape the work across the province. This collaborative planning and practice is a significant accomplishment as it removes barriers between institutions, organizations, and sectors and has fundamentally shifted the norm in collaborative planning from dividing mandates to co-creation. This work is supporting a shift towards a constructivist model to adapt concepts and principles to local context rather than replicating models or scaling up innovation.

Deepening understanding of evidence and types of evidence – the shift from “best practice” to principled practice

The DTFP has contributed to deepening understanding and respect for different types and sources of evidence that inform practice including Indigenous Knowledge, practice knowledge, intuition, client and family experience, and research knowledge.

Substance use clinical work is inherently culturally and contextually dependent, with heavy reliance on clinical judgement, reasoning, and discretion. Access to authentic and valuable sources of knowledge expands practice and builds resilience. This understanding is in opposition to the concept of a “best practice”. What has been uncovered through this initiative is the depth and breadth of evidence-informed practices that expand treatment options and a shift to principled practice – principles and aspirations that are congruent with a mission-driven workforce.

Importance of client and family engagement in planning

The caution learned in the first year of this initiative is that there is often a disconnect between the wish to support client-centered services and the meaningful participation of clients and families as active agents in the design, delivery and evaluation of system change initiatives. System change initiatives are often provider-biased. It was an uncomfortable eye-opener in review of the DTFP sector reporting in the first year of this initiative to realize we were not systematically reporting client and family participation.

58

Importance of distributed leadership

One of the key lessons learned is the amount of commitment and in-kind support required from very busy senior clinical leaders to collectively steward this initiative. BC is very fortunate to have senior visionary leaders with collective appreciation for the significance of this opportunity.

This initiative has been strengthened by identifying and supporting clinical champions from within the system and the recognition of the importance of social capital and existing formal and informal networks.

Finally, involving formal and informal leaders in planning, and evaluation on an ongoing basis has been critical to developing and strengthening partnerships.

Importance of relationship building and a whole systems approach

The intentional focus on relationship building during the early stages of change (consent to change / understanding the current context) and throughout the change process has been critical to the success of this initiative. Fostering linkages and exchanges across diverse systems of substance services and supports has a basis in social interactions. The attention to the importance of relationships and trust, and reflection during change in the initiative management, has also been critical for team processes and structures.

The whole sector approach reinforced by intentional cross-sector participation in KE activities has supported a collective goal of strengthening the system of care that privileges the client voice.

Importance of ongoing assessment of strategic alignment

Traditionally, the focus on strategic alignment occurs at the beginning of an initiative. Illustrating strategic alignment with other priorities / plans already endorsed, such as the Healthy Minds, Healthy People-A 10-year Plan to Address Mental Health and Substance Use in BC and other policy directives is critically important to establishing and maintaining consent to change.

In recognition of the dynamic nature of system change, ongoing re-assessment of strategic alignment is critical, both to guard against untended negative outcomes (e.g. initiatives working at cross-purposes), and to optimize links to strategic policy directives.

A-ha moments of BC’s DTFP community

“I think my a-ha moment is, I thought everybody out there was asleep, but they weren't. They're awake, and they're quite attentive to things that have promise and hold their attention. They only appear to be asleep because they kept getting the same old same old. And now the whole approach has really activated people and they're very excited about a multitude of things, and they speak in the language of knowledge exchange …. I'm hearing more and more people use those kinds of language. So I thought they were asleep, but they weren't. They just weren't interested because it was the same boring stuff they had before prior to the whole DTFP KE” ~ Director, Mental Health and Substance Use Services

“It’s not a medical process. I think in our language in our work now we are looking to ecological and social frameworks… even changing the name to supports and services and moving away from medical treatment language.”~ Addiction KE Leader

“We really moved from a manualized interaction to a meaningful interaction.” ~ Director, Mental Health and Substance Use Services

“The a-ha of the potential of unleashing community through supporting the development of communities of practice. You’re providing the space and legitimacy to share and learn together and do this across sectors. [CoP members] learn from one another, and they really, ten-toes-to-the-curb, figure out how to nimbly work across siloed, unsympathetic structures.” ~ Addiction KE Team Manager

59

Importance of ongoing management of expectations

In recognition of the dynamic nature of systems, this initiative was designed to be adaptive to change and includes a long list of system focus areas that align with existing provincial and regional priority areas. It has been critical to recognize the potential for overload, protect against scope creep and collectively manage expectations.

Importance of alignment within and across priority areas

Within and across Health Authorities there are a number of evidence-informed practices being strengthened (e.g., MI / Change Talk, Core Addition Practice, Mindfulness, Client-Directed Outcome Informed practice, Trauma Informed Practice) across a variety of settings and sectors. The volume of practice change targets and settings can be overwhelming and have negative consequences on an operations level. There is an intentional focus to reflect with leadership on how different initiatives are aligned, and to review and link curriculum to strengthen existing initiatives in an ongoing manner.

Conclusion Statement of Project Findings and how these findings have contributed to DTFP Outcomes

The progress towards outcomes has been substantive and shows the return of investment of the DTFP funds in BC to strengthen the system of care. We have been actively learning from practice that is supported by ongoing evaluation, rapid appraisal and feedback. Across priority areas of focus, we have been ambitiously opportunistic. For example, we have engaged 27 sectors in a three-year exploration and development of what it means to support clients and families with substance use concerns. This engagement has contributed to the development of multifaceted, multileveled strategies to address stigma, engage primary care, enhance linkages to Aboriginal communities, engage youth and families, and build workforce and evaluation capacity. With continued investment, the Addiction KE infrastructure and mechanisms developed to date in B.C. have the potential for increasing returns to strengthen the system of care.

Important considerations for sustainability

Advantages of ongoing investment in KE infrastructure/mechanisms include:

support KE across tiers, settings, sectors

support growth in knowledge base and expand evidence-informed practice

support workforce retention and resilience in complex, changing environments

support knowledge retention and generation within the system

support reflective practices, evaluative capacity, and critical systems thinking

support rapid appraisal and diffusion of innovation

provide forums to recognize and celebrate service excellence

create opportunities for emergence and service innovation (“butterfly effects”) beyond predefined outcomes

60

Sustainability Gains

Sustainability gains representing regional and provincial commitment to addiction KE infrastructure and mechanisms include:

KE being incorporated as business as usual to support workforce development (KE core component of Clinical Improvement Steering Committee in the Fraser region, KE coordinator positions created in the Interior region)

KE to support development of eight integrated primary care community clinics in the Interior region: success of DTFP-supported proposal to build on work initiated with the DTFP is now funded across the Interior with a KE lead position becoming a health authority-funded position

Clinical supervision: DTFP-funded appreciative inquiry contributing to KE to support exploration of clinical supervision across the Fraser region

Provincial Core Addiction Practice Steering Committee updated Core Addiction Practice facilitator and participant materials and continued exploration of ongoing provincial stewardship and development

Core Addiction Practice facilitator mentorship – regionally and provincially

Ministry of Health recognition for KE approach in the development of residential standards (Amanda Seymour, former ministry lead for the DTFP, received the BC Government Celebrating our Successes – People’s Choice Award – Stakeholders Relations category)

KE approach now being used to support planning and implementation of the BC Government Healthy Minds, Healthy People 10-year plan

Commitment by health authorities to provide transition funds to continue the Addiction KE Team work in BC

Recommendations for the Project Sponsor

Health Canada

Knowledge Exchange National Feedback Loop

We found the process of Health Canada required semi-annual reporting valuable for communicating progress towards outcomes and supporting shared learning at the provincial, health authority and organizational levels. The semi-annual progress towards outcomes communications and generative processes have supported shifts in language, introduced sensitizing concepts, and incorporated KE processes that are sensitive to culture and context. We recommend that Health Canada consider formalizing a KE approach around semi-annual reporting processes by creating feedback loops to individual jurisdictions and across jurisdictions. The sharing of learning across jurisdictions would support strengthening systems, growing our learning edge, and building our knowledge base about how to do this work. The core of this work involves the interplay of culture and context and these opportunities would help to expand possibilities, inspiration, and meaning, and opportunity.

61

Health Canada advocate for the DTFP funding focused on substance use as a health issue, not a criminal justice issue

There is a global evidence base that substance use is a population health concern and effective responses to supporting people and communities focus on strengthening the health of individuals and communities. Treating substance use as a criminal justice issue versus a health issue is not an evidence-based approach.

Shift structure and accountability framework for the DTFP from health service delivery processes to education and educational processes

Health Canada does not have a mandate to fund services. And the DTFP strengthening systems call is structured as if it is a service relationship. Many performance indicators for this initiative reflect metrics about the uptake or link to a specific tool/ product/ information. This information-centric approach is not a fit with how KE is conceptualized in our initiative. What Health Canada is investing in, at a systems level, is a learning process. "Individuals and systems change because they learn."14. Learning takes place in the zone of complexity where individuals engage with an uncertain situation in a meaningful way. We recommend a shift from health service delivery processes to education and educational processes, and structuring the relationship that way so our accountability is not to widgets, it is to learning. KE is a sensitizing concept that is measurable by exploring system level metrics that support understanding learning ecologies across sectors, tiers and settings where substance use service provision occurs.

Rethink evaluation framework and methodologies

Evaluation approaches that examine systems involve both monitoring and evaluating relationships, behaviours, programs, projects, initiatives, policies and sectors. As such, evaluations in a systems context will typically involve designing multifaceted and multileveled approaches designed to look at the interplay between interconnectedness and disconnectedness in the system.

Complex adaptive systems do not consist of a single theory of change, and usually encompass a collection of theories and constructs that have conceptual integrity within themselves. Understanding multiple contexts, system adaptations, dynamical change, ecologies, and characteristics of learning cultures will more than likely contribute to more effective and more utilization-focused monitoring and evaluation approaches. Attention to the development of purpose, intention, and transformation, the meta-level of systems work/change, would be an area worth exploring.

Engaging stakeholders throughout is crucial: engagement involves exploring meaning, developing evaluation questions and purpose, and collecting data with integrated reflective processes. A range of methodologies can support monitoring and evaluation approaches in a system context including developmental evaluation, Outcome Mapping, generative dialogues, and reflective practice – to name a few.

Linear evaluation approaches (focused on outputs, outcomes and impacts) are not typically an effective approach at reflecting complex and dynamical system change. It is more helpful to consider a contribution lens or how relationships, behaviours, programs, projects and initiatives have contributed to systems change. Logic models have limited value in system change; logic models are more effective

14

Senge, P. (1990). The Fifth Discipline: The Art and Practice of a Learning Organization. New York: Doubleday

62

when there is high agreement and high probability of intended outcomes which represents a very small fraction of the context in which system change occurs.

Ministry of Health - British Columbia

Support thinking about addictions as a provincial system of care with elements, of course, that are unique to health authorities, as there is far more common ground across the province in terms of what clients need.

Continue to use open, transparent planning processes in partnership with the BC Substance Use Network to facilitate the planning, development and implementation of any future DTFP proposals.

Continue to use and support KE processes in policy development and implementation that encourage and support shared learning and collective efforts focused on collectively supporting clients and families with substance use concerns.

Building on the success of the DTFP, continue to support whole systems approach to policy development and implementation across government ministries.

Building on the success of the DTFP, continue to use and support KE to develop and nurture evaluative capacity and systems thinking. Credible use of evaluation requires the ability to interpret data in a way that makes sense or gives meaning to practice, programs, and systems.

Recommendations for similar projects

Incorporate a whole systems approach with meaningful participation of clients and families and service providers across sectors, organizations, settings and tiers. Cross-sector forums provide the opportunity for participants to: be mindful of inclusion, compassion and engagement across settings, sectors, and tiers; address misattributions about the role of other sectors in providing substance use services and supports; increase understanding of other workplace cultures, settings and service contexts; explore and foster partnerships across sectors and silos and expand knowledge of evidence-based approaches.

Treat KE as a sensitizing concept to promote shared understanding and conceptual clarity. We recommend using an appreciative KE approach that celebrates practice excellence and unleashes the creative power of community.

Be humble and curious. Shift from an attribution to a contribution lens in ongoing planning, monitoring and evaluation that are appreciative of complexity and systems thinking.

Support distributed leadership and evaluative capacity building beyond traditional evaluators to empower champions across sectors, settings and tiers to recognize and support collective knowledge and emergent opportunities.

Shift from best practice to principled based practices to support adaptation of principles and concepts sensitive to different contexts and cultures.

63

Recommendations for the DTFP

Build on success and opportunities identified in work underway including approaches to evaluation. Expand spectrum of planning possibilities beyond the traditional planning processes built using gap analyses. We recommend using an opportunity analysis to build on the strengths of the current program to inform the next DTFP call.

Support KE and shared learning across DTFP jurisdictions / initiatives to share and consolidate learning in a meaningful way that informs the next call.

Incorporate systematized feedback mechanisms, such as reflective processes across jurisdictions as a required component of the next DTFP to support shared learning.

Incorporate project learning about system complexity and administrative requirements and expand time lines for both the development of the next DTFP call for proposals, and jurisdictional responses to the next DTFP call.

64

Presented in the order in which they appear in the text:

Achterbergh, J. & Vriens, D. (May-June 2002). Managing viable knowledge. Systems Research and Behavioral Science, 19(3), pp. 223

Healthy Minds, Healthy People – A ten year plan to address mental health and substance use in BC. BC Ministry of Health Services, Ministry of Children and Family Development, November 1, 2010

Earl, S., Carden, F., & Smutylo, T. (2001). Outcome Mapping. Ottawa, ON: International Development Research Centre. http://www.outcomemapping.ca/

Sensitizing Concepts Definition: http://tinyurl.com/celpf62

Patton, M. Q. (2010). Developmental Evaluation. New York, NY: Guilford Press (pp. 146-9)

Keeping Youth Connected, Healthy and Learning Report. http://www.carbc.ca/Portals/0/school/keepingyouthconnected.pdf

Pratt, D. (January 2012). Pedagogical Signatures in the Health Professions: Commonalities and Differences, Interprofessional Education Network Day

Routhieaux, R. L. & Tansik, D. A. (1997). The Benefits of Music in Hospital Waiting Rooms. Health Care Supervisor, 16(2)

Janulis, P. F. (2010). Understanding addiction stigma: Examining desired social distance toward addicted individuals. DePaul University, Chicago, Illinois. Theses and Dissertations. Paper 16. http://via.library.depaul.edu/etd/16

Link, B. G. & Phelan, J. C. (2001). Conceptualizing Stigma. Annual Review of Sociology, 27 (363-85)

Grigg, G. (2011) A Preliminary Scan Into Clinical Supervision in Health Authorities in BC: An Inquiry About Possibilities

Senge, P. (1990). The Fifth Discipline: The Art and Practice of a Learning Organization. New York: Doubleday

Appendix A. Sample of BC DTFP evaluation tools

Appendix B. Interim Process Evaluation Report

Appendix C-H. Health Authority Specific Cross Sector Engagement Graphs