What does this mean for you? Meeting the Challenges in Implementing Evidence-Based Practices and Treatments in Ontario

What does this mean for you?

Meeting the Challenges in Meeting the Challenges in Implementing Evidence-Based Implementing Evidence-Based

Practices and Treatments in OntarioPractices and Treatments in Ontario

Presentation ObjectivesPresentation Objectives

Introduction to Evidence-Based Practice• What is it?

• Why use it?

• What is needed?

• Evidence to practice process

Role of the Centre of Excellence in the uptake of Evidence-Based Practice

Truth or Dare …Truth or Dare …

If you’ve been doing something for twenty years and we tell you that what you’re doing is not effective – or may even cause harm – would you change your practice?

WHY or why not? What if we can show you strong evidence?

What kind of Evidence would convince you?

Those who would not change?

Would you then?

What’s preventing you?

Critical Decision-Making:Critical Decision-Making:How do you make decisions in your practice?

Think about a recent client (child, young person, family) you’ve worked with where you made a significant decision about their care/support.

What criteria did you use to make this decision?

Your intuition (gut feel) about what will work for them.

What you have heard from other professionals in informal exchanges.

Your previous experience with similar cases.

Data about outcomes from alternative care / support options.

What is usually offered by your service / agency.

Feedback from other similar clients about what is helpful.

What you are most familiar / comfortable with.

What you know about cases like this from reading research / theory.

What you learned to do in your own training (no matter how long ago)

© 2005, Research in Practice: Teamwise: Using Research Evidence

Critical Decision-Making:Critical Decision-Making:How do you make decisions in your practice?

NOW… imagine you have a potentially serious medical problem and you seek help from a doctor to recommend a treatment.

What criteria would you like the doctor to use to make this decision?

His/her intuition (gut feel) about what will work for you.

What he/she has heard from other professionals in informal exchanges.

His/her previous experience with similar cases.

Data about outcomes from alternative treatment options.

What is usually offered by the surgery / hospital / clinic.

Feedback from other similar patients about what is helpful.

What he/she is most familiar / comfortable with.

What he/she knows about cases like this from reading research / theory.

What he/she learned to do in his/her training (no matter how long ago).


Evidence-Based PracticeEvidence-Based Practice

How do we *know that what we do works?


(*not just believe)

Really!

……Do we have the evidence to back our beliefs?Do we have the evidence to back our beliefs?

Evidence-Based PracticeEvidence-Based PracticeIt’s not simple, either…

What is evidence?

Would you recognize evidence if it was right in front of

you?

What exactly does “evidence” mean to you?

How can you possibly know when you are using it?

You are not alone…You are not alone…

At the Centre, we are also grappling with defining models of construct relating to evidence-based practice. There is no universal definition when it comes to evidence as it relates to child & youth mental health. What do we mean by…

Evidence

Promising

Effective Model

Evidence-informed

Evidence-based

Emerging

Every child and youth has the right to

demand that anyone involved in practice

decisions knows what is most likely to work

for them, their families and their caregivers.

“That’s the best I can do. If you’d like to see

another child psychologist …”

Limitations of Evidence-Based PracticeLimitations of Evidence-Based Practice

Evidence does not always exist

Evidence derived from a lab in one place does not

necessarily apply elsewhere

Many promising practices have never had the opportunity

to be evaluated in such a way as to elevate them to EBP

Evidence can be developed/tested on homogeneous

samples that do not reflect real life

An effective intervention may not be feasible/appropriate in

certain communities and with certain populations

Culturally Sensitive ApproachesCulturally Sensitive Approaches

How best do you approach the treatment of clients whose characteristics and problems may differ from those of samples studied in research?

(Especially those from minority or marginalized populations.)

gender gender identity ethnicity race social class disability status sexual orientation religion disability status rural urban group

association (i.e. deaf culture) family context

Definition of Evidence: CHSRFDefinition of Evidence: CHSRF

Evidence is information that comes closest to the facts of a matter. The form it takes depends on context. The findings of high-quality, methodologically appropriate research are the most accurate evidence.

Because research is often incomplete and sometimes contradictory or unavailable, other kinds of information are necessary supplements to or stand-ins for research. The evidence base for a decision is the multiple forms of evidence combined to balance rigour with expedience—while privileging the former over the latter.

WEIGHING UP THE EVIDENCE

Making evidence-informed guidance accurate,

achievable, and acceptable

A summary of the workshop held on September 29, 2005

JANUARY 2006

Evidence Based ProgramsEvidence Based ProgramsU.S. Perspective

SAMHSA defines evidence-based programs according to three categories

What is Evidence-Based Practice? What is Evidence-Based Practice? U.K. Perspective

Sources: © 2003, Evidence Network: What Works for Children? © 2005, Research in Practice: Teamwise: Using Research Evidence

In these models: Awareness of research findings alone is not enough

Research evidence complements – not replaces – the many other

forms of knowledge

It’s not a magic bullet

And not simply about searching for the ‘right’ answer

It’s about understanding options available and, with children and youth

or their caregivers, choosing the course of action most likely to lead to

a good outcome

Most Common Evidence-Based Most Common Evidence-Based Practices?*Practices?*

According to Children’s Mental Health Ontario

Cognitive behaviour therapy (65%)

COPE (42.7%)

Wraparound (42.5%)

Behavioural parent training (41.2%)

Brief strategic family therapy (39.2%)

Narrative therapy (38.8%)

“The Incredible Years” (36.4%)

Multisystemic therapy (35.9%)

“Stop Now and Plan” (32.4%)

“Right from the Start” (29.3%)

What has been your motivation for using these interventions or therapies?

* Defined by CMHO as “a treatment that has been developed through research, is supported by the results of controlled treatment studies, and has guidelines and procedures related to its implementation”

Beyond the Evidence Beyond the Evidence What else is Needed?

Practitioners need to be willing and able to challenge

themselves and others

Organizations need to provide professional development

opportunities and validate the need to read, reflect and

debate

Leaders need to promote critical thinking, improve

access to research, identify and spread good practice

across their agencies

Teams should be able to tackle real practice dilemmas

collectively© 2005, Research in Practice: Teamwise: Using Research Evidence

Breakout Breakout Evidence-based Practice: Why or Why Not?

If you are using an EBP in your organization name it

If you are using EBP or not

• What are the barriers?

• What are the facilitators?

• What do you think are the key elements in creating a culture that

encourages and supports Evidence-Based Practice?

In groups discuss the pro’s and cons of integrating Evidence-Based Practices into your practice:

Consider 3 levels: individual, organizational, systemic

Barriers to Evidence-Based PracticeBarriers to Evidence-Based Practice

Limited access to relevant information/models

Evidence does not fit for my community

• Research done on pure samples

• Research done on different samples

• Intervention as researched is not feasible in my

community

Innovation does not equal evidence-based practice

Barriers to Evidence-Based PracticeBarriers to Evidence-Based Practice

Culture may not support this approach

Change is difficult at the best of times

Often viewed as an add-on that takes time away

from direct provision of service

Resources/expertise is often lacking

Access to training and ongoing support

What are we missing?

Surveyed leaders and

practitioners in 80 associated

Child and Youth Mental Health

Clinics across Ontario about

utilization of research-based

information and readiness

for change

Interviewed 12 experts in

implementation science

Made recommendations for

system of care improvements

© 2005, Children’s Mental Health Ontario

Resources

Leadership

Training

Practice Change

Readiness

Multiple methods of support

Culture of professional

development

Conclusion:Conclusion:

To implement, transfer, deploy evidence-based best practices to the field, we need:


Active strategies, collaboration, leadership are important

Sharing tacit knowledge face-to-face is powerful

Resistance to change – from the system, leaders, and practitioners – needs to be recognized and addressed

Change is complex and requires buy-in from a critical mass – create a tipping point!

Distillation of research knowledge into practice guidelines is insufficient to create practice change

Implementation requires a collaborative effort

Strategize on multiple levels: practice, organization, system, nature of evidence, the support plan

Literature Review Highlights:


What does this look like for

you?

Take It One Step at a time… Take It One Step at a time…

Formulate an answerable question Identify search terms for each component of

the question Appraise the evidence Apply the evidence Evaluate your practice

© 2003, Evidence Network: What Works for Children?

Start With a Plan… What is the problem we need to solve?

What is the question we need answered?

Select first choice resource where an answer may be found

(e.g. the Cochrane library)

Design search strategy and carry out search

Appraise and summarize the evidence

Select second choice resource

Poor or insufficient evidence

Good Evidence

Assess the relevance of the evidence to your local situation

and target groups

Consider implications for your practice and resources, and

those of the organizations you work with

Apply the evidence to practice

Evaluate your practice No improvement

Improvement

The Evidence to Practice Process

© 2003, Evidence Network: What Works for Children?

What might be the role of the Centre or others in helping you do this?

Evidence Source ChecklistEvidence Source Checklist

There’s no simple way to gather evidence:

Look to credible sources What are peers saying?

Look for reliable methods• large samples have power to generalize results• control groups – where possible and ethical

Where has this been published?

Is there a consensus?

Look for results over timeIs the entire process transparent and re-producible?

Is there a meta-analysis or systematic review?

Is there open discussion of results and gaps?

Evaluating What You FindEvaluating What You Find

Once you’ve gathered evidence you need to evaluate it:

Can it be implemented? Was random assignment applied?

Is it based on effective principles?

What was the longitudinal impact?

Was there multiple site replication?

Has dosage analysis been conducted?

Were comparison groups used?

What about meta-analysis, expert review and consensus?

Were clients satisfied?

Did the program change something?

D. Andrews, C. Buettner. Evidence-based Practice: Evaluating Supporting Evidence

Evaluating Feasibility: Evaluating Feasibility: CanCan it be it be implemented?implemented?

Is a particular intervention or therapy a good fit for your organization based on:

D. Andrews, C. Buettner. Evidence-based Practice: Evaluating Supporting Evidence

Availability

Affordability

Feasibility

Research and Development

• Relevant, evidence-based information, geared to a variety of audiences

• Best/promising practices

• Practical information (e.g., about funding sources)

• Funding creative, relevant research and building research capacity

“Facilitate knowledge dissemination”

Access to information and research was the number one need identified by regional conference participants.

What you told us:

How Can We Best Meet Your Needs?How Can We Best Meet Your Needs?

EBP at the Centre of ExcellenceEBP at the Centre of Excellence

Consulting services (Intervention)

• Connecting people with experts

• Consulting on projects

• Advising on program development

• Providing training

• Helping “natural leaders” develop to their full potential

“Provide leadership”Consulting and mentoring was the second most common need identified by regional conference participants.

What you told us:


Partnerships and Networks

• Facilitate and enhance existing and new partnerships and networks

• Create opportunities to connect and network

“Play a proactive role in partnership building”

Over one half of regional conference participants identified the need to develop linking, networking and partnerships.

What you told us:


Researchers Researchers DirectoryDirectory Launched October 19, 2005 Open to CYMH researchers in Ontario Must have, or be working on, a PhD Over 100 entries

*Datalife…Finding that perfect partner!*Datalife…Finding that perfect partner!

* With apologies to LavaLife!

Service Provider DirectoryService Provider DirectoryLaunched February 20, 2006Open to CYMH service providers in OntarioOver 145 entries

Education and Training

• Best practices

• More CYMH in more university programs

• More training for frontline CYMH workers

• More CYMH training in other sectors (e.g., justice)

• More opportunities to learn from each other

“Enhance training all the way down the line”

One third of regional conference participants identified education and training needs.

What you told us:


You can still support good research by:

Consuming research Facilitating research Partnering around research Identifying relevant applied questions

Don’t have research capacity?Don’t have research capacity?(… or have better things to do?)(… or have better things to do?)

The Centre seeks to increase capacity in order to increase

Evidence-Based Practice

Where do we go from here?Where do we go from here?

1. The process of implementing evidence-based practice

starts 1 step at a time – today is that first step.

2. You’re not alone – this is why developing a community

of practice around this issue is so important. The

Centre is part of this – as well as other organizations.

3. Working together, we can create a tipping point and

make things happen.

Documents

What does this mean for you? Meeting the Challenges in Implementing Evidence-Based Practices and Treatments in Ontario