Implementation research and science

This PPT and other resources from:

http://public.me.com/johnovr

1

John Øvretveit,Director of Research, Professor of Health Innovation

and Evaluation, Karolinska Institutet, Stockholm, Sweden

04/13/2023

Objectives – after the workshop I will be better able to:

Explain what implementation science and research is

Describe the elements of an implementation programme

Explain the strengths and limitations of an implementation research study

Plan an implementation study which is useful to the users of the study

Know where to go for more details

204/13/2023

How - Learning process

Use tool to analyse implementation programme(CESSiM)

Using tool to assess implementation (REAIM) Assess quality of research using DAGU tool Plan implementation and or research using these tools

Process – John explain, You try, We assess. Do you have Resources – outline, questions and cases?

Johnstone, Ammentor, Helm (1,3,2) 3,

Timetable Introductions what is implementation science and research?

you say “hallo”

Example implementation programme Group case analysis on elements CASSiM

Implementation assessment Group case analysis REAIM

Assess quality of research DAGU

Planning an implementation study – phases, your examples? Group exercise

Guide to resources & Q&A 404/13/2023

Is implementation about evidence based practice?

What is evidence based medicine, evidence based healthcare?

Professionals using research to make better informed decisions in their practice

Usually evaluations showing the most effective treatment for a patient

But - can be evaluations of service delivery model (eg team based care) or health promotion intervention

Implement = enable more practitioners to use a more effective method in their practice

504/13/2023

Implementation also about using improvement methods

An example of an “improvement method”?

PDSA testing: A quality team tests a change to reduce waiting times using a Plan

Do Study Act cycle (eg follow ups optional)

Flow process mapping A quality team maps patient pathway and reduces delays and errors

Implementation not just about getting more people to use effective treatments

6

Definitions: Implementation science and research

“Implementation research is the scientific study of

methods to promote the systematic uptake of

research findings and other evidence-based practices

into routine practice, and, hence, to improve the

quality and effectiveness of health” Eccles and Mittman, Implementation Science 2006.

704/13/2023

Implementation research

Mostly describes, evaluates and explains an

implementation in different real-life settings.

804/13/2023

Confusion warning: IS overlaps with:

‘‘translation research’’: ‘‘an effective translation of the new knowledge, mechanisms, and techniques generated by advances in basic science research into new approaches for prevention, diagnosis, and treatment of diseases essential for improving health’’ Many other related types of research

Knowledge translation research Quality improvement research

etc 9

04/13/2023

Say hallo to the person sitting next to you

Name, title and where you work Are you doing implementation research or

involved in an implementation project? What do you want to learn about

implementation research and practice? Specific question

1004/13/2023

Group rules

Choose chairperson, scribe to report, cuckoo clock

Chairperson says how much time left, when the

group will summarise and agrees the task and

questions

Make room for everyone to contribute.

Cell phones off!

1104/13/2023

Part 1: The elements of an implementation programmeExample: implementing and maintaining a care pathway for the last days of life

CESSiM Content?

Define What was the “thing” implemented?

Efficacious? Was it evidence-based: was the guideline for the care pathway

based on evidence of effective care?

Structure and systems What was the implementation structure: the people and groups

at different levels who took actions to enable others to follow the guideline

1204/13/2023

CESSiM - continued

Strategy: phases/steps, actions, Were there phases or steps in the implementation (different

things done at different times?) What were the implementation actions: what were the 5 main

things done by these people and groups to enable others to follow the guideline? (by whom and when?

implementation Methods: Did they use implementation methods – systematic ways to do

enable change?

1304/13/2023

Case

.

1404/13/2023

Points

1 Implementation is different from the change being

implemented

Before/after change – anti-biotic prophylaxis 1 hr before surgery

Implementation actions – training, reminders, performance data

collection and feedback

2 Efficacy X Implementation = Effectiveness

1504/13/2023

What does “fidelity” mean? Exact “digital” copy of proven practice

Key questions:

For which interventions is fidelity essential

For which is adaption needed

Are some parts of the intervention adaptable and some not (“core”) Eg must copy the change difference

But can implement it in different ways

Or do you have to copy implementation strategy as well?

1604/13/2023

Making a difference depends more on effective implementation than on an efficacious intervention

Drugs don’t work if the patient does not take them

Implementation is methods to enable patients to get, and take medicines in the way proven to be effective

An efficacious communication training for physicians will not improve patient-physician communication if it is not implemented effectively.

1704/13/2023

One Implementation assessment method Reach

(% target population participated)

Efficacy (outcome success rate if implemented as in original research)

Adoption (proportion of settings, practices that adopt the intervention)

Implementation (extent to which the intervention is implemented as intended in the real

world)

Maintenance (extent to which a program is sustained over time)

18

Case: Ammentorp & Kofoed 2011

Communication skills training in Danish hospitals

Only consider Phase IV and V studies Effectiveness studies: Intervention studies tested in a pre/post

design in two different departments (phase IV) Implementation study evaluated in large scale (phase V) Phase III was an RCT efficacy study

Group – Note for two departments (phase IV), what was Reach, Efficacy, Adoption, Implementation, Maintenance Is this a good method to use to assess implementation? Are all parts needed for implementation study?

1904/13/2023

Case: Ammentorp & Kofoed 2011

2004/13/2023

1 Effectiveness of treatment implementation is more important than efficacy of the improvement

2 Context factors affect each step  21

Use CASSiM and REAIM to explain Falls Guideline Failure (Helm et al 2006)

Groups – read out loud the abstract. From abstract in 15

mins summarise Content, Efficacious, Structure and systems, Strategy,

Implementation methods 15 mins summarise: Reach, Efficacy, Adoption,

Implementation, Maintenance

15 mins: what is your explanation for figure 2 and 3?

22

Case

Results: In the internal medicine ward, the target incidence of 6% was met for four of the 18 months in the intervention period. In the neurology ward, the inci- dence target of 11% was met in five months.

Discussion: Barriers to change and enabling factors may only become apparent during the implementation process itself. A strongly perceived need to change daily practice, a simple guideline, the hospital board’s support, an understanding of local barriers, monitoring of outcomes, a locally tailored multifaceted implemen- tation strategy, and voluntarily cooperating nurses are no guarantees for success. 2

304/13/2023

.Points .

2404/13/2023

Assessing the quality of a research study - DAGU Description

Could we copy this implementation?

Were the measures and data gathered the right ones (were resources used and saved measured?)

Attribution

What else could have explained the findings (how good was the design for the users questions)?

Generalisation Does the study explain in which settings similar findings could be

expected, and in which settings findings may be different?

Use Is it presented in a way users can easily understand and will it help with

users practical decisions? 25

Give 0-5 score for Helms et al 2006 study Description

Could we copy this implementation?

Were the measures and data gathered the right ones (were resources used and saved measured?)

Attribution

What else could have explained the findings (how good was the design for the users questions)?

Generalisation Does the study explain in which settings similar findings could be

expected, and in which settings findings may be different?

Use Is it presented in a way users can easily understand and will it help with

users practical decisions? 26

Plan research or an implementation

Group for designing an implementation project or programme

Group for planning a research study of an implementation

List implementations you want to plan

2704/13/2023

Groups Present timeline picture of main phases and data for monitoring

Group work:

Decide the customer for your plan – who is to use the design or the research when its done?

Use CASSiM to define elements

Use REAIM to define data needed to monitor or for research

Or look at frameworks at end of Johns Outline paper and choose which is most suited to the users questions

Use DAGU to decide how to improve your research design

2804/13/2023

Points .

2904/13/2023

. .

3004/13/2023

04/13/2023 31

Conclusions

Each person write down and then share in the group:

1. These were the main points…

2. This was new or surprising, for me…

3. The most useful idea for my work was…

4. What I would like to find out more about…

DETAILS

3204/13/2023

Resources

Handouts

Tools

Web sites

.

. .

3304/13/2023

Readings

Overview and guide to the literature: Fixsen et al 2005 .

. .

3404/13/2023

Research assessment – DAGU or AGUDMP Attribution

(extent to which intervention influenced the change in outcome data documented, & relative influence of components or synergistic influence)

Generalise Used

(Useful - matched to users questions – then gives information and presented to suit audience)

Description (theory informed data collection) Measures valid Presentation 3

504/13/2023

04/13/2023 36

Conclusions

Each person write down and then share in the group:

1. These were the main points…

2. This was new or surprising, for me…

3. The most useful idea for my work was…

4. What I would like to find out more about…