Randomized trials in complex settings: Lessons learned from developing a healthcare prevention intervention for frequent emergency department visitors

Nuffield conference: Evaluation of complex care

22 June 2015

Person-centred care and patient activation

Who are we?

1

Dr Gustaf Edgren

• MD PhD, Karolinska Institutet

• Associate professor of Epidemiology, Department of Medical Epidemiology and Biostatistics, Karolinska Institutet

• Scientific advisor, Health Navigator

Magnus Liungman

• UK Country Manager Health Navigator

• MSc Business Administration, Stockholm School of Economics

• UCLA Anderson School of Management, Los Angeles

We are presenting work from a 5-year scientific collaboration

2

5 county councils (regional payors) and 12,000 participating patients in the study

All 15 acute hospitals in the 5 county councils participate

Health Navigator develops and delivers the actual intervention program

Our main messages today

3

A randomised controlled trial is often the only viable option to achieve sufficiently strong evidence – especially if you want to measure effects of interventions

Different RCT variants come with different advantages and can sometimes provide different answers

With iterative, adaptive designs, randomized trials can be used to not only develop, but also implement new interventions

Starting point was the fact that a small part of population constitute a large share of the total healthcare cost

4

Note: Example from Stockholm County Council, Sweden. 1 £ = 12,8 SEK (April 2015) Source: Stockholm County Council, Swedish Bureau of Statistics, Health Navigator analysis

Population

10% of population (200 000 people), 75% of total healthcare cost (30 billion SEK)

1% of the population (20 000 people), 30% of total healthcare cost (12 billion SEK)

Total healthcare cost (Billion SEK)

2,000,000

0

40

20

30

10

1,000,000 500,000 1,500,000 0

The highest care consuming patient group is dynamic, which put constraints on possible solutions

5

Increasing frequency of A&E attendances

Period of non-elective activity starts, often repeated admissions

Integrated care package in place, healthcare utilisation stabilises

Patient flagged as high risk patient

Secondary prevention initiatives initiated

Healthcare cost per patient

Time

OUR AMBITION

Up to 1 year, often less

Our aim was to develop a new effective case management intervention for frequent emergency department visitors

6

The implementation of the intervention had a high degree of uncertainty on approach and target group

7

Given the uncertainty we needed an effective evaluation method and RCT was identified as most attractive option

8

Matched control

Historical controls

RCT

Uncontrolled trial

Options for study design

• Regression to the mean • High risk of residual confounding

We worked with two different RCT models to ensure our results

9

All patients

Target group

Invited patients

Participants

Excluded

Control

“Non-participants”

No

RCT, Zelen’s design

Intervention group: ”Intention to Treat”

No

Yes

Yes

Meets selection criteria?

Randomisation

No

Wants to participate?

Yes

All patients

Invited

Target group

Intervention

Excluded

“Nay-sayers”

Control

No

Intervention group

No

Yes

Yes

Meets selection criteria?

No

Wants to participate?

Yes

“Traditional” RCT

Randomisation

The RCT variants came with different advantages

10

• Zelen’s design is cheaper • Mimics real-life • Possible to ’sell’ as not

research

• Better statistical power • Provides estimates of efficacy • Less sensitive to non-

participation

RCT, Zelen’s design “Traditional” RCT

+ _ • More expensive

• Takes more time to get patients into the intervention

• Ethical aspects?

• May underestimate effects of the intervention

• Ethical aspects?

We used an adaptive approach during the study

11

Traditional approach Adaptive approach

VS

The protocol was iteratively improved through the use of interim analysis

12

Frequent interim analyses Assessing effect of intervention

New knowledge Improved opportunities to ensure efficient and working intervention

Modification Gradually modification and standardization of the intervention model

Delivery of intervention According to current protocol

The RCT methodology proved critical for the successful iteration and development of the intervention

13

-10

-5

0

5

10

15

20

25

30

2010 2011 2012 2013

Reduction in total healthcare cost (%)

• Improved selection • Improved training • Better support for

intervention delivery

• Standardisation • Better selection

and prediction • Experienced nurses

• New intervention • High enthusiasm • Small number of

patients

• Scale-up • Untrained nurses

Today, five years later…

14

• The case management intervention is permanently implemented in Stockholm

• Reducing A&E admissions with 2-4% on system-wide level • Readmissions within 30 days have declined with 15-20%

• Based on the results achieved in Sweden, Denmark has taken a national decision to implement the intervention

• It will be implemented as a huge RCT that starts now and will go on at least until 2018

• In UK we are now starting up RCTs to test and improve the interventions in several CCGs

To summarize…

15

A randomised controlled trial is often the only viable option to achieve sufficiently strong evidence – especially if you want to measure effects of interventions

Different RCT variants come with different advantages and can sometimes provide different answers

With iterative, adaptive designs, randomized trials can be used to not only develop but also implement new interventions