Cluster Randomised Trials:

Sources of Bias

Ada Keding

Professor David Torgerson

Problems with Cluster

Randomisation• Possible Selection Bias;

• Inadequate uptake of intervention by either

group reduces study power;

• Sample size needs to be increased

(typically between 50% to 100%), which

will often increase the cost and complexity

of a trial.

Selection Bias in Randomised

Trials• This should not occur in an individually

randomised trial unless the randomisation has been subverted.

• However, in cluster trials it is possible for selection bias to occur after successful cluster randomisation.

• This defeats the objective of randomisation.

Selection Bias in Cluster Trials

• Randomisation abolishes selection bias in

a cluster trial ONLY if all eligible cluster

members, or a random sample, within the

cluster are included in the trial.

Recruitment into cluster trials

• A key issue is individual participant

recruitment into cluster trials.

• There are a number of ways where biased

participant recruitment can occur, which

can lead to baseline imbalances in

important prognostic factors.

UK BEAM Trial

• The UKBEAM pilot study used a cluster design. Eligible patients were identified by GPs for trial inclusion.

• GP practices were randomised to usual care or extra training.

• The ‘primary care team’ were trained to deliver ‘active’ management of backpain.

UK BEAM Selection bias

• The pilot showed that practices allocated

to ‘active management’ were more adept

at identifying patients with low back pain

and including them in the trial.

• Patients had different characteristics in

one arm than the other.

UK BEAM participant

recruitment

Roland = 8.9

Aberdeen = 28.6

SF36 = 61.8

165 Recruited Participants

13 Active Management

102,063 registered patients

Roland = 10.3

Aberdeen = 34.2

SF36 = 55.2

66 Recruited Participants

13 Usual Care

106,834 registered patients

26 Practices

Type title here

P = 0.025

P = 0.001

P = 0.03

UKBEAM pilot study.

Recruitment by Practice Status

0

20

40

60

80

100

120

140

160

180

Apr 98 May 98 Jun 98 Jul 98 Aug 98 Sep 98 Oct 98 Nov 98 Dec 98 Jan 99 Feb 99 Mar 99 Apr 99

Nu

mb

er o

f p

arti

cip

an

ts

Another musculoskeletal trial

• SAPPHIRE trial of guidelines educating GPs on identification and management of shoulder pain

• Very similar design to UKBEAM

• At first TMC evidence of recruitment bias occurring – independent members of TSC recommended continuation of trial

Feb

Mar

April

May

June

July

Aug

Sept

OctNov

Dec

JanFeb

Mar

April

May

June

July

Cumulative actual - untrained 4 10 13 19 25 26 27 27 28 35 38 40 41 43 44 44 47

Cumulative actual - trained 10 22 29 37 44 50 57 66 67 73 78 83 88 93 97 104 104

0

20

40

60

80

100

120

Nu

mb

ers

recru

ited

Month of recruitment

Cumulative actual - untrained

Cumulative actual - trained

Review of Cluster Trials

• Because of the ‘BEAM’ problem York

Trials Unit decided to undertake a

methodological review of cluster trials.

• They identified all cluster trials published

in the BMJ, Lancet, NEJM since 1997.

Puffer et al. BMJ 2003;327:785.

Results

• We identified 36 relevant trials. ONLY 13 had

identified participants prior to randomisation.

• Of the 23 not identifying participants a priori 7

showed evidence of differential recruitment or

consent.

• Other biases included differential of inclusion

criteria or attrition.

• In total 14 (39%) showed evidence of bias.

Misleading trial

• One trial (Jorhdoy, Lancet 2000) where there

was no evidence of biased recruitment was later

found to have suffered recruitment bias in

another publication.

• This was an RCT for home care for terminally ill

patients.

• There was, no evidence, in the Lancet paper of

a problem. BUT…

Baseline Characteristics

Intervention Control

Live in Flat 40% 23% P < 0.001

Married 67% 59% P = 0.07

Access to

help

80% 70% P = 0.04

P values adjusted for clustering.

Jordhoy Palliative Medicine 2002 16:43-49.

Study Were patients

identified prior to

randomisation?

Identification prior to

randomisation possible?

Is recruitment bias likely?

Althabew1 No No Not likely –

Baquiw2 No No Not likely –

Bellaryw3 Yes Yes Not possible

Campbellw4 No Yes - Possible –

Cheynew5 No Yes Possible –

Clarkew6 No Yes - Possible –

Cummingw7 No No Not likely –

Daviesw8 No No Possible –

Doigw9 No No Not likely –

Hiscockw10 Yes Yes Not possible

Jewkesw11 No Yes Possible –

Jeyaratnamw12 No No Possible –

Kersew13 Yes Yes Not possible

Kumarw14 Unclear No Unclear

Melesew15 Yes Yes Not possible

Moetw16 Yes Yes Not possible

Pasanenw17 Yes Yes Not possible

Rahmanw18 No No Not likely –

Ruelw19 No Yes – Not likely –

Soligardw20 Yes Yes Not possible

SUMMITw21 Yes Yes Not possible

Tolw22 No Yes. Possible –

Woodw23 No Yes – Possible

Zengw24 Yes Yes Not possible

2008 cluster trials

Summary of 2008 trials

• Of the 24 trials, eight could have used

individual randomisation as an alternative

to cluster allocation. Seven studies could

have recruited participants prior to cluster

randomisation but did not. In 8 studies

where recruitment bias was possible more

than half (5) demonstrated some evidence

of differential recruitment rates.

Brierley et al, 2012 J Eval in Clin Prac 18:878.

Copyright ©2009 BMJ Publishing Group Ltd. Morrell, C J. et al. BMJ 2009;338:a3045

Fig 1 Trial profile of clusters, all women and women with six week EPDS score >=12 in control and intervention groups. Note: 597 women were not sent 12 month questionnaire as their baby was <12 months, and 1879 women were not sent 18 month questionnaire as

their baby was not 18 months when follow-up interval ended

Note: 35 per

cluster for

control group

and 44 per

cluster for

intervention –

26% difference

Continuing evidence of a

problem• In 2018 is there still a problem with cluster

randomised trials?

• Let’s look at the latest research on this

from Dept of Health Sciences at York.

Meta-analysis of age for

individually randomised trials

Difference = 0.005

(95% CI -0.026 to 0.035)

I2 = 0%

Meta-analysis of cluster trials of

age

Difference in age: -

0.05

(95% CI -0.057 to -

0.0426)

I2 = 93.2%

Bolzern et al

2018 – J

Clin Epi – in

press

Hip Protector Trial

650 in hip protector group

8 Clusters

1075 in control group

15 Clusters

1725 eligible participants

Kannus. N Eng J Med 2000;343:1506.

At this point trial is balanced for all co-variates

Hip Protector Trial

446

At baseline

69%

204 refused

(31%)

650 in hip protector group

8 Clusters

981

At baseline

91%

94 refused

(9%)

1075 in control group

15 Clusters

1725 eligible participants

Selection Bias

Figure 2

The Lancet 2016 388, 2264-2271DOI: (10.1016/S0140-6736(16)30384-1)

Copyright © 2016 Elsevier Ltd Terms and Conditions

Figure 1

The Lancet 2016 388, 2264-2271DOI: (10.1016/S0140-6736(16)30384-1)

Copyright © 2016 Elsevier Ltd Terms and Conditions

Summary

• Cluster trials are open to bias and too

many researchers do not design them

sufficiently well to avoid their vulnerability

to selection bias.

• Stepped wedge studies will probably be

more at risk than ‘normal’ cluster trials as it

may be more difficult to recruit before

randomisation.