Chul Ahn, PhD Song Zhang, PhD UT Southwestern … · Features of Pragmatic Clinical Trials 3. ... be signi cant after adjusting for ... patient outcome Greater chance of contamination

Chul Ahn, PhD Song Zhang, PhD

UT Southwestern Medical Center

1. Introduction 2. Features of Pragmatic Clinical Trials 3. Design of Pragmatic Trials 4. Types of Cluster Randomization Trials 5. Statistical Issues for Stepped-Wedge Design

}  Schwartz and Lellouch (1967) coined the phrase “pragma=c trial”.

}  Explanatory trials: test whether an interven=on works under op=mal situa=ons that control heterogeneity as much as possible to isolate treatment effect.

}  Pragma=c trials: evaluate the effec=veness of interven=ons in real-‐life rou=ne prac=ce condi=ons, which makes results widely generalizable.

}  Pragmatic and explanatory trials are not distinct concepts since trials can incorporate differing degrees of pragmatic and explanatory components.

}  For example, a trial may have strict inclusion/exclusion criteria, including only high-risk and compliant patients (explanatory aspect of a trial), but have no monitoring of practitioner adherence to the study protocol with no formal follow-up visits (pragmatic aspect of a trial).

North American Symptomatic Carotid Endarterectomy Trial (NASCET) of carotid endarterectomy in symptomatic patients with high-grade carotid stenosis

RCT of self-supervised and directly observed treatment of tuberculosis (DOT).

}  Primary outcome: Important to participants

}  Follow-up: Participant burden is no more than usual care

}  Primary analysis: Intention-to-treat analysis with all available data

} Use of electronic health records (EHRs) ◦  EHR database contains a wealth of informa=on that allow efficient and cost-‐effec=ve, recruitment, par=cipant communica=on & monitoring, data collec=on, and follow up.

} Use of Cluster Randomiza=on ◦ Unit of randomiza=on is different from that of analysis.

(example: cluster at clinic or provider level)

Peterson et al, ACC 2004

0

2

4

6

8

≤25 25–50% 50–75% ≥75

% In

-ho

spit

al M

ort

alit

y

Hospital Composite Quality Quartiles

Adjusted Unadjusted

Every 10% ↑ in guidelines adherence → 11% ↓ in mortality

9

What is PIECES™?

Parkland Intelligent e-‐Coordina=on and Evalua=on System

•  Sits on top of EHR/EPIC •  Natural language processing to read EHR •  Near real-‐=me risk stra=fica=on •  Automated protocol ac=va=on •  Pa=ent-‐tailored interven=ons •  Electronic ascertainment of outcomes

PIECES (Parkland Intelligent e-Coordination and Evaluation System)

PIECES

Investigators (Clinicians)

Center for Clinical Innovations Director: Ruben Amarasingham, MD

EHR Epic Other sources

Intervention Group Standard Care

Outcomes

Primary: All-cause hospitalizations Secondary: 30 day readmissions, disease-specific

hospitalizations, ER visits, cardiovascular events, deaths

Facilitated Care IT enhanced-Pieces Practice Facilitators

Weekly reports Care protocols Smart forms

Clinical measures reports

Practice Facilitator

Order sets

Patient reports

Cluster Randomization

PCP

Cluster Randomization: PIECES

}  The unit of randomiza=on is different from the unit of analysis.

}  CRT has become increasing popular in public health and clinical trial research and is used more widely for pragma=c trials.

}  CRT uses the smallest unit without contamina=on as a cluster.

}  Design considera=on: The number of clusters affects sta=s=cal power more than the number of individuals per cluster.

}  Smaller number of clusters increases the total number of pa=ents along with es=ma=on issues. More clusters are beZer if possible.

}  In a CRT we have to account for intracluster correla=on (ρ), which denotes the similarity of outcomes at a given site.

}  The number of required clusters (n) increases as ρ increases. Adding subjects to clusters doesn’t help much when ρ is large.

— For example, if everyone at a given cluster is expected to have exactly the same outcome (ρ=1), we need only 1 subject per cluster

}  Problem: oben difficult to es=mate ρ in planning a study

}  n = the number of clusters (clinics) }  m = the number of patients per cluster }  Hypothesis: H0 : µ1=µ2 vs. H1 : µ1≠µ2 at a

two-sided significance level of α and a power of 1-β

}  Let E(m)=θ, V(m)=τ2 , and γ= τ/ θ }  If τ=γ=0, equal cluster size }  Manatunga et al. (2001) provided the sample

size estimate for the number of clusters for continuous outcomes.

}  Let E(m)=θ, V(m)=τ , and γ= τ/ θ }  If τ=γ=0, equal cluster size }  Kang et al. (2003) provided the sample size

estimate for the number of clusters for binary outcomes. Test the hypothesis: H0 : p1=p2 vs. H1 : p1≠p2

}  Intervention trial to evaluate the effectiveness of PIECES program against standard medical care in patients with chronic kidney disease (CKD) at Healthcare system ABC.

}  Clinics are randomized to receive either PIECES intervention or standard medical care.

}  Patients from the same clinic receive the same intervention.

}  Endpoint= 1-year hospitalization rate

}  Suppose that the number of patients in each clinic is 100 (m=100, equal cluster size). To detect a difference of p1 - p2 =0.05 with p1 =0.1 and p2 =0.15, ρ=0.05, α=0.05, and power=80%, we need 41 clinics per group.

}  If the average number of patients per clinic are 100 with a standard deviation of 45, then we need 48 clinics per group (17% increase in the number of clinics compared with that of equal cluster size).

}  Most pragmatic trials used cluster randomization trials

- Simple cluster randomization trials - Stratified cluster randomization trials - Stepped wedge cluster randomization trials

Institution Project

Kaiser Foundation Research Institute

Strategies and opportunities to stop colon cancer in priority populations

Kaiser Foundation Research Institute

Collaborative care for chronic pain in primary care

University of Pennsylvania

Pragmatic trials in maintenance hemodialysis

University of California - Irvine

Decreasing bioburden to reduce healthcare-associated infections and readmissions

University of Washington A pragmatic trial of lumbar image reporting with epidemiology (LIRE)

University of Iowa Nighttime dosing of anti-hypertensive medications: A pragmatic clinical trial

Group Health Collaborative

Pragmatic trial of population-based programs to prevent suicide attempts

Institution Project

UT Southwestern Medical Center

Improving chronic disease management with PIECES (ICD-PIECES)

Brown University Pragmatic trial of video education in nursing homes

University of Washington A policy-relevant US trauma care system pragmatic trial for PTSD and comorbidity (Trauma Survivors Outcomes and Support [TSOS])

}  Stra=fied randomized pragma=c clinical trial of management of pa=ents with CKD, diabetes and hypertension with a clinician support model enhanced by technology support (PIECES) compared with standard of care

Intervention Group Standard Care

Outcomes

Primary: All-cause hospitalizations Secondary: 30 day readmissions, disease-specific

hospitalizations, ER visits, cardiovascular events, deaths

Facilitated Care IT enhanced-Pieces Practice Facilitators

Weekly reports Care protocols Smart forms

Clinical measures reports

Practice Facilitator

Order sets

Patient reports

Stratified Cluster Randomization

PCP

ICD-PIECES Study

Healthcare System # of Clinics or Prac:ce Sites

# of available pa:ents

Parkland 6 4,419

THR 40 3,288

ProHealth 13 5,805

VA 9 1,093

Parkland HHS

n=3,617

Texas Health Resources n=2,692

ProHealth n=4,752

VA North Texas n=895

CKD + Hypertension + Diabetes n=11,956 Patients to enroll

HCS

Clusters PRIMARY CARE CLINICS

Stratum Data Equal # of patients & Equal # of Clusters

Equal # of patients & Unequal # of Clusters

Unequal # of patients & Equal # of Clusters

Larger Cluster Size

A 4419, 6 3651, 17 3651, 17 1,460, 17 3651, 9

B 3288, 40 3651, 17 3651, 34 2921, 17 3651, 9

C 5805, 13 3651, 17 3651, 9 4382, 17 3651, 9

D 1093, 9 3651, 17 3651, 8 5842, 17 3651, 9

# of patients needed

11956 7889 9438 8708 11528

% of patients

82% 54% 65% 60% 79%

Statistical Issues for Stepped-Wedge Trial Design

1 Introduction

2 Analysis and Design Considerations

3 Summary (Pros and Cons)


Stepped-Wedge Cluster Trials

Mostly performed as a cluster design

All clusters start in the control group

At predefined time points (steps), subgroups of clusters switchto the intervention group in a random order

Clusters stay in the intervention group from the moment ofswitching to the end of study

Collect outcome measurements at each step


Scheme Illustration

Hemming, et al. (2015)


Motivations

Logistically more feasible to roll out intervention sequentially

Appropriate where there is already a belief that theintervention is beneficial and unlikely to do any harm

Eventually all clusters will receive the intervention, hence lessethical concern

Utilizes a natural implementation process but offersrandomized evidence of effectiveness

Allows assessment of treatment effect over time (trend)


Different Types of Stepped-Wedge Design

Cohort:Repeated measurements on the same cohort of individualsrecruited at the start and followed up throughout the study

Cross-sectional:Different participants at each step, each contributing onemeasurement

Open-cohort:Mixture of the above two


Example 1: Cross-sectional

The EPOCH Trial

To evaluate a service delivery intervention to improve the care of patientsundergoing emergency laparotomy (Pearse et al. 2013)http://www.nets.nihr.ac.uk/projects/hsdr/12500510

The intervention includes quality improvement and an integrated carepathway

90 hospitals sequentially switch from control to intervention every 5weeks at 15 different time points

The Primary outcome is 90 day mortality


Example 2: Open-cohort

Multi-Structured Depression Management

An intervention to promote the diagnosis and management of depressionin nursing homes (Leontjevas et al., 2013, Lancet)

17 nursing homes in Netherlands, randomly switch to intervention on oneof five dates

Most participants were recruited at the start of the trial and followed upover the five steps; others were recruited during the trial and followed upfor the remaining steps

The primary outcome was prevalence of depressionThe proportion of residents per unit with score>7 on the Cornell Scale forDepression in Dementia


Estimate Treatment Effect

Horizontally, each cluster serves asits own control (within-clustercomparison)

Vertically, compare across clustersto assess treatment effect(between-cluster comparison)

Combine the above two

Can analyze treatment effect astime average or slope


Correlation to Consider

Cross-sectional design:

Within-cluster correlation (ICC)Assume independence across steps

Cohort design:

Within-cluster correlationLongitudinal correlation amongrepeated measurements from thesame participants

Correlation Structures:

Compound symmetric (CS),exchangeable over time orparticipantsAR(1): assuming the correlation todecay over time


Impact of Correlation

Suppose treatment effect is evaluated as timeaverage,

For vertical comparisons, a largerwithin-cluster correlation leads to areduced power

For horizontal comparisons, a largerlongitudinal correlation leads to a greaterpower

Overall, the impact of within-clustercorrelation on power is less severe instepped wedge studies than in regularcluster studies


Confounding by Time

Time associated with exposure: moreunexposed/exposed observations atearlier/later stage

Underlying temporal trend (rising tide): A

seemingly effective intervention might not

be significant after adjusting for calendar

time. Possible explanation:

External to the study, there is ageneral move toward improvingpatient outcome

Greater chance of contamination

over time


Modeling Strategy

Approaches: Generalized linearmixed-effect model or generalizedestimating equation (GEE)

Modeling correlation:

Cluster random effects or/andsubject random effects

Specify the correlation matrix

directly

Trend under control:

intercept (constant)intercept+time (linear trend)

A separate intercept for each step

(arbitrary trend)

Treatment effect:

Indicator of treatment (shift inmean)

interaction of time and treatment

(shift in slope)Statistical Issues for Stepped-Wedge Trial Design

Other Considerations

Intention to treat principle: clustersshould be analyzed according to theirrandomized crossover time irrespective ofwhether crossover was achieved at desiredtime

Heterogeneity of treatment effect:utilizing within-cluster comparisons ofcontrol and intervention periods (usuallya secondary goal)

Missing data over time:

Monotone missing or independentmissing

Depending on the correlation

structure and testing hypothesis

(intercept or slope), the actual

information loss might be mitigated

or aggravated


Other Considerations

Options to handle transitional period:

1 During the transition the clustersare considered to be neitherexposed nor unexposed, hencetreated as missing data

2 The intervention gradually becomeembedded in the setting, henceincluding the length of time fromcrossover as an effect modifier


Reporting

Hemming et al. (2015) recommended that the estimatedintra-cluster correlation and time effect from the fitted model,although not of direct importance in the interpretation of the effectof the intervention, should be reported both for use in the designof future trials and to allow appreciation of any underlyingconfounding effects of calendar time.


Other Variations

Multiple layers of clustering:

Participants might be further clustered:hospital→clinics→physicians→patientsClusters themselves might be clustered: In the EPOCH trial,hospitals are geographically clustered

Various cluster sizes (# of subjects in each clusters)

Various step sizes (# of crossover clusters at each step)

Various lengths of step (length of interval between steps)

Different types of outcomes: continuous, binary, count, eventtime


Sample Size Calcualtion

In a recent systematic review (Mdege et al. 2013) ofstepped-wedge studies, out of 15 studies evaluated, sample sizecalculation was reported in 8 studies, only 3 of which account fordesign effect (considering intraclass correlation).

Sample size formulas for cross-sectional stepped-wedge studies(Hussey& Hughes, 2007; Woertman, et al. 2013)

Sample size formulas for cohort stepped-wedge studies

de Hoop et al. (2015)We are also working on a GEE sample size approach forstepped-wedge design (closed-form, flexible to account formissing data and correlation structures)

Simulation approach that can assess designs with anyparticular features (Baio et al. 2015)

Software implementation:STATA menu-driven program “steppedwedge”


Disadvantages of Stepped-Wedge Study

Takes a longer time to perform:duration of of a classic cluster trial × # of steps

Increased risk of attrition and contamination

Difficulty in blindingpatients and assessors are aware of the switch

Greater complexity in data analysis and trial design. Need toaccount for

confounding by timevarious sources of correlationmissing datatreatment assessed based on within- and between-clustercomparisons


Advantages of Stepped-Wedge Study

More ethical and culturally acceptalbecrossover is unidirectional, all clusters eventually receive the

intervention

More efficientbesides parallel comparisons (between-cluster), clusters act as their

own control

More manageableclusters gradually switch to intervention

More informationallows studying the effect of time on intervention effectiveness


Refrencnes (incomplete)

1 Brown C, Lilford R. The stepped wedge trial design: a systematic review. BMCMed Res Methodol. 2006;6:54

2 Mdege N, Man M, Brown C, Torgersen D. Systematic review of stepped wedgecluster randomised trials shows that design is particularly used to evaluateinterventions during routine implementation. J Clin Epidemiol. 2011;64:936-48

3 Hemming K, Haines T, Chilton A, Girling A, Lilford R. The stepped wedgecluster randomised trial: rationale, design, analysis and reporting. Br Med J.2015

4 Baio G, Copas A, Ambler G, Hargreaves J, Beard E, Omar RZ Sample sizecalculation for a stepped wedge trial, Trials (2015) 16:354

5 Hussey M, Hughes J. Design and analysis of stepped wedge cluster randomisedtrials. Contemporary Clin Trials. 2007;28:182-91

6 Hemming K, Lilford R, Girling A. Stepped-wedge cluster randomised controlledtrials: a generic framework including parallel and multiple-level design. StatMed. 2015 Jan 30;34(2):181-196

7 Woertman W, de Hoop E, Moerbeek M, Zuidema S, Gerritsen D, Teerenstra S.Stepped wedge designs could reduce the required sample size in clusterrandomized trials. J Clin Epidemiol. 2013;66(7):52-8

8 Zhou C Design Stepped Wedge Cluster Randomized Trials for QI Research - 101Part 1, 2013

9 Journal “Trial” published a thematic series on Stepped Wedge Trials in Aug2015, http://www.trialsjournal.com/series/SteppedWedge



Stepped-wedge design is getting into fashion!


Documents

Chul Ahn, PhD Song Zhang, PhD UT Southwestern … · Features of Pragmatic Clinical Trials 3. ... be signi cant after adjusting for ... patient outcome Greater chance of contamination