Accounting for Clinical Heterogeneity in Comparative Effectiveness Research

How Can One Examine a Trial for Heterogeneity of Treatment Effect (HTE)?

The Example of the BARI trial for CABG vs PTCA

September 28, 2010

Carlos Weiss, MD, MHS

AHRQ DEcIDE Project:

Methods to Study the Heterogeneity of Treatment Effects in Comparative Effectiveness Research

PI: Ravi Varadhan, PhD

Co-I: Jodi Segal, MD, MPH; Cynthia Boyd, MD, MPH; Al Wu, MD, MPH

Consultant: David Kent, MD, MPH

Technical Experts: Curt Furberg, MD, PhD; Bruce Psaty, MD, PhD

Task Order Officer: Parivash Nourjah, PhD

N=1,829

BARI Clinical Question

Population targeted: “Multivessel disease” with severe angina or ischemia

Intervention: PTCA (a form of PCI)

Comparator: CABG

Outcome: 5-yr Mortality

Questions to Audience - Set 1

What are sources of HTE?

How would pre-specification of analyses affect interpretation of results?

BARI Design for HTE

Protocol pre-specified 4 subgroup analyses:• angina severity

BARI Design for HTE

Protocol pre-specified 4 subgroup analyses:• angina severity• left ventricular function• number of diseased vessels• complex lesions

BARI Clinical Question: Sources of HTE in CABG vs PTCA

BARI Clinical Question: Sources of HTE in PTCA v CABG

• Patients– baseline risk– competing risks– risk of treatment harms– treatment responsiveness

>>Ideas drawn from Kravitz, Duan & Braslow, 2004, Milbank Quarterly

BARI Clinical Question: Sources of HTE in PTCA v CABG

• Patients – baseline risk– competing risks– risk of treatment harms– treatment responsiveness

• Treatment• Providers• Environments

PATIENTS

PROVIDERS ENVIRONMENTS

TREATMENT

BARI Results

5-yr Mortality:

Overall, no clinically significant nor statistically significant difference

CABG,+ treated diabetes

PTCA,+ treated diabetes

PTCA ,- treated diabetesCABG,- treated diabetes

Questions to Audience - Set 2

When should one be worried that a subgroup result is an error (Type I or Type II) ?

What can be done to lower error probabilities?

Proposed General Approach to Examining a Trial for HTE

1. HTE hypotheses pre-specified?

2. Design and measurement quality?

3. Modeling pre-specified?

Proposed General Approach to Examining a Trial for HTE

1. HTE hypotheses pre-specified?

2. Design and measurement quality?

3. Modeling pre-specified?

4. If No to 1, 2 or 3:

Validation study available?

Proposed General Approach to Examining a Trial for HTE

1. HTE hypotheses pre-specified?

2. Design and measurement quality?

3. Modeling pre-specified?

4. If No to 1, 2 or 3:

Validation study available?

5.a. If frequentist, test of interaction performed?

6.b. If Bayesian, pre-specified priors and variance acceptable?

Extra Slides

What is Heterogeneity of Treatment Effect?

Non-random variability in the direction or magnitude of a treatment effect

Ris

k if

Tre

ated

, ev

ents

per

100

p-y

5

10

Risk if UnTreated or “Baseline Risk”, events per 100 p-y105

Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale

Ris

k if

Tre

ated

, ev

ents

per

100

p-y

5

10─ Average Absolute Treatment Effect (ARR)

Risk if UnTreated or “Baseline Risk”, events per 100 p-y105

AR

Open circles - HTE absentClosed circles - HTE present

Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale

Ris

k if

Tre

ated

, ev

ents

per

100

p-y

5

10

Risk if UnTreated or “Baseline Risk”, events per 100 p-y105

- - Average Relative Treatment Effect (RRR)

Δy/Δx = RR

Open circles - HTE presentClosed circles - HTE absent

Treatment Effects for 2 Hypothetical Studies: Heterogeneity According to Scale