Noninferiority Trialssites.duke.edu/ebmworkshop/files/2019/04/Non-Inferiority.pdf•Noninferiority trials are more and more common •Noninferiority trials have important differences

Noninferiority TrialsTeaching and Leading EBM

Duke University EBM Workshop

5 April 2019

Jeff Kushinka, MD

CONFLICTS OF INTEREST

• None to disclose

OBJECTIVES

• Describe differences between noninferiority trials and traditional randomized controlled trials (RCTs)

• Assess validity of a noninferiority trial

• Demonstrate strategies for teaching critical appraisal in large group sessions

“Standard” RCTs

• Standard RCTs (often involving medications) investigate experimental intervention compared to placebo

• Goal is to show that intervention (treatment) is superior to placebo

• Using treatment improves outcomes, or reduces “badness”

• Examples: Statins in cardiovascular disease, ACE inhibitors in heart failure, protease inhibitors in HIV

When Not to Use Placebo

• Are there instances where we might avoid the use of placebo?

• If there is a treatment already known to be effective, should we test a new treatment vs. placebo or vs. existing treatment?

• If a new treatment doesn’t change primary outcome (e.g., MI, stroke, hospitalizations, etc.) but does reduce adverse effects, cost, monitoring burden, etc., we often want to compare to standard treatment instead of placebo

• More interest in comparative effectiveness trialshow does new treatment compare to standard treatment?

Why not conduct “standard” RCT?

• Why not demonstrate superiority of new treatment compared to standard treatment?

• If (bad) event rate is lowered by existing standard treatment, what does this mean for size/duration of trial with new treatment?

• We need either more patients or more time to see a difference with a new treatment

Why not conduct “standard” RCT?

• Why not demonstrate superiority of new treatment compared to standard treatment?

• If new treatment is truly similar to standard treatment, we may never see a difference in primary outcome no matter how large the trial (or how long it’s conducted)

• But there still may be reasons to choose new treatment: fewer side effects, less costly, easier to administer, easier to monitor, etc.

• Reducing “burden” of treatment is worthwhile goal

Enter the Noninferiority Trial

• Noninferiority trials test new intervention vs. active comparator

• Most commonly used for medications, but could be used for any therapeutic intervention

• Examples?

What does Noninferior Mean?

• Noninferiority does NOT mean “equal”

• How much worse can new intervention be and still be acceptable? How much of the benefit from standard treatment

are we willing to sacrifice to have lower treatment burden (less cost, fewer adverse events, less monitoring, etc.)?

RCT for Superiority

Treatment Difference

ARR = 0, RR = 1

Favors treatment Favors placebo

Treatment is superior Superiority not proven

RCT for Noninferiority


ARR = 0, RR = 1

Favors new treatment Favors standard treatment

New treatment superior

Standard treatment superiorOR

New treatment inferior

Noninferiority margin

How much worse than standard treatment will we allow new treatment to be?

Noninferiority Margin

• How much worse can new intervention be and still be acceptable? How much of the benefit from standard treatment

are we willing to sacrifice to have lower treatment burden (less cost, fewer adverse events, less monitoring, etc.)?

Noninferiority margin = maximum allowable amount of “badness” etc. compared to standard treatment

Choosing Noninferiority Margin

• No set rule for determining noninferiority margin

• Some sources suggest that experimental treatment should retain about 50% of smallest possible benefit of standard treatment vs. placebo

• Example: If warfarin lowers stroke risk by absolute risk difference of 4%...New drug should have no more than 2% risk increase in

stroke compared to warfarin

Preserve 50% of the 4% benefit seen in warfarin vs. placebo

Adapted from Users’ Guides to Medical Literature, 3rd ed.

Results of NoninferiorityTrials


Noninferiority margin


Superior

Noninferior

Noninferior

Inferior

ARR = 0, RR = 1

Adapted from Users’ Guides to Medical Literature, 3rd ed. and from Daniella Zipkin, MD

Results of NoninferiorityTrials


Non-inferiority margin


Noninferior but NOT equal!

ARR = 0, RR = 1

Choosing Noninferiority Margin

• What happens when we choose a larger noninferiority margin?

• We choose to preserve less of the benefit of standard treatment…

Wide Noninferiority Margin


ARR = 0, RR = 1


Wider noninferiority margin easier to “prove”!

Let’s Practice…

Comments on pre-marked article

• Can help move through an article quickly when time is limited

• Can focus on selected questions when assessing validity (recognizing that not all will agree on all questions)

• Can help intimidating topic seem more manageable

Some Starter Questions

#1. What primary question is this trial trying to answer?

Is a 5-day course of steroids similar to a 14-day course of steroids in COPD management (maximizing time to next COPD exacerbation)

#2. What is the primary outcome for this trial?Time to next COPD exacerbation within 6 month follow-up period

Some Starter Questions

#3. What was the noninferiority margin?

3a. What was the expected expected event rate in the standard treatment group (14 days of steroids)?

50% of patients would experience exacerbation within 6 months

3b. How much worse could the event rate be in the experimental group (5 days of steroids)?

15% absolute risk difference (increase) in the 5-day steroid group (65% of patients would experience exacerbation within 6 months)

Assessing Trial’s ValidityP a g e | 26

Duke Teaching and Leading EBM: A Workshop for Educators and Champions of Evidence-Based Medicine

Therapy Critical Appraisal Form, Non-inferiority trials Citation:

How serious is the risk of bias?

Did intervention and control groups begin the study with a similar prognosis?

Were patients randomized?

Was randomization concealed?

Were patients similar at baseline with respect to known prognostic factors?

Was prognostic balance maintained as the study progressed?

Were patients, caregivers, collectors of outcome data, adjudicators of outcome, and data analysts aware of group allocation?

Were groups prognostically balanced at the study’s conclusion?

Was follow-up complete?

Was the trial stopped early for benefit?

Were patients analyzed in the groups to which they were randomized?

Did the investigators guard against an unwarranted conclusion of non-inferiority?

Was the effect of the standard treatment preserved?

Did the investigators analyze patients according to the treatment they received, as well as to the groups to which they were assigned?

YesYes

#4 in article, Table 1

No

Assessing Trial’s Validity

P a g e | 26

















No; also review #5 in article (were patients treated similarly?)

#6

No

Yes…(and we’ll come back to this)


• Did the investigators guard against an unwarranted conclusion of noninferiority?

• What happens if the standard treatment doesn’t work as well as expected in this trial?

It’s easier to show noninferiority!


P a g e | 26

















#7…was the standard treatment as good as expected?

#8…why does this matter?

ITT vs. Per-protocol Analysis in Noninferiority Trials

• In standard superiority RCTs, ITT is a more “conservative” approach

• When including those who didn’t receive treatment and analyzing them as randomized, we are more likely to move toward the line of no difference

It’s “easier” to show noninferiority (outcomes are more similar)

• If ITT analysis and per-protocol analysis provide similar results (i.e., both noninferior), the conclusion of non-inferiority is strengthened

Results of Trial

Applying Results

P a g e | 27


What are the results?

How large was the treatment effect?

How precise was the estimate of the treatment effect?

How can I apply the results to my patient care?

Were the study patients similar to my patient?

Were all patient-important outcomes considered?

Are the likely advantages of the novel treatment worth the potential harms and costs?

Adapted from McMaster Evidence-based Clinical Practice Workshops and the Users' Guide to the Medical Literature 3rd Ed.

Exacerbations, mortality, FEV1, clinical performance status, hospital stay, need for mechanical ventilation

#9

Where was this trial conducted?

Conclusions from REDUCE trial

• Are you satisfied with validity of this noninferiority trial?

• Would you be comfortable with shorter duration of steroids in COPD exaberation?

Summary

• Noninferiority trials are more and more common

• Noninferiority trials have important differences from “standard” superiority trials

• A new treatment that is noninferior is not necessarily equivalent to standard treatment

• Careful critical appraisal can increase our confidence in results of noninferiority trials

• Pay attention to choice of noninferiority margin

Teaching Strategy Summary

• Use of graphical depiction to illustrate concept of non-inferiority margin

• Choice to do critical appraisal of actual trial to illustrate concepts

• Use of pre-marked article to permit critical appraisal in reasonable time in large-group setting

• Intentionally skipping (or not having group answer) some validity questions for time management

• Attempt to minimize jargon

Questions??

THANK YOU!

Documents

Noninferiority Trialssites.duke.edu/ebmworkshop/files/2019/04/Non-Inferiority.pdf•Noninferiority trials are more and more common •Noninferiority trials have important differences