View
1
Download
0
Category
Preview:
Citation preview
Noninferiority TrialsTeaching and Leading EBM
Duke University EBM Workshop
5 April 2019
Jeff Kushinka, MD
CONFLICTS OF INTEREST
• None to disclose
OBJECTIVES
• Describe differences between noninferiority trials and traditional randomized controlled trials (RCTs)
• Assess validity of a noninferiority trial
• Demonstrate strategies for teaching critical appraisal in large group sessions
“Standard” RCTs
• Standard RCTs (often involving medications) investigate experimental intervention compared to placebo
• Goal is to show that intervention (treatment) is superior to placebo
• Using treatment improves outcomes, or reduces “badness”
• Examples: Statins in cardiovascular disease, ACE inhibitors in heart failure, protease inhibitors in HIV
When Not to Use Placebo
• Are there instances where we might avoid the use of placebo?
• If there is a treatment already known to be effective, should we test a new treatment vs. placebo or vs. existing treatment?
• If a new treatment doesn’t change primary outcome (e.g., MI, stroke, hospitalizations, etc.) but does reduce adverse effects, cost, monitoring burden, etc., we often want to compare to standard treatment instead of placebo
• More interest in comparative effectiveness trialshow does new treatment compare to standard treatment?
Why not conduct “standard” RCT?
• Why not demonstrate superiority of new treatment compared to standard treatment?
• If (bad) event rate is lowered by existing standard treatment, what does this mean for size/duration of trial with new treatment?
• We need either more patients or more time to see a difference with a new treatment
Why not conduct “standard” RCT?
• Why not demonstrate superiority of new treatment compared to standard treatment?
• If new treatment is truly similar to standard treatment, we may never see a difference in primary outcome no matter how large the trial (or how long it’s conducted)
• But there still may be reasons to choose new treatment: fewer side effects, less costly, easier to administer, easier to monitor, etc.
• Reducing “burden” of treatment is worthwhile goal
Enter the Noninferiority Trial
• Noninferiority trials test new intervention vs. active comparator
• Most commonly used for medications, but could be used for any therapeutic intervention
• Examples?
What does Noninferior Mean?
• Noninferiority does NOT mean “equal”
• How much worse can new intervention be and still be acceptable? How much of the benefit from standard treatment
are we willing to sacrifice to have lower treatment burden (less cost, fewer adverse events, less monitoring, etc.)?
RCT for Superiority
Treatment Difference
ARR = 0, RR = 1
Favors treatment Favors placebo
Treatment is superior Superiority not proven
RCT for Noninferiority
Treatment Difference
ARR = 0, RR = 1
Favors new treatment Favors standard treatment
New treatment superior
Standard treatment superiorOR
New treatment inferior
Noninferiority margin
How much worse than standard treatment will we allow new treatment to be?
Noninferiority Margin
• How much worse can new intervention be and still be acceptable? How much of the benefit from standard treatment
are we willing to sacrifice to have lower treatment burden (less cost, fewer adverse events, less monitoring, etc.)?
Noninferiority margin = maximum allowable amount of “badness” etc. compared to standard treatment
Choosing Noninferiority Margin
• No set rule for determining noninferiority margin
• Some sources suggest that experimental treatment should retain about 50% of smallest possible benefit of standard treatment vs. placebo
• Example: If warfarin lowers stroke risk by absolute risk difference of 4%...New drug should have no more than 2% risk increase in
stroke compared to warfarin
Preserve 50% of the 4% benefit seen in warfarin vs. placebo
Adapted from Users’ Guides to Medical Literature, 3rd ed.
Results of NoninferiorityTrials
Treatment Difference
Noninferiority margin
Favors new treatment Favors standard treatment
Superior
Noninferior
Noninferior
Inferior
ARR = 0, RR = 1
Adapted from Users’ Guides to Medical Literature, 3rd ed. and from Daniella Zipkin, MD
Results of NoninferiorityTrials
Treatment Difference
Non-inferiority margin
Favors new treatment Favors standard treatment
Noninferior but NOT equal!
ARR = 0, RR = 1
Choosing Noninferiority Margin
• What happens when we choose a larger noninferiority margin?
• We choose to preserve less of the benefit of standard treatment…
Wide Noninferiority Margin
Treatment Difference
ARR = 0, RR = 1
Favors new treatment Favors standard treatment
Wider noninferiority margin easier to “prove”!
Let’s Practice…
Comments on pre-marked article
• Can help move through an article quickly when time is limited
• Can focus on selected questions when assessing validity (recognizing that not all will agree on all questions)
• Can help intimidating topic seem more manageable
Some Starter Questions
#1. What primary question is this trial trying to answer?
Is a 5-day course of steroids similar to a 14-day course of steroids in COPD management (maximizing time to next COPD exacerbation)
#2. What is the primary outcome for this trial?Time to next COPD exacerbation within 6 month follow-up period
Some Starter Questions
#3. What was the noninferiority margin?
3a. What was the expected expected event rate in the standard treatment group (14 days of steroids)?
50% of patients would experience exacerbation within 6 months
3b. How much worse could the event rate be in the experimental group (5 days of steroids)?
15% absolute risk difference (increase) in the 5-day steroid group (65% of patients would experience exacerbation within 6 months)
Assessing Trial’s ValidityP a g e | 26
Duke Teaching and Leading EBM: A Workshop for Educators and Champions of Evidence-Based Medicine
Therapy Critical Appraisal Form, Non-inferiority trials Citation:
How serious is the risk of bias?
Did intervention and control groups begin the study with a similar prognosis?
Were patients randomized?
Was randomization concealed?
Were patients similar at baseline with respect to known prognostic factors?
Was prognostic balance maintained as the study progressed?
Were patients, caregivers, collectors of outcome data, adjudicators of outcome, and data analysts aware of group allocation?
Were groups prognostically balanced at the study’s conclusion?
Was follow-up complete?
Was the trial stopped early for benefit?
Were patients analyzed in the groups to which they were randomized?
Did the investigators guard against an unwarranted conclusion of non-inferiority?
Was the effect of the standard treatment preserved?
Did the investigators analyze patients according to the treatment they received, as well as to the groups to which they were assigned?
YesYes
#4 in article, Table 1
No
Assessing Trial’s Validity
P a g e | 26
Duke Teaching and Leading EBM: A Workshop for Educators and Champions of Evidence-Based Medicine
Therapy Critical Appraisal Form, Non-inferiority trials Citation:
How serious is the risk of bias?
Did intervention and control groups begin the study with a similar prognosis?
Were patients randomized?
Was randomization concealed?
Were patients similar at baseline with respect to known prognostic factors?
Was prognostic balance maintained as the study progressed?
Were patients, caregivers, collectors of outcome data, adjudicators of outcome, and data analysts aware of group allocation?
Were groups prognostically balanced at the study’s conclusion?
Was follow-up complete?
Was the trial stopped early for benefit?
Were patients analyzed in the groups to which they were randomized?
Did the investigators guard against an unwarranted conclusion of non-inferiority?
Was the effect of the standard treatment preserved?
Did the investigators analyze patients according to the treatment they received, as well as to the groups to which they were assigned?
No; also review #5 in article (were patients treated similarly?)
#6
No
Yes…(and we’ll come back to this)
Assessing Trial’s Validity
• Did the investigators guard against an unwarranted conclusion of noninferiority?
• What happens if the standard treatment doesn’t work as well as expected in this trial?
It’s easier to show noninferiority!
Assessing Trial’s Validity
P a g e | 26
Duke Teaching and Leading EBM: A Workshop for Educators and Champions of Evidence-Based Medicine
Therapy Critical Appraisal Form, Non-inferiority trials Citation:
How serious is the risk of bias?
Did intervention and control groups begin the study with a similar prognosis?
Were patients randomized?
Was randomization concealed?
Were patients similar at baseline with respect to known prognostic factors?
Was prognostic balance maintained as the study progressed?
Were patients, caregivers, collectors of outcome data, adjudicators of outcome, and data analysts aware of group allocation?
Were groups prognostically balanced at the study’s conclusion?
Was follow-up complete?
Was the trial stopped early for benefit?
Were patients analyzed in the groups to which they were randomized?
Did the investigators guard against an unwarranted conclusion of non-inferiority?
Was the effect of the standard treatment preserved?
Did the investigators analyze patients according to the treatment they received, as well as to the groups to which they were assigned?
#7…was the standard treatment as good as expected?
#8…why does this matter?
ITT vs. Per-protocol Analysis in Noninferiority Trials
• In standard superiority RCTs, ITT is a more “conservative” approach
• When including those who didn’t receive treatment and analyzing them as randomized, we are more likely to move toward the line of no difference
It’s “easier” to show noninferiority (outcomes are more similar)
• If ITT analysis and per-protocol analysis provide similar results (i.e., both noninferior), the conclusion of non-inferiority is strengthened
Results of Trial
Applying Results
P a g e | 27
Duke Teaching and Leading EBM: A Workshop for Educators and Champions of Evidence-Based Medicine
What are the results?
How large was the treatment effect?
How precise was the estimate of the treatment effect?
How can I apply the results to my patient care?
Were the study patients similar to my patient?
Were all patient-important outcomes considered?
Are the likely advantages of the novel treatment worth the potential harms and costs?
Adapted from McMaster Evidence-based Clinical Practice Workshops and the Users' Guide to the Medical Literature 3rd Ed.
Exacerbations, mortality, FEV1, clinical performance status, hospital stay, need for mechanical ventilation
#9
Where was this trial conducted?
Conclusions from REDUCE trial
• Are you satisfied with validity of this noninferiority trial?
• Would you be comfortable with shorter duration of steroids in COPD exaberation?
Summary
• Noninferiority trials are more and more common
• Noninferiority trials have important differences from “standard” superiority trials
• A new treatment that is noninferior is not necessarily equivalent to standard treatment
• Careful critical appraisal can increase our confidence in results of noninferiority trials
• Pay attention to choice of noninferiority margin
Teaching Strategy Summary
• Use of graphical depiction to illustrate concept of non-inferiority margin
• Choice to do critical appraisal of actual trial to illustrate concepts
• Use of pre-marked article to permit critical appraisal in reasonable time in large-group setting
• Intentionally skipping (or not having group answer) some validity questions for time management
• Attempt to minimize jargon
Questions??
THANK YOU!
Recommended