Joel Singer, Programme Head, Methodology and Statistics, CIHR Canadian HIV Trials Network

Joel Singer,

Programme Head, Methodology and Statistics,

CIHR Canadian HIV Trials Network

What is a Non-inferiority Trial?

A trial to show that a particular therapeutic agent or policy is no worse than some other standard.


Type 1: Both agents have been shown superior to placebo/no treatment

• Type 2: Only one agent has been shown superior to placebo/no treatment


• The standard (STD) should have clinical efficacy: (i) that is of substantial magnitude; (ii) is precisely estimated; (iii) estimates that are relevant to the setting in which the trial is being conducted.

ICH Guidelines

• A suitable active comparator could be a widely used therapy whose efficacy in the relevant indication has been clearly established

• and

ICH: Constancy Assumption

• quantified in well-designed and well documented superiority trials and which can be reliably expected to have similar efficacy in the contemplated active control trial.

Example of non-Constancy

• Suppose vancomycin, has been shown to provide a large improvement, compared to no treatment, in cure rate in patients with various infections.


Suppose a high prevalence of vancomycin-resistant enterococci (VRE) has developed. NI trial: comparing an experimental antibiotic (EXP) with vancomycin is conducted where VRE is prevalent


A biased estimate of the efficacy of EXP would be obtained if the constancy assumption is falsely presumed to hold in the NI study.


• Because proof of exact equality is impossible, a prestated margin of noninferiority for the treatment effect in a primary patient outcome is defined.


• Non-inferiority trials are intended to

show whether a new treatment has at least as much efficacy as the standard or is worse by an amount less than Δ,often on the premise that it has some other advantage


• Both participants and outcome measures in a noninferiority or equivalence trial should be similar to those in trial(s) that established the efficacy of the reference treatment.


• The required sample size is calculated using the confidence interval (CI) approach, considering where the CI for the treatment effect lies with respect to both the margin of noninferiority and a null effect.


• Sample size depends on the level of confidence chosen, the risk of type II error (or desired power), and Δ.


• A prestated margin of noninferiority can be specified as a difference in means or proportions or the logarithm of an odds ratio, risk ratio, or hazard ratio.


• The meaning and clinical importance of measures such as relative risk, odds ratio and relative hazard will be dependent on the base rate of outcome


• A prestated margin of noninferiority is often chosen as the smallest value that would be a clinically important effect.


• The required size of non-inferiority trials is therefore usually larger than that for superiority trials,

• But:

• The actual calculation is the same as in superiority trials


• One should avoid features that might dilute true differences between treatments, thereby enhancing the risk of erroneously concluding non-inferiority


• For superiority trials, intention-to-treat (ITT) analysis (analyzing all patients within their randomized groups, regardless of whether they completed allocated treatment) is recommended


• In non-inferiority trials, ITT analysis will often increase the risk of falsely claiming non-inferiority (type I error), although not always


• In non-inferiority and equivalence trials, non-ITT analyses might be desirable as a protection from ITT’s increase of type I error risk (falsely concluding non-inferiority).


• Interpreting a non-inferiority trial’s results depends on where the CI for the treatment effect lies relative to both the margin of non-inferiority and a null effect. The observed treatment effect is not sufficiently informative.


• For superiority trials, intention-to-treat (ITT) analysis (analyzing all patients within their randomized groups, regardless of whether they completed allocated treatment) is recommended

Major Methodological Issues for Equivalence/Non-inferiority Trials

1. Difficult to get consensus on difference considered equivalence or non-inferior.

2. In superiority trials, lack of rigor in the conduct of a trial will tend to lead to equivalence between the comparators and will cause a conservative bias.

3. Lack of rigor or anything which causes background noise in an equivalence/noninferiority trial will bias in favour of the hypothesis of interest (no difference).

Important Variables for Defining Equivalence/non-inferiority

1. Outcome.

2. Prevalence of the disease.

3. Other: cost, invasiveness, ease of use.

Trial Design

n=220

NVP400 mg od

n=387

NVP200 mg bd

n=400

EFV600 mg od

n=209

NVP+ EFV400 / 800mg od

1216 ART naive patients

Nucleoside backbone: d4T and 3TC

Inclusion criteria: pVL > 5000 copies/mL

any CD4 cell count

any stage of CDC-classification

Outcome Measures% of patients with treatment failure at week 48:

• less than 1log10 decline in pVL in first 12 weeks

• 2 consecutive pVL > 50 copies/mL from week 24 onwards

• new CDC-C event or death

• change of allocated treatment

• All analyses Intention-to-Treat (unless stated otherwise)• all randomised patients

• pVL data: missing=failure

• Four pre-defined pairwise comparisons• NVP-bd vs EFV (primary)• NVP-od vs NVP-bd• NVP-od vs NVP+EFV• EFV vs NVP+EFV

Methods

Given that the 2NN trial was conducted as a non-inferiority trial, how does one go about interpreting the results?

The goal of an inferiority trial is to be able to rule out differences greater than the minimally important clinical difference (MICD).

Translated into a statistical statement, this means we can say with confidence (based on the 95% confidence interval) that the true difference is unlikely to exceed the difference deemed clinically important.

Methodological Concerns With the 2NN Trial

1. Choice of definition of outcome.

2. Background noise.

Concerns in 2NN Study re non-inferiority

1. patients who never received treatment (20 in EFV, 10 in NVP-BD).

2. early compliance failures.

3. Question inclusion of changes in d4T or 3TC as evidence of failure.

What Were the Primary Results of the 2NN Trial?

Treatment Success and Failure

NVP-od NVP-bd EFV NVP+EFV0

25

50

75

100

virologic

disease progression

change Rx

56.4 56.3 62.3 46.9

29.1 22.0 20.0 34.5 failure component:(whichever comes first)

success

11.4 18.915.3

16.3

% of patients

Success: only significant difference: EFV vs NVP+EFV, p< 0.001

% o

f pa

tient

s

There was no statistical difference for the primary comparison (NVP vs EFV) so one cannot say definitively that one treatment is better than the other.

The 95% confidence interval (1% in favour of nevirapine to 13% in favour of efavirenz) does not allow one to rule out a clinically important difference (at least as defined by the investigators) on the primary efficacy variable.

Conclusions• NVP and EFV have comparable potency in

suppressing HIV-1 replication

• NVP-od and NVP-bd show comparable efficacy

• Co-administration of NVP and EFV results in higher treatment failure due to increased toxicity

Comments

• Conclusion slide does not address primary outcome (combination of disease progression, virologic failure and change of therapy)

• Does not address issue of proof of non-inferiority

Some Technical Sample Size issues

• Typically, one wants the sample size to be large enough so that the bounds of the confidence interval will exclude values outside the non-inferiority range 80%-90% of the time


• If the outcome is continuous, and one assumes homogeneity of variance, then the sample size required for non-inferiority under the null hypothesis is the same as the sample size to detect the indicated difference under the alternative hypothesis


• If the outcome is dichotomous, then the sample size required for non-inferiority under the null hypothesis is not the same as the sample size to detect the indicated difference under the alternative hypothesis because the variability is a function of p1 and p2


• In a non-inferiority trial, the hypotheses are inverted.

• The null hypothesis is that there is a difference > m, and the alternative is the null hypothesis


• Generally, the clinical world is concerned about absolute differences between treatments, but depending on the statistical model being used, the null and alternative may be stated in other terms eg hazard ratios


• The choice of what is considered non-inferior in terms of a hazard ratio is typically driven by what one assumes the baseline rates are

• Eg if the hazard rates are relatively low, a somewhat large hazard rate may translate into a relatively low absolute difference

Summary of 2NN Study

1. Demonstrated a non-significant 6% advantage in the primary outcome in favour of efavirenz over nevirapine bid on intent-to-treat analysis.

2. Confidence interval around obtained difference between efavirenz and nevirapine bid include confidence bounds to 13% in favour of efavirenz. Cannot exclude non-inferiority on primary outcome set out by study criteria.

Documents

Joel Singer, Programme Head, Methodology and Statistics, CIHR Canadian HIV Trials Network