1

Superior Safety in Noninferiority Trials

David R. Bristol

To appear in Biometrical Journal, 2005

2

Abstract

Noninferiority of a new treatment to a reference treatment with respect to

efficacy is usually associated with the superiority of the new treatment to the

reference treatment with respect to other aspects not associated with efficacy.

3

Abstract

When the superiority of the new treatment to the reference treatment is with respect

to a specified safety variable, the between-treatment comparisons with

respect to safety may also be performed. Here techniques are discussed for the

simultaneous consideration of both aspects.

4

Background

ICH (1998) guidelines E-9 and E-10 discuss noninferiority trials, but only with

respect to the efficacy comparison. The efficacy problem has been discussed

by several authors.Bristol (1999) provides a review.

5

Notation

Treatment 0 = Reference treatment, (efficacious with an associated adverse

effect on a specified safety variable)Treatment 1 = New treatment.

6

GOAL

Show that Treatment 1 is superior to Treatment 0 with respect to the specified

safety variable and noninferior with respect to a specified efficacy variable.

7

Study Design

A randomized parallel-group study is to be conducted to compare Treatment 0

and Treatment 1, with n subjects / group.

A placebo group could be included in this design for completeness and sensitivity testing, but its inclusion will not have a direct impact on the primary analysis,

which is discussed here.

8

Notation

Let Xij and Yij denote the efficacy and safety responses, respectively, for

Subject j on Treatment i, i=0,1, j=1, …,n. It is assumed that

(Xij,Yij)' ~BVN(μXi, μYi, σ2X, σ2

Y, ρ),

where all parameters are unknown. Assume small values of efficacy and

safety are preferable.

9

Testing

It is desired to show that μX1 < μX0 +Δ and μY1 < μY0, where the noninferiority margin Δ is a specified positive number and is

defined by clinical importance, often as a proportion of the average efficacy seen

previously for Treatment 0.

10

Testing

This goal can be achieved by simultaneously testing

H0X: μX1 ≥μX0+Δ against H1X: μX1 < μX0 +Δ, and

H0Y: μY1 ≥μY0 against H1Y: μY1 < μY0.

11

Testing

Let H0=H0X U H0Y and let H1=H1X ∩H1Y.

It is desired to test H0 against H1.

12

Testing

The noninferiority (NI) aspect differs from that seen in most NI problems, as the

response is bivariate.

The reverse multiplicity (RM) aspect pertains to the “all-pairs” multiple

comparisons problem, where both H0X and H0Y must be rejected.

13

Test Procedures

Univariate approach composite score or a global statistic:

O’Brien (1984) Pocock, Geller, Tsiatis (1987)

And many others

14

Test Procedures

The multiplicity problem is solved by reducing the dimensionality of the

response variable used for the comparison. This approach suffers from

the possible impact of one variable on the new response variable. Thus, this

approach should not be considered for this problem. However, it is briefly

discussed for completeness.

15

Notation

Let

and

where and are (pooled) unbiased estimates of σ2

X and σ2Y, respectively.

 

1/ 21 0 ˆ(.5 ) ( ) /X XZ n X X

1/ 21 0 ˆ(.5 ) ( ) /Y YZ n Y Y

2ˆ X 2ˆY

16

Rejection Rule(s)

The rejection rule for efficacy is toReject H0X: μX1 ≥μX0 +Δ in favor of

H1X: μX1 < μX0 +Δ if ZX≤ -zα

and the rejection rule for safety is to Reject H0Y: μY1 ≥μY0 in favor of

H1Y: μY1 < μY0 if ZY≤ -zα,

where zα is the 100 (1-α)-th percentile of the standard normal distribution.

17

Notation

Let ΔX= μX1 -μX0 and ΔY = μY1 - μY0. Then the problem is to simultaneously test

H0X: ΔX≥ Δ against H1X : ΔX< Δ

and H0Y: ΔY ≥ 0 against H1Y: ΔY < 0.

18

Notation

(ZX,ZY)' ~

BVN((.5n)1/2(ΔX-Δ)/ σX,(.5n)1/2ΔY/σY,1,1,ρ).

(approx.)Tests could be based on linear

combinations of ZX and ZY.

Such tests will be inappropriate for the RM formulation.

19

Max Test (“Bivariate” Approach)

The simultaneous comparison is performed using a test based on

W=max{ZX,ZY}.

20

Max Test

The rejection rule is  

Reject H0 in favor of H1 if W≤ C,

where C is chosen such that P(Reject H0| ΔX =Δ and ΔY = 0)=α.

21

Max Test

Let G(.,.| ρ) is the joint cdf of a bivariate normal distribution with zero means, unit

variances, and correlation ρ.Then

P(Reject H0| ΔX =Δ and ΔY = 0) =G(C, C | ρ).

22

Max TestGiven ρ, C can be chosen such that

G(C,C| ρ)= α. However, ρ is unknown. The critical value

can be estimated by satisfying

where r is an estimate of ρ (pooled or average).

ˆ ˆ( , | ) ,G C C r

C

23

Stepwise Approach

Stepwise approaches to the multiple endpoints problem were considered by

Lehmacher, Wassmer, and Reitmer (1991) and several others.

However, because of the RM formulation, these results are not directly applicable. 

A stepwise procedure could be used here.

24

Stepwise Approach

(I) Test H0X.

If H0X is not rejected in favor of H1X, stop.

If H0X is rejected in favor of H1X,

(II) Test H0Y.

If H0Y is not rejected in favor of H1Y, stop.

If H0Y is rejected in favor of H1Y,

Reject H0 in favor of H1.

25

Stepwise Approach

The choice of level for each test has an important impact on the overall level, and

using an α-level test for each of the univariate tests results in the overall level

being much less than α.

The properties of this testing procedure are examined below using simulations.

26

Simulation Results

 The following results are based on 10,000 for each set of parameters, unit

variances and n=50 subjects per treatment. Each test is conducted at the

α=0.05 level. The simulations were conducted with the same seed for

comparison.

27

Simulation Results

Let PX & PY be the estimated power for the univariate tests based on X and Y

respectively.Let P denote the estimated power of the stepwise procedure of testing H0Y only if

H0X is rejected, where both tests are performed at the 0.05 level.

“Maximum” is test using W, with “pooled” or “average” estimate of correlation.

28

Power Estimates (%) Maximum ΔX -Δ ΔY ρ PX PY P Pooled Average

0 0 0.1 5.19 5.22 0.27 4.80 4.86 0.5 5.19 5.00 1.09 4.83 4.92 0.9 5.19 4.98 3.13 4.85 4.88

- 0.5 0 0.1 79.95 5.22 4.45 19.67 19.62 0.5 79.95 5.00 4.92 13.32 13.33 0.9 79.95 4.98 4.98 7.26 7.29

-0.25 -0.5 0.1 34.64 80.64 28.73 63.34 63.06 0.5 34.64 80.36 32.45 54.34 53.93 0.9 34.64 79.93 34.64 43.80 42.67

-0.5 -0.5 0.1 79.95 80.64 65.21 91.09 90.75 0.5 79.95 80.36 68.91 86.40 86.09 0.9 79.95 79.93 75.02 81.85 80.89

29

Discussion and SummaryNoninferiority trials are often conducted

when the new treatment has an advantage, other than efficacy, over the reference treatment. To simultaneously

test superiority with respect to safety and noninferiority with respect to efficacy, the single-stage testing approach based on

maximum is easy to use and easy to interpret.

30

THANK YOU