INTERIM ANALYSIS IN CLINICAL TRIALSBy: Aditya Chakraborty

Advisor: Dr. Subhash C. Bagui

CLINICAL TRIALS: AN OVERVIEW

Clinical trials are research studies that deals with whether a medical scheme, treatment, or device is safe and effective for humans. Clinical trials may also compare a new treatment to a treatment that is already available.

As an outcome of a treatment(drug), there are three possibilities generally can take place.

improves patient outcomes;

offers no benefit

causes unexpected harm

WHY CLINICAL TRIALS: Two basic questions are generally answered.

i) Does the new treatment work effectively for humans? If it does, doctors are also looking at how efficiently it works. Is it better and more improved than what is now being used to treat a certain disease? If it is not better, is it at least as good, while perhaps causing fewer side effects?

ii) Is the new treatment safe to use? In this context, we’ll discuss what is called an interim analysis in clinical trials.

Interim analysis in Clinical Trials

An interim analysis is any assessment of data done during the patient enrollment time or follow-up stages of a trial with the objective of assessing performance, the quality of the data collected, or treatment effects.

In a lucid sense, Interim analysis” or “early stopping” refers to the problem of interpreting the accruing information during a clinical trial.

Ethical and economic details are also taken into account to stop the trial early.

We want to make sure that the maximum number of patients receives the most effective treatment at the earliest stage.

Since clinical trials are expensive, there are also economic reasons to include as few patients as possible. One would not to spend additional money if he/she already have enough evidence.

Relevance: Why stop early?

Treatments are found to be convincingly dissimilar in nature.

Side effect or injuriousness is too severe to continue treatment, relative to the potential benefits.

Accrual is too sluggish to complete the study in timely manner.

Conclusive information is available from external study, making the trial unnecessary or unethical.

The scientific questions are no longer essential because of other study developments.

Adherence to the treatment is unacceptably poor, preventing an answer to the basic question.

Resources to perform the study are misplaced or no longer existing and/or the study integrity has been sabotaged by fraud or misconduct.

THE CONCEPT OF STATISTICAL POWER AND TYPE I ERROR

Whenever we conduct a hypothesis test, we'd like to make sure that it is a test of high quality. One way of quantifying the quality of a hypothesis test is to ensure that it is a "powerful" test.

A Type I error occurs if we reject the null hypothesis H0 (in favor of the alternative hypothesis HA) when the null hypothesis H0 is true. We

denote 𝛼 = P(Type I Error).

A Type II error occurs if we fail to reject the null hypothesis H0 when the alternative hypothesis HA is

true. We denote β = P(Type II Error).

Power = 1- β , i.e. rejecting H0 when it is false or accepting H0 when it is true.

In other words, Power is the probability of avoiding a Type II error.

THE PROBLEM WITH THE TYPE I ERROR

In interim analysis, we divide the whole sample size into some equal or unequal subsamples and then analysis is done based on those subsamples.

We can’t fix our alpha level as 0.05 throughout the study while performing an interim analysis.

We can’t just conclude that one treatment performs better than other if we see p-value (p) < 0.05?

Each time we look at the data, we have the likelihood of a committing type I error. If we see at the data multiple times, and we consider alpha as 0.05 as our standard significance level, then we have a 5% chance of stopping every time. Under the true null hypothesis and two looks at the data, we can approximate the error rates as:

Probability of stopping trial at first stage: 0.05

Probability of stopping trial at second stage: 0.95*0.05 = 0.0475

Hence, total probability of stopping becomes (.05+.0475) =0.0975

ALPHA SPENDING FUNCTION: A

concept to control type-I error

The main objective of the general group sequential methods is to control the type I error rate. The alpha spending function [4]

Deals with assigning some of the pre specified type I error to each interim analyses.

Allocates the total allowable type I error rate through a function depending on the evidence accumulated during the trial, such as the total number of observed patients or events.

Is dependent on the fraction of patients or events observed at a particular interim analysis out of the total number of patients or events anticipated or designed for this fraction.

All we need to guarantee is that the over-all size of

the test remains at level 𝜶.

PROMINENT CONTRIBUTORS: Armitage, Haybittle-Peto, Pocock, O'Brien and Fleming.

SOME ALPHA SPENDING FUNCTIONS[5][7]:

1. O'Brien and Fleming : 𝛼(t) = 2 − 2Φ(Z𝛼

2/t).5 ,

where Φ is the cumulative distribution function of a standard normal distribution.

2. Pocock: 𝛼(t) = 𝛼 ln[1 + (e − 1)t]

3. 𝛼(t) = 𝛼 ∗ t , Uniform Type

HEYBITTLE-PETO’S BOUNDARY

Reject H0 whenever p value ≤ .01 (i.e. no difference between treatments)

The final analysis though is performed at the usual level of significance .05

Advantage : Since it uses the usual 𝛼 = .05 in the final analysis, it become easier for researchers to interpret the result based on the outcome.

Disadvantage : According to some researchers, one obvious criticism of the approach is that the stopping a trial becomes too difficult resulting Heybittle-Peto approach(based on intuitive reasoning) a conservative one.

POCOCK’S APPROACH:

This approach came into being by the medical scientist Stuart Pocock in 1977.

In each interim analysis it uses the same threshold p value.

Some researchers dislike this approach because of it’s certain disadvantages.

The number of interim analysis has to be decided prior to the starting of any analysis and as soon as the analysis started, it is not possible to add any extra analysis to it.

Researchers and investigators sometimes get confused about how to interpret the p value.

As for example, suppose four analysis are planned and statistical significance has not reached in any of these. Suppose that the p value at the final analysis is 0.0364(>0.0182 from table). Under this scenario, if interim analysis had not been scheduled, then p value would be considered to project a statistical significant result(as 0.0364<.05)

O’BRIEN FLEMING’S APPROACH:

Most popularly used group sequential approach.

The overall significance level here approximately equals to the desired level of significance(.05) which is achieved by summing the significance levels of the previous interim steps.

Based on statistical reasoning

Not conservative as Heybittle-Peto’sapproach

LITERATURE REVIEWHere is an attempt towards Literature Review on some available Interim Tests.

Set Z* (I) = Z* (I,II)=……..=Z* (I,II,……,k) = 1.96. It is known that for k(# of stages) = 1, level attained = 0.05. It can be shown that for k= 2,3,4,5 respectively levels attained are given by .08, .14, .20, .35.

Suggested Ad Hoc Rule : Use Z* (I) = Z* (I,II) = ……….=Z* (I,II,……,k) = 2.6. For large k this yield approximate level 0.05

Haybittle- Peto Procedure : Use a common value 3.291 for all Z*’s above except for final stage when

1.96 is used. For K = 5 it is shown that this choice attains α= 0.05

Pocock’s Procedure : Use common value for Z*’s to attain exact levels α. For k = 5 and α= 0.05, common value is Z* = 2.413

O’ Brien – Fleming Procedure : Select a suitable value for Z* so that Z* (I) = Z* √(k) ,

Z*(I,II)=Z*√(k/2), Z* (I,II,III) = Z* √(K/3),……….Z*(I,II,…….,K) = Z* and attained level is α.

For k = 5 and α= 0.05, Z* = 2.04; Z* (I) = 4.555 , Z* (I,II) = 3.221 , Z* (I,II,III) = 2.63 , Z* (I,II,III,IV) = 2.037.

OUR PROBLEM

A testing problem involving the % cure rate for patients under a 'test treatment'(Drug A) as against a 'standard treatment'(Drug B) with both-sided alternatives has been developed to work out on explicit expressions for the sample size and the cut-off point considering alpha= .05 and power [against specified alternatives with a difference of 5%] = 90%. An interim analysis in four looks has been performed involving the two drugs each for each group.

Since the cure rate( proportion of individuals having a particular disease that are cured by a given treatment (drug), called the cure fraction or cure rate) for some life threatening diseases are very low, we assumed the cure rate to be .35 for the problem.

OBTAINING THE DATA : Data has been simulated using SAS and VBA. First we draw random numbers specifying patients and then we get the number of individuals cured using specific rules.

FULL ANALYSIS AND DETERMINATION OF SAMPLE SIZE

Hypothesis of interest : H0: PT=PS Vs HA: PT ≠ PS

PT: % cure in test treatment (Drug A) & PS: % Cure in standard treatment(Drug B) .

Given, α=0.05, Power (1- β )= .90 Δ=PS - PT= 0.05(clinically meaningful difference)

FULL ANALYSIS :

Determining the sample size :

Zα/2 = 1.96, β = .10, Zβ = 1.282 ,

Δ =0.05 , N= 2(Z α/2 + Zβ)2 P(1 – P)/Δ2

N: total sample size . P= 0.35 (suggestive cure rate)

Now, N= 2(1.96 +1.282)2(0.325)(1-0.35)/(.05)2

=4.61151/.0025 =1913

Conclusion: Each arm involves 1913 subjects for full experiment.

Interim Analysis (Two looks):Here each arm involves 956 subjects

DATA OF 1ST SET OF OBSERVATION OF 2ND LOOK

Data: T (340/956) & S (332/956) , n =956

PT(est) = 340/956=.355 ;PS(est) = 332/956=.347;

P (estimated) =(.355+.347)/2=.351

Z*_(look 2)obs =

(.335 - .347)/(√(.351*.649)*2/956)= -.012/.021 =-.57

Here IZ*_(look 2)obsI<Z*Conclusion:Hence H0 is accepted by all three rules suggested.

DATA OF 2ND SET OF OBSERVATION OF 2ND LOOK

Data: T (346/956) & S(322/956)

PT(est) = 346/956=.361 PS(est) = 322/956=.336P(est) =(.361+.336)/2 = .348

Z**_(look 2)obs =

(.361 - .336)/(√(.348*.652)*2)/956)= .025/.021=1.19

Here IZ**_(look 2)obsI <Z**Conclusion:Here also H0 is accepted by all three rules.

Interim Analysis (Three looks):

Now we have 638 patients in each arm.Here 1st we compute Z(i)(look 3)obs . (Z(i)(look 3)obs is the observed cut off points in 3 looks interim analysis problem, i=*,**,***) and each arm involves 638 subjects.

Data of 1st set of observation of 3rd look are following:

Data : T(227/638), S(220/638)

PT(est)= 227/638= .355 ,PS(est)= 220/638=.344P(estimated)= (.355+.344)/2= .35

Z*_(look 3)obs=

(.355-.344)/ √(.35*(1-.35)*2/638)=.011/.026=.42

Here IZ*_(look 3)obsI<Z*_1

So H0 is accepted by three rules.Implies we go to the 2nd set of observations.

Data of 2nd set of observation of 3rd look are following:

Data: T(234/638), S(234/638)

Progressive proportion for T and S:

PT(estimated): (227+234)/1276 =.36PS(estimated) : (220+234)/1276 = .35P(estimated) = (.36+.35)/2=.355Z**_(look3)obs =

(.36-.35)/(√(.355*.645*2)/1276) = .01/.02=.5

Here IZ**_(look3)obsI<Z**_2

Hence again H0 is accepted by all three rules. Conclusion:Hence we’ll go to third set of observations.

Data of 3rd set of observation of 3rd look are following:

Data: T(206/638) S(235/638)

Progressive proportion of T and S

PT(estimated) = (227+234+206)/1914=.348PS(estimated)= (220+234+235)/1914=.359P(estimated)= (.348+.359)/2= .353

Z***_(look 3)0bs =

(.359-.348)/(√(2* .353*.647)/1914)=.011/.015=.73

Here IZ***_(look 3)0bsI<Z***_3

So, again H0 is accepted by all three rules.

Interim Analysis (Four looks):

Data of 1st set of observation of 4th look are following:

Data : T(163/478) S(174/478)Result : Accept H_0 and proceed furtherData set for 2nd set of observation for 4th look:Data: T(172/478) S(165/478)Result : Accept H_0 and proceed furtherData set for 3rd set of observation for 4th look:Data: T(179/478) S(167/478)Result : Accept H_0 and proceed furtherData set for 4th set of observation for 4th look:Data :T(160/478) S(178/478)Result : Accept H0 finally. Z_c Hebittle-peto Pocock O’Brien-

Fleming

Z*_1 3.291 2.361 4.084

Z*_2 3.291 2.361 2.888

Z*_3 3.291 2.361 2.358

Z*_4 2.0 2.361 2.042

‘

CONCLUSION AND FURTHER STUDY

CONCLUSION: Since we assumed the average cure rate (35%) to be

very small for this problem statistically as well as intuitively it seems that cure rate for both the drugs are same. Patient enrollment would have stopped immediately if at any stage our Z − value exceeded the cut-off points suggested by Haybittle-Peto, O’Brien Fleming and Pocock.

Ideas For Further Study:

Like alpha-spending function discussed previously for controlling type I error margin, is there any rule to get maximum power between and within looks?

Among the three rules discussed earlier which rule is the most efficient with respect to economic as well as statistical point of view? Different estimators using the rules can be defined and their precision can be compared.

Is there any other simple sampling algorithm for obtaining the boundary values of Interim Analysis?

The Big Picture

References:

[1] Michael R. Chernick The Essentials of Biostatistics for Physicians, Nurses, and Clinicians. August 4, 2011,John Wiley & Sons

[2] Sophie D. Foss and Eva Skovlund Journal Of Clinical Oncology, clinical Oncology

[3] Guosheng Yin Clinical Trial Design: Bayesian and Frequentist Adaptive Methods June 7, 2013 John Wiley & Sons

[4] David L. DeMets and Gordon Lan The alpha spending function approach to interim data analyses alpha spending function

[5] O’Brien, P.C. Fleming, T.R. (1979) A Multiple Testing Procedure for Clinical Trials Biometrics 35: 549556

[6] Christopher Jennison Bruce W. Turnbull Group Sequential Methods with Applications to Clinical Trials September 15, 1999 CRC Press

[7] Pocock SJ (1983) Clinical Trials: A Practical Approach. New York: Wiley.

[8] Cynthia O. Sin and K.K Gordon Lan Flexible Interim Analysis For Sample Size Re-estimation and Early Stopping: A Conditional Power Approach New York,NY 10017-5755,New London,CT 06320

[9] Shein-Chung Chow, Hansheng Wang, Jun Shao Sample Size Calculations in Clinical Research (Chapman & Hall/CRC Biostatistics Series) 2 Rev Exp Edition

[10] A.S Hedayat and Bikas K. Sinha Interim Statistical Analysis In Clinical Trials

[11] Haybittle JL (1971 )Repeated assessment of results in clinical trials of cancer treatment Br J Radiol 44:793- 797.

[12] Pocock SJ (1977)Group sequential methods in the design and analysis of clinical trials Biometrika 64:191-199.

THANK YOU