Interim Analyses ofClinical Trials

A Requirement

Outline

• Background and how DSMBs function

• Group sequential methods

• Examples

Suggested Reading

DeMets DL, Furberg CD, Friedman LMData Monitoring in Clinical Trials. A

Case Studies Approach

Springer, 2006

Fleming TR, Neaton JD, et al, JAIDS, 1995.

Example: ddI/ddC Trial, Good Thing We Did Not Stop Early

5.73.6

0.80

Relative Risk (on a log scale)

1.00 1.25

FINAL 9/20/92157:152 (ddI:ddC)

DSMB #5 8/21/92130:130 (ddI:ddC)

DSMB #4 2/13/9277:91 (ddI:ddC)

DSMB #3 11/08/9150:66 (ddI:ddC)

DSMB #2 8/29/9119:39 (ddI:ddC)

ddC Better ddI Better

Confidence Intervals:Repeated95%

Lessons Learned: ddI/ddC

• Early findings can be unreliable

• Statistical monitoring can be useful for assessing whether interim trends are conclusive.

• Importance of restricting interim results of trials to an independent DMC.

Example: Prevention Trial of Toxoplasmic Encephalitis, A Trial Stopped 2, Maybe 3, Times

750 Patients

Unblinded

375 Clindamycin Arm 375 Pyrimethamine Arm

Blinded Blinded

250 ActiveTreatment

250 ActiveTreatment

125Placebo

125Placebo

TOXO Protocol HistorySeptember 1990: Enrollment beginsFebruary 1991: 1st DSMB reviewMarch 1991: 2nd DSMB review

Clindamycin arm discontinueddue to dose-limiting toxicities; and

protocol amended

Case 25 in Case Studies BookLancet 1992;339:333-334.JID 1994; 169:384-394.

TOXO Trial:Revised Design and Recruitment Goals

600 Patients

PyrimethamineN=400

PlaceboN=200

Study re-powered for one placebo group (2:1) and patientsin the clindamycin arm were offered re-randomization.

TOXO Protocol History (cont.)August 29, 1991: 3rd DSMB review. Low overall TE event

rate noted

February 13, 1992: 4th DSMB review – DSMB recommends stopping the study due to low TE event rate among controls, and higher mortality on active drug. DSMB uncertain,also recommend un-blinding thechair to obtain his opinion.

TOXO Protocol History (cont.)February 19, 1992: Protocol chair proposes to

continue study based on review of interim analyses. He will no

longer enroll patients.March 17, 1992: 5th DSMB review (special teleconference) to discuss chair’s position and new data; DSMB recommends termination of study again.

Number of Deaths and Rateby Treatment Group and DSMB Review

08-29-91 07-31-91 3 (5) 6.5 (10.8) 2 (2) 7.8 (7.8)02-13-92 12-31-91 22 (28) 19.3 (24.5) 8

(8) 13.7 (13.7)03-17-92 01-31-92 28 (35) 21.4 (26.7) 10 (10) 14.1(14.1)

Pyrimethamine PlaceboDate ofReview

CutoffDate No. Rate* No. Rate*

( ) = actual numbers updated after review*Per 100 person years

TOXO Protocol History (cont.)March 30, 1992: Chair and Protocol Team concur with DSMB recommendation.

April 3, 1992: Clinical alert sent to sites.

Number of Deaths and Rateby Treatment Group and Report

DSMB 01-31-9228 (35) 21.4 (26.7) 10 (10) 14.1 (14.1)

Clinical Alert 03-24-9234 (46) 20.9 (28.3) 12 (14)13.7 (16.0)

Final 03-30-92 48 28.9 14 15.7

Pyrimethamine Placebo

ReportCutoffDate No. Rate* No. Rate*

( ) = actual numbers updated after review*Per 100 person years

Lessons Learned: TOXO

• Importance of ensuring timely reporting of outcomes.

• Unexpected results may require unblinding of one or more investigators before making a decision.

Example: CAST Study, A Trial Stopped for Harm

• 1st DSMB meeting before enrollment started

– Recommended =0.025 (one-sided) for benefit

– Added =0.025 lower symmetric boundary for harm

– Lan-DeMets spending function used with expected number of cardiac sudden deaths (425) used to estimate information fraction

Case 13 in Case Studies Book

CAST Sequential Boundaries

CAST Study: 2nd DSMB Meeting

• 6 months after study began

• No outcome data available for review

• DSMB decided to remain partially blinded in the review of future interim data to maintain objectivity

• Could be unblinded if necessary for deliberations

CAST Study: 3rd DSMB Meeting

• 15 months after study began

• 1147 patients randomized, ~ ¼ of target

• Treatment assignment blinded to DSMB in reviewing data

• Sudden Death: - DSMC report: 3/576 in drug X, and 19/571 in Drug Y - Data sweep: 10 in drug X, and 22 in Drug Y

• DSMB decided no recommendations were appropriate

• Planned to meet again 6 months later

CAST Study: Events Following 3rd DSMB Review

• CAST Coordinating Center monthly summarized the data for internal monitoring

• 3 months after 3rd review, the results became more extreme

• One month later, unblinded primary data presented to NHLBI

• One month later, DSMB was notified the unblinded harmful findings by conference call

• DSMB requested verification of treatment coding, additional detailed analysis and sweep of the clinical centers for yet unreported primary outcome

CAST Study: 4th DSMB review

• 33 sudden deaths in encainide and flecainide arms combined and 9 on placebo

• Logrank Z=-3.22, crossed the harm boundary -3.04

• DSMB recommended dropping the encainide and flecainide arms

• NHLBI, principal investigators, drug regulatory agencies were notified the harmful findings

• The public was alerted because many non-study patients were being treated with these drugs

• Primary report was published rapidly

CAST Study: Interim and Final Results for Sudden Cardiac Mortality

Interim Review

Ecanide/Flecanide Placebo Z

1 22 10 1.77

2 33 9 3.70 vs. (3.22 for logrank)

Final 43 16 3.52

Z (normal approx. to Poisson) = a - b √ a + b

a= no. deaths on encainide/flecainideb= no. deaths on placebo

Lesson Learned: CAST

• A 1-sided boundary is usually not appropriate (DMC modified it)

• DSMBs need to be able to un-blind treatment codes

• Timely review of data important (unblinded statistician alerted DMC)

Example: CPCRA NuCombo Study, a Trial in Which DSMB’s Shared Data

Saravolatz L, N Eng J Med, 1996

Unblinded

Blinded Blinded

ddI+AZT

ddC+AZT

Placebo (ddI)+AZT

Placebo (ddC)+AZT

ddI Arm ddC Arm

NuCombo Interim Results: Late 1993

No. AIDS or Death

B-P

166

42

30.6

B

332

80

31.2

A-P

167

55

46.4

A

330

74

29.2

No. Patients

HR (A-P/B-P) = 1.6; 95% CI: 1.1 to 2.4; p=0.03

Treatment Group

Armitage P, Cont Clin Trials, 1999

Rate(per 100)

Lessons Learned: NuCombo

• In some situations (e.g., a major safety concern), exchange of information between DMCs can be helpful.

• In most situations, external data considered by DMCs should be limited to published information. (see Dixon D and Lagakos S, Cont Clin Trials, 2000; 21:1-6)

Example: MRC/BHF Heart Protection Study, a Trial Some Felt Should Have Been Stopped

Early• 20, 536 men and women in the UK with CHD,

other occlusive arterial disease or diabetes with cholesterol > 135 mg/dl

• Randomized equally to simvastatin or placebo

• Primary comparisons were of effects from all cause mortality, CHD mortality and from all other causes.

Heart Protection Study Collaborative Group, Lancet, 2002.

DMC Monitoring Guidelines

With consideration of the study data and the results from other studies, the DMC was to advise the investigators if…

• “Proof beyond reasonable doubt” that either for all participants or for some specific type of participant, use of statin therapy was clearly indicated or contraindicated in terms of the net difference in all-cause mortality.

• Evidence that might reasonably be expected to influence materially the management of patients by many clinicians.

Case Examples of Trials That Had to React to External Information

• BEST (beta-blockers for heart failure) N Engl J Med 2001

• Concorde (zidovudine for asymptomatic HIV)

Lancet 1994 and Cont Clin Trials1999

• CPCRA 023 (CMV prophylaxis)

AIDS 1998 and Cont Clin Trials 2003

General Requirements for Informed Consent

Significant new findings developed during the course of the research that may relate to the subject’s willingness to continue participation will be provided.

WHAT IF THERE ARE NEW FINDINGS?You will be told about any new information learned during the study that might cause you to change your mind about staying in the study.

Code of U.S. Federal Regulations Part 46, Subpart A, Section 46.116

Types of New External Information

• A finding from a randomized study with clinical outcomes in the same target population?

• A trial in a different target population?

• A non-randomized study?

• Changes in labeling due to adverse events (e.g., a modification to RISKS and/or DISCOMFORT section of consent)?

All New Information is Not Equal: A Hierarchy of Evidence Should be Considered in Assessing

Significance

• Coherence of evidence from multiple sources• Systematic review of well-designed, large randomized

trials• Strong evidence from one large randomized trial• Systematic review of small trials (e.g., surrogate

outcome studies)• Systematic review of well-designed cohort studies• Strong evidence from one cohort study• Unsystematic observations (expert opinions)

Adapted from Devereaux PJ et al, Evidence-Based Cardiology, 2nd Edition BMJ Books 2003.

Responsibilities of the Data Monitoring Committee (DMC) Concerning

External Information

“A DMC may be asked to consider the impact of external information on the study being monitored. Release of results of a related study may have implications for the design of the ongoing study, or even its continuation.”

“The role of the DMC in considering interim changes to a study protocol or other aspects of a study conduct in response to external information raises additional issues that merit consideration.”

FDA Guidance for Clinical Trial Sponsors. Establishment and Operation of Clinical Trial Data Monitoring Committees.

Consideration of External Information by the DMC: An Issue

that Merits Consideration • Possible unblinding of study participants,

investigators and sponsor.

• In most cases, the study team and sponsor (if blinded): – Should make the assessment since they are

blinded to treatment differences.– Should notify the DMC.

Case Example: Beta-Blocker Evaluation of Survival Trial (BEST)

Control (standard of care) arm:Optimal medical therapy and placebo for beta-blockerSetting: Beta-blockers had been avoided due to effects on heart rate and BP in heart failure patients, but data emerges indicating they may be effective treatment for heart failure. Several ongoing studies sponsored by pharmaceutical companies but most patients enrolled outside the U.S.

Case Example: BEST

Randomization of Patients with NYHA Class III or IV Heart Failure

Bucindolol Hydrochloride+

Optimal Medical Management(ace-inhibitor digitalis, diuretic)

(N=1354)

Placebo+

Optimal Medical Management(N=1354)

Case Example: BESTTimeline of Events

• May 1995: enrollment initiated• March 1998: Cardiac Insufficiency Bisprolol Study (CIBIS-II)

(N=2647) stopped early due to significant mortality benefit of bisprolol (monitoring guidelines were changed in 1998 to a nominal 0.05 for primary outcome of mortality)

• October 1998: MERIT-HF trial (N=3991) stopped early due to significant benefit of metoprolol on survival and HF hospitalization

• July 1999: DMC recommends early termination due “information…from other studies of beta-blockers…and by a concern about the equipoise of the trial”. Sponsors (NHLBI and VA) concurred. (p-value at termination=0.16 at time of recommendation)

Mortality Results of CIBIS-II, MERIT-HF and BEST

Treatment PlaceboCIBIS-II (Bisoprolol) 156/1327 228/1320

MERIT-HF (Metoprolol) 145/1990 217/2001

BEST (Bucindolol) 411/1354 449/1354

Case Study: BEST Aftermath

• Do benefits of beta-blockers diminish in patients with advanced heart failure?- Copernicus trial of patients with advanced heart failure (N=2289) stopped early in March 2000 for mortality benefit of carvedilol as compared to placebo (130 vs 190 deaths)

• Is bucindolol fundamentally different from other beta-blockers?

• Does beta-blockade work in African-Americans with HF? (23% black in BEST vs < 5% in other studies)

Case Example: Concorde(Lancet, 1994)

Control (standard of care) arm:

Deferred zidovudine (placebo): open-label treatment provided after AIDS/ARC

Setting:

Several ongoing trials of zidovudine.

Case Example: ConcordeDesign Schematic

Randomization of Asymptomatic Patients

with HIV

Immediate Zidovudine(N=877)

Deferred Zidovudine (Placebo)(N=872)

Open label treatment following AIDS/ARC

October 1989 Amendment:• Open-label treatment after 2 consecutive CD4+

counts <500 • Primary prophylaxis for PCP

Case Example: Concorde DMC Deliberations-1(Armitage P, Cont Clinical Trials, 1999)

• March 1988, enrollment begins.

• September 1989: (987 participants randomized) Concorde DMC informed by NIAID of early termination ACTG 019 (<500 CD4+)- DMC recommends trial should continue as planned

• October 1989, ACTG senior investigators meet with Concorde DMC and coordinating committee- Concorde DMC again recommends trial should continue as planned- Protocol amendment allowing open-label treatment after two consecutive CD4+ counts <500

- Letter sent to trial participants describing ACTG 019 results and why it was important to continue Concorde

• October 1991 (1715 participants), 1st formal interim analysis- DMC recommends trial continue as planned and that recruitment end

Case Example: Concorde DMC Deliberations-2(Armitage P, Cont Clinical Trials, 1999)

• February 1992: DMC asked to consider another trial, European-Australian Collaborative Group Study, (CD4+ >400) which was terminated early based on CD4+ differences

- DMC recommends continue as planned. • June 1992: (1749 participants), 2nd formal interim

analysis- DMC recommends follow-up end in December 1992 due to growing lack of equipoise among investigators and growing use of open-label treatment

Case Example: ConcordeACTG 019 and Interim Concorde Results

in September 1989

ACTG 019+

ConcordeInterim Data

September 1989++

500 mg

ZDV

1500 mg ZDV Placebo

IMM ZDV (1000mg)

DEF ZDV

AIDS, ARC or death

17 19 38 7 13

AIDS 11 14 33 4 7Death 1 3 4 0 0

No. patients 453 457 428 494 493

Avg follow-up:

51-61 weeks 23 weeks

+ N Engl J Med, 1990++ Cont Clin Trials, 1999

Final Concorde Study Results

Immediate ZDV Deferred ZDV

Deaths 95 76

AIDS or death 175 171

ARC, AIDS or death 263 284

CD4 difference over 3 years of follow-up (immediate – deferred) = 30 cells (p<0.0001)

Lancet, 1994

Overview of Trials of ZDV vs. Placebo (Immediate vs. Deferred)

Immediate ZDV

Deferred ZDV(placebo)

Risk Ratio

Deaths 734 917 1.04AIDS/death 1026 882 0.96

No. patients 4431 3291

Lancet 353: 2014-2025, 1999.

Case Example: CPCRA 023(CMV Prophylaxis)

Control (standard of care) arm:

Placebo (no prophylaxis for CMV)

Setting:

Ganciclovir and foscarnet approved for treatment, but not prophylaxis

A similar, but not identical, trial sponsored by (Syntex) pharmaceutical sponsor is also conducted. CPCRA study is confirmatory.

Case Example: CPCRA 023(CMV Prophylaxis)

Randomization of HIV Patients with CD4+ ≤100

Oral Ganciclovir(3g daily)(N=662)

Placebo(N=332)

Background treatment: ART and prophylaxis for other

infections

September 1994 amendment:Open label treatment allowed

Case Example: CPCRA 023 (CMV Timeline of Events Prophylaxis)

(Hillman & Louis, Cont. Clin Trials, 2003)

• November 1992: Syntex trial initiated• March 1993: CPCRA trial initiated• June 1994: Early termination of Syntex trial• July 1994: CPCRA provided results of Syntex study; DMC is

notified and holds special meeting- Interim results shared with study co-chairs, and the DMC recommended continuing CPCRA trial

• September 1994: - Protocol amendment- Patients informed of Syntex study results and DMC recommendation- Patients also informed that CPCRA study did not show that CMV disease or mortality differed (no numeric results given)

• Patients given 3 options:1) Continue blinded treatment 2) Stop blinded treatment and receive open-label3) Stop blinded treatment and no not take open-label

Case Example: CPCRA 023 (CMV Prophlyaxis)

Syntex and Interim CPCRA 023 Results in July 1994

Syntex Trial+

CPCRA 023 Interim Data July, 1994++

No. Patients 725 994No. with CMV 148 62HR (95% CI) CMV 0.51 ( 0.36-0.73)

0.87 (0.52-1.46)

HR (95% CI) Death

0.81 (0.61-1.07) 1.27 (0.78-2.07)

+ N Engl J Med, 1996++ Cont Clin Trials, 2003

Final CPCRA 023 Results(AIDS, 1998)

Ganciclovir(N=662)

Placebo(N=332)

HR (95%CI)

CMV 101 55 0.92 (0.65-1.27)

Death 222 132 0.84 (0.67-1.04)

Open-label ganciclovir (%)

39% 37%

Median months on ganciclovir

9.2 2.0

Summary - 1• Plan for interim looks in advance and state monitoring

guidelines in protocol.

• Recognize that every study is different and no single principle (except assurance of patient safety) can be identified to guide decisions – need to be flexible

• Consider all of the internal evidence (multiple outcomes and subgroups), data quality and timeliness before recommending early termination.

• Be cautious of trends and subgroups even though this may be agonizing – in the end want a definitive answer.

• Timely data collection and summaries are critical.

Summary - 2• If new information arises, even from a randomized

trial, it may be important to continue the ongoing study.– Different target population– Longer follow-up– Replication

• Treatments used lifelong require long-term studies to assess risk/benefit; external information is more likely to arise in long-term studies. Implications:

– Study designs need to anticipate new information that might arise.

– Study investigators and DMCs must be prepared to respond.