Using an Using an Independent Independent

Statistician to Statistician to Support a Data Support a Data

Monitoring Monitoring CommitteeCommittee

Patrick D. O’Meara, Patrick D. O’Meara, Ph.D.Ph.D.Pat O’Meara Associates, Pat O’Meara Associates, Inc.Inc.pat@patomeara.compat@patomeara.com

FDA/Industry WorkshopFDA/Industry Workshop14-16 September 200514-16 September 2005

OutlineOutline

IntroductionIntroduction ChecklistChecklist 2 examples2 examples RecommendationsRecommendations

““. . . the integrity of the trial is best . . . the integrity of the trial is best protected when the statistician protected when the statistician preparing unblinded data for the preparing unblinded data for the DMC is external to the sponsor, DMC is external to the sponsor, especially for the critical studies especially for the critical studies intended to provide definitive intended to provide definitive evidence of effectiveness.”evidence of effectiveness.”

- Draft Guidance On the Establishment and Operation of - Draft Guidance On the Establishment and Operation of

Clinical Trial Data Monitoring CommitteesClinical Trial Data Monitoring Committees

IntroductionIntroduction

There’s more to protecting the There’s more to protecting the integrity of the study than just integrity of the study than just engaging an independent engaging an independent statistician to perform a statistician to perform a survival analysis for the survival analysis for the DSMB.DSMB.

IntroductionIntroduction

ChecklistChecklist

CharterCharter ContractContract Documentation from SponsorDocumentation from Sponsor DatabaseDatabase Support from the SponsorSupport from the Sponsor

DSMB CharterDSMB Charter

Independent statistician’s roleIndependent statistician’s role Who directs the IS work?Who directs the IS work?

Sufficient freedom to create tables/analyses Sufficient freedom to create tables/analyses requested by the DSMBrequested by the DSMB

In effect the IS acts as an employee of the DSMBIn effect the IS acts as an employee of the DSMB Roles of other organizations who Roles of other organizations who

provide dataprovide data CROCRO IVRSIVRS Central clinical labCentral clinical lab Biomarker labBiomarker lab

ContractContract

Tasks assigned to the ISTasks assigned to the IS Programming of the tables, graphs, & listingsProgramming of the tables, graphs, & listings

Who will write them?Who will write them? How will they be verified ON THE IS’s SYSTEM?How will they be verified ON THE IS’s SYSTEM? If written by the Sponsor, what changes are If written by the Sponsor, what changes are

allowed?allowed? Who is in responsible for the data?Who is in responsible for the data?

Especially important if a CRO has been retained to Especially important if a CRO has been retained to monitor the study and to prepare the data for monitor the study and to prepare the data for interim analysis:interim analysis:

Adding variables to an analysis databaseAdding variables to an analysis database Correcting errors, inconsistencies Correcting errors, inconsistencies Interpretation Interpretation

ContractContract

Time to assemble the reportTime to assemble the report From time IS receives the dataFrom time IS receives the data Once agreed Sponsor must not let it Once agreed Sponsor must not let it

slide.slide. Time to prepare a presentationTime to prepare a presentation

Especially important at formal interim Especially important at formal interim analysis when auxiliary analyses may be analysis when auxiliary analyses may be needed to support result.needed to support result.

Documentation from Documentation from SponsorSponsor

Protocol & blank CRFProtocol & blank CRF Informed consent formsInformed consent forms Formal Statistical Analysis PlanFormal Statistical Analysis Plan

Includes a clear statement of decision Includes a clear statement of decision rules for any interim analysis:rules for any interim analysis:

Null and Alternative HypothesesNull and Alternative Hypotheses Significance levelSignificance level Test statistics and methodsTest statistics and methods

Futility, efficacyFutility, efficacy Adjudication process & rulesAdjudication process & rules Investigator’s BrochureInvestigator’s Brochure

Documentation from Documentation from SponsorSponsor

Proposed list of Tables, Graphs, Proposed list of Tables, Graphs, ListingsListings

Any special consideration in the Any special consideration in the studiesstudies The definition and processing of SAEs The definition and processing of SAEs

in a clinical endpoint studyin a clinical endpoint study For example, strokeFor example, stroke

Biomarker lab dataBiomarker lab data

DatabaseDatabase

Sources of the dataSources of the data CRF database (SAS)CRF database (SAS) Analysis database (SAS)Analysis database (SAS) Clinical laboratory (SAS, text)Clinical laboratory (SAS, text) IVRS (SAS, text, or Excel)IVRS (SAS, text, or Excel) Biomarker or specialty laboratory (text, Biomarker or specialty laboratory (text,

Excel)Excel) SAE database (expedited review – SAE database (expedited review –

pharmacovig.)pharmacovig.) Current death list (Excel)Current death list (Excel) 24-hour reports of SAE/Clinical endpoints24-hour reports of SAE/Clinical endpoints

DatabaseDatabase

Detailed specifications – derivations of Detailed specifications – derivations of derived variablederived variable

Annotated CRFAnnotated CRF Description of special processesDescription of special processes Topics that Sponsor and IS should discussTopics that Sponsor and IS should discuss

Frequency of updatesFrequency of updates Timing before reportsTiming before reports Robustness of interim cuts of the databaseRobustness of interim cuts of the database

Do AEs disappear?Do AEs disappear? Are some data sources more reliable or current Are some data sources more reliable or current

than others?than others?

Support from the Support from the SponsorSponsor

Availability by telephone or email ofAvailability by telephone or email of A Statistician who can speak with authority A Statistician who can speak with authority

about the study and proposed analysis.about the study and proposed analysis. A Statistician/Data analyst/Programmer who A Statistician/Data analyst/Programmer who

can answer detailed questions about the data.can answer detailed questions about the data. Face-to-face meeting with key project team Face-to-face meeting with key project team

members data management and statistics members data management and statistics Learn the system that produces the data for Learn the system that produces the data for

DSMBDSMB Especially useful when resolving Especially useful when resolving

inconsistencies.inconsistencies.

Example 1Example 1

Two treatments; Planned subjects: ~2200; Two treatments; Planned subjects: ~2200; 28-day all-cause mortality28-day all-cause mortality Two interim analyses plannedTwo interim analyses planned Protocol, SAPProtocol, SAP Tables, listings from Sponsor’s standard Tables, listings from Sponsor’s standard

librarylibrary The Job: IS to reproduce TLG and presentThe Job: IS to reproduce TLG and present Randomization schedule from IVRS groupRandomization schedule from IVRS group 4 data sets for analysis with detailed 4 data sets for analysis with detailed

specification specification 8 days to prepare and ship report8 days to prepare and ship report

Example 1Example 1

Statistician assigned to database quality Statistician assigned to database quality Sponsor’s project statistician provided Sponsor’s project statistician provided

SAS code to implement formal interim SAS code to implement formal interim analysis.analysis.

A dummy r.s. using A: odd, B: evenA dummy r.s. using A: odd, B: even 4-5 test shipments of data before 14-5 test shipments of data before 1stst

interiminterim Using last test shipment, 100% check of Using last test shipment, 100% check of

all tables against set produced by all tables against set produced by Sponsor. Sponsor.

Example 1Example 1

11stst Interim Report and Analysis Interim Report and Analysis Timeline for report was squeezed by 1-2 daysTimeline for report was squeezed by 1-2 days DSMB decision to continue study without DSMB decision to continue study without

changechange 22ndnd Interim Report and Analysis Interim Report and Analysis

Efficacy was demonstrated.Efficacy was demonstrated. Auxiliary analyses demonstrated consistent Auxiliary analyses demonstrated consistent

trends across many subgroupstrends across many subgroups DSMB recommended stopping for efficacyDSMB recommended stopping for efficacy IS presented results to Sponsor executive IS presented results to Sponsor executive

comm.comm. Study stopped.Study stopped.

Example 1Example 1

Lessons learnedLessons learned Well-defined roles and responsibilities Well-defined roles and responsibilities

contributed to team environment.contributed to team environment. Extra data transfers allowed practice so Extra data transfers allowed practice so

that final transfers went smoothly and that final transfers went smoothly and tight timelines could be met.tight timelines could be met.

Time spent reviewing database was a Time spent reviewing database was a big contributor to success of the project.big contributor to success of the project.

100% check of tables using dummy r.s. 100% check of tables using dummy r.s. was essential.was essential.

Example 2Example 2

Two treatments; ~1800 patientsTwo treatments; ~1800 patients All-cause mortalityAll-cause mortality DSMB meetings every 3 months at beginningDSMB meetings every 3 months at beginning Protocol, no SAPProtocol, no SAP Review imbalance every 8 deaths for 1Review imbalance every 8 deaths for 1stst 100 100 IVRS, Sponsor’s DM, Biomarker, Clinical labIVRS, Sponsor’s DM, Biomarker, Clinical lab Monthly updates of clinical database via FTP Monthly updates of clinical database via FTP Sponsor provided SAS programs for data from Sponsor provided SAS programs for data from

Clinical database (primary motivation $$)Clinical database (primary motivation $$)

Example 2Example 2

Programs designed for VAX; local Programs designed for VAX; local system PC.system PC. File references in every program had to be File references in every program had to be

changed.changed. Each program contained an extensive block Each program contained an extensive block

of code to merge in randomization schedule.of code to merge in randomization schedule. Logic errors in several programs.Logic errors in several programs. After a month, client resent programs with After a month, client resent programs with

changes but the changes were not changes but the changes were not documented and all the file references had documented and all the file references had to be changed again.to be changed again.

Example 2Example 2

Meanwhile, Biomarker data had been Meanwhile, Biomarker data had been coded by the lab to prevent inadvertent coded by the lab to prevent inadvertent blinding.blinding.

Coded results were manually transcribed Coded results were manually transcribed into database – no source record of into database – no source record of original value.original value.

Coded with C++ algorithm using radix-32.Coded with C++ algorithm using radix-32. When decoded found that there had been When decoded found that there had been

many transcription errors --- invalid many transcription errors --- invalid values.values.

Example 2Example 2

DSMB expressed concern about decision DSMB expressed concern about decision rule for stopping for safety.rule for stopping for safety. Requested Monte Carlo study.Requested Monte Carlo study. Decided to meet again in 10 days to discuss.Decided to meet again in 10 days to discuss. Monte Carlo showed that under reasonable Monte Carlo showed that under reasonable

assumptions, chances of stopping were less assumptions, chances of stopping were less than .05.than .05.

Recommended changing rule so Recommended changing rule so Pr(Stopping) ~ .15.Pr(Stopping) ~ .15.

Next DSMB, no improvement in safety – Next DSMB, no improvement in safety – study stopped.study stopped.

Example 2Example 2

Lessons learned:Lessons learned: Transferring SAS programs between systems Transferring SAS programs between systems

requires careful planning and lots of work.requires careful planning and lots of work. Need to specify who is responsible for Need to specify who is responsible for

correct implementation on local system.correct implementation on local system. Nothing in contract that said IS could do Nothing in contract that said IS could do

additional analyses suggested by DSMB. additional analyses suggested by DSMB. Serious decisions by DSMB could result in Serious decisions by DSMB could result in

liability.liability. Unforeseen problems can heavily influence Unforeseen problems can heavily influence

the amount of time spent on a project.the amount of time spent on a project.

RecommendationsRecommendations

Provide the independent statistician with all Provide the independent statistician with all the available information about the study:the available information about the study: ProtocolProtocol Statistical Analysis PlanStatistical Analysis Plan

Assign project team members to with the IS.Assign project team members to with the IS. Charter should state IS works for the DSMB.Charter should state IS works for the DSMB. IF Sponsor decides to provide programs, IF Sponsor decides to provide programs,

Work with IS when designingWork with IS when designing State in contract & charter who is responsible State in contract & charter who is responsible

Contract should indemnify IS.Contract should indemnify IS.

The EndThe End