111 Update on Clinical Trials Registration and Results Reporting Requirements Deborah A. Zarin, M.D. ClinicalTrials.gov February 24, 2009

111

Update on Clinical Trials Registration and

Results Reporting Requirements

Deborah A. Zarin, M.D.ClinicalTrials.gov

February 24, 2009

Context: Continuing Concerns About Lack of

Transparency of Clinical Trial Information

Recent Events: Lack of Transparency in Clinical Research

Source: Figure 1A. Turner et al. (NEJM, 2008)

The investigation was launched following concerns…

• …although the ENHANCE trial ended in April 2006, the data had not yet been released.

• …[the sponsors] did not register the clinical trial in a timely manner.

• …[the sponsors] attempted to change the study endpoints, and thus the study results, prior to the public release of the results.

ENHANCE

Source: Kastelein JJ, Sager PT, de Groot E, Veltri E. Am Heart J. 2005 Feb;149(2):234-9.

12

3

Primary Outcome Measures:

• Change in ultrasound-determined average carotid artery intima-media thickness (IMT) on a per subject basis between baseline and endpoint. [ Time Frame: 24 months ]

Primary Outcome Measures:

• Change in ultrasound-determined average carotid artery intima-media thickness (IMT) on a per subject basis between baseline and endpoint. [ Time Frame: 24 months ]

Source: Silverstein FE et al. JAMA. 2000 Sep 13;284(10):1247-55.

Kaplan-Meier estimates for ulcer complications according to traditional definition. Results are truncated after 12 months, no ulcer complications occurred after this period. Adapted from Lu 2001.

Source: Jüni P, Rutjes AW, Dieppe PA. BMJ. 2002 Jun 1;324(7349):1287-8.

10

“The Neurontin Legacy – Marketing through Misinformation and Manipulation”

• Studies “designed and commissioned specifically to promote Neurontin use.”

• Seeding trial “to give neurologists the opportunity to titrate to higher doses [up to twice the FDA-approved limit] when needed”

• Delayed publication of negative results to mitigate damage to “neurontin’s marketing success”

• Summary: “the documentation of comprehensive manipulation of research and publication related to Neurontin is remarkable.”

Landefeld CS, Steinman MA. NEJM. N Engl J Med 360:103-6

11

(WSJ, Aug 2008)

PLoS Med. 2008;5(11): e217

Lee K, Bacchetti P, Sim I. PLoS Med. 2008;5(9): e191

161616

History of ClinicalTrials.gov

• FDAMA 113 (1997): Mandates Registry– IND trials for serious and life-threatening diseases or

conditions

• ClinicalTrials.gov Launched in February 2000• Calls for Increased Transparency of Clinical Trials

– Maine State Law; State Attorneys General– Journal Editors (2004)

• ClinicalTrials.gov Accommodates Other Policies• FDAAA 801 (2007): Expands Registry and Adds

Results Database

1717

Policies and Users

ClinicalTrials.gov

FDAMA 113BPCA

Maine

FDAAA Sponsor Policy(e.g., NIH, VA)

ICMJE

WHO

OttawaStatement

Recruitment (e.g., patients,

physicians)Journal Editors

Research Funders

IRBsHealth Policy

Makers

18

ClinicalTrials.gov Statistics(as of 02/03/2009)

Number Percent

Total 67,064 100%Type of Trial*

Observational 10,690 16%Interventional 57,119 84%– Drug & Biologic 42,684 – Surgical Procedure 8,585 – Behavioral, Gene Transfer, Other 7,997 – Device** 3,862

International Sites (161 countries)US only 32,772 49%Non-US only 23,109 34%US & Non-US mixed 4,064 6%Missing 7,119 11%

*171 records missing Study Type information**173 device trials – “delayed posting”

19

ClinicalTrials.gov Statistics (cont.)(as of 02/03/2009)

User Statistics

Page Views per month 40 Million Unique visitors per month 500,000

Number Percent

Trials by Data Provider

US Federal (including NIH)18,088 27%Industry 21,072 31%University, Other 28,820 42%

Total 67,980

Search Features• Powerful Search Engine

– Synonymy• E.g., can find V501 studies by searching “Gardasil”

– Hierarchies• E.g., can find Crohn’s disease by searching for IBD

– Spelling relaxation

• Interface allows for field specific searching– E.g., can find “condition” without finding

“exclusion criterion”

Update on Clinical Trial Registration

26262626

Public Law 110-85Sec.801 Expanded Clinical Trial Registry

• Enacted on September 27, 2007• Requires Trial Registration (Dec 2007)

– Phase II-IV drug and device trials for all diseases– Data elements: ClinicalTrials.gov + ~ WHO/ICMJE

• Requires Results Reporting (Sept 2008)– Trials of FDA-approved or cleared drugs and devices– “Basic” Results: Baseline Characteristics, Primary &

Secondary Outcomes, Statistical Analyses– Adverse Events (Sept 2009)– “Expansion” of results by rulemaking (Sept 2010)

• Added enforcement provisions

Enforcement Provisions

• Notices of non-compliances

• Civil monetary penalties up to $10,000/day

• Withholding of NIH grant funds

Key Terms

• Applicable Clinical Trials– Interventional trials– Phase 2-4 drug, biologic, device– >= one site in U.S.– Ongoing as of 9/27/07, or later

• Responsible Party– Sponsor, grantee– PI if designated

• (Primary) Completion Date

Wood AJJ. Progress and deficiencies in the registration of clinical trials. NEJM. 2009

Wood AJJ. Progress and deficiencies in the registration of clinical trials. NEJM. 2009

What Information Is In the Trial Registry?

• Basic Protocol Details– Condition, intervention(s), design, outcome

measures, key dates

• Administrative Information– NCT#, other IDs, Responsible Party, PI

• Recruitment and location status• Linkages

– PubMed, FDA resources, consumer health info

31

New Registrations Continue to IncreaseNumber of New Records Since May 1, 2005

0

10,000

20,000

30,000

40,000

50,000

60,000

70,000

start to 12/31/2005

1/30/2005

3/6/2005

4/10/2005

5/15/2005

6/19/2005

7/24/2005

8/28/2005

10/2/2005

11/6/2005

12/11/2005

1/15/2006

2/19/2006

3/26/2006

4/30/2006

6/4/2006

7/9/2006

8/13/2006

9/17/2006

10/22/2006

11/26/2006

12/31/2006

2/4/2007

3/11/2007

4/15/2007

5/20/2007

6/24/2007

7/29/2007

9/2/2007

10/7/2007

11/11/2007

12/16/2007

1/20/2008

2/24/2008

3/30/2008

5/4/2008

6/8/2008

7/13/2008

8/17/2008

9/21/2008

10/26/2008

Registration of Phase 1 and Device Trials

• 2100 device trials registered between 9/07 and 1/09– 175 are in “lock box”

• 162 Industry• 13 Other

• Phase 1 trials– 186/month in fy 2008 (73% increase from 07)– 205/month in fy 2009* (10% increase from 08)

* First four months

Key Points: Memo from Dr. Kington, Acting Director, to NIH Grant Awardess

• “For grants, NIH is generally not the sponsor … and, as such, NIH would not be the responsible party.”

• “Responsible parties who have not yet registered their clinical trials should do so immediately.”

• “Thank you for your attention to this important matter and your commitment to helping enhance the transparency of NIH funded clinical trials.”

Status of Stanford Trials

• 73 may be “applicable”– 65 have outcome measure– 2 have RP– 62 have start date– 1 has “primary completion date”– 8 have “completion date”

• How many “results” are due?

35

Enforcement Provisions

• Notices of non-compliances

• Civil monetary penalties up to $10,000/day

• Withholding of NIH grant funds

Bottom Line

• Register prior to enrollment:– Phase 2-4 interventional trials that include a drug,

device or biologic– Regardless of whether or not the trial is being used

to support an FDA application

• Report results:– Any trial described above once the drug, device or

biologic has been approved; OR– Within one year of “primary completion date”

• Keep all information up to date!37

38

Basic Results Database

3939393939

Basic Results Database: General Characteristics

• Results of “applicable clinical trials” of FDA-approved/cleared medical products

• Generally, submission within 12 months of the earlier of estimated or actual trial completion date (of primary outcome)

• Delayed Submission of Results– Seeking initial approval – Seeking approval of a new use – Extensions for “good cause”

Basic Results Modules

• Participant Flow • Baseline and Demographic Characteristics• Outcome Measures• Adverse Events (summary data)• Other Information

– “Certain Agreements” Restricting Results Disclosure

– Overall Limitations and Caveats– Results Point of Contact

4141

Current Status – “Basic Results”(as of 02/06/09)

• Functional Web-based Data Entry System • Launched in September 2008• Ongoing system of feedback and

improvements

• 410 Results Records have been submitted• Industry: 293 records from 72 data providers• Other: 117 records from 80 data providers• Anticipate greatly increased rate of

submission

424242

Sample Posted Results

43

Published Participant Flow

Source: Kimmick GG et al. Breast J. 2006 Mar-Apr;12(2):114-22.

Reasons Not Completed

Milestone

Arms

Multiple “Periods”

Crossover Design

Published Baseline Data

Source: Richter JE et al. Am J Gastroenterol. 2001 Mar;96(3):656-65.

User-SpecifiedMeasure

“Default” RequiredMeasures

Categories

Arms

5050

51

Published Primary Outcome

Source: Kimmick GG et al. Breast J. 2006 Mar-Apr;12(2):114-22.

Categories

Statistical Analysis

Statistical Analysis

Published Adverse Events

57

595959

Results Data Entry Process

Technical Issues

60

Design Requirements

• Display consists of data tables with minimal text—must be self-explanatory

• System must accommodate range of study designs and facilitate comparison across studies

• NLM directed to: – Consider different methods of display based on

principles of risk communication for different audiences

– Ensure the data are searchable in many ways

• Structured data entry required to facilitate search and display needs 60

61

Design Features

• Tables are “constructed” by the data provider– Columns are pre-set as study arms, but can

be changed by the data provider– Rows are measures—some are pre-set,

others are customized for each study– Type of measure determines specific design

of “cells”

• Attempt to balance fixed structure with flexibility

62

Principles for Using the Basic Results Database

• Submitted data are used to develop basic tables for the public display

• Tables must be interpretable by people not familiar with each particular study

• Labels for rows, columns, and units of measure must be meaningful and precise

Who is the Audience?PI and Clinical Research Team (You!)

Other Medical Researchers in same field

[Study Sponsor]

Other Medical Researchers in other fields

Other Readers of the medical literature

Science Writers

Lay Public (readers of consumer health literature)

Resources to Help Data Providers

• “Helpful Hints”– Illustrates process for entering different study

designs (parallel, crossover, diagnostic accuracy, bioequivalence—in progress)

• Webinar• “Common Errors” • Individual discussions regarding particular

studies• Presentations

72

Interesting Findings to Date

• Large numbers of submitted Outcome Measures and Statistical Analyses

• Power of Defaults (e.g., “Baseline Measures”)– Age > 65– Race and Ethnicity– Region of Enrollment

• Problems with imprecise entries

Lessons Learned from Early Submissions of Basic Results

• Many iterations with the QA staff are necessary to reach minimal quality standards and to correct serious flaws

• Data Providers must be able to understand the study design and data analysis– Typically, the investigator and a

statistician will need to be involved

73

Quality Assurance Challenges

• Data tables will be the public representation of the study—must be clear and informative;

• NLM QA Focuses on:– Apparent Validity (when possible)– Meaningful Entries– Internal consistency/logic– Format

7575

Common Quality Concerns

• Reporting of percentage without reporting absolute numbers

• Improper use of terms• Incidence• Proportion and Ratio• Frequency

• Reporting a change—lack of specificity• Subtraction: minuend vs. subtrahend• Ratio: nominator vs. denominator

• Complicated outcomes that cannot be understood

Registration and Results Data Must be Consistent

• Participant Flow Numbers and Enrollment

• Study Design and Results Tables– Number of Arms

7777

Actual Enrollment: 229Study Start Date: June 2006Study Completion Date: October 2007Primary Completion Date: October 2007 (Final data collection date for primary outcome measure)

Summary Protocol Section:

Basic Results Section:

Placebo Drug X

STARTED 220 211

COMPLETED 218 210

NOT COMPLETED 2 1

Participant Flow: Initial Treatment

Actual enrollment (229) displayed in the protocol section does not match total number started in the basic results section (220 + 211 = 431)

BEFORE Revision (Public View)

Table Structure Must be Logical

7979

Measured ValuesDrug X, Week 10

Drug X, Change from Week 10 to 18

Number of Participants Analyzed 88 80

Treatment Satisfaction QuestionnaireAfter 18 Weeks of Treatment

[units: Score]

Mean ± Standard Deviation

81 ± 17.46 7.9 ± 12.16

Inconsistency between columns and rows: Measure “at week 10” and Measure “after 18 weeks of treatment”


Data Must Make Sense

• Outcome Measure Name, Description, Units and Data are Compatible

8181

Measured ValuesIntervention X Control


Hours Per Day of Sleep

[units: Average Hours per Day]


823 ± 92 864 ± 106

Inconsistency between Units of Measure, “average hours per day,” and Measure Data: value provided is greater than the total number of hours in a day


8282

Measure Type Secondary

Measure Name Use of Community Health Resources

Measure Description Evaluation of visits to primary care pediatrician, hospital emergency and re-hospitalization

Time Frame Up to 3 months after discharge

Safety Issue No

Secondary Outcome Measure: Use of Community Health Resources

Measured ValuesEarly

DischargeStandard Discharge


Use of Community Health Resources

[units: Number]

4.4% 10.5%

• Data are inconsistent: percentages of what?

• Invalid entry: needs to be numerical (cannot include “%”)


Implies number of health resources used – how was it measured?

8383

Measure Type Primary

Measure Name Frequency and Magnitude of Antibody Response

Measure Description Nasal secretions to Virus A/12 and B/14. Antibody Response: Three-fold increase after immunization

Time Frame Visit 3 (Week 15)

Safety Issue Yes

Secondary Outcome Measure: Frequency and Magnitude of Antibody Response

Measured ValuesVaccine,

Low Dose

Vaccine, High Dose


Frequency and Magnitude of Antibody Response

[units: Participants]

17 21

May mean “three-fold or greater increase”


Same unit cannot represent measures of “frequency” and “magnitude”

“Participants” is not a unit of measure for “frequency” or “magnitude”

Best to provide both categories for a dichotomous measure:

• < 3x increase• ≥ 3x increase

Best to provide both categories for a dichotomous measure:

• < 3x increase• ≥ 3x increase


Measure Name Maximum Tolerated Dose (MTD) Determination as Measured by Dose Limiting Toxicity (DLT)

Measure Description The primary variable for determination of the MTD was the occurrence of DLT during the first treatment cycle. MTD has been exceeded if >=2 of 6 patients experience a DLT.

Time Frame Cycle 1

Safety Issue Yes

Primary Outcome Measure: Maximum Tolerated Dose (MTD) Determination …

Measured ValuesDose A.1

Dose A.2

Dose B.1

Dose B.2

Dose C.1

Dose C.2

Number of Participants Analyzed 6 7 3 6 7 4

Maximum Tolerated Dose (MTD)…

[units: Participants]

Number (#) of DLT 1 3 0 2 1 0

# Patients at dose level < MTD 0 0 0 0 7 4

# Patients at dose level = MTD 6 0 3 0 0 0

# Patients at dose level > MTD 0 7 0 6 0 0

Tables Must Be Informative

• Scales should include:– Full name– Construct or domain (e.g., pain)– Direction of scores (Best/Worst Value)– Other information as necessary

• Measures Have Useful Descriptions

• Avoid Abbreviations

8686

Investigational Drug X

GOG Performance Status

[units: Score]

0 48

1 27

2 4


Need information about this scale• Full Name• Construct/domain• Range and directionality

Baseline MeasuresNeed information about these values(e.g., is “0” better or worse than “2”?)

Need to change to “participants” – data represent “number of participants” with a particular score

Are these the only possible scores?

Open Label

Number of Participants Analyzed 403

Duration (Days)

[units: Days]


195.5 ± 43.87


Measure Name Duration (Days)

Measure Description Extent of Exposure for All Treated Subjects

Time Frame Duration of Study

Safety Issue No

Secondary Outcome Measure: Duration (Days)

Measured Values


Needs description: Duration of what?

Needs Arm Label: What is the intervention?

Measure Information Must be Precise and Accurate

• Avoid misuse of terms, e.g., – proportion– ratio– incidence

• State what is being measured and how– Do not provide results in measure description

field

Drug X Drug Y Drug X + Y

Number of Participants Analyzed 351 361 384

Proportion of Patients with Controlled SBP

[units: Participants]186 135 287


Measure Name Proportion of Patients with Controlled SBP

Measure Description Controlled SBP defined as SBP < 130 mmHg

Time Frame Baseline to 12 weeks

Safety Issue No

Primary Outcome Measure: Proportion of Patients with Controlled SBP

Spell out acronym

Not a proportion


Not a proportionNot a proportion

Drug X Drug Y Drug X + Y

Number of Participants Analyzed 351 361 384

Change in Sitting DBP From Baseline to End of Study

[units: mmHg]

Lease Squares Mean ± Standard Error

-8.4 ± 0.2 -6.7 ± 0.2 -11.2 ± 0.3


Measure Name Change in Sitting DBP From Baseline to End of Study

Measure Description Change in Sitting DBP

Time Frame Baseline to 12 weeks

Safety Issue No

Primary Outcome Measure: Change in Sitting DBP From Baseline to End of Study

Spell out acronym

Specify calculation details: which value was subtracted from which?



Measure Name To Compare Drug X and Drug Y for Efficacy

Measure Description

Time Frame 4 months

Safety Issue No

Secondary Outcome Measure: To Compare Drug X and Drug Y for Efficacy

State what is being measured, not the purpose


Needs description: what is being measured and how?

Data in All Tables Must be Internally Consistent and Logical

• Participants must “flow”

• “Number analyzed” must be consistent with participant flow data

• Avoid Illogical Entries

93

Placebo Drug X

STARTED 301 299

COMPLETED 291 285

NOT COMPLETED 10 14

Participant Flow: First Period Number of participants STARTED in second period of Participant Flow needs to be the same as numbers COMPLETED in the first period


Placebo Drug X

STARTED 298 290

COMPLETED 288 278

NOT COMPLETED 10 12

Participant Flow: Second Period

9494

Measured ValuesIntervention X Control

Number of Participants 28 27

Hours Per Day of Sleep

[units: Average Hours per Day]


823 ± 92 864 ± 106

Inconsistency between Units of Measure, “average hours per day,” and Measure Data: value provided is greater than the total number of hours in a day


Statistical Analyses

• Must be Logical

• Compatible with Data

• Informative (report informative metrics)

9696

Groups Early Discharge vs. Standard Discharge

Method ANOVA

P-Value 0.05

Mean Difference (Net) 9

Statistical Analysis 1 for Parental Stress

Measured ValuesEarly Discharge Standard Discharge


Parental Stress

[units: Points on a Likert Scale]


9.3 ± 1.2 7.8 ± 2.1

Inconsistency between Measure Data and Method of Estimation

• Reported Mean Difference: “9”• By Inspection: 9.3 – 7.8 = 1.5


9797

Placebo Investigational Drug X


Time to Relapse of a Mood Episode

[units: Days]

Median (Inter-Quartile Range)

219 (83 to NA) NA (173 to NA)


Measure Name Time to Relapse of a Mood Episode

Measure Description

Time Frame 24 months

Safety Issue No

Secondary Outcome Measure: Time to Relapse of a Mood Episode

Measured Values

Needs description

Invalid entry


98

Where is the Quality Line?

Domains of Quality:

Quality of Entries

Scope of Entries

Not Meaningful Meaningful

Minimal Comprehensive

QA Staff Resources

Who is the Audience?PI and Clinical Research Team (You!)

Other Medical Researchers in same field

[Study Sponsor]

Other Medical Researchers in other fields

Other Readers of the medical literature

Science Writers

Lay Public (readers of consumer health literature)

Bottom Line

• Register within 21 days of enrollment:– Phase 2-4 interventional trials that include a drug,

device or biologic– Regardless of whether or not the trial is being used to

support an FDA application

• Report results:– Any trial described above within one year of “primary

completion date” OR– once the drug, device or biologic has been approved;

• Keep all information up to date!

100

101101101101101

Additional Information

• Email LISTSERV and other FDAAA information:– http://prsinfo.clinicaltrials.gov/fdaaa.html

• Other general information:– http://prsinfo.clinicaltrials.gov

• Questions?– [email protected]

102102

Finding Results at ClinicalTrials.gov

• From Homepage – Go to “Search for Clinical Trials”– Select “Advanced Search”– Select “Studies with Results” from the menu

for the Study Results field– Select study record from results list– Click “Study Results” tab

103103

104

http://prsinfo.clinicaltrials.gov/fdaaa.html

105105

106106

107107

Documents

111 Update on Clinical Trials Registration and Results Reporting Requirements Deborah A. Zarin, M.D. ClinicalTrials.gov February 24, 2009