Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
1
Harmonizing Regulatory Approaches to Clinical Trials to Improve Quality and Compliance
Leslie K. Ball, MDDirector, Division of Scientific Investigations
Office of Compliance, CDER, FDAFDA Enforcement Summit October 22, 2009
2
Disclaimer
The views expressed are my own.
3
Quality and Compliance in Clinical Trials: Overview
• Where are we now?– FDA’s responsibility for ensuring data quality– Public expectations and role of enforcement
• Where are we going?– Quality risk management
• Sponsors and clinical investigators• FDA and other regulatory authorities
4
Historical Perspective: Role of FDA in Ensuring High Quality Research
• 1961: 1st FDA inspection of clinical investigator in Maryland– CI convicted after it was determined that most of results produced
at kitchen table (> 25 sponsors) • Early 1960’s; MER-29 (cholesterol lowering agent)
approved– Subsequent human use revealed balding, impotence, cataracts,
and liver damage– Sponsor found guilty of falsifying source data and failing to submit
data on toxicity• These high profile cases of data fraud by clinical
investigators and sponsors* led to the recognition that FDA had responsibility for ensuring data quality via inspections
*http://www.fda.gov/downloads/AboutFDA/CentersOffices/CDER/UCM164151.pdf
5
Role of FDA In Ensuring High Quality Research
• Approvals and labeling of a new drug or biologic products are risk-benefit decisions
• Based on scientific and clinical review of data collected and presented by Sponsors
• Relies on public participation in clinical trials and public confidence in the clinical trial enterprise
6
FDA Bioresearch Monitoring Inspection Program
• Inspections of clinical investigators, sponsors, IRBs, GLP and BE– Evaluate data quality and integrity– Assess whether rights, safety and welfare of subjects have been
protected• Types of inspection:
– Data validation in support of NDAs/BLAs/ANDAs• Clinical Investigators• Sponsors/Contract Research Organizations• Bioequivalence
– Surveillance (Systems-based inspections)• GLP• Institutional Review Boards
– For cause (complaints)• All of the above
7
Quality and Compliance in Clinical Trials: Overview
• Where are we now?– FDA’s responsibility for ensuring data quality– Public expectations and role of enforcement
• Where are we going?– Quality risk management
• Sponsors and clinical investigators• FDA and other regulatory authorities
8
Public Expectations:How effective is the FDA*?
*http://www.gallup.com/poll/121886/cdc-tops-agency-ratings-federal-reserve-board-lowest.aspx
9
Public Expectations (cont.)• Congress
– House Energy and Commerce Committee/Govt Oversight Subcommittee
• Ketek (February 2008)• Independent IRBs (March 2009)
• OIG– September 2007: FDA’s Oversight of Clinical Investigators– January 2009: FDA’s Oversight of Clinical Investigators’ Financial
Information– Ongoing: FDA’s Oversight of Foreign Clinical Trials
• GAO– March 2009 Independent IRBs– Report to be released soon: FDA’s Oversight of Clinical
Investigators (DQ process)
10
Enforcement
11
Enforcement (cont.)“…A strong FDA has credibility with the public.A strong FDA is transparent in explaining its
decisions.A strong FDA pursues creative solutions to
longstanding problems and is always looking for novel ways to prevent illness and promote health.
And a strong FDA enforces the law…”
Margaret Hamburg, M.D., Commissioner of Food and DrugsFDLI "Effective Enforcement and Benefits to Public Health"August 6, 2009
12
0
100
200
300400
500
600
700
800
FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08
Spon
CIIRB/RDRC
BIOEQGLP
CDER BIMO Inspections* FY 2000-2008
549 556690723 672 667647 674690
*Based on inspection start date 2/5/09
13
CDER Clinical InvestigatorOAI Actions (Warning/NIDPOE Letters*)
02468
1012141618
2000 2001 2002 2003 2004 2005 2006 2007 2008 2009
WL
NIDPOE
*Based on Letter Issued (FY) 9-23-09
14
Enforcement • Is misconduct by clinical investigators
increasing?– ORA and DSI inspection data indicate that
rate of significant non-compliance (as evidenced by inspections with field classifications of OAI) has been stable, but rate of enforcement (final classifications of OAI including WLs) has been increasing
15
Quality and Compliance in Clinical Trials: Overview
• Where are we now?– FDA’s responsibility for ensuring data quality– Public expectations and role of enforcement
• Where are we going?– Quality risk management
• Sponsors and clinical investigators• FDA and other regulatory authorities
16
Data Validation Inspections for New Drug Applications: Old Paradigm
• Sites selected for inspection using qualitative risk-based approach
– Most often selected highest enrolling sites for inspection
• Typically 4-5 sites inspected per application• If significant violations identified at 1-2 sites
such that data deemed unreliable, FDA performed sensitivity analysis excluding those sites from efficacy analysis
• At remaining sites in application, data assumed to be reliable
17
The World Has Changed• Increase in multiregional, multicenter
studies with no single site driving efficacy• Limiting enrollment at clinical sites as risk-
mitigation strategy• If pivotal study has 200 sites and 2 of 4
inspected sites reveal significant GCP violations impacting data integrity, are data unreliable from 50% of sites across application?
18
Data Validation Inspections for New Drug Applications: Current Paradigm
• Use risk-based approach to select clinical investigator sites for inspection
• If first round on inspections of clinical investigator sites reveal significant violations affecting data integrity, conduct second round of inspections
• Conduct systems-based inspection of sponsor and/or CRO?– Were identified problems corrected in real time?
• Request 3rd Party audit of representative sample of sites• Compare 3rd Party audit with sponsor monitoring reports and FDA
inspection results• Assess whether violations were limited to sites inspected by FDA or
extend across other sites as determined by 3rd party audit• Hold sponsor and CRO accountable for inadequate monitoring,
failure of quality systems (Including rejection of data, WarningLetters, Follow up Inspections)
19
Limitations of Current Approach• Can’t “inspect in” quality after clinical trial
is completed• Labor and time intensive
– May lead to delays in product approval– Costly in terms of inspectional resources
• Lacks adequate incentives to prevent or correct non-compliance in real time– “Compliance is a reverse lottery that sponsors
choose not to play”
20
• How can we collectively ensure that clinical trials produce high quality data and protect the rights, safety, and welfare of research subjects?
21
What is Quality?• Definition of quality:
– In clinical trial context: The ability to effectively and efficiently answer the intended question about the benefits and risks of a medical product (therapeutic or diagnostic) or procedure while assuring protection of human subjects.*
*Clinical Trials Transformation Initiative at http: //www.trialstransformation.org/scope
22
Data Quality and Data Integrity• Data Quality: Basic Elements (ALCOA)
– Attributable– Legible– Contemporaneous– Original– Accurate
• Data integrity: A dimension of data contributing to trustworthiness and pertaining to the systems and processes for data capture, correction, maintenance, transmission, and retention*
• Focus on significant endpoint data supporting safety and efficacy determinations in FDA decision-making
*http://www.cdisc.org/glossary/CDISC2008GlossaryVersion7.0.pdf
23
High Quality Data• No perfect dataset• Alternate definition of high quality data: data that
sufficiently supports conclusions and interpretations equivalent to those derived from error-free data* (data fit for purpose)– Sufficiently accurate to support FDA regulatory
decisions, sponsor claims about a product, labeling• Data quality and integrity together determine
fitness for purpose*Assuring Data Quality and Validity in Clinical Trials for Regulatory DecisionMaking: Workshop Report, IOM 1999 at http://www.iom.edu/CMS/3740/5583.aspx
24
Implementation of Quality Systems*• Can’t “inspect in” quality • Requires coordination of all
stakeholders to build in quality • ISO Quality Management System
– Iterative process • Plan: Define objectives-Systematic
identification of errors throughout process
• Do: Implement changes to address errors
• Check: Measure results• Act: Correct and improve in continuous
feedback loop (in as close to real time as you can get)
*http://www.iso.org/iso/iso_catalogue/management_standards/understand_the_basics.htm
25
Clinical Trial Error Types with Potential to Influence Data Reliability and Subject Safety*
• Design error: Poor protocol design • Procedural error: Failure to follow investigational plan,
failure to conduct proper monitoring• Recording error: Data collection
– Random– Systematic
• Analytical error: Data analysis
A quality system must address each of these*Adapted from Baigent et al., Clinical Trials 2008;5:49-55
*Adapted from Baigent et al., Clinical Trials 2008;5:49-55
26
Effect of Errors • Fraud is rare and usually isolated in scope (i.e.
single clinical investigator)– But when it occurs, it can be dramatic and undermine
public confidence in clinical trials• Errors other than fraud such as poorly designed
and executed protocols, inadequate recordkeeping, or faulty statistical analysis may be more systematic and can render data unreliable– Non-inferiority study designs: “Sloppiness” in clinical
trials may obscure difference between study drug and comparator
27
What is the acceptable error rate?
• No “one size fits all” in clinical trials– Depends on risk; ie, What is the risk of getting
this wrong?• For example, in adverse event reporting, risk of
incorrectly recording a headache is different from missing a subject death
28
FDA Regulations Support a Risk-Based Approach
• Risk management aids in prioritizing our work
• Allows for better use of finite resources! Less time and resources spent on low risk and low
impact areas means more time and resources for meaningful high impact activities
• Within regulatory framework there is flexibilitypermitted in approach and implementation
29
Implementing a Quality Risk Management System
• Identity key areas of risk to data quality/integrity and human subject protections
• Develop an integrated framework– Protocol design, Site selection and training, Data and Safety
Monitoring Plan, Data Management Plan, Quality Assurance Plan, Data Analysis Plan
– Use of information technology in detecting data irregularities• Address human factors in systems
– Hire experienced, qualified staff– Avoid/manage financial conflicts of interest – Decrease number of times data are handled– Clinical trial monitoring: alternative approaches to traditional
industry model using a quality management system approach– Single points of accountability, including appropriate
management of outsourcing
30
Implementing a Quality Risk Management System (2)
• Create systems that limit opportunity for errors– Simplify protocol and outcomes assessed– Standardize systems and formats where possible– Use validated instruments/definitions– Keep amendments to a minimum and check the
CRFs and consent form against each change– Think very carefully about unblinding procedures– Insist on training and then test it– Have a disaster plan
– Do beta-testing/dry-runs
– Monitor and correct errors in real time
31
Quality and Compliance in Clinical Trials: Overview
• Where are we now?– FDA’s responsibility for ensuring data quality– Public expectations and role of enforcement
• Where are we going?– Quality risk management
• Sponsors and clinical investigators• FDA and other regulatory authorities
32
Data Validation Inspections for New Drug Applications: New Paradigm
• FDA and other regulatory authorities– Use Inspection Risk Model to predict how
inspectional findings from a few sites translates across entire application• Model contains data across all sites in
application, based on risk scores violations observed at a single or a few sites can be extrapolated
• Include inclusion of random sites • Develop a learning algorithm so risk attributes
and weights will evolve over time
33
Site Selection Tool incorporates factors in a multi-attribute decision analysis algorithm
Risk-based Site Selection Tool Algorithm Illustration: Note that Actual Model Differs in Details
Attributes Risk Functions Hierarchical Weighting Schema1 2 3
# Risk Attribute1 Enrollment2 Efficacy Outcome3 Protocol Violations4 Application Type
5 Non-Serious Adverse Events
6 Time Since Last Inspection
7 OAI History8 VAI History9 Protocol Violations
10 Site Type
1% 5% 25% 50% 75% 95% 99%Highest!Risk O O
Very!High!Risk
High!Risk O O
Medium!Risk
Low!Risk O O
Very!Low!Risk
Lowest!Risk O
Enrollment
Efficacy Outcome
PI Complaints
Financial Disclosures
75%
25%
50%
50%
! Column Weights = 100%
30%
20%
Efficacy Drivers
Site Quality
! Column Weights = 100%
Subset of entire risk tree
Site Percentile
Risk Score4
0 0.25 0.5 0.75 1
site a
site b
site c
site e
site f
site g
site h
site i
site j
site d
Risk Attribute Total ContributionPatient Population 30%
Historic Violations 20%
Safety Data Reporting 20%
Site Location 10%
Complexity of Study Design 5%
Complaints 5%
Sample Inspection
Target
34
Data Validation Inspections for New Drug Applications: New Paradigm (cont.)
• Conduct inspections of sponsor’s quality systems– Select products in early phase of development, mid phase, and
at NDA submission; link with clinical investigator inspections– Require submission of sponsor’s quality risk management
system for new drug applications, including monitoring plan• Conduct “real time” inspections of phase 3 clinical
studies– Feedback to site to correct problems in real time
• Involve DSI reviewers earlier in drug development process; i.e. End of Phase 2– Evaluate protocol for data quality and human subject protection
issues• Partner with other regulatory authorities
35
EMEA/FDA Collaboration
• EMEA/FDA(CDER)– Develop a joint GCP program to forecast, coordinate, schedule
and conduct joint and/or collaborative GCP inspections– Establish and implement procedures to share information with
EMEA related to applications for approval of new drugs• Challenges
– Consider legal limitations with information sharing• Potential benefits
– Optimize use of limited GCP inspection resources – Enhance consistency in regulatory approaches – May decrease burden on Sponsors/New Drug Applicants– Extension of partnership to other regulatory authorities
35
36
EMEA Inspections by Region (1997-2008)
21 21
8 7 9 10
21
1 3
10 9
3 2
2
4
1
13
3
19
8
41
5
1
3
0
10
20
30
40
50
60
2008 2007 2006 2005 2004 2003 2002 2001 2000 1999 1997
Rest of the world
North America
EU/EEA/AFTA
Courtesy Dr. Fergus Sweeney and Ana Rodriguez, EMEA
37
“…Economics has been too isolated and too stuck on the view that markets are efficient and self-regulating. It has derailed our thinking…”
“... Rules that are imposed from the outside or unilaterally dictated by powerful insiders have less legitimacy and are more likely to be violated. Likewise, monitoring and enforcement work better when conducted by insiders than by outsiders. These principles are in stark contrast to the common view that monitoring and sanctions are the responsibility of the state andshould be conducted by public employees.”
38
Take Home Messages1. FDA has an important role in ensuring high
quality data • Enforcement efforts should be transparent, timely
and appropriate to the observed violations2. Inspections alone cannot ensure the quality
of clinical research and high quality data 3. Quality of clinical trial enterprise must be built
into the process all at phases of clinical development and involving all stakeholders • Prioritize and manage risks for maximum impact
4. At stake is public confidence and participation in the drug development and ultimately the availability of safe and effective drug products