What Is the Role for Analyses of What Is the Role for Analyses of Administrative Data in AssessingAdministrative Data in Assessing

Drug Safety in Post-MarketDrug Safety in Post-MarketCommitment (PMC) Studies?Commitment (PMC) Studies?

Cathy W. Critchlow, PhDCathy W. Critchlow, PhDGlobal Epidemiology, Amgen, Inc.Global Epidemiology, Amgen, Inc.September 29, 2006September 29, 2006

OutlineOutline

Why should we consider additional Why should we consider additional approaches (e.g., analyses of administrative approaches (e.g., analyses of administrative data) to post-market commitment studies?data) to post-market commitment studies?What are situations where analyses of What are situations where analyses of administrative data can be used to administrative data can be used to supplement, or even replace, clinical post-supplement, or even replace, clinical post-market commitment studies?market commitment studies? Study design issuesStudy design issues Strengths and limitations of administrative data Strengths and limitations of administrative data

analyses as a component of post-market analyses as a component of post-market surveillancesurveillance

Post-Market Surveillance –Post-Market Surveillance –Continuing Assessments of Safety or EfficacyContinuing Assessments of Safety or Efficacy

Post-market surveillance

Post-market commitment studies Routine surveillance

Spontaneous reports• Unexpected or rare AEs

Observational studies

Registries or studies conducted to ‘complete’ pre-market assessments

• Rate of expected/unexpected AEs• Relative rate of ‘hard’ endpoints

• Dynamic response to emerging issues, hypotheses

Can analyses using automated databases help meet these needs?

Many examples???

Need for Reevaluation of Post-MarketNeed for Reevaluation of Post-MarketAssessment StrategiesAssessment Strategies

Withdrawal of Cox-2 inhibitors after several years Withdrawal of Cox-2 inhibitors after several years and several million patient exposures contributes to and several million patient exposures contributes to “perception of crisis that has compromised the “perception of crisis that has compromised the credibility of FDA and the pharmaceutical industry”*credibility of FDA and the pharmaceutical industry”*

Public’s loss of faith in the ability of industry to Public’s loss of faith in the ability of industry to deliver safe and effective drugsdeliver safe and effective drugs††

13% think pharmaceutical companies are “generally 13% think pharmaceutical companies are “generally honest and trustworthy”honest and trustworthy”

60% not confident that drug companies will publicly 60% not confident that drug companies will publicly disclose safety data in a timely mannerdisclose safety data in a timely manner

*IOM Report: The Future of Drug Safety: Promoting and Protecting the Health of the Public

†Harris Poll, 2004

Need for Reevaluation of Post-MarketNeed for Reevaluation of Post-MarketAssessments Strategies (2)Assessments Strategies (2)

Reliance on regulation alone to demonstrate long-Reliance on regulation alone to demonstrate long-term safety has not workedterm safety has not worked Unmet phase 4 commitmentsUnmet phase 4 commitments

114 (9.6%) of 1,191 open PMCs met*114 (9.6%) of 1,191 open PMCs met* Confirmation of efficacy using hard endpoints in phase 4 Confirmation of efficacy using hard endpoints in phase 4

commitment studies for drugs receiving “fast-track” commitment studies for drugs receiving “fast-track” approval based on surrogate measuresapproval based on surrogate measures

FDA Critical Path InitiativeFDA Critical Path Initiative Use of database registries and electronic medical record Use of database registries and electronic medical record

systems to compare outcomes among relevant patient systems to compare outcomes among relevant patient groups in post-market drug evaluationsgroups in post-market drug evaluations

* Federal Register 2005;70:8379-81.

Crux of the Issue…..Crux of the Issue…..Post-Approval Drug SafetyPost-Approval Drug Safety

Typically, 500 - 3000 patients exposed to drug in Typically, 500 - 3000 patients exposed to drug in phase 3 testingphase 3 testing Drug effects detectable with an incidence ~1–6 per 1000Drug effects detectable with an incidence ~1–6 per 1000

To quantify the risk of an event with incidence of 2 To quantify the risk of an event with incidence of 2 per 10,000/year (precision per 10,000/year (precision ±1 per 10,000) with 95% ±1 per 10,000) with 95% probability, need ~80,000 subjects followed for 1 yearprobability, need ~80,000 subjects followed for 1 year

Difficult to conduct studies this large in a timely Difficult to conduct studies this large in a timely fashionfashion

OpportunitiesOpportunities

Consider additional or alternative strategies to Consider additional or alternative strategies to demonstrate long-term drug safety or efficacydemonstrate long-term drug safety or efficacy Analyses of administrative data* in post-market Analyses of administrative data* in post-market

surveillancesurveillance

Collaboratively establish high standards for the Collaboratively establish high standards for the conduct of observational data analyses conduct of observational data analyses conducted as part of post-market commitmentsconducted as part of post-market commitments

Demonstrate safety and effectiveness of drugs Demonstrate safety and effectiveness of drugs in ‘real-world’ settingsin ‘real-world’ settings

*Claims data (commercial insurance, Medicare, Medicaid), medical record data, national databases

Qualities of the Ideal DatabaseQualities of the Ideal Database

ComprehensiveComprehensive Inpatient & outpatient care; ER visits; lab & Inpatient & outpatient care; ER visits; lab &

radiological tests; prescribed & OTC drugs; mental radiological tests; prescribed & OTC drugs; mental health care; alternative therapyhealth care; alternative therapy

Large, stable populationLarge, stable populationUnique identifiers for linkageUnique identifiers for linkageRegular, frequent updatesRegular, frequent updatesVerifiable, reliableVerifiable, reliableCapacity for chart review or patient interviewsCapacity for chart review or patient interviews Confounder dataConfounder data ComplianceCompliance

But, few databases are ideal….But, few databases are ideal….(some are better than others)(some are better than others)

Problematic situationsProblematic situations Illnesses that do not reliably come to medical attentionIllnesses that do not reliably come to medical attention Inpatient drug exposuresInpatient drug exposures Outcomes poorly defined by ICD-9 codingOutcomes poorly defined by ICD-9 coding When necessary confounder data cannot be obtainedWhen necessary confounder data cannot be obtained Very long latency eventsVery long latency events

Need to understand the limitations of any databaseNeed to understand the limitations of any database Purpose for which database was createdPurpose for which database was created Data quality, validity, completenessData quality, validity, completeness Availability of confounder dataAvailability of confounder data Patient follow-upPatient follow-up Access to source informationAccess to source information

Issues to Consider in the Design of PMCsIssues to Consider in the Design of PMCs

What is the objective?What is the objective? Hypothesis generation (descriptive or exploratory) Hypothesis generation (descriptive or exploratory)

vs.vs. confirmation? confirmation?

Evaluating expected vs. unexpected eventsEvaluating expected vs. unexpected events Apriori Apriori specification of events of interestspecification of events of interest

Timing of events of interestTiming of events of interest Short-term Short-term vs.vs. long latency outcomes long latency outcomes

Most PMCs are Observational…..Most PMCs are Observational…..Issues of Study ValidityIssues of Study Validity

BiasBias Selection biasSelection bias Information biasInformation bias

Misclassification of covariates, exposure, outcomeMisclassification of covariates, exposure, outcome Data validityData validity

ConfoundingConfounding By disease severity, treatment indication, comorbid conditionsBy disease severity, treatment indication, comorbid conditions Unmeasured covariatesUnmeasured covariates Time-dependent confoundingTime-dependent confounding

Physician prescription patterns (‘channeling’)Physician prescription patterns (‘channeling’)

Dosing variability according to patient responsivenessDosing variability according to patient responsiveness

What are appropriate comparator groups?What are appropriate comparator groups?

Other Considerations……Other Considerations……

Urgency of need for dataUrgency of need for data Drug first in class or are other relevant data Drug first in class or are other relevant data

available?available?

Risk Risk vs.vs. benefit profile benefit profile

Numbers of persons to be exposedNumbers of persons to be exposed

Expected AE incidence rateExpected AE incidence rate

Signal detection – what constitutes a safety Signal detection – what constitutes a safety signal?signal?

Implications for study design, sample size, comparators, interim analyses, scientific rigor required

What are situations where analyses What are situations where analyses of administrative data can be used of administrative data can be used

to supplement, or even replace, to supplement, or even replace, clinical PMC studies?clinical PMC studies?

PMC Scenario 1: Single-Arm Prospective, PMC Scenario 1: Single-Arm Prospective, Observational Clinical RegistryObservational Clinical Registry

Characterize long-term safety profile of approved Characterize long-term safety profile of approved drugdrug Incidence of various adverse eventsIncidence of various adverse events

Drug utilization in the ‘real-world’Drug utilization in the ‘real-world’ Special populations, e.g., childrenSpecial populations, e.g., children Effect of comorbid conditionsEffect of comorbid conditions Drug-drug interactionsDrug-drug interactions

PMC Scenario 1: Single-Arm Prospective, PMC Scenario 1: Single-Arm Prospective, Observational RegistryObservational Registry

Clinical RegistryClinical Registry

Hypothesis generatingHypothesis generating

Pre-specified outcomes; Pre-specified outcomes; unexpected eventsunexpected events

Modest sample sizeModest sample size Will not observe rare eventsWill not observe rare events

Effect measureEffect measure Absolute incidence rateAbsolute incidence rate SIR (external comparator)SIR (external comparator)

Prospective data collectionProspective data collection Difficult to assess long latency Difficult to assess long latency

eventsevents

Virtual (Database) RegistryVirtual (Database) Registry

Hypothesis confirmationHypothesis confirmation

Post-hoc analyses of Post-hoc analyses of unexpected eventsunexpected events

Large sample sizeLarge sample size Study rare eventsStudy rare events

Effect measureEffect measure Absolute risk in select populationAbsolute risk in select population Relative risk (internal Relative risk (internal

comparator)comparator)

Retrospective study designRetrospective study design Potential for answers soonerPotential for answers sooner

Objective: Characterize drug safety post-approval

PMC Scenario 1: Single-Arm Prospective, PMC Scenario 1: Single-Arm Prospective, Observational Registry (2)Observational Registry (2)

Clinical RegistryClinical Registry

Data qualityData quality Outcome adjudicationOutcome adjudication Covariate data can be obtainedCovariate data can be obtained Regulatory definitions of AEsRegulatory definitions of AEs

Sources of bias, confoundingSources of bias, confounding Potential selection, recall or Potential selection, recall or

information biasinformation bias Differential loss-to-followup with Differential loss-to-followup with

respect to risk of outcomes?respect to risk of outcomes?

Virtual (Database) RegistryVirtual (Database) Registry

Data qualityData quality Validity of algorithms assessing Validity of algorithms assessing

drug exposure, disease severity, drug exposure, disease severity, outcomesoutcomes

Comparable ascertainment of Comparable ascertainment of data from exposed and data from exposed and comparator groupscomparator groups

Data from all medical care Data from all medical care providersproviders

Sources of bias, confoundingSources of bias, confounding Relevant covariate data Relevant covariate data

available?available? Confounding by indication for Confounding by indication for

treatment, comorbiditiestreatment, comorbidities Stability of populationStability of population

Objective: Characterize drug safety post-drug approvalObjective: Characterize drug safety post-drug approval

When/What Could Analyses of Administrative Data Contribute to this PMC Scenario?

When…When… Large sample size needed to assess rare eventsLarge sample size needed to assess rare events Events specifiedEvents specified apriori apriori Objective lab-driven diagnosesObjective lab-driven diagnoses Confounder data availableConfounder data available Denominator needed to calculate population incidence Denominator needed to calculate population incidence

rates rates

What…..What….. Background incidence ratesBackground incidence rates Attributable risk of eventsAttributable risk of events

Objective: Characterize drug safety post-drug approval

PMC Scenario 2: Controlled PMC Scenario 2: Controlled Studies Further Assessing EfficacyStudies Further Assessing Efficacy

Obtain additional data regarding Obtain additional data regarding meaningful clinical endpointsmeaningful clinical endpoints Confirm estimates of efficacy of drugs Confirm estimates of efficacy of drugs

receiving “fast-track” approval based on receiving “fast-track” approval based on surrogate measuressurrogate measures

Head-to-head comparisonsHead-to-head comparisons New vs. existing drugNew vs. existing drug

PMC Scenario 2: Controlled StudiesPMC Scenario 2: Controlled StudiesFurther Assessing EfficacyFurther Assessing Efficacy

Clinical Study Clinical Study

Modest sample sizeModest sample size Expense of large study of Expense of large study of

infrequent outcomesinfrequent outcomes

Effect measureEffect measure Relative risk compared to Relative risk compared to

placebo or standard of careplacebo or standard of care

Prospective data collectionProspective data collection Potential ethical issues in Potential ethical issues in

randomized trial with placebo randomized trial with placebo control and/or long follow-upcontrol and/or long follow-up

Database StudyDatabase Study

Large sample sizeLarge sample size Study rare eventsStudy rare events

Effect measureEffect measure Relative riskRelative risk More easily do ‘head-to-head’ More easily do ‘head-to-head’

comparisons comparisons

Retrospective cohort studyRetrospective cohort study Potential for answers soonerPotential for answers sooner

Objective: Compare incidence of clinical (efficacy) endpoints among exposed and unexposed groups

PMC Scenario 2: Controlled StudiesPMC Scenario 2: Controlled StudiesFurther Assessing Efficacy (2)Further Assessing Efficacy (2)

Clinical Study Clinical Study

Data qualityData quality Outcome adjudicationOutcome adjudication Covariate data can be Covariate data can be

obtainedobtained

Sources of bias, Sources of bias, confoundingconfounding Potential selection, recall Potential selection, recall

or information biasor information bias Differential loss-to-Differential loss-to-

followup with respect to followup with respect to risk of outcome?risk of outcome?

Database StudyDatabase Study

Data qualityData quality Validity of algorithms Validity of algorithms

assessing drug exposure, assessing drug exposure, disease severity, outcomesdisease severity, outcomes

Sources of bias, Sources of bias, confoundingconfounding Unmeasured confounders?Unmeasured confounders? Confounding by indication Confounding by indication

for treatment, comorbiditiesfor treatment, comorbidities Stability of populationStability of population

Objective: Compare incidence of clinical (efficacy) endpoints among exposed and unexposed groups

When/What Could Analyses of Administrative Data Contribute to this PMC Scenario?

When…When… Large sample size needed to assess rare outcomes, long-Large sample size needed to assess rare outcomes, long-

latency outcomeslatency outcomes Valid ascertainment of outcomeValid ascertainment of outcome Short-term effectsShort-term effects Recall or interviewer bias could effect associationRecall or interviewer bias could effect association

What…..What….. Timely validation of surrogate markersTimely validation of surrogate markers Head-to-head comparison of outcome incidenceHead-to-head comparison of outcome incidence

Objective: Compare incidence of clinical (efficacy) endpoints among exposed and unexposed groups

Opportunities - RevisitedOpportunities - Revisited

Potential role for analyses of administrative Potential role for analyses of administrative data in post-market surveillancedata in post-market surveillance

Collaborative establishment of regulatory Collaborative establishment of regulatory thresholds for the conduct, analysis and thresholds for the conduct, analysis and interpretation of observational data analyses interpretation of observational data analyses conducted as part of post-market commitmentsconducted as part of post-market commitments

Demonstrate safety and effectiveness of drugs Demonstrate safety and effectiveness of drugs in timely fashionin timely fashion

Thank you!