Embed Size (px)
DESCRIPTION
Learning Objectives Identify a framework for analyzing RCT’s in the learning environment Discuss major sources of bias in RCT’s Define validity and generalizability and be able to begin to assess these in real-world article. Use these skills in a small group environment
Citation preview
Making Randomized Clinical Trials Seem LessRandom
Andrew P.J. Olson, MDAssistant Professor
Departments of Medicine and PediatricsUniversity of Minnesota Medical School
Disclosures
• I have no financial interests to disclose.
• I will not discuss off label or investigational product use.
Learning Objectives
• Identify a framework for analyzing RCT’s in the learning environment
• Discuss major sources of bias in RCT’s• Define validity and generalizability and be able
to begin to assess these in real-world article.
• Use these skills in a small group environment
$
Agenda
• Overview of RCT’s• Randomization• Blinding• Outcome measurement• Analysis – Intention to treat?
Let’s start with a roadmap.
RCT Roadmap
PopulationPeople at risk for
heart attacks
Sample
1000 Statins
1000 Placebo
50 heart attacks
25 heart attacks
Randomization Treatment Outcome
Follow-up
Randomization is Key
• By randomizing subjects to different groups, both known (measured) and unknown (unmeasured) variables should be randomly distributed.
• This controls for known and unknown confounding variables
What is Confounding?
• A confounding variable is associated with the receipt of treatment and the outcome.
• Statin trial:– Smoking, exercise, hypertension medications
Validity and Generalizability
• Validity: In the studied population, was the study performed in a way that the results are valid?
• Generalizability: Are these results applicable to my patients?
Elements of a Randomized Controlled Trial
Element Best Case Scenario(Described in paper)
Validity or Generalizability
Subject Selection Recruitment Procedures and Entrance Criteria
specified
Generalizability
Randomization Random SequenceAllocation Concealment
Validity
Treatment Feasible, safe, delineated Generalizability
Element Best Case Scenario(Described in paper)
Validity or Generalizability
Follow up Complete (all accounted for) and similar between groups
Validity
Co-intervention Same between groups and relevant co-interventions described
Validity
Blinding Subjects, Providers, and Outcome assessors
Validity
Element Best Case Scenario(Described in paper)
Validity or Generalizability
Outcomes Measurable? Validity
Meaningful? Generalizability
Analysis and Power Intention to treat? Validity
Adequately powered? Validity
Statistical Methods described and appropriate?
Validity
Randomized Controlled Trials
• Overview of RCT’s• Randomization• Blinding• Outcome measurement• Analysis – Intention to treat?
Randomization
• Is the randomization of a subject to a group really random?– If allocation is truly random, it cannot be predicted– Random number table or generator
– Examples of non-random allocation:• Even or odd MRN• Days of the week• Morning or afternoon patients• First patient the day
RandomizationAllocation Concealment
• The sequence of allocation to different groups cannot be seen by subjects or providers
• Examples:– Sealed, opaque envelopes– Central voice-response system– Online systems
Randomized Controlled Trials
• Overview of RCT’s• Randomization• Blinding• Outcome measurement• Analysis – Intention to treat?
Blinding• Ideally, the only difference between groups is the
treatment (which no one knows about!)
• Triple Blinding is Ideal– No one knows the treatment group allocation
• Provider• Subject• Outcomes assessor
• Blinding protects against bias from:– Different receipt of co-interventions between groups– Differential outcome ascertainment
Co-interventions
PopulationPeople at risk for
heart attacks
Sample
1000 Statins
1000 Placebo
50 heart attacks
25 heart attacks
60% take aspirin
30% take aspirin
Co-interventions
• By not knowing which group a subject is assigned to, subjects in different groups should be treated the same
• Neither those giving or receiving treatment know the assignment
Blinding in Treatment Studies
Blinding
PopulationPeople at risk for
heart attacks
Sample
1000 Statins
1000 Placebo
50 heart attacks
25 heart attacks
Randomization Treatment Outcome
Follow-up
In treatment studies, it is usually necessary to have a placebo or sham procedure
Blinding
• If subjects and providers are unaware of which group the patient is allocated to, co-interventions should be the same on average
• Differences in co-interventions, if there is proper randomization and blinding, will be due to chance.
Don’t forget about the third blind team member!
• Subjects and Providers can be difficult to blind, especially with certain treatments
• However, those who are analyzing the outcomes can almost always be blinded– Analysis of medical records– If they know the group assignment, their view of
an outcome can be biased
Randomization and Blinding
Population
Sample
TreatmentA
Treatment B # Events
# Events
Randomization Treatment Outcome
Follow-up
Similar at baseline? Similar during followup?
Randomized Controlled Trials
• Overview of RCT’s• Randomization• Blinding• Outcome measurement• Analysis – Intention to treat?
Outcomes• Outcomes are prespecified
– Measurable - Validity– Meaningful – Generalizability
• Easily measurable:– Mortality, MI, cancer recurrence, blood pressure, lipids
• Less easily measured:– Quality of life, pain, disability– Validated tool?
• Meaningful: – Do they matter to a patient?
Surrogate Outcomes
• Sometimes a meaningful outcome is difficult to measure:– Time for followup, hard to quantify
• So a surrogate outcome is used:– FDA Definition:• A laboratory measurement or physical sign that is used
as a substitute for a clinically meaningful outcome because it is expected to predict the effect of therapy on a clinically meaningful outcome.
Surrogate Outcome
Population
Sample
TreatmentA
Treatment B # Events
# EventsSurrogate
Surrogate
Surrogate Outcome
Treatment
Treatment No Change Events
↑Surrogate
↓Surrogate
↑Harm
An Example of Surrogate Outcomes
• There is significant mortality from arrhythmias after myocardial infarctions
• PVC’s can be a marker of arrhythmias
• Antiarrhythmic medications decrease PVC’s
• Thus, it makes sense that using antiarrhythmic medications after myocardial infarctions might decrease mortality
A Classic Example ofSurrogate Outcomes
CAST Trial
Cardiac Mortality
All Cause Mortality
Randomized Controlled Trials
• Overview of RCT’s• Randomization• Blinding• Outcome measurement• Analysis – Intention to treat?
Intention to Treat Analysis5000 Patients Screened
1000 Randomized
500 Placebo500 Metoprolol
OutcomeOutcome
23 withdraw consent
14 lost to followup
22 stop taking medicine
10withdraw consent
4 lost to followup13 stop taking
medicine
?
Intention to Treat
• All randomized subjects are included in the analysis, regardless of actual receipt of treatment
• This means some subjects who didn’t get the intervention are still included in the analysis
• Preserves the randomization
Intention to Treat• All subjects should be able to be accounted for while
you read the paper– High rate of participation– Few are “lost to followup”
• If a subject is lost to followup:– Search for vital statistics– Perform advanced analyses to determine what probably
happened to these subjects
• Most importantly, patients must NOT be removed from the study in a non-random way!
Small Group Activity
Small Group Activity• Was the assignment of patients to treatments randomized?• Were the groups similar at the start of the trial?• Except for the allocated treatment, were the groups treated equally?• Were all patients who entered the trial accounted for and were they
analyzed in the groups to which they were randomized?• Were the measures objective?• Were the patients and clinicians kept blind to which treatment was
being received?• How large was the treatment effect?• How precise was the estimate of the treatment effect?• Will the results help me in caring for my patients?
