HARMHARM
Indonesia Clinical epidemiology and Indonesia Clinical epidemiology and Evidence Based Medicine (ICEEvidence Based Medicine (ICE--EBM)EBM)
HARMHARM
Jarir At Thobari (FK UGM)Jarir At Thobari (FK UGM)
EfficaciousEfficacious
Medical interventionMedical interventionHarmHarm
HARM
“Unintended physical injury resulting from
or contributed to by medical care, that
requires additional monitoring, treatment
or hospitalization, or that results in death
which is hospital acquired.”which is hospital acquired.”
Definition modified from the IHI definition of Harm by the QUEST
Harm Workgroup
“Do oral contraceptives cause breast “Do oral contraceptives cause breast cancer?”cancer?”
“Do calcium“Do calcium antagonists increase the antagonists increase the risk of death or cancer?”risk of death or cancer?”
HarmHarm
risk of death or cancer?”risk of death or cancer?”
“Do vasectomies increase the risk of “Do vasectomies increase the risk of prostate cancer?”prostate cancer?”
“Do“Does Rosiglitazone cause myocardial es Rosiglitazone cause myocardial infactioninfaction?”?”
Summary estimates of risk of breast cancer in premenopausal women and women younger than 50 years associated with ever use of oral contraceptives
Kahlenborn C et al. Mayo Clin Proc. 2006;81:1290-1302
♦ Meta analysis OR=1.19; 95% CI, 1.09-1.29
– Parous OR=1.29; 95% CI: 1.20-1.40
– Nulliparous OR=1.24; 95% CI: 0.92-1.67
♦ Parous women, OCs were used
Risk of breast cancer in premenopausal women and women younger than 50 years associated with ever use
of oral contraceptives
♦ Parous women, OCs were used
– Before FFTP* OR=1.44 (1.28-1.62)
– after FFTP OR=1.15 (1.06-1.26)
♦ used OCs 4 or more years before FFTP (OR=1.52 (1.26-1.82).
*first full-term pregnancy
Mayo Clin Proc. 2006;81(10):1290-1302
Primary and Other Outcomes: DREAM
Rosiglitazone
group
(n=2635)
Placebo
group
(n=2634)
HR (95% CI) p
Cardiovascular events composite* 75 (2.9%) 55 (2.1%) 1.37 (0.97-1.94) 0.08
Myocardial infarction 15 (0.6%) 9 (0.3%) 1.66 (0.73-3.80) 0.2
Stroke 7 (0.3%) 5 (0.2%) 1.39 (0.44-4.40) 0.6
Cardiovascular death 12 (0.5%) 10 (0.4%) 1.20 (0.52-2.77) 0.7
Confirmed heart failure‡ 14 (0.5%) 2 (0.1%) 7.03 (1.60-30.9) 0.01
New angina 24 (0.9%) 20 (0.8%) 1.20 (0.66-2.17) 0.5
Revascularization 35 (1.3%) 27 (1.0%) 1.29 (0.78-2.14) 0.3
Myocardial infarction, stroke, or
cardiovascular death
32 (1.2%) 23 (0.9%) 1.39 (0.81-2.37) 0.2
DREAM investigators. Lancet 2006;368:1096-1105.
Data are number (%). *Rows are not mutually exclusive for components of the
composite—if a participant had more than one component of the composite then they are
counted in the relevant row. †Regression implies achieving a normal fasting glucose
concentration (as defined in both rows) and 2-h plasma glucose level. ‡Defined as acute
treatment with at least two of the following criteria: typical signs and symptoms, typical
radiological evidence, use of diuretics, vasodilators, or inotropes.
Adverse Events, Hospitalization, and Death: ADOPT
Variable
Rosiglitazone (N=1456) Metformin (N=1454) Glyburide (N=1441)
Serious Events
Total Events
Serious Events
Total Events
Serious Events
Total Events
Adverse events — number of patients (%)
Total events 346 (23.8) 1338 (91.9) 331 (22.8) 1341 (92.2) 308 (21.4) 1321 (91.7)
Cardiovascular disease 49 (3.4) 62 (4.3) 46 (3.2) 58 (4.0) 26 (1.8)† 41 (2.8)
Myocardial Infarction
Fatal 2 (0.1) 2 (0.1) 2 (0.1) 2 (0.1) 3 (0.2) 3 (0.2)
Nonfatal 22 (1.5) 25 (1.7) 18 (1.2) 21 (1.4) 11 (0.8) 15 (1.0)Nonfatal 22 (1.5) 25 (1.7) 18 (1.2) 21 (1.4) 11 (0.8) 15 (1.0)
Congestive heart failure(investigator-reported) 12 (0.8) 22 (1.5) 12 (0.8) 19 (1.3) 3 (0.2)† 9 (0.6)†
Stroke 13 (0.9) 16 (1.1) 17 (1.2) 19 (1.3) 12 (0.8) 17 (1.2)
Peripheral vascular disease 7 (0.5) 36 (2.5) 6 (0.4) 27 (1.9) 4 (0.3) 31 (2.2)
Gastrointestinal events 8 (0.5) 335 (23.0) 7 (0.5) 557 (38.3)‡ 3 (0.2) 316 (21.9)
Death and HospitalizationHospitalization for any reason
Patients — no. (%) 169 (11.6) 172 (11.8) 150 (10.4)
Events — no. 251 267 203
Deaths from any cause—no. 34 31 31
ADOPT study group. N Engl J Med 2006;355:2427-2443.
† P<0.05 for the comparison between this treatment group and the rosiglitazone group; ‡ P<0.01 for the
comparison between this treatment group and the rosiglitazone group.
HOME | SUBSCRIBE | CURRENT ISSUE | PAST ISSUES | COLLECTIONS | HELP | Search Term Advanced Search
Institution: Syrian Arab Republic | Sign In as Individual | Contact Subscription Administrator at Your
Institution | FAQ
.100):1(357;2007A correction has been published: N Engl J Med
Institution | FAQ
Effect of Rosiglitazone on the Risk of Myocardial
Infarction and Death from Cardiovascular Causes
Steven E. Nissen, M.D., and Kathy Wolski, M.P.H
Number 24June 14, 2007Volume 356:2457-2471
Search NEJMSEA RC H
Meta Analysis: Rates of Myocardial Infarction and Death from Cardiovascular Causes
Rosiglitazone
Group
Control
Group
Odds Ratio
(95% CI) P Value
no. of events/total no. (%)
Myocardial Infarction
Small trials combined 44/10,285 (0.43) 22/6106 (0.36) 1.45 (0.88–2.39) 0.15
DREAM 15/2,635 (0.57) 9/2634 (0.34) 1.65 (0.74–3.68) 0.22
ADOPT 27/1,456 (1.85) 41/2895 (1.42) 1.33 (0.80–2.21) 0.27
Overall 1.43 (1.03–1.98) 0.03
Death from
Cardiovascular Causes
Small trials combined 25/6,845 (0.36) 7/3980 (0.18) 2.40 (1.17–4.91) 0.02
DREAM 12/2,635 (0.46) 10/2634 (0.38) 1.20 (0.52–2.78) 0.67
ADOPT 2/1,456 (0.14) 5/2895 (0.17) 0.80 (0.17–3.86) 0.78
Overall 1.64 (0.98–2.74) 0.06
Nissen and Wolski. N Engl J Med 2007;356:2457-2471.
Hierarchy of EvidenceHierarchy of Evidence
Systematic
Reviews of
Randomized
Controlled Trials
(Meta-analysis)
Single Randomized Single Randomized
Controlled Trial (RCT)
Systematic Review of
Observational Studies Addressing
Patient-Important Outcomes
Single Observational Study
Addressing Patient-Important Outcomes
Physiologic Studies
Unsystematic Clinical Observations
Level Therapy/Prevention, Aetiology/Harm
Prognosis Diagnosis Differential diagnosis/symptom
prevalence study
Economic and decision analyses
1a SR (with homogeneity*) of RCTs
SR (with homogeneity*) of inception cohort studies; CDR† validated in different populations
SR (with homogeneity*) of Level 1 diagnostic studies; CDR† with 1b studies from different clinical centres
SR (with homogeneity*) of prospective cohort studies
SR (with homogeneity*) of Level 1 economic studies
1b Individual RCT (with narrow Confidence Interval‡)
Individual inception cohort study with > 80% follow-up; CDR† validated in a single population
Validating** cohort study with good††† reference standards; or CDR† tested within one clinical centre
Prospective cohort study with good follow-up****
Analysis based on clinically sensible costs or alternatives; systematic review(s) of the evidence; and including multi-way sensitivity analyses
1c All or none§ All or none case-series Absolute SpPins and SnNouts†† All or none case-series Absolute better-value or worse-value analyses
2a SR (with homogeneity*) of cohort studies
SR (with homogeneity*) of either retrospective cohort studies or untreated control groups in RCTs
SR (with homogeneity*) of Level >2 diagnostic studies
SR (with homogeneity*) of 2b and better studies
SR (with homogeneity*) of Level >2 economic studies
2b Individual cohort study (including low quality RCT;
Retrospective cohort study or follow-up of untreated control
Exploratory** cohort study with good†††reference standards;
Retrospective cohort study, or poor follow-up
Analysis based on clinically sensible costs or alternatives; limited review(s) (including low quality RCT;
e.g., <80% follow-up)follow-up of untreated control patients in an RCT; Derivation of CDR† or validated on split-sample§§§ only
good†††reference standards; CDR† after derivation, or validated only on split-sample§§§or databases
poor follow-up costs or alternatives; limited review(s) of the evidence, or single studies; and including multi-way sensitivity analyses
2c "Outcomes" Research; Ecological studies
"Outcomes" Research Ecological studies Audit or outcomes research
3a SR (with homogeneity*) of case-control studies
SR (with homogeneity*) of 3b and better studies
SR (with homogeneity*) of 3b and better studies
SR (with homogeneity*) of 3b and better studies
3b Individual Case-Control Study
Non-consecutive study; or without consistently applied reference standards
Non-consecutive cohort study, or very limited population
Analysis based on limited alternatives or costs, poor quality estimates of data, but including sensitivity analyses incorporating clinically sensible variations.
4 Case-series (and poor quality cohort and case-control studies§§)
Case-series (and poor quality prognostic cohort studies***)
Case-control study, poor or non-independent reference standard
Case-series or superseded reference standards
Analysis with no sensitivity analysis
5 Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles"
Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles"
Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles"
Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles"
Expert opinion without explicit critical appraisal, or based on economic theory or "first principles"
Clinical Scenario
♦Osteoarthritis
– Osteoarthritis is the most common type of
arthritis, especially among older people
– People with osteoarthritis usually have joint – People with osteoarthritis usually have joint
pain and limited movement
– Unlike some other forms of arthritis,
osteoarthritis only affects joints, and not
internal organs
– Usually need long-term analgesics
– Is there any side effects of using long-term
analgesics?
Clinical QuestionClinical Question
PP•Patient and the problem: osteoarthritis
II•Intervention: selective Cox-2 inhibitor (rofecoxib)II (rofecoxib)
CC•Comparison: other analgesic and anti-inflammatory agents
OO•Outcome of interest: congestive heart failure
BMJ 2005; 330:1370
How to assessHow to assess HARM?HARM?
Is there evidence on Is there evidence on cause & effect cause & effect relationship?relationship?relationship?relationship?
ValidValid ImportanceImportanceDirect Direct
relevancerelevance
Establishing ValidityEstablishing Validity
1. Were there clearly defined group of 1. Were there clearly defined group of
patients, similar in all important ways other patients, similar in all important ways other
than exposure to the treatment or other than exposure to the treatment or other
cause?cause?
♦Randomization??
♦ The choice of comparison groups has ♦ The choice of comparison groups has an enormous influence on the credibility of the results
♦ The design of the study determines the comparison groups
Randomized Controlled Trials (RCT)
♦Not suitable for harm studies:
– Unethical
– Expensive (Rare events, long period of – Expensive (Rare events, long period of time)
– Need large samples
♦ Some RCTs for therapy might have shown harm unintentionally (stopped)
Cohort Studies
♦ Prospective, Non-randomized exposure
♦ Useful when subjects cannot be assigned to an
exposure group
♦ Exposed patients might differ from non-exposed ♦ Exposed patients might differ from non-exposed
for important determinants of outcome
(CONFOUNDERS)
♦ Investigator must document the characteristics of
each group and demonstrate comparability, and
use statistical techniques to adjust for known
confounders
Confounders Confounders ((Atthobari et al, Nephrol. Dial. Transp 2006; 21:3106-14))
Case-Control
♦Useful for RARE outcomes that take LONG time to develop
– Ex: DES and vaginal adenocarcinoma in femalefemale
♦ Susceptible to unmeasured confounders
♦Retrospective
♦Recall Bias
♦Can be used to study potentially causal relationships between risk factors & outcomes
♦Measure prevalence (not incidence)
Cross Sectional Studies
♦Measure prevalence (not incidence)
♦ Exposures (risk factors) and outcomes (disease) are measured simultaneously
– Temporal relationship is lost
Case Series and Case Reports
♦No comparison group!
♦Unusual/dramatic outcome (Phocomelia in offsprings of mothers receiving Thalidomide)(Phocomelia in offsprings of mothers receiving Thalidomide)
♦ Sufficient for hypothesis generation (Need more studies)
DesignDesign Starting Starting PointPoint
AssessmentAssessment StrengthsStrengths WeaknessesWeaknesses
CohortCohort Exposure Exposure statusstatus
Outcome event Outcome event statusstatus
Feasible when Feasible when impossible to impossible to randomizerandomize
--Susceptible to Susceptible to biasbias
--Limited validityLimited validity
CaseCase-- Outcome Outcome Exposure statusExposure status --Overcomes Overcomes --Susceptible to Susceptible to CaseCase--
ControlControl
Outcome Outcome event event statusstatus
Exposure statusExposure status --Overcomes Overcomes temporal temporal delaysdelays
--May only May only requires small requires small sample sizesample size
--Susceptible to Susceptible to biasbias
--Limited validityLimited validity
RCTRCT Exposure Exposure statusstatus
Adverse event Adverse event statusstatus
Low Low susceptibility susceptibility to biasto bias
--FeasibilityFeasibility
--GeneralizabilityGeneralizability
2. Were treatment/exposures & clinical outcomes 2. Were treatment/exposures & clinical outcomes
measured in the same ways in both groups?measured in the same ways in both groups?
(Was the assessment of outcomes either objective (Was the assessment of outcomes either objective
or blinded)or blinded)
– How was the exposure
ascertained?
• Recall Bias or
Study hypothesisStudy hypothesis
BlindBlind
• Recall Bias or
Interviewer Bias
• Blinding
– How was the outcome
ascertained?
• Surveillance Bias
3. Was the follow up of the study patients sufficiently 3. Was the follow up of the study patients sufficiently long (for the outcome to occur) and complete?long (for the outcome to occur) and complete?
Short termShort term
Follow upFollow up20% lost to 20% lost to follow upfollow up
Long termLong termcelecoxib, rofecoxib, celecoxib, rofecoxib,
NSAIDNSAID
CHFCHF
No CHFNo CHF
TemporalityTemporality
Strength of associationStrength of association
Biologic PlaucibilityBiologic Plaucibility
CAUSE
4. Do the results satisfy some “diagnostic tests 4. Do the results satisfy some “diagnostic tests for causation”? for causation”?
Biologic PlaucibilityBiologic Plaucibility
SpecificitySpecificity
DoseDose--responseresponse
ConsistencyConsistency
Cessation of exposureCessation of exposureEFFECT
ARE THE VALID RESULTS OF THIS HARM STUDY IMPORTANT?
♦ If the study is valid in showing harm/association,… how bad and how accurate is the harm?
♦ What is the magnitude and precision of the association between exposure and outcome?
…. In other words…
The Estimate of the Magnitude Depends on
the Study Design
Incidence or risk
CohortRCT
Harm No IIexpexp = 20/1000 = 0.02= 20/1000 = 0.02Harm No
Treatment 20 1000
No treatment 2 1000
IIexpexp = 20/1000 = 0.02= 20/1000 = 0.02
IInonnon--expexp = 2/1000 = 0.002= 2/1000 = 0.002
RR=RR= IIexpexp/ I/ Inonnon--expexp== 1010
Exposed patients are 10 times more likely to suffer from the adverse event than non-exposed
Celecoxib
CHFCHF No CHFNo CHF
EXPOSE TO TREATMENT
ADVERSE OUTCOME
aa bb
cc ddNSAID TREATMENT cc dd
RR = RR = a / (a+b)a / (a+b)--------------------c/ (c+d)c/ (c+d)
OR = OR = ad /bcad /bc
Case-Control StudiesOdds Ratio (Relative Odds)
Adverse Outcome
PresentPresent AbsentAbsent TotalsTotals
ExposureExposure aa bb a+ba+b
OR = (a/c)/(b/d) = ad/bc
ExposureExposure
YesYes
aa bb a+ba+b
ExposureExposure
NoNo
cc dd c+dc+d
TotalsTotals a+ca+c b+db+d a+b+c+a+b+c+dd
Are the Results Important?
Case-Control Studies
♦Out of 100 affected cases
– 90 were exposed to the agent (a)
– 10 were not exposed (c)
♦Out of 100 non-affected cases (controls)
– 45 were exposed to the agent (b)
– 55 were not exposed (d)
Case Control Studies – Odds Ratio
Adverse Outcome♦ Out of 100 affected cases
– 90 were exposed
– 10 were not exposed
♦ Out of 100 controls
– 45 were exposed
Present Absent Totals
ExposureYes
90 45 135– 45 were exposed
– 55 were not exposed
♦ Odds Ratio:– OR = (a/c)/(b/d) = ad/bc = (90x55)/(45x10) = 11
The odds of experiencing the adverse event for patients exposed is 11x that of those unexposed
ExposureNo
10 55 65
Totals 100 100 200
Odds Ratio (OR)
♦ OR > 1 represents an increased risk or association
♦ Describes the relative harm of an exposure independent of disease prevalence
♦ When the prevalence of the outcome of interest is rare in the population from which the sample was drawn (often the reason for using a case-control study), the OR closely approximates the RR
Criteria to Examine RR & OR
♦Magnitude– Cohort and Case-Control are both biased
– What if RR or OR’s value preferable?•R.R. > 3
•O.R. > 4Impressive!!
•O.R. > 4
– Recalculate adjusting for a known confounder, if increased or the same after adjustment � more believable
– If the adjustment results in large decline in RR or OR � We should be suspicious of both of them
♦Precision
Impressive!!
Adjusting for Confounders
♦ Assuming the study is attempting to examine the association between SSRIs and Upper GI bleed.
♦ 100 cases with GI bleed, 90 were on SSRIs, 100 controls with no GI bleed, 45 were on 100 controls with no GI bleed, 45 were on SSRIs � OR = 11
♦ If in the study 7 subjects were also on Ibuproffen (confounder) (2 in the control group one on SSRI, 5 in the cases group 3 on SSRI)
♦ You can calculate an adjusted OR
Adjusted OR
Present Absent Totals
ExposureYes
90 45 135
Exposure 10 55 65
87 44 131
8 54 62
7 subjects on Ibuproffen
2 in the control group one on SSRI (b)
Upper GI Bleed
ExposureNo
65
Totals 100 100 200
The adjusted OR = ad/ad/bcbc = (87x54)/(44x8) = 13= (87x54)/(44x8) = 13
What is your conclusion????What is your conclusion????
8 54 62
95 98 193
5 in the cases group 3 on SSRI (a)
Criteria to Examine RR and OR
♦Magnitude
♦Precision
– Is highest when C.I. given is narrow and remains within a clinically important remains within a clinically important increased risk
Examples of C.I. – Is There Association??
♦ SSRIs and Upper GI bleed:
– Adjusted OR 3.0 (95% CI, 2.1-4.4)
♦ Non-SSRIs and Upper GI bleed:
– Adjusted OR 1.4 (95% CI, 1.1-1.9)
♦ NSAIDs and Upper GI bleed:
– Adjusted OR 3.7 (95% CI, 3.2-4.4)– Adjusted OR 3.7 (95% CI, 3.2-4.4)
♦ SSRIs + NSAIDs and Upper GI bleed:
– Adjusted OR 15.6 (95% CI 6.6-36.6)
♦ Long acting Benzodiazepines and risk of fall in elderly patients:
– RR 2.0 (95% CI 1.6-2.5)
♦ Calcium channel blockers and MI:
– OR 1.9 (95% CI 0.9-5.5)
Number Needed to Harm (NNH)
♦ The number of patients who, if they received the experimental treatment, would lead to one additional patient being “harmed”, compared with being “harmed”, compared with patients who received the comparison treatment.
♦ 1/ARI
NNH – For RCT and Cohort
♦One way to report the evidence to your patients
♦ Same formula as NNT♦ Same formula as NNT
– NNH = 1/ARI = 1/ (Iexp- Inonexp) =
1 [a/(a+b)-c/(c+d)]
ARI = Absolute Risk Increase
Number Needed to Harm (NNH)Number Needed to Harm (NNH)
NNHNNH = = aa
a + ba + bcc
c + dc + d
11
1 1 -- [PEER x (1 [PEER x (1 -- OR)]OR)]NNH = NNH = ----------------------------------------------------------
(1 (1 -- PEER) x PEER x (1 PEER) x PEER x (1 -- OR)OR)
1 1 ++ [PEER x (OR[PEER x (OR -- 11)])]NNH = NNH = ----------------------------------------------------------
(1 (1 -- PEER) x PEER x (ORPEER) x PEER x (OR -- 11))If OR > 1If OR > 1
If OR < 1If OR < 1
NNH - For Case Control
♦ PEER = Patient Expected Event Rate = Adverse event rate in non exposed
♦ Table of values available (Do NOT Need to ♦ Table of values available (Do NOT Need to Memorize)
♦ Note: For the same OR, NNH can be different for different PEERs:– Example: For OR = 0.9
• NNH = 2000 for PEER of 0.005
• NNH = 40 for PEER of 0.40
Converting OR to NNH
For OR greater than 11.1 1.25 1.5 1.75 2 2.25 2.5
0.05 212 86 44 30 23 18 16
0.1 113 46 24 16 13 10 9
0.2 64 27 14 10 8 7 6
Calculator available at:
http://www.cebm.utoronto.ca/practise/ca/statscal/orToNnt.htm
PEER
0.2 64 27 14 10 8 7 6
0.3 50 21 11 8 7 6 5
0.4 44 19 10 8 6 5 5
0.5 42 18 10 8 6 6 5
0.7 51 23 13 10 9 8 7
0.9 121 55 33 25 22 19 18
The Flawless Study
♦ A young investigator is designing a case-control study on the association between SSRIs and upper GI bleed. He recruited his cases and controls from a GI clinic. Both cases and controls were interviewed retrospectively for their use of SSRIs. In order to blind
FlawfulFlawful
retrospectively for their use of SSRIs. In order to blind himself to the subjects response, interviews were conducted by the clinic nurse who would ask the subjects a set of questions while checking them in for the visit. The investigator reported a RR of 2.5 (95% CI 0.5-4.0) and concluded that there is a positive association between SSRIs and upper GI bleed and that 1 in 2-3 patients treated with SSRIs will have an upper GI bleed caused by the medication
Study Flaws
♦Controls might have other GI diseases that predisposes them to GI bleed
♦ Interviewer not blinded
♦CI wide and lower end < 1
♦Used RR in a case-control design
♦ Interpretation of RR is wrong
Summary
♦ Harm is unavoidable but can be minimized
♦ Every intervention has some degree of harm
♦ Criteria for establishing validity in a harm study
♦ RCT, while excellent design, might be unethical ♦ RCT, while excellent design, might be unethical in harm
♦ Cohort/ Case-Control (Rare outcome or outcome that takes long time to develop)
♦ Association versus Causation
Summary
♦ Magnitude: – RCT or Cohort � RR = Iexp/Inonexp = [a/(a+b)]/[c/(c+d)]
– Case-Control � OR = (a/c)/(b/d) = ad/bc
♦ Magnitude: RR or OR > 1 (positive association)– Preferable association: (because of confounders)– Preferable association: (because of confounders)
• RR > 3
• OR > 4
– More criteria needed for CAUSATION
– Adjustment for confounders
♦ Precision: C.I. (Narrow and do not straddle “1”)
♦ NNH = 1/ARI = 1/ (Iexp- Inonexp)
THANK YOU