The Value of Observational Research A Case Study Approach Hal V. Barron, MD

The Value of Observational Research A Case Study Approach

Hal V. Barron, MD

OverviewReview what we can learn from observational data

• Examine associations and attempt to speculate on causality when RCTs are not feasible

– when RCTs are unethical (Does smoking really cause cancer?)– when the sample size needed for a RCT is prohibitive

• Examine associations for hypothesis generation• Describe what is happening in the “real world”

– Safety surveillance : Identification of rare events or subgroup analysis– Drug utilization patterns– Natural history of disease– Efficacy vs Effectiveness

OverviewReview what we can learn from observational data

• Examine associations and attempt to speculate on causality when Examine associations and attempt to speculate on causality when RCTs are not feasibleRCTs are not feasible

– when RCTs are unethical (Does smoking really cause cancer?)when RCTs are unethical (Does smoking really cause cancer?)– when the sample size needed for a RCT is prohibitive

• Examine associations for hypothesis generation• Describe what is happening in the “real world”

– Safety surveillance : Identification of rare events or subgroup analysis– Drug utilization patterns– Natural history of disease– Efficacy vs Effectiveness

Examine associations and attempt to speculate on causality when RCTs are not feasible

• Studies have demonstrated the importance of establishing and maintaining a patent infarct related artery in the setting of acute myocardial infarction (AMI) complicated by cardiogenic shock.

• The purpose of the present study was to determine whether the use of Intra-aortic baloon pumping (IABP) is associated with a survival advantage in patients with AMI complicated by cardiogenic shock.

• Why not do a RCT???

National Registry of Myocardial Infarction (NRMI) :

IABP Use and Outcome

• Using data from the National Registry of Myocardial Infarction 2 (NRMI 2), we evaluated 23,180 patients who presented with or developed cardiogenic shock during the hospitalization.

NRMI : IABP Use and Outcome

69.2

49.544.8

47.5

0

10

20

30

40

50

60

70

Mo

rta

lity

(%)

TT PPTCA

Type of Reperfusion Therapy

IABP - IABP +

OverviewReview what we can learn from observational data

• Examine associations and attempt to speculate on causality when Examine associations and attempt to speculate on causality when RCTs are not feasibleRCTs are not feasible

– when RCTs are unethical (Does smoking really cause cancer?)– when the sample size needed for a RCT is prohibitivewhen the sample size needed for a RCT is prohibitive

• Examine associations for hypothesis generation• Describe what is happening in the “real world”

– Safety surveillance : Identification of rare events or subgroup analysis– Drug utilization patterns– Natural history of disease– Efficacy vs Effectiveness

Data from the TIMI 2 Study

0

0.8

2.2

5.2

0

1

2

3

4

5

6

100mg tPA 150 mg tPA

BB+ BB-

Examine associations and attempt to speculate on causality when RCTs are not feasible

• Do beta-blockers reduce intra-cranial hemorrhage ICH rates when given immediately following tPA for AMI

– Does this meet the FINER criteria?

– What is the rate of ICH following tPA?

– Is a 30% reduction meaningful?

– What size trial would need to be conducted?

NRMI: BB Use and ICH

0

0.5

1

1.5

2

2.5

<65 65-74 75

Immediate BB No Immediate BB

Un

adju

sted

IC

H R

ate

(%)

AGE

NRMI: BB Use and ICH

00.20.40.60.8

11.21.41.61.8

Male Female

Immediate BB No Immediate BB

Una

djus

ted

ICH

Rat

e

NRMI and Beta-blocker UseMultivariate Analysis: Effect of Drug Therapy Administered Within 24 Hours on Intracranial Hemorrhage Rate

Medication Adjusted OR (95% Cl)

blocker 0.69 (0.57-0.84)*

ACE inhibitor 0.75 (0.55-1.03)

Calcium channel antagonist 1.27 (0.98-1.64)

Lidocaine 0.93 (0.76-1.13)

Intravenous magnesium 1.05 (0.76-1.45)

Intravenous nitroglycerin 0.86 (0.69-1.09)

*p<0.001.

Cl = confidence intervals; OR = odds ratio; other abbreviation as in Table 1.

OverviewReview what we can learn from observational data

• Examine associations and attempt to speculate on causality when RCTs are not feasible

– unethical studies– sample size is prohibitive

• Examine associations for hypothesis generationExamine associations for hypothesis generation• Describe what is happening oin the “real world”

– Safety surveillance : Identification of rare events or subgroup analysis– Drug utilization patterns– Natural history of disease– Efficacy vs Effectiveness

The Association Between White Blood Cell Count, Epicardial Blood Flow, Myocardial Perfusion, and Clinical

Outcomes in the Setting of Acute Myocardial Infarction

Hal V. Barron, M.D.; Christopher P. Cannon, M.D.; Sabina A. Murphy, M.P.H.; Susan J. Marble, M.S., R.N.; Eugene Braunwald, M.D.; and C. Michael Gibson, M.S., M.D.;

for the TIMI 10 Study Group

• Background: Patients with elevated white blood cell (WBC) counts during acute myocardial infarction (AMI) have a higher risk of adverse outcomes.

• Objectives: The goal of this study was to determine the relationship between the WBC count and angiographic characteristics to gain insight into this pathophysiology of this clinical observation.

• Methods: Angiographic and clinical data from 936 patients in the TIMI 10A and TIMI 10B trials was used to evaluate these relationships

0

4.9

3.8

10.4

0

2

4

6

8

10

0-5 5-10 10-15 >15

Death

0

7.9 7.8

20.9

0

5

10

15

20

0-5 5-10 10-15 >15

Death or CHF

• Results : The development of new congestive heart failure was associated with significantly higher WBC counts (13.3 8.9, n=64 vs 10.8 3.5, p<0.0001), an observation which remained significant in a multivariable model adjusting for all potential confounding variables (O.R. 1.2 per 1 unit increase in WBC count, p<0.001).

OverviewReview what we can learn from observational data

• Examine associations and attempt to speculate on causality when RCTs are not feasible

– unethical studies– sample size is prohibitive

• Examine associations for hypothesis generation• Describe what is happening oin the “real world”Describe what is happening oin the “real world”

– Safety surveillance : Identification of rare events or subgroup analysisSafety surveillance : Identification of rare events or subgroup analysis– Drug utilization patterns– Natural history of disease– Efficacy vs Effectiveness

ICH Risk following t-PA:NRMI 2

Gurwitz et al. 1998 Annals Int Med. 129; 597-604.

0

2

4

6

8

10

12

M<65 F<65 M 65-74 F 65-74 M>75 F>75

Adj

ust

ed O

R

OverviewReview what we can learn from observational data

• Examine associations and attempt to speculate on causality when RCTs are not feasible

– unethical studies– sample size is prohibitive

• Examine associations for hypothesis generation• Examine associations to identify treatment modifiers • Describe what is happening oin the “real world”

– Safety surveillance : Identification of rare events or subgroup analysis– Drug utilization patterns– Natural history of disease– Efficacy vs Effectiveness

All Patients in NRMI 2

30.1%

Thrombolytic Therapy

No Initial Reperfusion

63.0%

Primary PTCA6.8%

Background

• Initial reperfusion therapy (IRT) is beneficial for patients with acute myocardial infarction (AMI)

• A minority of patients with AMI receive IRT.

• Underutilization could be related to:

– the absence of clear indications,

– perceived contraindications and

– physicians’ reluctance to prescribe IRT.

Hypothesis

• To determine what percent of patients identified as having clear indications for initial reperfusion therapy (IRT) do not receive this life-saving therapy and

• To identify patient subgroups who are at greatest risk for not receiving IRT.

Methods - Study Population

SymptomsSymptomsHosp <6 hrsHosp <6 hrs

ST Segment ST Segment or LBBBor LBBB

Contraindications to thrombolytic RxContraindications to thrombolytic Rx

No IRTNo IRTN=20,319N=20,319

IRTIRTN=64,344N=64,344

LBBB

No CP

Age > 75

Prior CHF

Prior MI

Prior Stroke

Killip 3#

Killip 2#

Prior Angina

Diabetes

Female

Prior Revasc.

Anterior MI*

Prior HTN

Caucasian

Current Smoker

Prehospital ECG

Sx < 3 hrs. 0.2 0.4 0.6 0.8 1.0 1.2 1.4 1.6 1.8

Less Likely More LikelyReperfusion Therapy

Underutilizing of IRT In High Risk Patients

VariableOR: In-Hospital

DeathOR:

Receiving IRT

Female Gender 1.47 0.88

History of CHF 1.23 0.35

History of Stroke 1.76 0.66

History of Diabetes 1.12 0.78

Current Smoker 0.68 1.34

Anterior Wall MI 1.76 1.59

Rales On Initial PE 2.06 0.69

Pulmonary Edema onInitial PE

2.19 0.34

Conclusions

• At least 31% of patients presenting with AMI are appropriate for IRT

• 1 in 4 patients appropriate for IRT do not receive this life-saving therapy.

• The underutilization is particularly evident in the elderly, women and other patients at increased risk for in-hospital mortality.

OverviewReview what we can learn from observational data

• Examine associations and attempt to speculate on causality when RCTs are not feasible

– when RCTs are unethical (Does smoking really cause cancer?)– when the sample size needed for a RCT is prohibitive

• Examine associations for hypothesis generation• Describe what is happening in the “real world”

– Safety surveillance : Identification of rare events or subgroup analysis– Drug utilization patterns– Natural history of disease– Efficacy vs Effectiveness

Sex Differences in Early Mortality After Myocardial Infarction

Vaccarino et al. N Engl J Med 1999;341:217-25

Background

• Literature is conflicting about whether short-term mortality after MI is higher in women than in men after adjusting for age and other prognostic factors

• Traditional approach: compare all the men and all the women, adjusting for age and other factors

Specific Aims

• To test the following hypotheses:

1. the mortality of women relative to men is not constant across ages

2. the younger the age of the patients, the higher the risk of death in women relative to men

• To identify factors that may account for the higher mortality rates of women compared with men

Data Source

• Second National Registry of Myocardial Infarction (NRMI-2)

• 1,658 participating U.S. hospitals

• N=691,995 MI patients enrolled up to 1/31/98

Study Sample

EXCLUSIONS:

• Age <30 and > 90

• Patients transferred from other hospitals

• Patients transferred to other hospitals

• N for analysis: 384,878

Methods of Analysis

Multiple logistic regression with hospital death as outcome

1. Traditional analysis approach: main effect of female sex after adjusting for age

2. Test for sex-age interaction

3. Sequential adjustment for other covariables

RESULTSSelected Patient Characteristics by Sex

Women Men

Mean age 72 66

History of MI (%) 24 28

History of CHF (%) 21 13

History of HTN (%) 59 47

History of diabetes (%) 33 25

Chest pain (%) 63 72

ST elevation (%) 38 42

CHF or cardiog. shock (%) 27 19

Hospital mortality (%) 17 11

Factors Disproportionately more Common in Women at Younger Ages

• Demographic factors– Non-White race

– Medicaid insurance

• Medical history– Hx of CHF

– Hx of diabetes

– Hx of stroke

• Admission data– Delay to presentation >6 hrs

– No ST elevation

– CHF, pulmonary edema

– Hypotension or cardiogenic shock

• Treatments– No coronary reperfusion therapy

– No use of IV beta-blockers

History of Diabetes

0

5

10

15

20

25

30

35

40

30-59 60-69 70-79 80-89

Age

%MenWomen

Presentation After 6 hrs from Symptom Onset

05

1015202530354045

30-59 60-69 70-79 80-89

Age

%MenWomen

Hypotension on Admission

0

1

2

3

4

5

6

30-59 60-69 70-79 80-89

Age

%MenWomen

Overall Effect of Female Sex on Mortality (traditional approach)

OR of MortalityWomen Vs. Men (95% CI)

Unadjusted 1.54 (1.51-1.57)

Age adjusted 1.14 (1.12-1.17)

Hospital Mortality Rates by Sex and Age (Unadjusted)

0

5

10

15

20

25

<50 50-54 55-59 60-64 65-69 70-74 75-79 80-84 85-89

Age

Ho

sp

ita

l M

ort

ali

ty (

%)

Men

Women

Sex-Age Interaction: P<0.001

Effect of Female Sex on Mortality by Age (Unadjusted)

Age

0.5

1.0

1.5

2.0

2.5

3.0O

R (W

omen

Vs.

Men

)

PredictedActual

30 35 40 45 50 55 60 65 70 75 80 85 90

0.5

1.0

1.5

2.0

2.5

3.0

Unadjusted

Impact of Overall Adjustment

Adjusted

Age

OR

(W

om

en

Vs

. Men

)

30 35 40 45 50 55 60 65 70 75 80 85 90

• A higher risk of death in women relative to men is seen in the younger age groups only

• There is a linear increase of risk for women relative to men going from older to younger age

• The younger the patients’ age, the higher the risk of death of women relative to men

• Adjustment for covariables explains only 1/3 of the higher mortality risk for women at younger ages

Summary / Conclusions

OverviewReview what we can learn from observational data

• Examine associations and attempt to speculate on causality when RCTs are not feasible

– when RCTs are unethical (Does smoking really cause cancer?)– when the sample size needed for a RCT is prohibitive

• Examine associations for hypothesis generation• Examine associations to identify treatment modifiers • Describe what is happening in the “real world”

– Safety surveillance : Identification of rare events or subgroup analysis– Drug utilization patterns– Natural history of disease– Efficacy vs Effectiveness

PP=0.06=0.06PP=0.06=0.06 PP=0.02=0.02PP=0.02=0.02

Grines CL, et al. N Engl J Med. 1993;328:673-679.

Trials Comparing Primary PTCA With Fibrinolytic Therapy: PAMI Cohort

2.6

6.5

5.1

0

2

4

6

8

10

12

In-hospital death In-hospital death or nonfatalreinfarction

PTCA

t-PA

12.0

9.6

13.7 14.1

16.1

02468

1012141618

30 days 6 months

PTCA

t-PA

P=0.033 P=NS

GUSTO-IIb Angioplasty Substudy Investigators. N Engl J Med. 1997;336:1621-1628.

Trials Comparing Primary PTCA With Fibrinolytic Therapy: GUSTO-IIb Cohort

Co

mp

os

ite

Ou

tco

me

(%

)

Rate %Rate %Rate %Rate %

Study Study GroupGroupStudy Study GroupGroup PTCA Lytic

TherapyOdds Ratio

(95% CI)

Absolute Risk Reduction, %

(95% CI)

Streptokinase 4.0 5.9 0.66 (0.29 to1.50) 1.9 (-2.7 to 4.1)

3- to 4-hour t-PA 3.5 5.7 0.60 (0.24 to1.41) 2.2 (-2.2 to 4.3)

Accelerated

t-PA5.0 7.2 0.68 (0.42 to 1.08) 2.2 (-0.5 to 4.0)

Total 4.4 6.5 0.66 (0.46 to 0.94) 2.1 (0.4 to 3.4)

Meta-analysis of Mortality Benefit With Primary PTCA Versus Fibrinolytic Therapy

Weaver WD, et al. JAMA. 1997;678:2093-2098.

70

75

80

85

90

95

100

Su

rviv

al (

%)

Primary AngioplastyFibrinolytic Therapy

Time After Discharge (years)Time After Discharge (years)Every NR, et al. N Engl J Med. 1996;335:1253-1260.

P=NS

Trials Comparing Primary PTCA With Fibrinolytic Therapy: MITI Cohort

0 0.5 1 1.5 2 2.5 3 3.5 4

PPTCA versus tPA :NRMI 2

• 4,939 nontransfer pts underwent PPTCA within 12 hrs from Sx onset

• 24,705 pts received tPA

• Lytic ineligable and shock pts were excluded

Randomized Trial Results Versus Community-Setting Results: NRMI-2 Cohort

Tiefenbrunn AJ, et al. J Am Coll Cardiol. 1998;31:1240-1245.

5.2 5.4 5.6 6.2

0

2

4

68

In-hospital mortality In-hospital mortality ornonfatal stroke

PTCA t-PA

PP=NS=NS PP=NS=NS

Pe

rce

nt

n=2958, lytic eligible, no shock at presentation n=2958, lytic eligible, no shock at presentation

Overall 5.4 5.2STE or LBBB 1st ECG 5.3 5.5

Age < 75 yr. 3.4 3.5Age > 75 yr. 16.5 14.4

Male 4.5 5.2Female 9.6 8.9

Inferior MI 3.9 3.9Anterior MI 7.6 7.1

Low Risk 2.9 2.8Not Low Risk 7.5 7.4

rt-PA PTCAMortality (%)

0.5 1.0 1.5

Odds Ratio and 95% CI

rt-PA better PTCA better

PPTCA versus tPA (Death and Nonfatal Stroke)

6.25.6 6.1 5.9

18.4

14.6

4.1 3.9

0

2

4

6

8

10

12

14

16

18

20

All STE or LBBB Age >75 Age <75

t-PA PPTCA

Efficacy vs Effectiveness

• Why might they differ?

1

3.7 46.4

14.1

0

5

10

15

20

25

60 61-75 76-90 >91 PTCA notperformed

Berger PB, et al. Circulation. 1999;100:14-20.

PP=0.001=0.001PP=0.001=0.001

Door-to-Balloon Time (minutes)

Importance of Door-to-Balloon Time: 30-Day Mortality in the GUSTO-IIb Cohort

Mo

rta

lity

(%)

<

Treatment effect modifiers

5.85 6.21 5.87

4.465.37

3.37

0

1

2

3

4

5

6

7

8

Low Volume Medium Volume High Volume

Thrombolytic Therapy Primary Angioplasty

Rates of Death during Hospitalization for Myocardial Infarction among patients treated with thrombolytic therapy versus primary angioplasty. The interaction between reperfusion strategy and primary angioplasty volume was significant (p<.01).

Dea

th d

uri

ng

Hos

pit

aliz

atio

n (

%)

Hospital-specific primary angioplasty volume category

OverviewReview what we can learn from observational data

• Examine associations and attempt to speculate on causality when RCTs are not feasible

– when RCTs are unethical (Does smoking really cause cancer?)– when the sample size needed for a RCT is prohibitive

• Examine associations for hypothesis generation• Describe what is happening in the “real world”

– Safety surveillance : Identification of rare events or subgroup analysis– Drug utilization patterns– Natural history of disease– Efficacy vs Effectiveness

Conclusions

• Observational research studies can be very valuable – They provide information not obtainable from RCTs – They provide important information when RCTs are not

feasible

• Observational research studies can be very misleading as well– They can never really clarify causality (only

associations)– Measured and especially unmeasured confounders can

be a VERY BIG problem!-more to come on this