1

EPI235: Epi Methods in HSR

April 12, 2007 L4

Evaluating Health Services using administrative data 3: Advanced Topics on Risk Adjustment and Sensitivity Analysis (Dr. Schneeweiss)

Risk adjustment in studies using administrative databases is limited to observed confounders. Dr. Schneeweiss will illustrate theory and practice of assessing the sensitivity of epidemiologic risk estimates towards unobserved confounding. An interactive Excel program will be used for illustration.

Background reading: •Walker AM: Observation and inference, Chapter 9. Epidemiology Resources, Newton Lower Falls, 1991.•Schneeweiss S, Glynn RJ, Tsai EH, Avorn J, Solomon DH. Adjusting for unmeasured confounders in pharmacoepidemiologic claims data using external information: The example of COX2 inhibitiors and myocardial infarction. Epidemiology 2005;16:17-24.

2

Unmeasured (residual) Confounding

Confounding factors that are not measured are hard to adjust for in observational analyses

If unadjusted they lead to residual confounding

3

Unmeasured (residual) Confounding:

[smoking,healthy lifestyle, etc.]

Drug exposure

Outcome

RREO

OREC RRCO

CU

CM

4

Unmeasured Confounding in Claims Data

Database studies are criticized for their inability to measure clinical and life-style parameters that are potential confounders in many pharmacoepi studies OTC drug use BMI Clinical parameters: Lab values, blood pressure, X-

ray Physical functioning, ADL (activities of daily living) Cognitive status

5

Strategies to Discuss Residual Confounding

Qualitative discussions of potential biasesSensitivity analysis

SA is often seen as the ‘last line of defense’ A) SA to explore the strength of an association as a

function of the strength of the unmeasured confounder B) Answers the question “How strong must a

confounder be to fully explain the observed association”

Several examples in Occupational Epi but also for claims data

Greenland S et al: Int Arch Occup Env Health 1994

Wang PS et al: J Am Geriatr Soc 2001

6

Foot-in-Mouth Award (Economist ‘04): “… there are known knowns; there are things we know we know. We also know that there are known unknowns; that is to say we know that there are some things we do not know. But there are also unknown unknowns – the ones we don’t know we don’t know. …, it is the latter category that tend to be the difficult ones.”

 

(Wisely unknowing) Donald Rumsfeld

7

Notation

RR Fully adjusted (“true”) exposure relative

risk

ARR Apparent exposure relative risk

RRCD Association between confounder and

disease outcome

PC Prevalence of confounder

PC1 Prevalence of confounder in the

exposed

PC0 Prevalence of confounder in the

unexposed

PE Prevalence of drug exposure

OREC Association between drug use category

and confounder

8

A simple sensitivity analysis

The apparent RR is a function of the adjusted RR times ‘the imbalance of the unobserved confounder’

After solving for RR we can plug in values ofr the prevalence and strength of the confounder:

1)1(

1)1(

0

1

CDC

CDC

RRP

RRPRRARR

1)1(

1)1(

0

1

CDC

CDC

RRP

RRP

ARRRR

9

A made-up example

Association between TNF-a blocking agents and NH lymphoma in RA patients Let’s assume and observed RR of 2.0 Let’s assume 50% of RA patients have a more

progressive immunologic disease … and that more progressive disease is more likely

to lead to NH lymphoma Let’s now vary the imbalance of the hypothetical

unobserved confounder

10

Bias by residual confounding

4.5

2.5

0.8

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.0

1.0

2.0

3.0

4.0

5.0

6.0

RRadjusted

RRCD

PC1

Fixed:ARR = 2.0

PC0 = 0.5

11

2. Array approach

fix X Y fix Z2 Z1

ARR RRCD PC1 PC0 RRadjusted % Bias % Bias = [(ARR-RRadj.)/(RRadj.-1)]*100

2.0 4.5 0.0 0.5 5.5 -77.782.0 4.0 0.0 0.5 5.0 -75.002.0 3.5 0.0 0.5 4.5 -71.432.0 3.0 0.0 0.5 4.0 -66.672.0 2.5 0.0 0.5 3.5 -60.002.0 2.0 0.0 0.5 3.0 -50.002.0 1.5 0.0 0.5 2.5 -33.332.0 1.0 0.0 0.5 2.0 0.002.0 0.8 0.0 0.5 1.8 33.332.0 0.5 0.0 0.5 1.5 100.002.0 4.5 0.1 0.5 4.1 -67.472.0 4.0 0.1 0.5 3.8 -64.862.0 3.5 0.1 0.5 3.6 -61.542.0 3.0 0.1 0.5 3.3 -57.142.0 2.5 0.1 0.5 3.0 -51.062.0 2.0 0.1 0.5 2.7 -42.112.0 1.5 0.1 0.5 2.4 -27.592.0 1.0 0.1 0.5 2.0 0.002.0 0.8 0.1 0.5 1.8 25.812.0 0.5 0.1 0.5 1.6 72.732.0 4.5 0.2 0.5 3.2 -55.262.0 4.0 0.2 0.5 3.1 -52.942.0 3.5 0.2 0.5 3.0 -50.002.0 3.0 0.2 0.5 2.9 -46.152.0 2.5 0.2 0.5 2.7 -40.912.0 2.0 0.2 0.5 2.5 -33.332.0 1.5 0.2 0.5 2.3 -21.432.0 1.0 0.2 0.5 2.0 0.002.0 0.8 0.2 0.5 1.8 18.752.0 0.5 0.2 0.5 1.7 50.002.0 4.5 0.3 0.5 2.7 -40.582.0 4.0 0.3 0.5 2.6 -38.712.0 3.5 0.3 0.5 2.6 -36.362.0 3.0 0.3 0.5 2.5 -33.332.0 2.5 0.3 0.5 2.4 -29.272.0 2.0 0.3 0.5 2.3 -23.532.0 1.5 0.3 0.5 2.2 -14.812.0 1.0 0.3 0.5 2.0 0.002.0 0.8 0.3 0.5 1.9 12.122.0 0.5 0.3 0.5 1.8 30.772.0 4.5 0.4 0.5 2.3 -22.582.0 4.0 0.4 0.5 2.3 -21.432.0 3.5 0.4 0.5 2.3 -20.002.0 3.0 0.4 0.5 2.2 -18.182.0 2.5 0.4 0.5 2.2 -15.792.0 2.0 0.4 0.5 2.1 -12.502.0 1.5 0.4 0.5 2.1 -7.692.0 1.0 0.4 0.5 2.0 0.002.0 0.8 0.4 0.5 1.9 5.882.0 0.5 0.4 0.5 1.9 14.292.0 4.5 0.5 0.5 2.0 0.002.0 4.0 0.5 0.5 2.0 0.002.0 3.5 0.5 0.5 2.0 0.002.0 3.0 0.5 0.5 2.0 0.00

4.5

3.5

2.5

1.5

0.8

0.0 0.

2 0.4 0.

6 0.8 1.

0-100

-50

0

50

100

150

200

250

300

350

% Bias

RRCD

PC1

Fixed:ARR = 2.0

PC0 = 0.5

4.5

2.5

0.8

0.0 0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

0.0

1.0

2.0

3.0

4.0

5.0

6.0

RRadjusted

RRCD

PC1

Fixed:ARR = 2.0

PC0 = 0.5

1)1(

1)1(

0

1

.

CDC

CDC

adj

RRP

RRP

ARRRR

12

Pros and cons of “Array approach”

Very easy to perform using ExcelVery informative to explore confounding with

little prior knowledge Problems: It usually does not really provide an answer

to a specific research question4 parameters can vary -> in a 3-D plot 2

parameter have to be kept constantThe optical impression can be manipulated

by choosing different ranges for the axes

13

Same example, different parameter ranges

3.0

1.7

0.8

0.2

0.3

0.3

0.4

0.4

0.5

0.5

0.6

0.6

0.7

0.7

0.0

0.5

1.0

1.5

2.0

2.5

3.0

RRadjusted

RRCD

PC1

Fixed:ARR = 2.0

PC0 = 0.5

14

Conclusion of “Array Approach”

Great tool but you need to be honest to yourself

For all but one tool that I present today: Assuming conditional independence of CU and CM

given the exposure status If violated than residual bias may be overestimated

Drug exposure

Outcome

RREO

OREC RRCO

CU

CM

Hernan, Robins: Biometrics 1999

?

15

More advanced techniques

Wouldn’t it be more interesting to know How strong and imbalanced does a confounder

have to be in order to fully explain the observed findings?

RRCO OREC

16

Example:

Wang et al: JAGS 2001;49:1685

Zolpidem use and hip fractures in older people.

The issue:

Are there any unmeasured factors that may lead to a preferred prescribing of zolpidem to people at higher risk for falling and fracturing?

> Frailty is a hard to measure concept in claims data

RRCO

OREC

ARR

PC

17

How do we do that?

We want to express as a function of , ARR, PC, PE

OREC

RRCO

Walker AM: Observation and Inference. Epidemiology Resources Inc., Newton, 1991

18

A s s u m i n g a 2 - b y - 2 t a b l e o f a d i c h o t o m o u s e x p o s u r e a n d a d i c h o t o m o u s

c o n f o u n d e r , l e t e b e t h e p r e v a l e n c e o f t h e c o n f o u n d e r a m o n g e x p o s e d ( P C 1 | E 1 ) . T h e

a s s o c i a t i o n b e t w e e n t h e c o n f o u n d e r a n d e x p o s u r e c a n t h e n b e m e a s u r e d b y t h e

c o n f o u n d e r - e x p o s u r e o d d s r a t i o o r O R C E , w h i c h i s a f u n c t i o n o f e a n d t h e m a r g i n a l

p r o b a b i l i t i e s o f e x p o s u r e P r ( E ) a n d c o n f o u n d e r P r ( C ) :

])][Pr()[Pr(

])Pr()Pr(1[

eEeC

eECeOR CE

( 1 )

A s s u m i n g n o u n d e r l y i n g t r u e e x p o s u r e - d i s e a s e a s s o c i a t i o n o r R R E D = 1 ,

W a l k e r s h o w e d t h a t t h e a p p a r e n t R R E D ( A R R E D ) i s a f u n c t i o n o f e , t h e m a r g i n a l

p r o b a b i l i t i e s P r ( E ) a n d P r ( C ) , a n d t h e c o n f o u n d e r - d i s e a s e a s s o c i a t i o n R R C D :

)Pr(

)Pr(1

1)Pr(]1][)[Pr(

)Pr(]1[

E

E

ERReC

ERReARR

CD

CDED

( 2 )

19

])][Pr()[Pr(

])Pr()Pr(1[

eEeC

eECeOR CE

( 1 )

)Pr(

)Pr(1

1)Pr(]1][)[Pr(

)Pr(]1[

E

E

ERReC

ERReARR

CD

CDED

( 2 )

I f t h e p r i m a r y i n t e r e s t i s t o e x p l o r e t h e r e l a t i o n s h i p b e t w e e n O R C E a n d R R C D f o r a g i v e n A R R E D , R R C D ,

P r ( C ) , a n d P r ( E ) t h e n w e n e e d t o s o l v e e q u a t i o n ( 2 ) f o r e ,

)Pr())(Pr(1))Pr())(Pr(

))(Pr())(Pr())Pr()(Pr())Pr()(Pr()()Pr( 22

ERRERRARRRREARRE

ARREARREARRCEARRRRCEEPREe

CDCDEDCDED

EDEDEDEDCD

a n d s u b s t i t u t e t h e d e r i v e d t e r m f o r e i n e q u a t i o n 1 .

20

3. Rule Out Residual ConfoundingHow strong does an unmeasured confounder have to be to fully explain the observed findings?

The relationship between OREC and RRCD for a given ARR, RRC, PC, PE.

Data from Wang et al.: Zolpidem use and hip fractures in older people. J Am Geriatri Soc 2001;49:1685-90.a(prim)

RRCD PC PE ARR=1.95 OREC ARR=1.09 OREC

1.2 0.2 0.01 1.95 1.09 8.09 0.0438671.5 0.2 0.01 1.95 1.09 2.65 0.019562 0.2 0.01 1.95 1.09 1.78 0.011458

2.5 0.2 0.01 1.95 1.09 1.54 0.0087573 0.2 0.01 1.95 24.23 1.09 1.43 0.007406

3.5 0.2 0.01 1.95 13.10 1.09 1.36 0.0065964 0.2 0.01 1.95 9.51 1.09 1.32 0.006056

4.5 0.2 0.01 1.95 7.74 1.09 1.29 0.005675 0.2 0.01 1.95 6.69 1.09 1.27 0.005381

5.5 0.2 0.01 1.95 5.99 1.09 1.25 0.0051566 0.2 0.01 1.95 5.49 1.09 1.24 0.004976

6.5 0.2 0.01 1.95 5.12 1.09 1.22 0.0048287 0.2 0.01 1.95 4.83 1.09 1.22 0.004706

7.5 0.2 0.01 1.95 4.60 1.09 1.21 0.0046028 0.2 0.01 1.95 4.41 1.09 1.20 0.004513

8.5 0.2 0.01 1.95 4.26 1.09 1.19 0.0044369 0.2 0.01 1.95 4.13 1.09 1.19 0.004368

9.5 0.2 0.01 1.95 4.01 1.09 1.19 0.00430910 0.2 0.01 1.95 3.91 1.09 1.18 0.004256

0.00

2.00

4.00

6.00

8.00

10.00

0 2 4 6 8 10

RRCDO

RE

C

ARR=1.95

ARR=1.09

21

Example:

Psaty et al: JAGS 1999;47:749

CCB use and acute MI.

The issue:

Are there any unmeasured factors that may lead to a preferred prescribing of CCB to people at higher risk for AMI?

OREC

RRCO

ARR = 1.57

ARR = 1.30

22

3. Rule Out Residual ConfoundingHow strong does an unmeasured confounder have to be to fully explain the observed findings?

The relationship between OREC and RRCD for a given ARR, RRC, PC, PE.

Data from Psaty et al.: Assessment and control for confounding by indication in observational studies. J Am Geriatri Soc 1999;47:749-54.a(prim)

RRCD PC PE ARR=1.57 OREC ARR=1.3 OREC

1.2 0.2 0.01 1.57 1.3 0.0311771.5 0.2 0.01 1.57 1.3 23.90 0.0143442 0.2 0.01 1.57 28.79 1.3 5.11 0.008733

2.5 0.2 0.01 1.57 9.03 1.3 3.42 0.0068633 0.2 0.01 1.57 5.97 1.3 2.78 0.005928

3.5 0.2 0.01 1.57 4.73 1.3 2.45 0.0053674 0.2 0.01 1.57 4.06 1.3 2.25 0.004993

4.5 0.2 0.01 1.57 3.65 1.3 2.12 0.0047255 0.2 0.01 1.57 3.36 1.3 2.02 0.004525

5.5 0.2 0.01 1.57 3.15 1.3 1.94 0.0043696 0.2 0.01 1.57 2.99 1.3 1.88 0.004244

6.5 0.2 0.01 1.57 2.87 1.3 1.84 0.0041427 0.2 0.01 1.57 2.77 1.3 1.80 0.004057

7.5 0.2 0.01 1.57 2.68 1.3 1.77 0.0039858 0.2 0.01 1.57 2.61 1.3 1.74 0.003924

8.5 0.2 0.01 1.57 2.56 1.3 1.71 0.003879 0.2 0.01 1.57 2.51 1.3 1.69 0.003824

9.5 0.2 0.01 1.57 2.46 1.3 1.68 0.00378210 0.2 0.01 1.57 2.42 1.3 1.66 0.003746

0.00

2.00

4.00

6.00

8.00

10.00

0 2 4 6 8 10

RRCD

OR

EC

ARR=1.57

ARR=1.3

23

Caution!

Psaty et al. concluded that it is unlikely that an unmeasured confounder of that magnitude exists

However, the randomized trial ALLHAT showed no association between CCB use and AMI

Alternative explanations: Joint residual confounding may be larger than

anticipated from individual unmeasured confounders Not an issue of residual confounding but other biases,

e.g. control selection?

24

Pros and cons of “Rule Out Approach”

Very easy to perform using Excel Meaningful and easy to communicate

interpretationStudy-specific interpretationProblems:Still assuming conditional independence of CU

and CM “Rule Out” lacks any quantitative assessment

of potential confounders that are unmeasured

25

External Adjustment

One step beyond sensitivity analysesUsing additional information not available in

the main studyOften survey information

26

Strategies to Adjust residual con-founding using external information

Survey information in a representative sample can be used to quantify the imbalance of risk factors that are not measured in claims among exposure groups

The association of such risk factors with the outcome can be assess from the medical literature (RCTs, observational studies)

Velentgas et al: PDS, under review

Schneeweiss et al: Epidemiology, in press 2004

27

How do we do that?

We want to express ARR as a function of , , ARR, PC, PE

ORECRRCO

Walker AM: Observation an Inference. Epidemiology Resources Inc., Newton, 1991

28

])][Pr()[Pr(

])Pr()Pr(1[

eEeC

eECeOR CE

( 1 )

)Pr(

)Pr(1

1)Pr(]1][)[Pr(

)Pr(]1[

E

E

ERReC

ERReARR

CD

CDED

( 2 )

I f t h e p r i m a r y i n t e r e s t i s t o e s t i m a t e A R R E D a s a f u n c t i o n o f O R C E , R R C D , a n d t h e m a r g i n a l p r o b a b i l i t i e s

P r ( E ) a n d P r ( C ) t h e n w e n e e d t o s o l v e e q u a t i o n ( 1 ) f o r e ,

0)Pr()Pr(]1)Pr()Pr()Pr()Pr([)1(2 EORCCEOREORCeORe CECECECE

a b c

a n d e c a n b e f o u n d a s t h e s o l u t i o n o f a q u a d r a t i c e q u a t i o n o f t h e f o r m

a

acbbe

2

42

w h i c h w i l l t h e n b e s u b s t i t u t e d f o r e i n e q u a t i o n 2 .

29

Example: COX-2 inhibitors use and MI

Ray et al., Lancet 2002: >25mg roficoxib vs. non NSAID users, RR=1.9 (1.1-

3.4) Medicaid patients, new users

Solomon et al., Circulation in press: >25mg roficoxib vs. non NSAID users, RR=1.6 (1.04-

2.4) Medicare patients with drug coverage through PACE

Can these associations be due to confounding by factors not measured in claims data? e.g. BMI, OTC aspirin use, smoking, education etc.

30

In our example:

Rofecoxib Acute MI

RREO

From Survey data in a

subsampleFrom medical

literature

OREC RRCO

[smoking,aspirin, BMI, etc.]

CU

CM

31

Where can we get detailed information on unmeasured confounders?

MCBS: Medicare Current Beneficiary Survey Representative Sample 12,000 Medicare beneficiaries each year (majority

> 65y) Face-to-face interview in beneficiary’s home ‘Cost and Use’ file include drug utilization 98% response rate >95% data completeness Low cost ($900 / year) Readily available, but 2-year lag time)

32

Unobserved confounders in our example

Independent predictors of MI: Aspirin use Smoking BMI Educational attainment Income status

Expl. 2: Independent predictors of hip fracturs: Cognitive impairment Physical impairment Restrictions in ADL (Rubinstein L)

33

Our survey population

1999 MCBSRestricted to >64 yearsRestricted to community sample (no proxi

interviews)N = 8,785

34

Distribution of unmeasured confounders among drug users

Any COX-2 Inhibitors Any non-selective NSAIDs Non-users

(N=872) (N=1,302) (N=6,611) N % N % N %

Gender Female 607 69.6 777 59.7 3746 56.7 Male 265 30.4 525 40.3 2865 43.3 Age 65 to 74 436 50.0 731 56.1 3335 50.5 75 and older 436 50.0 571 43.9 3276 49.6 BMI Not Obese (BMI<30) 662 75.9 982 75.4 6484 83.0 Obese (BMI30) 206 23.6 315 24.2 1096 16.6 Aspirin Aspirin Use 80 9.2 133 10.2 614 9.3 No Aspirin Use 792 90.8 1169 89.8 5997 90.7 Smoking status Current 71 8.1 127 9.8 669 10.1 Former 396 45.4 661 50.8 3278 49.6 Never 405 46.4 514 39.5 2663 40.3 Education High school or less 603 69.2 937 72.0 4566 69.1 College or more 266 30.5 357 27.4 1976 29.9 Income $20,000 415 47.6 730 56.1 3507 53.1 > $20,000 457 52.4 572 43.9 3104 47.0

35

Celecoxib vs. Rofecoxib users … Celecoxib only Rofecoxib only (N=562) (N=244) N % N %

Gender Female 381 67.8 175 71.7 Male 181 32.2 69 28.3 Age 65 to 74 289 51.4 119 48.8 75 and older 273 48.6 125 51.2 BMI Not obese (BMI<30) 423 75.3 197 80.7 Obese (BMI30) 136 24.2 47 19.3 Aspirin Aspirin Use 46 8.2 28 11.5 No Aspirin Use 516 91.8 216 88.5 Smoking status Current 49 8.7 17 7.0 Former 254 45.2 120 49.2 Never 259 46.1 107 43.9 Education High school or less 388 69.0 161 66.0 College or more 174 31.0 80 32.8 Income $20,000 260 46.3 116 47.5 > $20,000 302 53.7 128 52.5

36

Literature estimates of RRCO

Adjustment of primary

estimate Potential confounder Relative

risk Age-sex adjusted

Multivariate adjusted

Obesity (BMI30) 1.7 a) yes

Aspirin use (non-use vs. use) 0.7 b) yes

Smoking (current vs. never) 3.1 c) yes

Educational attainment ( high school vs. >high school)

2.1 d) yes

Income ($20,000 vs. >$20,000) 2.1 e) yes

* In case of conflicting literature estimates the more extreme estimate was used. This will potentially lead to an overestimation of the magnitude of bias.

37

Calculating bias

Cox-2 inhibitor use vs. non-selective NSAID use and myocardial infarction.

RRCD p(C)

Adjusted OREC**

True RRED p(E)

Apparent RRED

† % Bias†† Data source: Literature MCBS MCBS assumed MCBS

Potential confounder:*

Obesity (BMI30 vs. BMI<30)

1.7 0.24 0.99 1.00 0.40 0.99 -0.11

Aspirin use (use vs. non-use)

0.7 0.10 0.90 1.00 0.40 1.00 0.29

Smoking (current vs. never)

3.1 0.09 0.87 1.00 0.40 0.98 -1.97

Educational Attainment ( high school vs. >high school)

2.1 0.71 0.83 1.00 0.40 0.98 -2.36

Income status ($20,000 vs. >$20,000)

2.1 0.53 0.92 1.00 0.40 0.99 -1.44

38

More contrasts

% Bias††

COX-2 (872) vs.

non-selective NSAIDs (1,302)

COX-2 (872) vs.

non-users (6,611)

COX-2 (872) vs.

naproxen (238)

Rofecoxib (244) vs.

naproxen (238) Potential confounder:*

Obesity (BMI30 vs. BMI<30)

-0.11 4.31 2.42 0.01

Aspirin use (use vs. non-use)

0.29 -0.08 -0.34 -1.28

Smoking (current vs. never)

-1.97 -2.41 -2.36 -0.61

Educational Attainment ( high school vs. >high school)

-2.36 -1.13 -3.67 -5.61

Income status ($20,000 vs. >$20,000)

-1.44 -1.08 -1.47 -1.65

Net confounding:

Sum of all negative biases: -5.88 -4.69 -5.08 -9.15

Weighted average: -1.56 -0.54 -1.86 -3.15

Sum of all positive biases: 0.29 4.31 -0.34 0.01

39

What does it mean?

Ray et al.: RR of 1.9 is an underestimation of the unconfounded RR by 5% (max) So the effect estimate corrected for 5 unobserved

confounders would be about 2.0

Solomon et al.: RR of 1.6 would move to 1.7

40

Sensitivity of Bias as a Function of a Misspecified RRCD :

-20

-15

-10

-5

0

5

10

15

20

1 1.5 2 2.5 3 3.5 4 4.5RRCD

Bia

s o

f R

RE

D i

n %

COX-2 vs. non-selective NSAIDsCOX-2 vs. non-usersCOX-2 vs. naproxenRofecoxib vs. naproxen

Literature estimate

RRCD = 1.7

Obesity (BMI >=30 vs. BMI<30)

41

Sensitivity towards a misspecified RRCO from the literature:OTC aspirin use (y/n)

-20

-15

-10

-5

0

5

10

15

20

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

RRCD

Bia

s o

f R

RE

D i

n %

COX-2 vs. non-selective NSAIDsCOX-2 vs. non-usersCOX-2 vs. naproxenRofecoxib vs. naproxen

Literature estimate

RRCD = 0.7

42

4. External AdjustmentGiven external information for selected factors on OREC from survey data and RRCD from the literature,

how much confounding is caused by not controling for these factors?

Data from Schneeweiss et al.: Assessment of bias by unmeasured confoundersin pharmacoepidemiologic claims data studies using external information. Epidemiology 2004, in press.

Unmeasured covariate: Aspirin (use vs. non-use) Bias as a function of misspecification of the RRCD from the literature:Data source: Lit MCBS MCBS assumed MCBSParameter: RRCD p(C) OREC true RRED p(E) app RRED CRR % biasSensitivity: varry const const const const calc calc

COX vs. 0.1 0.1 0.9 1 0.4 1.0093282 1.009 0.933

NSAID 0.2 0.1 0.9 1 0.4 1.0081979 1.008 0.820

0.3 0.1 0.9 1 0.4 1.0070929 1.007 0.709

0.4 0.1 0.9 1 0.4 1.0060124 1.006 0.601

0.5 0.1 0.9 1 0.4 1.0049555 1.005 0.496

0.6 0.1 0.9 1 0.4 1.0039215 1.004 0.392

0.7 0.1 0.9 1 0.4 1.0029096 1.003 0.291

0.8 0.1 0.9 1 0.4 1.0019192 1.002 0.192

0.9 0.1 0.9 1 0.4 1.0009495 1.001 0.095

1 0.1 0.9 1 0.4 1 1.000 0.000COX vs. 0.1 0.09 1.03 1 0.12 0.997608 0.998 -0.239

non-user 0.2 0.09 1.03 1 0.12 0.9978943 0.998 -0.211

0.3 0.09 1.03 1 0.12 0.9981752 0.998 -0.182

0.4 0.09 1.03 1 0.12 0.9984507 0.998 -0.155

0.5 0.09 1.03 1 0.12 0.998721 0.999 -0.128

0.6 0.09 1.03 1 0.12 0.9989863 0.999 -0.101

0.7 0.09 1.03 1 0.12 0.9992468 0.999 -0.075

0.8 0.09 1.03 1 0.12 0.9995024 1.000 -0.050

0.9 0.09 1.03 1 0.12 0.9997535 1.000 -0.025

1 0.09 1.03 1 0.12 1 1.000 0.000COX vs. 0.1 0.09 1.15 1 0.79 0.9892571 0.989 -1.074

naproxen 0.2 0.09 1.15 1 0.79 0.9905337 0.991 -0.947

0.3 0.09 1.15 1 0.79 0.9917884 0.992 -0.821

0.4 0.09 1.15 1 0.79 0.9930216 0.993 -0.698

0.5 0.09 1.15 1 0.79 0.9942339 0.994 -0.577

0.6 0.09 1.15 1 0.79 0.9954259 0.995 -0.457

0.7 0.09 1.15 1 0.79 0.996598 0.997 -0.340

0.8 0.09 1.15 1 0.79 0.9977507 0.998 -0.225

0.9 0.09 1.15 1 0.79 0.9988846 0.999 -0.112

1 0.09 1.15 1 0.79 1 1.000 0.000Rofecox vs. 0.1 0.1 1.6 1 0.51 0.9597175 0.960 -4.028

naproxen 0.2 0.1 1.6 1 0.51 0.9644945 0.964 -3.551

0.3 0.1 1.6 1 0.51 0.9691917 0.969 -3.081

0.4 0.1 1.6 1 0.51 0.9738113 0.974 -2.619

0.5 0.1 1.6 1 0.51 0.9783551 0.978 -2.164

0.6 0.1 1.6 1 0.51 0.9828249 0.983 -1.718

0.7 0.1 1.6 1 0.51 0.9872227 0.987 -1.278

0.8 0.1 1.6 1 0.51 0.99155 0.992 -0.845

0.9 0.1 1.6 1 0.51 0.9958085 0.996 -0.419

1 0.1 1.6 1 0.51 1 1.000 0.000

Unmeasured covariate: BMI (obese vs. non-obese)Data source: Lit MCBS MCBS assumed MCBSParameter: RRCD p(C) OREC true RRED p(E) app RRED CRR % bias

-20

-10

0

10

20

1 2 3 4 5 6 7 8

R RCD

-20-15-10-505101520

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

R RCD

-20

-10

0

10

20

1 1.5 2 2.5 3 3.5 4 4.5

R RCD

-20

-10

0

10

20

1 1.5 2 2.5 3 3.5 4 4.5

R RCD

-20

-10

0

10

20

1 1.5 2 2.5 3 3.5 4 4.5

R RCD

-20

-10

0

10

20

0 1 2 3 4 5 6 7 8

R RCD

COX-2 vs. non-selective NSAIDs

COX-2 vs. non-users

COX-2 vs. naproxen

Rofecoxib vs. naproxen

-20

-15

-10

-5

0

5

10

15

20

0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

RRCD

Bia

s o

f R

R ED i

n %

COX-2 vs. non-selective NSAIDsCOX-2 vs. non-users

COX-2 vs. naproxenRofecoxib vs. naproxen Literature estimate

RRCD = 0.7

43

Variance estimate of externally adjusted parameters

RR = ARR * F

Var (ARR) from main study

Var (F) from survey study and medical literature

Var (RR) = Var (ARR * F)

= ARR2 Var(F) + F2 Var(ARR)

44

Summary External Adjustment

This method provides a quantitative assessment of the effect of selected unobserved confounders

Easy to use (Excel program available from author)

MCBS is available from CMS for $900 per annual survey

Should be more frequently used in Pharmacoepi studies using claims data

45

Limitations (1)

Example is limited to 5 potential confounders No lab values, physical activity, blood pressure etc. What about the ‘unknow unknowns’?

We currently explore NHANES ’99/’00 Lab values, dietary suppl. (Ca2+), Drug data quality?

To assess the bias we assume an exposure–disease association of 1 (null hypothesis) The more the truth is away from the null the more bias in

our bias estimate However the less relevant unmeasured confounders

become

46

Limitations (2)

Validity depends on representativenes of sampling with regard to the unmeasured confounders

We could not consider the joint distribution of confounders

Limited to a binary world

47

Solving the Main Limitations

Need a method That addresses the joint distribution of several

unmeasured confounders That can handle binary, ordinal or normally distributed

unmeasured confounders Lin et al. (Biometrics 1998):

Can handle a single unmeasured covariate of any distribution

But can handle only 1 covariate Sturmer, Schneeweiss et al. (AJE 2005 in press):

Propensity Score Calibration can handle multiple unmeasured covariates of any distribution