70
Please take an i>clicker from the box in front of the room

Please take an i>clicker from the box in front of the room

Embed Size (px)

Citation preview

Page 1: Please take an i>clicker from the box in front of the room

Please take an i>clicker from the box in front of the room

Page 2: Please take an i>clicker from the box in front of the room

Classification Schemes for Error

• Szklo and Nieto– Bias (Systematic error)

• Selection Bias• Information/Measurement Bias

– Confounding– Chance (Random error)

• Other Common Approach– Bias (Systematic error)

• Selection Bias• Information/Measurement Bias• Confounding Bias

– Chance (Random error)

Think of the “BIG 4” in all of your work

Page 3: Please take an i>clicker from the box in front of the room

• Descriptive studies– Objective: Estimate measures of disease occurrence

(e.g., prevalence or incidence)

• Analytic studies– Objective: Estimate measures of association between

exposure (predictors) and outcome (e.g., disease)

Page 4: Please take an i>clicker from the box in front of the room

Bias in Clinical Research: Measurement Bias

• In descriptive studies

• In analytic studies

– Misclassification of dichotomous exposure & outcome variables

• non-differential misclassification

• differential misclassification

• magnitude and direction of bias

– Mis-measurement of interval scale variables

– Advanced topics (mention only)

• misclassification of multi-level categorical variables

• misclassification of confounding variables

• back-calculating to the truth

Page 5: Please take an i>clicker from the box in front of the room

Measurement Bias

• Definition– bias that is caused when any measurement collected about or

from subjects is not completely reproducible or valid (accurate)• any type of variable: exposure, outcome, or confounder

– aka: misclassification bias; information bias (S&N text); identification bias

• misclassification is what happens when there is error in measurement of a categorical variable, for which everyone is “classified”

Page 6: Please take an i>clicker from the box in front of the room

Misclassification of Dichotomous Variables

Page 7: Please take an i>clicker from the box in front of the room

What does d/(b+d) refer to?

Sensit

ivity

- A

Positiv

e pr

edict

ive va

lue -

C

Specif

icity

- B

Negat

ive p

redic

tive

value

- D

Characterizing the Measurement of a Dichotomous Variable (e.g., present vs absent)

Page 8: Please take an i>clicker from the box in front of the room

What does d/(b+d) refer to?

Sensit

ivity

- A

Positiv

e pr

edict

ive va

lue -

C

Specif

icity

- B

Negat

ive p

redic

tive

value

- D

Characterizing the Measurement of a Dichotomous Variable (Present vs Absent)

Page 9: Please take an i>clicker from the box in front of the room

Terms Used to Characterize Measurement/Classification of Dichotomous Variables (Terms for Validity)

• Sensitivity

– the ability of a measurement to identify correctly those who

HAVE the characteristic (disease or exposure) of interest.

• Specificity

– the ability of a measurement to identify correctly those who

do NOT have the characteristic of interest

• Applies to any dichotomous variable, not just diagnoses

Gold Standard Present Absent

Your Present a b Measurement Absent c d

Sensitivity = a/(a+c) Specificity = d/(b+d)

Positive predictive value = a/(a+b)

Negative predictive value = d/(c+d)

Page 10: Please take an i>clicker from the box in front of the room

Causes for Misclassification• Questionnaire problems

– inaccurate recall– socially desirable responses– ambiguous questions– under or overzealous interviewers

• Biological specimen collection– problems in specimen collection or processing or storage

• Biological specimen testing– inherent limits of detection– faulty instruments

• Data management problems in coding• Study design or analytic problems (See Problem Set)

– incorrect time period assessed, particularly for exposure– lumping of outcome variables (composite variables)

Page 11: Please take an i>clicker from the box in front of the room

SOURCE POPULATION = CALIFORNIA

STUDY SAMPLE = PRE-ELECTION POLL(Field Poll)

Descriptive Study: Measurement Bias

Deukmejian

Bradley +7%

1982 California Governor Election

Page 12: Please take an i>clicker from the box in front of the room

SOURCE POPULATION = CALIFORNIA

STUDY SAMPLE = PRE-ELECTION POLL(Field Poll, one of the largest pre-election surveys)

Descriptive Study: Measurement Bias

“Bradley Effect” = Respondents who favored Deukmejian sought to avoid

appearing racist and hence did not state true choice in pre-election survey

Deukmejian

Deukmejian 49% Bradley +7%

Bradley 48%

1982 California Governor Election

Page 13: Please take an i>clicker from the box in front of the room

SOURCE POPULATION

STUDY SAMPLE

Contrast with Selection Bias

Uneven dispersion of arrows

e.g., Dewey backers were

over-represented

Page 14: Please take an i>clicker from the box in front of the room

Descriptive Biomedical Studies: Measurement Bias

• e.g., Prevalence of:– Flossing– Condom use– Exercise– Etc.

• “Social desirability bias”– Humans tend to give socially desirable responses

Page 15: Please take an i>clicker from the box in front of the room

Bias in Clinical Research: Measurement Bias

• Measurement bias in descriptive studies

• Measurement bias in analytic studies

– Misclassification of dichotomous exposure & outcome variables

• non-differential misclassification

• differential misclassification

• magnitude and direction of bias

– Mismeasurement of interval scale variables

– Advanced topics (mention only)

• misclassification of multi-level categorical variables

• misclassification of confounding variables

• back-calculating to the truth

Page 16: Please take an i>clicker from the box in front of the room

Diseased

Exposed

+ -

+

-

SOURCE POPULATION

STUDY SAMPLE

Non-Differential Misclassification of Exposure: Imperfect Sensitivity

Problems with sensitivity in measurement of exposure - independent of disease status

e.g., case-control study

exposure = alcohol abuse

Evenly weighted arrows =

non-differential

Page 17: Please take an i>clicker from the box in front of the room

Non-differential Misclassification of Exposure

Truth: No misclassification (100% sensitivity/specificity)

Exposure Cases ControlsYes 50 20No 50 80

OR= (50/50)/(20/80) = 4.0

Presence of 70% sensitivity in exposure classification

Exposure Cases ControlsYes 50-15=35 20-6=14No 50+15=65 80+6=86

OR= (35/65)/(14/86) = 3.3

Effect of non-differential misclassification of dichotomous exposures: Bias “toward the null” value of 1.0

Page 18: Please take an i>clicker from the box in front of the room

Diseased

Exposed

+ -

+

-

SOURCE POPULATION

STUDY SAMPLE

Non-Differential Misclassification of Exposure: Imperfect Specificity

Problems with specificity of exposure measurement - independent of disease status

e.g., exposure = self-reported second-hand smoke exposure

Page 19: Please take an i>clicker from the box in front of the room

Non-differential Misclassification of Exposure

Truth: No misclassification (100% sensitivity/specificity)

Exposure Cases ControlsYes 50 20No 50 80

OR= (50/50)/(20/80) = 4.0

Presence of 70% specificity in exposure classification

Exposure Cases ControlsYes 50+15=65 20+24=44No 50-15=35 80-24=56

OR= (65/35)/(44/56) = 2.4

Effect of non-differential misclassification of dichotomous exposures: Bias toward the null value of 1.0

Page 20: Please take an i>clicker from the box in front of the room

Diseased

Exposed

+ -

+

-

SOURCE POPULATION

STUDY SAMPLE

No misclassification

e.g., exposure = self-reported second-hand smoke exposure

50

50

20

80 OR = 4.0

Page 21: Please take an i>clicker from the box in front of the room

Diseased

Exposed

+ -

+

-

SOURCE POPULATION

STUDY SAMPLE

Non-Differential Misclassification of Exposure: Imperfect Specificity

e.g., exposure = self-reported second-hand smoke exposure

OR = 2.4

65

50 35

44

80 56

differences become blurred

Page 22: Please take an i>clicker from the box in front of the room

Diseased

Exposed

+ -

+

-

SOURCE POPULATION

STUDY SAMPLE

Non-Differential Misclassification of Exposure: Imperfect Specificity and Sensitivity

Problems with sensitivity - independent of disease status

Problems with specificity - independent of disease status

Page 23: Please take an i>clicker from the box in front of the room

Assuming no sampling error, what will the observed OR be?

OR = 3

.1 -

A

OR = 2

.4

- C

OR = 2

.8

- B

OR = 1

.6

- D

Exposure Cases ControlsYes 50 20No 50 80

True OR = (50/50) / (20/80) = 4.0

But, now assume non-differential exposure misclassification problemswith both sensitivity and specificity

Sensitivity = 70%Specificity = 70%

OR = 1

.2

- E

Page 24: Please take an i>clicker from the box in front of the room

Assuming no sampling error, what will the observed OR be?

OR = 3

.1 -

A

OR = 2

.4

- C

OR = 2

.8

- B

OR = 1

.6

- D

Exposure Cases ControlsYes 50 20No 50 80

True OR = (50/50) / (20/80) = 4.0

But, now assume non-differential exposure misclassification problemswith both sensitivity and specificity

Sensitivity = 70%Specificity = 70%

OR = 1

.2

- E

Page 25: Please take an i>clicker from the box in front of the room

Exposure Cases ControlsYes 50 20No 50 80 True OR = (50/50) / (20/80) = 4.0

True Cases Controls Distribution exp unexp exp unexp (gold standard) 50 50 20 80

Study distribution: Cases ControlsExposed 35 15 50 14 24 38Unexposed 15 35 50 6 56 62

sensitivity 0.70 0.70 0.70 0.70 or specificity

Exposure Cases ControlsYes 50 38No 50 62 Observed OR = (50/50) / (38/62) = 1.6

Non-Differential Misclassification of Exposure: Imperfect Sensitivity and Specificity

SOURCE POPULATION

STUDYSAMPLE

Sensitivity = 0.7

Specificity = 0.7

Page 26: Please take an i>clicker from the box in front of the room

Assuming no sampling error, what will the observed OR be?

OR = 3

.5 -

A

OR = 3

.0

- C

OR = 3

.2

- B

OR = 2

.8

- D

Exposure Cases ControlsYes 50 20No 50 80

True OR = (50/50) / (20/80) = 4.0

But, now assume non-differential exposure misclassification problemswith both sensitivity and specificity

Sensitivity = 90%Specificity = 80%

OR = 2

.4

- E

Page 27: Please take an i>clicker from the box in front of the room

Assuming no sampling error, what will the observed OR be?

OR = 3

.5 -

A

OR = 3

.0

- C

OR = 3

.2

- B

OR = 2

.8

- D

Exposure Cases ControlsYes 50 20No 50 80

True OR = (50/50) / (20/80) = 4.0

But, now assume exposure misclassification problemswith both sensitivity and specificity

Sensitivity = 90%Specificity = 80%

OR = 2

.4

- E

Page 28: Please take an i>clicker from the box in front of the room

Non-Differential Misclassification of Exposure: Imperfect Sensitivity and Specificity

Exposure Cases ControlsYes 50 20No 50 80 True OR = (50/50) / (20/80) = 4.0

True Cases Controls Distribution exp unexp exp unexp (gold standard) 50 50 20 80

Study distribution: Cases ControlsExposed 45 10 55 18 16 34Unexposed 5 40 45 2 64 66

sensitivity 0.90 0.80 0.90 0.80 or specificity

Exposure Cases ControlsYes 55 34No 45 66 Observed OR = (55/45) / (34/66) = 2.4

SOURCE POPULATION

STUDYSAMPLE

Sensitivity = 0.9

Specificity = 0.8

Seemingly respectable Sn and Sp result in

substantial bias

Page 29: Please take an i>clicker from the box in front of the room

Non-Differential Misclassification of Exposure: Imperfect Sensitivity & Specificity and Uncommon Exposure

Exposure Cases ControlsYes 50 20No 500 800 True OR = (50/500) / (20/800) = 4.0

True Cases Controls Distribution exp unexp exp unexp (gold standard) 50 500 20 800

Study distribution: Cases ControlsExposed 45 100 145 18 160 178Unexposed 5 400 405 2 640 642

sensitivity 0.90 0.80 0.90 0.80 or specificity

Exposure Cases ControlsYes 145 178No 405 642 Observed OR = (145/405) / (178/642) = 1.3

SOURCE POPULATION

STUDYSAMPLE

e.g., radon exposure

Sensitivity = 0.9

Specificity = 0.8

Higher exposure prevalence is more balanced and more resilient to misclassification

Page 30: Please take an i>clicker from the box in front of the room

Non-differential Misclassification of Exposure: Magnitude of Bias on the Odds Ratio

True OR=4.0

2.20.080.900.90

2.80.200.900.90

3.00.370.900.90

1.90.200.600.90

3.20.200.950.90

1.90.200.850.60

2.60.200.850.90

Observed ORPrev of Exp in controls

SpecificitySensitivity

Page 31: Please take an i>clicker from the box in front of the room

Bias as a function of non-differential imperfect sensitivity and specificity of exposure measurement

0.9

0.7

0.5

Sensitivity of exposure measurement

Specificity of exposure measurement

Copeland et al. AJE 1977

True OR = 2.67

Case-control study

Prevalence of exposure in controls = 0.2

Ap

par

ent

Od

ds

Rat

io

2.8

2.5

2.2

1.9

1.6

1.3

1.0

.50 .55 .60 .65 .70 .75 .80 .85 .90 .95 1.00

Page 32: Please take an i>clicker from the box in front of the room

Bias as a function of non-differential imperfect sensitivity and specificity of exposure measurement

0.9

0.7

0.5

Sensitivity of exposure measurement

Specificity of exposure measurement

Copeland et al. AJE 1977

True OR = 2.67

Prevalence of exposure in controls = 0.2

Ap

par

ent

Od

ds

Rat

io

2.8

2.5

2.2

1.9

1.6

1.3

1.0

.50 .55 .60 .65 .70 .75 .80 .85 .90 .95 1.00

When does OR fall

below 2?

Page 33: Please take an i>clicker from the box in front of the room

Non-Differential Misclassification of Exposure in a Cohort Study: Effect of Sensitivity, Specificity and Prevalence of Exposure

Flegal et al. AJE 1986

True Risk Ratio = 10

App

aren

t Ris

k R

atio

U = sensitivity; V = specificity

All RR < 8

If Pe >.25, ↑ Sn. influ.

Dependence upon Prev.

Sn Sn Sn Sn Sn

Sp Sp Sp Sp Sp

Sn Sp

Sn and Sp

Page 34: Please take an i>clicker from the box in front of the room

In the presence of non-differential misclassification of exposure (e.g., sensitivity and specificity of 80%), what can we say in our Discussion section about any measures of association derived from the exposure?

Are a

n un

dere

stim

ate

of tr

uth

- A

Tend

to u

nder

estim

ate

truth

- C

Will,

on

aver

age,

unde

resti

mat

e tru

th -

B

Are a

n ov

eres

timat

e of

trut

h -

D

Need

mor

e inf

orm

ation

- E

Page 35: Please take an i>clicker from the box in front of the room

Are a

n un

dere

stim

ate

of tr

uth

- A

Tend

to u

nder

estim

ate

truth

- C

Will,

on

aver

age,

unde

resti

mat

e tru

th -

B

Are a

n ov

eres

timat

e of

trut

h -

D

Need

mor

e inf

orm

ation

- E

In the presence of non-differential misclassification of exposure (e.g., sensitivity and specificity of 80%), what can we say in our Discussion section about any measures of association derived from the exposure?

Page 36: Please take an i>clicker from the box in front of the room

Non-differential misclassification of exposure and “Bias towards the null”

• In any single study, non-differentiality by itself does not guarantee that the observed measure of association is falsely low

• Reason: in any single study, the observed results are a function of bias plus CHANCE– Only if a study is repeated many times over and the findings averaged, can we

say that the observed measure of association is biased towards the null

• Don’t say: “Because we had non-differential misclassification of exposure, our findings are an underestimate of the true measure of association.” (i.e., do not be definitive)

• Instead, say: “Because imperfect sensitivity and specificity of <the exposure measurement> was generally the same irrespective of outcome, our findings tend to be (or, “on average are likely to be) an underestimate of the true association.”

Jurek et al. IJE 2005

Page 37: Please take an i>clicker from the box in front of the room

Non-Differential Misclassification of Exposure: Rules of Thumb Regarding Sensitivity & Specificity

Exposure Cases ControlsYes 50 100No 50 300 True OR = (50/50) / (100/300) = 3.0

SOURCE POPULATION

Sens + Spec = 1 gives OR = 1 (no effect)

Sensitivity Specificity Observed OR

0.8 1.0 2.6

0.8 0.8 1.9

0.4 0.6 1.0

0.4 0.4 0.82

0 0 0.33

Sens + Spec >1 but <2 gives attenuated effect

Sens + Spec < 1 gives reversal of effect

Coding error

Page 38: Please take an i>clicker from the box in front of the room

Diseased

Exposed

+ -

+

-

SOURCE POPULATION

STUDY SAMPLE

Non-Differential Misclassification of Outcome

Problems with outcome sensitivity -independent of exposure status

Problems with outcome specificity - independent of

exposure status

Evenly weighted arrows =

non-differential

Page 39: Please take an i>clicker from the box in front of the room

Bias as a function of non-differential imperfect sensitivity and specificity of outcome measurement in a cohort study

Sensitivity of outcome measurement0.9

0.7

0.5

Specificity of outcome measurementCopeland et al. AJE 1977

True risk ratio = 2.0

Cumulative incidence in unexposed = 0.05

Steep bias with change in

specificity

Relatively less influence

from sensitivity

App

aren

t Ris

k R

atio

Page 40: Please take an i>clicker from the box in front of the room

Non-Differential Misclassification of Outcome: Effect of Incidence of Outcome

Copeland et al. AJE 1977

Specificity of outcome measurement

0.2 0.1

0.1 0.05

0.05 0.025

Cumulative incidence of outcome

Exposed Unexposed

True risk ratio = 2.0

Sensitivity of outcome measurement held fixed = 0.9

App

aren

t Ris

k R

atio

Page 41: Please take an i>clicker from the box in front of the room

Special Situation In a Cohort or Cross-sectional Study

Misclassification of outcome• If specificity of outcome measurement is 100%• Any degree of imperfect sensitivity, if non-differential, will not

bias the risk ratio or prevalence ratio• e.g.,

• Risk difference, however, is changed by a factor of (1 minus sensitivity), in this example, 30% (truth=0.1; biased = 0.07)

DiseaseNoDisease

Exposed 20 80 100Unexposed 10 90 100

2.0

1001010020

ratio )prevalence (or Risk

DiseaseNoDisease

Exposed 20-6=14 80+6=86100Unexposed 10-3=7 90+3=93100

2.0

1007

10014

ratio )prevalence (or Risk

Truth

70% sensitivity

Page 42: Please take an i>clicker from the box in front of the room

When specificity of outcome is 100% in a cohort or cross-sectional study

Sensitivity of outcome measurement0.9

0.7

0.5

Specificity of outcome measurementCopeland et al. AJE 1977

True risk ratio = 2.0

Cumulative incidence in unexposed = 0.05

App

aren

t Ris

k R

atio

Page 43: Please take an i>clicker from the box in front of the room

When specificity of outcome measurement is 100% in a cohort or cross sectional study

• Worth knowing about when defining outcomes, such as choosing cutoffs for continuous variables on ROC curves

• Choosing most specific cutoff (or 100% cutoff) will lead to least biased ratio measures of association

0.0 0.1 0.2 0.3 0.4 0.5

0.5

0.6

0.7

0.8

0.9

1.0

1.0 0.9 0.8 0.7 0.6 0.5

Sen

sit

ivit

y

1 - Specificity

00.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

OD: 0.06 Specificity: 84 % Sensitivity: 100 %

OD: 0.19 Specificity: 95 % Sensitivity: 94 %

OD: 0.49 Specificity: 100 % Sensitivity: 74 %

0.0 0.1 0.2 0.3 0.4 0.5

0.5

0.6

0.7

0.8

0.9

1.0

1.0 0.9 0.8 0.7 0.6 0.5

Sen

sit

ivit

y

1 - Specificity

00.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

OD: 0.06 Specificity: 84 % Sensitivity: 100 %

OD: 0.19 Specificity: 95 % Sensitivity: 94 %

OD: 0.49 Specificity: 100 % Sensitivity: 74 %

OD: 0.06 Sensitivity = 100%OD: 0.06 Specificity = 84%

OD: 0.19 Sensitivity = 94%OD: 0.19 Specificity = 95%

OD: 0.49 Sensitivity = 74%OD: 0.49 Specificity = 100%

1.0 0.9 0.8 0.7 0.6 0.5

Specificity

Example of ROC curve

Page 44: Please take an i>clicker from the box in front of the room

What should you choose as your primary outcome variable?

>5 da

ys o

f cou

gh -

A

Abnor

mal

ches

t x-ra

y - C

Micr

obiol

ogic

diagn

osis

of

pertu

ssis

- B

>5 da

ys o

f cou

gh +

micr

obiol

ogic

diagn

osis

of p

ertu

ssis

- D

Efficacy of a pertussis (whooping cough) vaccine in adults

• RCT of: Approved (in kids) pertussis vaccine vs. control vaccine for the prevention of pertussis in adults

Group No. of subjects Person-years Pertussis vaccine 1391 2421 Control 1390 2444

Ward et al. NEJM 2005

Page 45: Please take an i>clicker from the box in front of the room

What should you choose as your primary outcome variable?

>5 da

ys o

f cou

gh -

A

Abnor

mal

ches

t x-ra

y - C

Micr

obiol

ogic

diagn

osis

of

pertu

ssis

- B

>5 da

ys o

f cou

gh +

micr

obiol

ogic

diagn

osis

of p

ertu

ssis

- D

Efficacy of a pertussis (whooping cough) vaccine in adults

• RCT of: Approved (in kids) pertussis vaccine vs. control vaccine for the prevention of pertussis in adults

Group No. of subjects Person-years Pertussis vaccine 1391 2421 Control 1390 2444

Ward et al. NEJM 2005

Page 46: Please take an i>clicker from the box in front of the room

Efficacy of a pertussis (whooping cough) vaccine in adults

• Outcome: Cough > 5 days– No. of events: 2672 (and lots of statistical power)– Result: No significant difference between groups

• Outcome: Cough + microbiologic pertussis confirmation– No. of events: 10– Result: rate ratio = 0.08 (92% vaccine efficacy) (95% CI = 0.01 to 0.68)

• Acellular vaccine vs. control vaccine for the prevention of pertussis in adults (Ward et al. NEJM 2005)

Group No. of subjects Person-years Pertussis vaccine 1391 2421 Control 1390 2444

Page 47: Please take an i>clicker from the box in front of the room

Pervasiveness of Non-Differential Misclassification

• Direction of this bias is typically towards the null

• Therefore, called a “conservative” bias

• Goal, however, is to get the truth

• Consider how much underestimation of effects must be occurring in research

• How many “negative” studies are truly “positive”?

Page 48: Please take an i>clicker from the box in front of the room

Differential Misclassification of ExposureWeinstock et al. AJE 1991• Nested case-control study in Nurses Health Study cohort

• Cases: women with new melanoma diagnoses

• Controls: women w/out melanoma - by incidence density sampling

• Measurement of exposure: questionnaire about self-reported

“tanning ability”; administered shortly after melanoma development

Melanoma

No Melanoma

No tan to light tan 15 77 Medium to dark tan 19 157

1.6

157771915

OR

Page 49: Please take an i>clicker from the box in front of the room

• Question asked after diagnosis

• Question asked before diagnosis (NHS baseline)

MelanomaNoMelanoma

No tan to light tan 9 79Med to dark tan 25 155

0.7

15579259

OR

MelanomaNoMelanoma

No tan to light tan 15 77Med to dark tan 19 157

1.6

157771915

OR

Virtually unchanged

Substantially changed

Page 50: Please take an i>clicker from the box in front of the room

Melanoma

Tanningability

+ -

No

Yes

SOURCE POPULATION

STUDY SAMPLE

“Tanning Ability” and Melanoma:

Differential Misclassification of Exposure

Imperfect specificity of exposure measurement - mostly in cases

Bias away from the null

leading to spurious

association

Page 51: Please take an i>clicker from the box in front of the room

Congenital Malformation

Exposed

+ -

+

-

SOURCE POPULATION

STUDY SAMPLE

Differential Misclassification of Exposure:

Exposures During Pregnancy and Congenital Malformations

Cases more likely than controls to remember a variety of exposures

Cases might be more likely than controls to falsely state a

variety of exposures

Uneven weighting of

arrows = differential

Page 52: Please take an i>clicker from the box in front of the room

Differential Misclassification of Exposure: Magnitude of Bias on the Odds Ratio

True OR=3.9

Exposure Classification

Sensitivity Specificity

Cases Controls Cases Controls OR

0.90 0.60 1.0 1.0 5.79

0.60 0.90 1.0 1.0 2.22

1.0 1.0 0.9 0.70 1.00

1.0 1.0 0.7 0.90 4.43

Prevalence of Exposure in Controls = 0.1

Page 53: Please take an i>clicker from the box in front of the room

Misclassification of Dichotomous Exposure or Outcome: Summary of Bias

Misclassification Bias on Ratio Measure of Association

Non-differential Exposure Towards null Outcome Towards null*

Differential

Exposure Away or towards null Outcome Away or towards null

*Exception: When specificity is 100%, no effect on risk ratio or prevalence ratio regardless of sensitivity

Page 54: Please take an i>clicker from the box in front of the room

Relating Last Week to This Week:Relating Validity / Reproducibility of Individual

Dichotomous Measurements to Measurement Bias in Inferences in Analytic Studies

• Validity– How sensitivity and specificity of a measurement results

in measurement bias covered in prior slides

• Reproducibility– Recall that a measurement with imperfect reproducibility

will typically lack perfect validity when used in practice -- (unless it is repeated many many times)

Page 55: Please take an i>clicker from the box in front of the room

Reproducibility and Validity of a Measurement

With only one shot at the measurement, most of the time you

will be off the center of the target

Page 56: Please take an i>clicker from the box in front of the room

GoodB-Ball

PoorB-Ball

>6 ft 10 30 40 +1 10 +3 30<6 ft 10 50 60 10 +1 50 +5

20 80 100 20 80

P

GoodB-Ball

PoorB-Ball

>6 ft 10 32 42<6 ft 10 48 58

20 80 100

Truth = Prevalence Ratio= (10/40) / (10/60) = 1.5

Observed = Prevalence Ratio = (10/42) / (10/58) = 1.38

10% Misclassification

Imperfect reproducibility leads

to 90% sensitivity and 90% specificity of

height measurement –non-differential with respect to outcome

Page 57: Please take an i>clicker from the box in front of the room

Reproducibility

of Measurement

Validity of

Measurement in Practice

Validity of

Analytic Inferences

Derived from Measurement

“Measurement Bias”Systematic

Error of

Measurement

Page 58: Please take an i>clicker from the box in front of the room

Bias in Clinical Research: Measurement Bias

• Measurement bias in descriptive studies

• Measurement bias in analytic studies

– Misclassification of dichotomous exposure & outcome variables

• non-differential misclassification

• differential misclassification

• magnitude and direction of bias

– Mismeasurement of interval scale variables

– Advanced topics (mention only)

• misclassification of multi-level categorical variables

• misclassification of confounding variables

• back-calculating to the truth

Page 59: Please take an i>clicker from the box in front of the room

Effect of Lack of Validity and Reproducibility in Interval Scale Measurements

• Lack of Validity (Systematic Error)– Measurements systematically off truth by some multiplicative factor or absolute difference

x xtruth observed

xxx x x x xtruth observedobserved

• Lack of Reproducibility– Measurements off truth by some random factor or difference

Page 60: Please take an i>clicker from the box in front of the room

Relating the Validity and Reproducibility of Measurements to Measurement Bias in Analytic Studies – Interval Scale Variables

Validity (Systematic error)• Result moves systematically up or down scale by given factor or absolute

difference• e.g., systematic error in an interval scale outcome variableMean Ratio of Means Difference in Means

Truth Exposed 100

Unexposed 50 2 50

Measurement off by factor of 10 Exposed 1000

Unexposed 500 2 500

Measurement off by difference of 10 Exposed 110

Unexposed 60 1.83 50

Bias depending

upon measure of association

Page 61: Please take an i>clicker from the box in front of the room

Relating the Reproducibility and Validity of Measurements to Measurement Bias in Analytic Studies – Interval Scale Variables

Reproducibility (Random error)

e.g., random error in an exposure variableAssuming:

• Exposure is normally distributed with variance, 2True

• Random error is normally distributed with variance, 2E

• Then, the observed regression coefficient is equal to the true regression coefficient times:

• i.e., the greater the measurement error, the greater the attenuation (bias) towards the

null (e.g., if ICC is 0.5, the measure of association is halved)

22

2

ETrue

True

(i.e. reproducibility, the intraclass correlation coefficient)

Truth and Error

Truth

Regression Dilution Bias

Page 62: Please take an i>clicker from the box in front of the room

Relating the Reproducibility and Validity of Measurements to Measurement Bias in Analytic Studies – Interval Scale Variables

• See Extra Slides for Additional Examples

Page 63: Please take an i>clicker from the box in front of the room

Advanced Topics• Misclassification of multi-level categorical exposure variables

– some of the rules change regarding direction of bias– See Extra Slides for examples

• Mis-measurement of confounding variables– When confounding variables are mis-measured, the net result is failure

to fully control (adjust) for that variable• You are left with “residual confounding”• You have not fully adjusted for the variable• “Adjusted” measures of association may be over or under-estimated

– Very common problem• Researchers focus mainly on optimal measurement of exposure & outcome• By the time confounders surface, everyone is too exhausted

– e.g., when controlling for smoking, does classification of people into smokers and non-smokers based on current smoking capture the essence of the exposure?

Page 64: Please take an i>clicker from the box in front of the room

Advanced Topics

• Back-calculating to unbiased results– thus far, truth about measurement quality and the relationships

between exposure/outcome variables have been assumed • We have then predicted what the bias will be in observed results

– In practice, we have observed results and sometimes a guess about the measurement quality

– when extent of classification errors (e.g., PPV, NPV, sensitivity & specificity, ICC) are known, it is possible to back-calculate to truth

– if exact classification errors are not known, it is possible to perform sensitivity analyses to estimate a range of study results given a range of possible classification errors

– “Quantitative bias analysis”

Page 65: Please take an i>clicker from the box in front of the room

Poor Reproducibility

Poor Validity

Good Reproducibility

Good Validity

Managing Measurement Bias• Prevention and avoidance are critical

– study design phase is critical – use state-of-the-art techniques, blinding, SOPs, and replicates

• Little to be done after study (but back-correction may be possible)

• Become an expert in the measurement of your primary variables

• For the other variables, seek out the advice of experts (teams)

• Optimize the reproducibility/validity of your measurements!

Page 66: Please take an i>clicker from the box in front of the room

Extra slides

Page 67: Please take an i>clicker from the box in front of the room

Mismeasurement of Interval Scale Variables: Summary of Bias When Relating to Perfectly Measured Variables

Issue Bias on Measure of Association

Imperfect validity (systematic error)

Exposure Depends upon characterization of exposure (per unit or per factor change)

Outcome Depends upon measure (ratio or difference) Imperfect reproducibility (random error)

Exposure Towards null (“regression dilution”)* Outcome No effect*

*Also affects precision

Page 68: Please take an i>clicker from the box in front of the room

• Correlating one interval scale measurement to another

– e.g., weight and cholesterol

• Correlation is attenuated directly proportional to ICC of measurements (r = correlation coefficient)

– e.g., If ICC of both weight and cholesterol is 0.80

– 20% attenuation

yxtrueyxobservedyx ICCICCrr ;, ;,

Relating the Reproducibility of Measurements to Measurement Bias in Analytic Studies – Interval Scale Variables

)8.0)(8.0( ; , ; , truelcholesteroweightobservedlcholesteroweight rr

)8.0( ; , ; , truelcholesteroweightobservedlcholesteroweight rr

Page 69: Please take an i>clicker from the box in front of the room

Non-differential Misclassification of Multi-level Exposure

Cases ControlsOddsRatio

None 100 100 1.0

Low 200 100 2.0

High 600 100 6.0

Exposure

Cases ControlsOddsRatio

None 100 100 1.0

Low 440 140 3.1

High 360 60 6.0

Misclassification between adjacent exposure categoriesTruth

Bias away from the nullDosemeci et al. AJE 1990

Page 70: Please take an i>clicker from the box in front of the room

Misclassification of Multi-level Exposure

Cases ControlsOddsRatio

None 100 100 1.0

Low 200 100 2.0

High 600 100 6.0

Exposure

Cases ControlsOddsRatio

None 420 180 1.0

Low 120 60 0.86

High 360 60 2.57

Misclassification between adjacent and non-adjacent exposure categories

Truth

Appearance of J-shaped relationshipDosemeci et al. AJE 1990