Epid 600 Class 11 Screening

8/14/2019 Epid 600 Class 11 Screening

1/65

EPID 600; Class 11Screening

University of Michigan School of Public Health

1


2/65

The New York TimesSunday, October 31, 1999; pg. 5

Bedtime stories. Telephone bills. Life as usual. It ends quickly with the trauma of a breast biopsy

even though most breast biopsies turn out to be benign. This fact has inspired clinical trials of an

adjunctive breast screening device designed to distinguish benign from malignant lesions without a

breast biopsy. So, life can return to normal for a little sooner for everyone. We invite you to help us.

If youre scheduled for a breast biopsy, ask your doctor about participating in our clinical trials. 2


3/65

Why screen?

To find people with the disease (or at risk of the disease)

who dont know it

In other wordsto find people who are pre-symptomatic

3


4/65

Why try to find asymptomatic diseased

people?

To treat disease

To cure disease

To prevent disease spread

To slow down disease progress

To study disease natural history

4


5/65

Different from identifying people at risk

but without disease

Identifying people at risk of disease but without disease is

done to prevent the disease altogether, to delay disease

onset, or to study the precondition state

5


6/65

Additional thought...

Why else might we encouragescreening or promote a

specific screening test?

6


7/65

Additional thought...

Why else might we encouragescreening or promote a

specific screening test?

Because we want to do somethingBecause we can

For money, fame, and glory

7


8/65

A digression (1)....

What is a disease?

Colon cancer

Myocardial infarction

What is a condition?

High blood pressure

High cholesterol

What is a marker?

High Prostate Specific Antigen

8


9/65

A digression (2)...

Binary tests

Yes vs No

Continuous measures

Multiple values; may require the choice of a cutoff point

9


10/65

A note: primary vs. secondary

prevention

Primary prevention

Screening that aims to identify risk factors or etiologic

factors for disease so that disease occurrence can be

prevented

Secondary prevention

The early detection of disease in the hope of improvingprognosis

10


11/65

Natural history of a disease

Detectable

Preclinical Phase(DPCP)

Onset Symptomatic DeathDetectableby

Screening

11


12/65

Test

?

Pos

Neg

Screening

Disease

POSITIVES

NEGATIVES

12


13/65

Yes No

Pos

Neg

TESTTP FP

FN TN

DISEASE

Screening

13


14/65

Sensitivity (Sn)

Probability of test positive if disease is present

TP true positives

Sn = =

TP + FN everyone with disease

14


15/65

Specificity (Sp)

Probability of a negative test if disease is not present

TN true negatives

Sp = =TN + FP everyone without disease

15


16/65

Sensitivity and specificity

Sensitivity and Specificity are characteristics of TEST itself,

i.e., how good is the test

Changing cutoffs generally increases one at the expense of

the other

16


17/65

0 6 8 10 12 14

Disease

Test Yes No

Pos (+) TP FP

Neg () FN TN

Changing cutoffs

Disease

No Disease


18/65

No disease

0 6 8 10 12 14

Disease

Test Yes No

Pos (+) 5 FP

Neg () FN TN

Changing cutoffs

Disease


19/65

Disease

No disease

0 6 8 10 12 14

Disease

Test Yes No

Pos (+) 5 3

Neg () FN TN

Changing cutoffs


20/65


21/65

Disease

No disease

0 6 8 10 12 14

91Neg ()

35Pos (+)

NoYesTest

Disease

Changing cutoffs


22/65

Disease

No disease

0 6 8 10 12 14

Disease

Test Yes No

Pos (+) 5 3

Neg () 1 9

Sn = 5/(5+1) = 0.83

Sp = 9/(9+3) = 0.75

Changing cutoffs


23/65

Disease

No disease

0 6 8 10 12 14

Disease

Test Yes No

Pos (+) TP FP

Neg () FN TN

Changing cutoffs


24/65

Disease

No disease

0 6 8 10 12 14

Disease

Test Yes No

Pos (+) 3 1

Neg () 3 11

Sn = 3/(3+3) = 0.50

Sp = 11/(11+1) = 0.92

Changing cutoffs


25/65

Score on Screen

NumberScre

ened

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

Non-

CasesCases

25


26/65

Score on Screen

NumberScreened

Screening Level Set at >5

Screening Level Set at >7

Cases

Non-

Cases

Overlapping Area

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16

26


27/65

What is gold standard that actually determines if disease is

present or not?

Cost of false positives and false negatives

Anxiety/emotional distressInconvenience

Subsequent testing and mortality

Issues about sensitivity vs. specificity

27


28/65

yes no

Disease

TP FP

TN

TN

FP +TN

TP

TP + FN

Sensitivity = Specificity =

FN

Classification of test results

28


29/65

Characteristics of tests

Validity (accuracy)

How close does the test result get to the correct (true) number

Reliability (precision)How close are repeat measurements on the same sample?

29


30/65

Validity vs Reliability

XXXX

X X

X

X

X

XXXX

X

XX

X

X

Valid and

reliable

Valid but not

reliable

Not valid but

reliable

Not valid and not

reliable

Well calibratedscale

Allowed to settlebefore measurement

recorded

Well calibratedscale

Not allowed tosettle before

measurementrecorded

Scale 6oz offAllowed to settlebefore measurement

recorded

Scale 6oz offNot allowed tosettle before

measurement

recorded

Baby scale examples

Truth = 8lbs Biased = 7lbs 6oz

X X

30


31/65

Four sources of variability

Biological variation

Test method itself

Intra-observer

Inter-observer

31


32/65

BP

Lowest

Highest

Casual

Patient A

86/47

126/79

108/64

Patient C

123/78

153/107

137/103

Example...blood pressure variability

32


33/65

If we screen apopulation, what percent of people with

the disease, and without the disease, will be correctly

identified by our test?

How well does the test work in a population?

Question addressed so far...

33


34/65

If a specific patient has a positive test, what is the

probability that this patient really has the disease?

The clinical question however is

34


35/65

Yes No

Pos

Neg

TESTTP FP

FN TN

DISEASE

Screening...

35


36/65

Likelihood that disease is present IF test is positive

TP true positives

PPV = =TP + FP all positives

Positive predictive value

36


37/65

Likelihood that disease is NOT present IF test is negative

TN true negatives

NPV = =TN + FN all negatives

Negative predictive value

37


38/65

TEST

(Screening

Survey)

pos

neg

TP FP

TN

Predictive

Value

(positive)

Predictive

Value

(negative)

TPTP + FP

TN

FN +TNFN

Classification of screening test results

38


39/65

PPV and NPV

PPV and NPV are characteristics of test and of disease

prevalence

PPV is influenced by disease prevalence and more by the

specificityof test*

The greater the prevalence and the specificity, the greater is thePPV

NPV is influenced by disease prevalence and more by the

sensitivityof test*The lower the prevalence and the greater the sensitivity, the

greater is NPV

*when disease is rare39


40/65

Test Sensitivity = 99%

Test Specificity = 95%

True Status

Test Result

Sick

+ 495

- 9405

10,000

Not-Sick Total

Total

99

1

594

9406

9900100

=99 + 495

99Positive

PredictiveValue

= 17%

Disease Prevalence = 1%

PPV, example 1

40


41/65



True Status

Test Result

Sick

+ 475

- 9025

10,000

Not-Sick Total

Total

495

5

970

9030

9500500

=495 + 475

495Positive

PredictiveValue

= 51%

Disease Prevalence = 5%

PPV, example 2

41


42/65

Prevalence Rate = 1% Predictive Value (positive) = 17%

Prevalence Rate = 5% Predictive Value (positive) = 51%



True Status

Test Result

Case

+ TP FP

- FN TN

10,000

Non-Case Total

Total

Relationship of disease prevalence

to predictive value of a positive test

42


43/65

TEST(Screening

Survey)

pos

neg

yes no

Disease

TP FP

TN

Predictive

Value

(positive)

Predictive

Value

(negative)

TPTP + FP

TN

FN +TN

TN

FP +TN

TP

TP + FN

Sensitivity = Specificity =

FN

Classification of screening test results

43


44/65

Epidemiologic approach to the

evaluation of screening programs

Key question: do patients benefit from early detection of disease?

1. Can the disease be detected early?2. What are the sensitivity and specificity of the test?3. What is the predictive value of the test?4. How serious is the problem of false-positive results?5. What is the cost of early detection in terms of funds, resources, and

emotional impact?

6.

Are the subjects harmed by the screening tests?7. Do the individuals in whom disease is detected early benefit from

the early detection, and is there an overall benefit to those who are

screened?

44


45/65

Mammography and mortality reduction

The US recommends annual screening for breast cancer for women

above age 40

From a public health perspective it may be argued that this is

justifiable only ifscreening reduces breast cancer mortality

If screening is offered to all women in the target group, no welldefined control group is available

A study was done in Denmark to examine the varying estimates of

breast cancer mortality reduction based on different control groups

Olsen et al. Estimating the benefits of mammography screening: the impact of study design. Epidemiology. 2007; 18: 487-492

45


46/65


The study population included all women invited to screen

in Copenhagen from April 1991 to March 2001

The women were followed for breast cancer mortality

Person years at risk counted as date of first invitation untildate of death, emigration from Denmark, or end of follow-

up (March 2001)


46


47/65


Control group 1: Concurrent regional. Women in the same

age group living in Denmark from April 1991-2001, outside

the region of organized screening programs

Control group 2: Local historical. These were women from

the same age group living at any time between April 1981

and March 1991 (10 years before the program)

Control group 3: Historical-regional. These women were in

the same age group and living in Denmark, from

1981-1991, living outside of the region that later

implemented organized screening programs


47


48/65


1. Local historical. This analysis showed a reduction of

20%; the lesser benefit was probably due to the increase

in incidence in breast cancer over time

2. Concurrent regional. This analysis yielded a reduction in

breast cancer mortality of9%. Breast cancer incidence and

mortality was higher in Copenhagen than in the rest of

Denmark before screening.

3. Historical regional. This analysis estimated a 25%

decrease in breast cancer mortality. This controlled for time

and region. Probably the best method.


48


49/65

Factors influencing epidemiologic approach to

the evaluation of screening programs

1. Natural history of disease2. Pattern of disease progression3. Methodologic issues4. Study designs for evaluation of screening5. Problems in assessing sensitivity and specificity of tests6. Interpreting study results that show no benefit of

screening7. Cost benefit analysis of screening

49


50/65

Natural history

To discuss

methodologic

issues involved in

evaluating the

benefit of screening,we need to

understand natural

history of disease

50


51/65

Natural history

51


52/65

Pattern of disease progression

52


53/65

Methodologic issues

There are concerns particular to screening and an understanding of

why decisions about whether or not to use screening tests are

controversial requires consideration of the biases that can arise with

screening

Detection

Lead time bias

Length time bias

53


54/65

Detection

Screening appears to have a positive effect since disease precursor is

detected in persons who would not ultimately develop symptoms or die

from the disease

Initiation Disease

Detectable

by Screening

NO Clinical

Symptoms

Death from

other causes

NO Complications

from disease

Screening dx

54


55/65

Detection

Screening appears to have a positive affect since disease

precursor is detected in persons who would not ultimately

develop symptoms or die from the disease

Example: Blood pressure screening leads to people with high

blood pressure being told that they have hypertension. While

people with hypertension are more likely to develop diseases

such as stroke, not all of them will.

55


56/65

Lead-time bias

Survival appears to be increased among screen-detected cases

because diagnosis was made earlier in the disease

Initiation Disease

detectable

by screening

Clinical

symptoms

DeathComplications

from the disease

Screening dx Usual dx

56


57/65

Lead-time bias

Screening for lung cancer with chest X-rays is an example of lead time

bias. When tumors can be detected earlier, screening will seem to

prolong life compared to persons who are not screened and in whom

disease is detected later

Lead Time Bias

Positive Screening

Outcomes

57


58/65

Lead-time bias and 5 year survival

58


59/65

Length-time bias

People with a more protracted preclinical phase have a greater

probability of coming to screening. If a protracted preclinical phase is

associated with a better prognosis or survivorship, then screening may

actually look better than it is because of its affiliation with a protracted

preclinical phase.

Initiation Disease Detectable

by Screening

Clinical

Symptoms

Death

Complications from the

disease

Initiation Disease

Detectable

by Screening

Clinical

Symptoms

DeathComplications from the

disease

59


60/65

Length-time bias example

Example: Length-time bias may occur when carcinomas-in-situ are

picked up with breast screening. These may be slow-growing

precursors to cancer. Their early detection and treatment may appear

to improve mortality from the disease.

60

E id i l i t d d i t l t


61/65

Epidemiologic study designs to evaluate

screening

Non randomized studies

Case-control

Individuals with and without disease are compared; controls should be

representative of the population from which disease cases emerged

CohortCompare the rate of disease in those who chose to be

screened vs. who choose not to be screened

Randomized studies

Randomized trials

Most evidence about the efficacy of screening comes from non-

experimental designs: randomize to screening vs. no screening and

compare rates of disease61

P bl i i th S iti it


62/65

Problems in assessing the Sensitivity

and Specificity of tests

New screening programs are

frequently initiated after a

screening test becomes

available for the first time.

Usually claims are made (bymanufacturers of test kits,

investigators etc.) that the test

has high Sn and Sp. However,

not always easy to demonstrate.

62

I t ti t d lt th t h


63/65

Interpreting study results that show no

benefit of screening

The apparent lack of benefit may be inherent in the natural history of

the disease (e.g., the disease has no detectable preclinical phase or

an extremely short detectable preclinical phase).

The therapeutic intervention currently available may not be any more

effective when it is provided earlier than when it is provided at the time

of usual diagnosis.

The natural history and currently available therapies may have the

potential for enhanced benefit, but inadequacies of the care provided

to those who screen positive may account for the observed lack of

benefit (that is, there is efficacy, but poor effectiveness).

63


64/65

Cost-benefit analysis of screening

Cost issues when evaluating screening include financial but

also non-financial issues.

1. There must be good evidence that each test or procedurerecommended is medically effective in reducing morbidity and

mortality

2. The medical benefits must outweigh risks3. The costs of each test or procedure must be reasonable compared

to expected benefits4. The recommended actions must be practical and feasible

Source: American Cancer Society 64


65/65

Screening conclusions

Screening assumes that we can do something with the

positive screen

There are real costs of false negatives and false positives

We should not be screening just because we can

65

Documents

Epid 600 Class 11 Screening