Andi Marmor, MD, MSEdThomas B. Newman, MD, MPHOctober 18, 2012

What are screening tests supposed to do? Definition and spectrum of screening What are the potential harms of

screening?Evaluating screening tests

Study designs Survival vs mortality Biases in studies of screening tests

Common definition: “Testing to detect asymptomatic disease”

A better definition?*: “Application of a test to detect a potential

disease or condition in people with no known signs or symptoms of that disease or condition”

“ Condition” includes a risk factor for a disease…

*Common screening tests. David M. Eddy, editor. Philadelphia, PA: American College of Physicians, 1991

Risk factor

Recognized symptomatic disease

Presymptomatic disease

Unrecognized symptomatic disease

Fewer people Easier to demonstrate benefit Less potential for harm to exceed benefit

Risk factor treatment disease Does risk factor predict disease? Does treatment reduce risk factor? Does identification/treatment of risk factor

reduce disease? Potential for harm exceeding benefit

greatest when screening for risk factors!

Caution: risk factors as surrogate outcomes

Are PVC’s after MI a risk factor for sudden death? Yes

Do encainide and flecainide decrease PVCs? Yes

Do these drugs save lives? NO! RCT showed total mortality

after 10 months higher in treated group vs placebo: 8.3% vs. 3.5% (P <0.0001)

Echt DS et al. N Engl J Med. 1991;324:781-8Moore TJ. Deadly Medicine. NY: Simon and Schuster, 1995

Does screening detect risk factor? Yes

Benefits to screening? Not studied

Possible risks to children/society? Cost, testing,

distraction from other priorities

Detect disease in earlier stage than would be detected by symptoms Only possible if an early detectable

phase is present (latent phase)Begin treatment earlier

Only beneficial if earlier treatment is more effective than later treatment

Do this without greater harm than benefit

Natural history heterogeneous Screening test may pick up slower

growing or less aggressive cancers Not all patients diagnosed with cancer

will become symptomatic “Pseudodisease”

Diagnosis is subjective There is no gold standard

Malignant

Benign

MalignantCan’t tellBenign

Why Not?

To those with a negative resultTo those with a positive resultTo all

The general teaching: Maximize sensitivity for

screening tests This is true IF

Goal is not to miss anyone with the disease

HOWEVER…. NPV already good in

low-prevalence population

Copyright restrictions may apply.

Schwartz, L. M. et al. JAMA 2004;291:71-78.

38% had experienced at least 1 false-positive;

>40% described that experience as "very scary”/"scariest time of my life.”

98% were glad they had had the screening test.

73% would prefer a total-body CT over $1000

Organisation for Economic Co-operation and Development. “OECD Health Data: Health Expenditures and Financing”, OECD Health Statistics Data from internet subscription database. http://www.oecd-library.org, data accessed on 08/23/12.

EconomicPoliticalPublic/culturalHealth care providers

Ad sponsored by Schering: company that

makes interferon.

2009: USPSTF changed age for routine mammogram from 40 to 50 For women 40-49 over 11 years of follow

up:▪ 1900 women invited, 20,000 visits, 2000 FP

mammograms = one death prevented Recommendations criticized by

Radiologists American Cancer Society The public

Quanstrum, Hayward. Lessons from the mammography wars. NEJM, 2010

What are screening tests supposed to do? Definition and spectrum of screening What are the potential harms of

screening?Evaluating screening tests

Study designs Survival vs mortality Biases in studies of screening tests

Screening test

Detect disease early

Treat disease

Patient outcome

Screening test

Detect disease early

Treat disease

Patient outcome

Screening test

Detect disease early

Treat disease

Patient outcome

Ideal Study: Randomize patients to screened/

unscreened Compares outcome (eg: mortality) in

ENTIRE screened group to ENTIRE unscreened group

Screened

Not screened

MortalityR

D+D-

D-D+ Mortality

Survival: Denominator is patients with the disease Introduces multiple biases

Mortality: Denominator often a population (eg:

those randomized to screening vs controls)

May include patients without the disease Can be total or cause-specific…

Survival (patients with disease) Compare those diagnosed by screening vs

those diagnosed by symptoms Compare those with disease in screened

group vs those with disease in unscreened group

Stage-specific survival in screened vs unscreened

Mortality (all enrolled) Compare outcomes in all screened patients

vs all unscreened patient

Screened

Not screened

MortalityR

Not screened

Screened

D+

D-

D-

D-

D-D+

D+

D+

R

Mortality

Mortality

Mortality

D-

D-

Screened

Not screened

R

D+

D+ Survival with disease

Survival with disease

Volunteer biasLead time biasLength biasStage migration biasPseudodisease

People who volunteer for screening differ from those who do not (generally healthier)

Example of the effect HIP Mammography study (1960’s):

randomized 60,000 women▪ 2/3 randomized to screening accepted

Among invited group, those who GOT mammography had lower cardiovascular death rates

Multicenter Aneurysm Screening Study (Problem 6.3)

Men aged 65-74 were randomized to either receive an invitation for an abdominal ultrasound scan or not

Ashton, et al 2002

Randomize patients to screened and unscreened Intent to treat – analyze as randomized

Control for factors (confounders) which might be associated with receiving screening AND the outcome eg: family history, level of health

concern, other health behaviors

Screening test

Detect disease early

Treat disease

Patient outcome

Screened

Not screened

R

D+

D+ Survival with disease

Survival with diseaseD-

D-

Latent Phase

Onset of symptoms DeathDetectable by screening

Detected by screening

Biological Onset

Survival After Diagnosis

Survival After Diagnosis

Lead Time

Lead Time Bias

Contribution of lead time to survival measured from diagnosis

Only present when survival from diagnosis is compared between diseased persons Screened vs not screened Diagnosed by screening vs by symptoms

Avoiding lead time bias Measure outcome from time of

randomization or entry into study (in entire group)

Depends on relative lengths of latent phase (LP) and screening interval (S)

Screening interval shorter than LP:

ScreenScreen Screen Screen

Depends on relative lengths of latent phase (LP) and screening interval (S)

Screening interval shorter than LP: Maximum false increase in survival = LP Minimum = LP – S

Screening interval longer than LP: Max = LP Proportion of disease dx by screening =

LP/S

Figure 2: Maximum lead time bias possible when screening interval is longer than latent phase

Max = LPProportion of disease diagnosed by screening: P = LP/S

SLP

Max

Screen ScreenScreen

Slowly progressive cases spend more time in presymptomatic phase Disproportionately picked up by

screeningHigher proportion of less aggressive

disease in screened group creates appearance of improved survival even if treatment is ineffective

TIME

Disease onset Symptomatic disease

Screen 1 Screen 2TIME

Survival in patients detected by screening

Survival in patients detected by symptoms

Only present when Survival from diagnosis is compared AND disease is heterogeneous

Lead time bias usually present as wellAvoiding length bias:

Compare mortality in the ENTIRE screened group to the ENTIRE unscreened group

Screening test

Detect disease early

Treat disease

Patient outcome

(Survival)

A condition that looks just like the disease, but never would have bothered the patient Type I: Disease which would never cause

symptoms Type II: Preclinical disease in people who will die

from another cause before disease presentsThe Problem:

Treating pseudodisease will always be successful Treating pseudodisease can only cause harm

RCT of lung cancer screening9,211 male smokers randomized

to two study arms Intervention: CXR and sputum

cytology every 4 months for 6 years (75% compliance)

Usual care: recommendation to receive same tests annually

*Marcus et al., JNCI 2000;92:1308-16

Marcus et al., JNCI 2000;92:1308-16

After 20 years of follow up, there was a significant increase (29%) in the total number of lung cancers in the screened group Excess of tumors in early stage No decrease in late stage tumors

Overdiagnosis (pseudodisease)

Black, cause of confusion and harm in cancer screening. JNCI 2000;92:1280-1

Marcus et al., JNCI 2000;92:1308-16

Appreciate the varying natural history of disease, and limits of diagnosis

Impossible to distinguish from successful cure of (asymptomatic) disease in individual patient

Clues to pseudodisease: Higher cumulative incidence in screened

group No difference in overall mortality between

screened and unscreened groups Schwartz, 2004: 56% said they would

want to be tested for pseudodisease !

New test

Stage disease

Treat disease

“Stage-specific”patient outcome

(stratified analysis)

Also called the "Will Rogers Phenomenon” "When the Okies left Oklahoma and moved

to California, they raised the average intelligence level in both states.”

Can occur when New test classifies severity of disease

differently AND outcomes are stratified by severity of

disease (ie: stage-specific survival)

Stage 1

Stage 2

Stage 3

Stage 4

Stage 0Stage 0

Stage 2

Stage 3

Stage 4

Stage 1

Old test New test

You are evaluating a new policy to admit COPD patients with CO2> 40 to the ICU rather than ward

Deaths in both ICU and ward go DOWN

Is this policy effective?

Admitted to ICU

Admitted to ward

Admitted to ward

Admitted to ICU

Before new policy After new policy

You are evaluating a new policy to admit COPD patients with CO2> 40 to the ICU rather than ward

Deaths in both ICU and ward go DOWN

Is this policy effective?

You want to know overall survival, before and after the policy…

Looking harder for disease, with more advanced technology, results in: Higher disease prevalence Higher disease stage (severity) Better (apparent) outcome for each stage

Stage migration bias does NOT affect Mortality in entire population Survival in ENTIRE screened group vs

ENTIRE unscreened group

D-

D-

Not screened

Screened

D-

D-

D+

D+

RMortality

Mortality

Screened

Not screened

R

D+D-

D-D+ Mortality

Mortality

Screened

Not screened

R

D+

D+ Survival with disease

Survival with disease

Screened

Not screened

R

D+D-

D-D+

What about the “Ideal Study”? Quality of randomization Cause-specific vs total mortality

Screened

Not screened

R

D+D-

D-D+ Mortality

Mortality

Edinburgh mammography trial (1994) Randomization by healthcare practice

7 practices changed allocation status Highest SES:

26% of women in control group 53% of women in screening group

Evidence: 26% reduction in cardiovascular mortality in mammography group

Problems: Assignment of cause of death is

subjective Screening and/or treatment may have

important effects on other causes of death

Bias introduced can make screening appear better or worse!

“Sticky diagnosis” bias: If pt has a cancer, death more often attributed

to cancer Effect: overestimates cancer mortality in

screened group (makes screening look WORSE) “Slippery linkage” bias:

Linkage lost between death and screening/diagnosis (eg: death from complications of screening result)

Effect: underestimates cancer mortality in screened group (makes screening look BETTER)

Meta-analysis of 40 RCT’s of radiation therapy for early breast cancer* Breast cancer mortality reduced in

patients receiving radiation (20-yr ARR 4.8%; P = .0001)

BUT mortality from “other causes” increased (20-yr ARR -4.3%; P = 0.003)

Does radiation help women?*Early Breast Cancer Trialists Collaborative Group. Lancet 2000;355:1757

Mortality from other causes generally exceeds disease-specific mortality

Effect on condition of interest more difficult to detect

Screening may be promoted due to economic, political or public interest rather than evidence

We must consider: size of effect and balance of benefits/harms to patient and society

Studies of screening efficacy: Ideal comparison: RCT of screened vs

unscreened population Biases possible when survival measured in

diseased patients only, or stratified by stage Mortality less subject to bias than survival