Validity of a screening test

Validity Of A Screening Test

Dr. Kulrajat Bhasin

1

• Definitions• Objective• Sensitivity & Specificity• Concept of false positives and false negatives• Tests for continuous variables• Combination of tests and effects• Predictive value of a test• Determining cut-off point• ROC curve 2

Plan Of Presentation

Definition

• The process by which unrecognized disease or defects are identified by means of rapidly applied tests, examination or other procedures in apparently healthy individuals on a large scale.

• Differs from periodic health exam as:i. Capable of wide applicationii. Relatively inexpensiveiii.Physician has to only interpret it and hence saves time.

3

Aims And Objectives

• To sort out from a large group of apparently healthy people, those likely to have disease or are at an increased risk.

• To bring those who are apparently abnormal under supervision, confirmation and treatment if required.

4

Three Key Measures Of Validity

1. SENSITIVITY2. SPECIFICITY3. PREDICTIVE VALUE

5

• The above figure is a bimodal distribution of a variable in the normal and diseased populations.

• Below is a fig representing unimodal distribution.6

7

Validity Of Screening Tests

• The validity of a screening test is defined as its ability to distinguish between who has a disease and who does not and has 2 components:

Sensitivity : defined as ability of a test to identify correctly those who have the disease or true positives.

Specificity : defined as the ability of a test to identify correctly those who do not have a disease or true negatives.

8

9

Screening test results

Diagnosis Total

Present Absent

Positive a (True positive) b (False positive) a + b

Negative c (False negative) d (True negative) c + d

Total a + c b + d a+b+c+d

SensitivityIt is defined as the ability of a test to identify correctly

all those who have the disease, that is "true positive". x 100 x100

• A test has 90% sensitivity means: 90 per cent of the

diseased people screened by the test will give a "true

positive" result and the remaining 10 per cent a "false

negative" result.

(TP)

(TP+FN)

(a)

(a + c) OR

10

Specificity

It is defined as the ability of a test to identify correctly those who do not have the disease, that is, "true negatives".

x 100 x 100

• 90 per cent specificity means: 90 per cent of the non diseased

persons will give "true negative" result and 10 per cent of non-

diseased people screened by the test will be wrongly classified as

"diseased" when they are not (False positives).

(TN)

(FP+TN)OR

(d)

(b + d)

11

12

Percentage Of False Positives:

x

• Means that patients who do not have the disease

are told that they have the disease.

• Further diagnostic test.

• Inconvenience, discomfort, anxiety & unnecessary

expense.

(b)

(b + d)100

13

Percentage Of False Negatives:

x

• Means that patients who actually have the disease are

told that they do not have the disease.

• False Re-assurance.

• Might ignore the development of signs & symptoms.

• Postpone treatment.

• IMPORTANT IN SERIOUS DISEASE.

(c)(a + c)

100

14

• At times we need to test for a continuous variable, e.g. BP or blood sugar level for which there is no “positive” or “negative result”.

• A decision must therefore be made in establishing a cut-off level above which a test result is considered positive and below which a result is considered negative.

15

Tests For Continuous Variables

16

3 2

7 8

7 5

3 5

DIABETIC

DIABETIC DIABETICNON-DIABETIC

NON-DIABETIC

NON-DIABETIC

HIGHHIGH

HIGH

DIABETIC

DIABETIC NON-DIABETIC

NON-DIABETIC

BLOOD SUGAR BLOOD

SUGAR

BLOOD SUGAR

LOW

LOW

LOW10 10

10 10

+

+

-

-

Screening For Diabetes In Hypothetical Population With A Prevalence Of 50 %.Effects Of Choosing Different Cutoff Levels For A Positive Test:

A B

C

17

HIGHHIGH

HIGH

BLOOD SUGAR

BLOOD SUGAR

BLOOD SUGAR

LOW LOW

LOW

HIGH

BLOOD SUGAR

LOW

D E

F G

18

19

Combination Of Tests

• Two or more tests can be used in combination to enhance the sensitivity or specificity of screening. E.g. for syphilis, pts. first evaluated by RPR test, whose sensitivity is high but still gives false positives, hence the second test is applied, e.g. FTA – ABS which is a more specific test and the resultant positives are true positives.

• Sequential (two stage) and simultaneous testing.

20

• We can calculate the net sensitivity and specificity by using both tests in sequence.• After finishing both tests 315 people of 500 actual diabetics in the 10,000 population

have been correctly called positive, hence 63 % net sensitivity (loss).• 7600 of 9500 in this population who never had DM were correctly called negative by

the first test and were not tested further, while an addl 1710 of those 9500 were correctly called negative by 2nd test hence 7600 + 1710/9500 x 100, i.e. 98 % net specificity (gain).

21

Sequential (Two Stage) Testing

22

Simultaneous TestingTotal population: 1000 ; prevalence: 20%

…In Summary

23

• In sequential testing, there is a net loss in sensitivity but a net gain in specificity compared with results by either test alone.

• In simultaneous testing, there is a net gain in sensitivity but a net loss in specificity compared with either tests done alone.

• Hence the decision to use either of the above methods depends upon whether they’re being done for screening or diagnosis as well as on considerations of practicality, i.e. setting in which testing is being done, costs, length of hospital stay, degree of invasiveness, as well as third party insurance coverage.

Predictive Values Of A Test :• Useful to know what proportion of patients with abnormal test

results are truly abnormal.• They reflect the diagnostic power of a test.

Positive Predictive Value: The "predictive value of a positive test" indicates the probability that a patient with a positive test result has, in fact, the disease in question.

• Proportion of patients with positive test results who are correctly diagnosed.

• PPV of 90% means that 90% of the patients who are diagnosed to be positive by the test in fact have the disease in question.

x(a)

(a + b)100

24

Negative Predictive Value:

The "predictive value of a negative test" indicates the probability that a patient with a negative test result doesn't have disease in question.

• NPV of 90% means that 90% of the patients who are diagnosed to be negative by the test do not have the disease in question.

x(d)

(c + d)100

25

Calculating The Rates

A test is used in 50 people with disease and 50 people without. These are the results:

Disease

+ -

Test

+ 48 3 51

- 2 47 49

50 50 10026

Sensitivity = 48/50 = 96%Specificity = 47/50 = 94%Positive predictive value = 48/51 = 94% Negative predictive value = 47/49 = 96%

Disease

+ -

Test

+ 48 3 51

- 2 47 49

50 50 100

27

Effect of Prevalence :

• Predictive values depend strongly on prevalence of the

condition.

• As the prevalence of the condition increases positive

predictive value increases and thus more chances of getting

true positive results.

• If the condition is uncommon it is more sure that the

negative test indicates no abnormality.28

• Higher the prevalence, higher is the predictive value.• Hence a screening program is most productive and

efficient when it is directed to high risk target population.

29

Relationship B/W Predictive Value And Disease Prevalence

Amount of previous unrecognized disease diagnosed as a result of screening effort.

•Depends on Sensitivity, specificity.Prevalence of disease. Participation of individuals.

• High risk populations selected for screening – 40 yrs. above adults for diabetes.

30

Yield

Determining The Cutoff PointThe factors to be considered are :

• Disease prevalence : when prevalence of the disease is

high in the community the cut-off point is set at low

level.

This will increase the sensitivity.

• The disease : if the disease is very lethal and early

intervention markedly increases the prognosis, cut off

point is set at lower level.31

Receiver Operating Characteristic (ROC) Curve

• The cutoff point for a disease is best determined using ROC curve.

• As no single value cutoff point of an individual test can be expected to have both perfect sensitivity and specificity, it is often necessary to determine which value or cut off point is most appropriate for a given purpose.

• Receiver Operating Characteristic (ROC) Curve is a graph displaying the relationship between true positive rate (on vertical axis) and false positive rate (on horizontal axis).

• The ROC curve was first used during world war 2 for the analysis of RADAR signals for the detection of enemy objects entering the battle field.

32

• A perfect test can be represented by drawing a line along the Y-axis and horizontal line on the top.

• ROC curve of most of the tests appear as a curved line. Better the test, closer the curve approaches depiction of perfect test.

33

34

Uses Of ROC Curve:

1. ROC curve helps to choose the critical cut-off value which best discriminates the presence or absence of a disease. - At which the curve’s deviation is maximum. -Where the perpendicular distance from the line equality is maximum.

35

Test A Test B

Uses Of ROC Curve:

2. Another use of ROC curve is to compare two indicators.

The curve that contains a large area below it is a better predictor than

one below with a smaller area.

References

• Park K. Park’s textbook of preventive and social medicine. 22nd ed. Jabalpur(India): Banarsidas Bhanot; 2012. p127-134.

• Leon Gordis, Text book of epidemiology.4th ed.Saunders: Elsevier Inc.; 2009.p.85-108

• Sukon Kanchanaraksa.Evaluation of diagnostic and screening tests: validity and reliability, 1st edition. John hopkins bloomberg school of public health. 2011. p 1 – 124.

36

37