Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 1

1

APIII

Use of a ‘Mathematical Microscope’ to Understand Radiologists’ Errors in

Breast Cancer Detection

Claudia Mello-ThomsClaudia Mello-Thoms, MSEE, PhD, MSEE, PhD

Department of Biomedical InformaticsDepartment of Biomedical Informatics

University of PittsburghUniversity of Pittsburgh

mellothomsc@upmc.edumellothomsc@upmc.edu

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 2

2

APIIIMotivationMotivation

Breast cancer is the most common type of non-skin cancer among women in the US, and the 2nd most deadly.

Many factors affect the correct detection of breast cancer:

– low disease prevalence in screening population;– high variability in the appearance of cancer;– experience level of the radiologist;– and many others.

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 3

3

APIII

It has been shown that 10-30% of all cancers detected by mammography screening are visible in retrospect.

Given that in 2007 an estimated 178,480 new breast cancers were expected to be detected in the US, and assuming that all these cancers were found at mammography screening:

– 10% = 17,848;– 30% = 53,544.

For a long time it was assumed that radiologists only missed lesions that they did not visually inspect.

Eye-position tracking has been used to confirm this.

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 4

4

APIII

It showed that these misses could be divided in 3 categories:– ‘search’ errors: lesion did not attract any amount of visual attention;– ‘perceptual’ errors: lesion attracted visual attention, but not long enough

for object recognition to occur;– ‘decision making’ errors: lesion attracted visual attention long enough for

object recognition to occur; lesion was incorrectly identified as being either benign or a variation of normalcy.

Eye-position tracking studies have shown that 70% of the missed lesions attracted visual attention. Thus, most errors were either ‘perceptual’ or ‘decision making’.

A model of medical image perception states that the decision to report or to dismiss a perceived finding is based upon:

– the local characteristics of the finding;– a comparison strategy of the finding with selected areas of the

background.

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 5

5

APIIIResearch QuestionsResearch Questions

1. Can we differentiate malignant breast lesions that were correctly reported from the ones that were visually inspected but not reported?

2. Are there any differences between malignant breast lesions that lead to ‘perceptual’ errors and the ones that lead to ‘decision making’ errors?

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 6

6

APIIIMethodsMethods

Four MQSA-certified expert breast radiologists from the Breast Imaging Division, Department of Radiology, University of Pittsburgh, participated in this experiment.

The radiologists read a case set of 40 two-view (CC and MLO) digitized mammograms. From these:

– 30 cases contained a biopsy-verified malignant breast mass;– 10 cases were lesion-free and had been stable for 2 years.

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 7

7

APIII

The experiment had 2 phases:– 1. Visual Search:

» Eye tracking;» This phase ended once the radiologist indicated their initial

impression of the case, that is, whether it was ‘normal’ or ‘abnormal’.

– 2. Report and Localization:» No eye tracking;» The radiologists used a mouse-controlled cursor to mark

the (x,y) locations of all malignant masses that they wished to report in each case, in both views.

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 8

8

APIIIData Analysis Data Analysis

i) Segmentation: – Each segmented area corresponded to 5° of visual angle (i.e., the

‘useful visual field’). In this setting, 256 x 256 pixels;– Two types of areas were segmented: 1) areas that yielded a local

response; and 2) areas that were used in the radiologist’s sampling of the background;

– Local Responses:» Areas that attracted visual attention and yielded marks by the

observer (True and False Positive decision outcomes);» Areas that contained a malignant mass that attracted visual

attention but was not reported by the observer (False Negative decision outcomes);

i) Segmentationi) Segmentation

ii) Wavelet filteringii) Wavelet filtering

iii) Statistical Analysesiii) Statistical Analyses

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 9

9

APIII

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 10

10

APIIIData AnalysisData Analysis

i) Segmentation:– Each segmented area corresponded to 5° of visual angle (i.e.,

the ‘useful visual field’). In this setting, 256 x 256 pixels;– Background Sampling:

» Lesion-free areas that attracted prolonged visual dwell (>330 ms) but that did not yield a mark by the observer (that is, that were correctly interpreted to be lesion-free).

i) Segmentationi) Segmentation

ii) Wavelet filteringii) Wavelet filtering

iii) Statistical Analysesiii) Statistical Analyses

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 11

11

APIII

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 12

12

APIIIData AnalysisData Analysisi) Segmentationi) Segmentation

ii) Wavelet filteringii) Wavelet filtering

iii) Statistical Analysesiii) Statistical Analyses

Level 0

Level 1 Level 2

sfb N

sfb 2

sfb 1

sfb 3

energy

energy

energy

energy

energy

energy

energy

energy

log

log

log

log

log

log

log

log

le1

le2

le3

leN

Scalar vector

Spatial frequency bands

normalizationNonlinear transformation

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 13

13

APIIIData AnalysisData Analysis

For the statistical analyses, the segmented areas were represented as:

– Local responses: 20 scalar values that represented TPs, FPs and FNs in different ‘spatial frequency bands’;

– Background sampling: 20 scalar values that represented each area of the background fixated by the radiologist for 330 ms or longer in different ‘spatial frequency bands’.

i) Segmentationi) Segmentation

ii) Wavelet filteringii) Wavelet filtering

iii) Statistical Analysesiii) Statistical Analyses

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 14

14

APIIIResultsResults

Research Question 1. Can we differentiate malignant breast lesions that were correctly reported (TP) from the ones that were visually inspected but not reported?

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 15

15

APIIIUsing Analysis of VarianceUsing Analysis of Variance

Differences between lesions visually inspected and correctly reported (TP) and those not reported due to:

– Search errors: 10 sfbs;– Perceptual errors: borderline on 2 sfbs (p=0.0548);– Decision Making errors: none.

Differences in background sampling between lesions visually inspected and correctly reported (TP) and those not reported due to:

– Perceptual errors: none;– Decision Making errors: 10 sfbs.

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 16

16

APIIIResultsResults

Research Question 2. Are there any differences between malignant breast lesions that lead to ‘perceptual’ errors and the ones that lead to ‘decision making’ errors?

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 17

17

APIIIUsing Analysis of VarianceUsing Analysis of Variance

Differences between lesions visually inspected but not reported due to ‘perceptual’ errors vs. those not reported due to ‘decision making’ errors:

– None!!

Differences in background sampling between lesions visually inspected but not reported due to ‘perceptual’ errors vs. those not reported due to ‘decision making’ errors:

– Statistically significant differences in 10 sfbs!!

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 18

18

APIIIUsing Correlation AnalysisUsing Correlation Analysis

Area of Lesion Area of Lesion

Perceptual Errors

Decision Making Errors

Background Sampling

NONE

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 19

19

APIIIConclusionsConclusions

1. We used a ‘mathematical microscope’ to study the differences between malignant breast masses that were visually inspected and correctly reported (TP) and those that (i) did not attract any amount of visual attention (‘search errors’); (ii) did attract visual attention but not long enough for object recognition to occur (‘perceptual errors’); or (iii) attracted visual attention long enough for object recognition to occur, but were ultimately dismissed by the radiologist (‘decision making errors’).

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 20

20

APIII

2. Our data suggests that most local differences existed between the representation of the cancers that were correctly reported (TP) and those that did not attract any amount of visual attention (‘search errors’).

3. We have previously shown that radiologists adhere to specific error patterns, and that these error patterns are characteristic to each radiologist.

4. Taking these findings together, the current results suggest that local image processing, aiming to increase the conspicuity of certain lesions, may be useful, as long as it takes into account the individual variability of the different radiologists.

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 21

21

APIII

5. In addition, the current results also suggest that most differences between correctly reported lesions (TP) and visually inspected but dismissed lesions (FN) can be found in the radiologists’ comparison strategy of the perceived finding with selected areas of the background.

6. In this case, most differences were found between the TP and the ‘decision making’ errors, but also a significant number of differences appeared between the ‘perceptual’ and ‘decision making’ errors themselves.

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 22

22

APIII

7. Finally, the missed lesions that were visually inspected but not reported could be completely characterized by:– Perceptual Errors: NO object recognition = NO change in

search strategy» Bkgd sampling x Area of Lesion = No changes;

– Decision Making Errors: ACTUAL decision = ACTUAL change in search strategy

» Bkgd sampling x Area of Lesion = Decreased Correlation;

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 23

23

APIIITake Home MessageTake Home Message

Local Image Processing may be useful to increase conspicuity of certain lesions, but there is no ‘one-size-fits-all’!!! It needs to be tailored to each radiologist!

In the radiologist’s decision making process regarding how to proceed about a perceived finding (‘report it?’, ‘dismiss it?’), as important as the local characteristics of the lesion is the comparison strategy used. Often, local differences do not exist that can explain why certain lesions were reported while others, virtually identical, were dismissed. However, if one looks at how the radiologist searched the parenchyma, the explanation is right there!

Department of Biomedical InformaticsDepartment of Biomedical InformaticsSlide 24

24

APIII