Radiological Expertise

Paul Taylor

Centre for Health Informatics and Multiprofessional Education

Overview

• Exposition: Signal-Symbol problem

• Cognitive studies of radiological expertise

• Perceptual studies of radiological expertise

• Coda: Radiology and ontology

Decision Making

knowledge based computer aid for making decisions about calcifications on mammograms

Identifying terms to use in arguments

METHOD:

• 11 radiologists recorded ‘thinking aloud’ while reading 20 sets of mammograms

• Verbal reports transcribed, descriptors

extracted & grouped

– 50 descriptors

Identifying arguments

• 10 radiologists read 40 sets of mammograms on which calcifications highlighted– asked to characterize calcifications using our

descriptors– test capacity of descriptors to discriminate benign and

malignant

• Identified physical dimensions underlying the descriptors– e.g. ‘size’ underlies ‘large’ and ‘small’

• Implemented image processing measures of those dimensions

Image Processing for Certain Arguments

Benign Malignant

well defined pleomorphic homogeneous segmental big within fat similar density curvilinear with a rim

Strong

Strong

isolated variable density lucent centre branching 1-5 flecks variable size scattered ill-defined vascular linear no finding clustered in skin towards nipple larger cluster few specks adjacent

Weak

oval/round

Weak

Implementing image processing

Size Variation AreaSD >= 0.0749 variable size AreaSD < 0.0749 similar size

72%

Density Variation

d = 0.126*ProminenceSD-0.997 d<-0.20485 variable density d>0.36265 similar density

45%

• Use discriminant analysis to derive classification rules

• Some rules work quite well, others less so

Evaluation of the prototype

• For purposes of the evaluation count the arguments to obtain a diagnosis

Actual diagnosisMalignant Benign

Malignant 5 8CADMIUM 2Benign 1 7

6 cases produced ties (5 benign and 1 malignant)

Conclusions

• Radiologists vary in the cues that they use to identify and characterise abnormalities

• Radiological terms may not map to a consistent definition in terms of ranges on physical characteristics– small calcifications are not always smaller

than large calcifications

Cognitive Studies of Radiological Expertise

• Radiology– Lesgold et al. 1988– Azevedo et al. 1998– Raufaste, Raufaste et Eyrolle 1998

• Others have looked at analogous specialties– Pathology

• Crowley et al. 2003

– Dermatology• Kulatunga-Moruzi, Brooks and Norman 2004

Cognitive Studies of Radiological Expertise

• Novices fail – to invoke appropriate schema– to test appropriate schema– to follow the schema through

• Intermediates able to identify features but fail to achieve diagnostic closure

• Experts – Generate hypotheses– perform longer chains of reasoning involving better integrated

clusters of findings– better able to switch schema

• All groups– mix forward and backward reasoning

Kulatunga-Moruzi et al. Cognition and Perception in Dermatology

Visual after verbal

Given comprehensive and accurate description

of image features

Asked to make diagnosis

Given photograph of case

Asked to make diagnosis

Visual

Given photograph of case

Asked to make diagnosis

Verbal

Given comprehensive and accurate description

of image features

Asked to make diagnosis

Kulatunga-Moruzi et al. Cognition and Perception in Dermatology

striped shading = dermatologist; solid shading = family practitioner; open shading = resident

Lesgold et al: Perceptual Differences

• Asked to draw key feature• Experts agree on size and

position• Half residents did not identify a

region that even approximately matched experts– their interpretations account

for an abnormality different to that perceived by experts

Perceptual and the Visual Hierarchy

Striate cortex, simple cells

Striate cortex, complex cells

Inferotemporal cortex

Anterior Inferotemporal cortex

Perceptual Learning and the Visual Hierarchy

Learning specific toorientation and location

Learning generalised across orientation, location and form

Reverse Hierarchy TheoryAhisser and Hochstien 2004

• Experiments show that: – more demanding task conditions result in

more stimulus-specific learning • Ergo, in difficult conditions learning occurs at lower

levels

– when tasks are interleaved, learning on harder tasks only occurs after learning on easier

• Ergo, learning at higher levels has to happen first

Reverse Hierarchy TheoryAhisser and Hochstien 2004

First glimpse able to access high level representationsHigh-level learning generalises over stimulus parameters

Later scrutiny accesses feature information from low level processingLow-level learning is more parameter specific

Reverse Hierarchy TheoryAhisser and Hochstien 2004

• Expert perception is immediate, holistic– Suggesting high level

representations

• Expert perception does not generalise– Based on low level

changes

Sowden, Davies and Roling (2000)

• Radiologists better than non-radiologists at detecting dots placed at random against a complex background – radiologists’ conceptual knowledge

irrelevant

• Limited experience leads to increase in the performance of novices– improvement did not transfer to

images in which the contrast was reversed

Mello-Thoms at al. Eye movement studies

• Phase 1:– Track readers’ gaze while they

inspect image

• Phase 2– Readers indicate where

abnormalities lie on image

• Responses classed by decision outcome– True Positive

• Lesion identified correctly

– False Negative• Missed lesion

– False Positive• Normal region incorrectly

labelled as containing a lesion

– True Negative• Region inspected at Phase 1 but

no lesion indicated in Phase 2

Two phases of search

• Relationship between accuracy and time to decision is different for each group

• Experts seem perceptually more sensitive, better tuned vlsual recognition

Can we identify image features that predict decision outcomes?

• Bank of filters sensitive to scale and orientation

• Compute:– Local features

• a vector of log(energy) of response across spatial frequency bands for each inspected region

– Global features• For each reader compute the mean

vector for all inspected regionsWavelet Packets

Can we identify image features that predict decision outcomes?

Can we identify image features that predict decision outcomes?

• Spatial frequency signature, combining local and global features predicts decision outcome– ANN identified 79% of TP decisions

• Visual saliency is observer-dependent– Observers attracted to different but similar areas, but

make different decisions– Observers respond differently to spatial frequency– Personal profile guides each radiologist

Overall Conclusions

• Expertise - in part - depends on perceptual learning– Training should involve extensive practice– Learning may be enhanced if easy cases

learnt first

• Salience is observer-dependent– Individuals’ representations of normal

background differ

Copyright ©Radiological Society of North America, 2005

Haller, S. et al. Radiology 2005;236:983-989

Haller and Radue 2005

• Captured fMRI scans of radiologists and non-radiologists carrying out a simple decision task – on X-ray images – on electron microscopy

images

Copyright ©Radiological Society of North America, 2005

Haller, S. et al. Radiology 2005;236:983-989

Difference between radiology and other images

in non-radiologists’ brains in radiologists’ brains

Copyright ©Radiological Society of North America, 2005

Difference between radiologists’ and non-radiologists’ brains on

non-radiology images

Haller and Radue: conclusions

• X-rays excite regions in the radiologists’ brains not activated in non-radiologists’– bilateral middle and inferior temporal gyrus, bilateral medial and

middle frontal gyrus, and left superior and inferior frontal gyrus– associated with visual attention and memory retrieval

• X-rays automatically attract attention, invoke memories

• control images invoke different patterns of activity in the two groups, – radiological expertise leads to the development of general visual

processing skills, • E.g. enhanced capacity of mentally rotating visual stimuli

Overall Conclusions

• Expertise - in part - depends on perceptual learning– Training should involve extensive practice– Learning may be enhanced if easy cases

learnt first

• Salience is observer-dependent– Individuals’ representations of normal

background differ

Radiology and Ontology

• Various initiatives require the development of ontologies or of controlled terminologies for biomedical images– RadLex– Birads– RITI– SNOMED-CT– DICOM

Difficult to describe what we see

• Design Criteria for Ontologies– Ontologies communicate the intended

meaning of defined terms– Definitions of terms should be objective

• Some applications require annotations at a level of granularity for which an objective definition of terms may not be possible

Examples of terms from RadLex• Findings

– Visual Features• Finding Related Features

– Shape» Lobular» Nodular» Pedunculated

– Margins» Poorly-defined

– Morphology» Blunted» Eroded

“These terms describe features on the image that can be described without reference to specific physical, anatomic, or pathological processes or structures”

Oh, and another thing

• Perception may be an important element of expertise in other specialties

• Doctors may need to learn to perceive all manner of clinical signs