Tutorial on Medical Image Retrieval- application domains-

Medical Informatics Europe 2005

28.08.2005

Henning Müller, Thomas Deselaers

Service of Medical InformaticsGeneva University & Hospitals, SwitzerlandAachen Technical University, Germany

2©2005 Hôpitaux Universitaires de Genève

Overview

•• Current applicationsCurrent applications

•• Tools to manage archivesTools to manage archives•• Semi-automatic coding, DICOM header correction, Semi-automatic coding, DICOM header correction, ……

•• TeachingTeaching•• Access to teaching files for lecturersAccess to teaching files for lecturers

•• …… and for students and for students

•• ResearchResearch•• Find good examples, quality controlFind good examples, quality control

•• Include visual features into studiesInclude visual features into studies

•• Diagnostic aidDiagnostic aid•• Very focused domain, evidence-based medicine, case-basedVery focused domain, evidence-based medicine, case-based

reasoningreasoning

•• Example systems and fieldsExample systems and fields

•• OthersOthers

3©2005 Hôpitaux Universitaires de Genève

Current applications

•• This should rather be This should rather be emptyempty

•• No programs for visual information retrieval areNo programs for visual information retrieval arecurrently used in clinical routine, at least to mycurrently used in clinical routine, at least to myknowledgeknowledge•• Assert on lung image retrievalAssert on lung image retrieval

•• IRMA in image classification and semi-automatic codingIRMA in image classification and semi-automatic coding

•• Research applications Research applications and large number ofand large number ofprojectsprojects•• MelanomaMelanoma

•• Pathology slidesPathology slides

•• Mammography, lung Mammography, lung CTsCTs

•• PACS-like databasesPACS-like databases

4©2005 Hôpitaux Universitaires de Genève

Tools to manage archives

•• NavigationNavigation in large archives in large archives

•• Find lost images (without/with wrong annotations)Find lost images (without/with wrong annotations)

•• DICOM is not enoughDICOM is not enough

•• Semi-automatic Semi-automatic codingcoding

•• Propose codes of visually similar imagesPropose codes of visually similar images

•• Quality controlQuality control

•• Control the codes and find images with abnormalControl the codes and find images with abnormal

codes based on visual similaritycodes based on visual similarity

•• DICOM headers contain errors (~16% in anatomicDICOM headers contain errors (~16% in anatomic

region) that can be correctedregion) that can be corrected

5©2005 Hôpitaux Universitaires de Genève

Semi-automatic annotation (IRMA)

6©2005 Hôpitaux Universitaires de Genève

Teaching

•• Manage teaching filesManage teaching files

•• myPACSmyPACS.net, MIRC (Medical Imaging Resource Center,.net, MIRC (Medical Imaging Resource Center,

RSNA), HEAL, RSNA), HEAL, PathoPicPathoPic, , ……

•• Resource for Resource for studentsstudents to find and explore to find and explore

databases and casesdatabases and cases

•• Casimage Casimage (used for exams, teaching CDs, (used for exams, teaching CDs, ……))

•• Resource for Resource for lecturerslecturers to find optimal images for to find optimal images for

teachingteaching

•• Share images among lecturersShare images among lecturers

•• Find visually similar images with varying diagnosesFind visually similar images with varying diagnoses

7©2005 Hôpitaux Universitaires de Genève

myPACS (http://www.mypacs.net/)

8©2005 Hôpitaux Universitaires de Genève

MIRC – Medical Image Resource Center

•• http://http://mircmirc..rsnarsna.org/.org/

•• RRadiological adiological SSociety ociety NNorth orth AAmericamerica

•• Ten+ databases are made available for text-Ten+ databases are made available for text-based search in database fields or as free textbased search in database fields or as free text•• Based on Internet standardsBased on Internet standards

•• Software is open sourceSoftware is open source

•• Goal is to create a Goal is to create a worldwide repositoryworldwide repository of cases of casesfor teachingfor teaching

•• Visual retrieval would be a good complement toVisual retrieval would be a good complement tothe textthe text•• Multi-lingual retrieval is currently impossibleMulti-lingual retrieval is currently impossible

9©2005 Hôpitaux Universitaires de Genève

CasImage (http://www.casimage.com/)

10©2005 Hôpitaux Universitaires de Genève

Research

•• Optimize the Optimize the selection of casesselection of cases for research for research

•• Find visually similar casesFind visually similar cases

•• Browse databases through example casesBrowse databases through example cases

•• Find misclassified casesFind misclassified cases

•• Include Include visual features into research studiesvisual features into research studies

•• Find unknown connectionsFind unknown connections

•• Features need to have a rather high levelsFeatures need to have a rather high levels

•• Correspond roughly to diseasesCorrespond roughly to diseases

•• Visual data miningVisual data mining

•• Visual knowledge managementVisual knowledge management

11©2005 Hôpitaux Universitaires de Genève

Diagnostic aid

•• Case-based reasoningCase-based reasoning

•• Evidence-based medicineEvidence-based medicine

•• Supply Supply similar casessimilar cases as a help for practitioners as a help for practitioners•• Has shown to help inexperienced practitionersHas shown to help inexperienced practitioners

•• Aisen Aisen et al., et al., RadiologyRadiology

•• This is possible in fields where visual low-levelThis is possible in fields where visual low-levelsimilarity is importantsimilarity is important•• High resolution lung CTHigh resolution lung CT

•• Dermatology, Pathology, MammographyDermatology, Pathology, Mammography

•• Fractures (treatment planning)Fractures (treatment planning)

•• Problem: Advances in medical imaging equipmentProblem: Advances in medical imaging equipment

12©2005 Hôpitaux Universitaires de Genève

Example: case-based reasoning

?

Emphysema

Micro nodules

Emphysema

Macro nodules

13©2005 Hôpitaux Universitaires de Genève

Assert

•• Diagnostic aid on lung Diagnostic aid on lung CTsCTs

14©2005 Hôpitaux Universitaires de Genève

Dermatology

•• ABCDABCD rule (Asymmetry, Border, Color, rule (Asymmetry, Border, Color,

Differential structures)Differential structures)

•• Hair removal, boundary detection, textureHair removal, boundary detection, texture

analysis, analysis, ……

15©2005 Hôpitaux Universitaires de Genève

UPittsburg, Pathology; IGDS Rutgers

16©2005 Hôpitaux Universitaires de Genève

Mammography

•• Less image retrieval, but rather Less image retrieval, but rather detection ofdetection of

regionsregions with abnormal characteristics with abnormal characteristics

•• micro calcificationsmicro calcifications

•• Local analysis is importantLocal analysis is important

•• Large databases with Large databases with preclassified preclassified image regionsimage regions

existexist

•• England: England: MammogridMammogrid

17©2005 Hôpitaux Universitaires de Genève

Case-based rather than image-based retrieval

•• Currently the input is mostly one imageCurrently the input is mostly one image

•• MD might have MD might have several imagesseveral images (RX, CT, (RX, CT, ……) for a) for a

same patientsame patient

•• Cases stored in the patient record also oftenCases stored in the patient record also often

have more than one imagehave more than one image

•• Plus other data: text, numerical values (lab)Plus other data: text, numerical values (lab)

•• Also, Also, entire seriesentire series (CT, MRI) as an input and not (CT, MRI) as an input and not

selected imagesselected images

•• Slice selection based on what a Slice selection based on what a ““normalnormal”” image would image would

be likebe like

18©2005 Hôpitaux Universitaires de Genève

Other applications

•• Parameter settingsParameter settings for segmentation, etc. for segmentation, etc.

•• Based on a large number of known, well-segmentedBased on a large number of known, well-segmented

casescases

•• Show me if this case needs further attention,Show me if this case needs further attention,

dissimilaritydissimilarity retrieval against healthy cases retrieval against healthy cases

•• Needs a large number of healthy casesNeeds a large number of healthy cases

•• Create a model for a Create a model for a ““healthyhealthy”” image image

•• ……

19©2005 Hôpitaux Universitaires de Genève

Conclusions

•• Image retrieval is at the moment mainly anImage retrieval is at the moment mainly an

academicacademic problem problem

•• Information explosions is happening in the medicalInformation explosions is happening in the medical

domain (multimedia in digital form)domain (multimedia in digital form)

•• We need We need toolstools and we need to imagine how to use and we need to imagine how to use

themthem

•• There are There are many applicationsmany applications for image retrieval for image retrieval

•• We need to start the clinical We need to start the clinical integrationintegration

•• Visual systems will not replace text butVisual systems will not replace text but

complement itcomplement it