Computational Aspects of Medical Imaging

Classification Level JASON 2003JASON 2003

Christopher Stubbs Michael BrennerAl DespainStan FlatteRobert HendersonDarrel LongJohn TonryPeter Weinberger

JASON Summer 2003

Computational Aspects of Medical Computational Aspects of Medical ImagingImaging

JASON 2003JASON 2003

22

JASON will undertake a study for the DOE and the NIH National Institute JASON will undertake a study for the DOE and the NIH National Institute for Biomedical Imaging and Bio-engineering on the role of computation for Biomedical Imaging and Bio-engineering on the role of computation (broadly defined to include raw computational capabilities, mass (broadly defined to include raw computational capabilities, mass storage needs, and connectivity) for medical imaging. This study will storage needs, and connectivity) for medical imaging. This study will address the computational requirements in three general areas:address the computational requirements in three general areas:

The fusion of image data of varying modalities, over differing The fusion of image data of varying modalities, over differing spatial and spatial and temporal scales and resolutions.temporal scales and resolutions.

The extraction and display of quantitative information, with The extraction and display of quantitative information, with associated associated uncertainties.uncertainties.

Data archiving: raw vs. extracted parameters, metadata Data archiving: raw vs. extracted parameters, metadata standards.standards.

JASON will assess the present status of computational, storage and JASON will assess the present status of computational, storage and connectivity needs for existing tools and techniques, and will project connectivity needs for existing tools and techniques, and will project likely computational demands for the future. The imaging systems likely computational demands for the future. The imaging systems under consideration include both diagnostic and real-time clinical tools. under consideration include both diagnostic and real-time clinical tools.


33

ContextContext

• Bio-computing is a vast enterprise: protein Bio-computing is a vast enterprise: protein folding, genetic databases, medical records...folding, genetic databases, medical records...

• Other committees have reviewed bio-Other committees have reviewed bio-computing:computing:

Biomedical Information Science and Biomedical Information Science and Technology Initiative (BISTI)Technology Initiative (BISTI)

Coalition for Advanced Scientific Computing Coalition for Advanced Scientific Computing (CASC) (CASC)

President’s Information Technology Advisory President’s Information Technology Advisory Committee (PITAC)Committee (PITAC)

• Our focus was narrower, dealt only with Our focus was narrower, dealt only with biomedical imagingbiomedical imaging


44

FindingsFindings

• We were impressed with existing efforts! We were impressed with existing efforts!

Combination of applied mathematics, Combination of applied mathematics, computational techniques, biological computational techniques, biological sciencessciences

• Computational demand (processing, storage) Computational demand (processing, storage) of generating, analyzing and displaying of generating, analyzing and displaying biomedical image data is within capabilities biomedical image data is within capabilities of current high-end systems (e.g. Linux of current high-end systems (e.g. Linux clusters)clusters)

• A caveat: this is a huge field, and there are A caveat: this is a huge field, and there are counterexamples to our generalizations. counterexamples to our generalizations. However, there are also clear trendsHowever, there are also clear trends


55

Relevant communitiesRelevant communities

• MathematiciansMathematicians

• Physicists and EngineersPhysicists and Engineers

• Biological ScientistsBiological Scientists

• Computer Scientists and Computer Computer Scientists and Computer EngineersEngineers

• Clinical PhysiciansClinical Physicians

• You and me!You and me!


66

So what So what isis computationally hard? computationally hard?

Evolving from qualitative to quantitative analysisEvolving from qualitative to quantitative analysis

Using common metrics for algorithm appraisalUsing common metrics for algorithm appraisal

Integrating across modalities and length scalesIntegrating across modalities and length scales

Evolving towards an accepted metadata standardEvolving towards an accepted metadata standard

Database architectures that accommodate image Database architectures that accommodate image datadata

Connectivity across federations of distributed data Connectivity across federations of distributed data setssets

Cultural issues – data access, open source...Cultural issues – data access, open source...


77

BriefersBriefers• Richard Leahy, Neuroimaging Research Group, USC Richard Leahy, Neuroimaging Research Group, USC

• Christopher Johnson, Director, Scientific Computing and Imaging Christopher Johnson, Director, Scientific Computing and Imaging Institute, Univ. of UtahInstitute, Univ. of Utah

• Michael Miller, Director, Center for Imaging Science, Johns Michael Miller, Director, Center for Imaging Science, Johns Hopkins Univ. Hopkins Univ.

• Mark Ellisman, Director, National Center for Imaging and Mark Ellisman, Director, National Center for Imaging and Microscopy Research, UCSDMicroscopy Research, UCSD

• Larry Frank, Center for functional MRI imaging, UCSDLarry Frank, Center for functional MRI imaging, UCSD

• Michael Vannier, Chair, Dept of Radiology, Univ. of IowaMichael Vannier, Chair, Dept of Radiology, Univ. of Iowa

• Richard Martino, Director, Division of Computational Bioscience, Richard Martino, Director, Division of Computational Bioscience, Center for Information Technology, NIHCenter for Information Technology, NIH

• Judith Niland, Director, Division of Information Sciences, Judith Niland, Director, Division of Information Sciences, City of Hope Hospital, LACity of Hope Hospital, LA


88

Domains of Medical ImagingDomains of Medical Imaging

Physical scale (m)

Bas

ic

C

lini

cal

10-9 10-8 10-7 10-6 10-5 10-4 10-3 10-2 10-1 1

BodyOrgans

Vascular

Cellular

Intra-cellular

Molecular


99

Important Distinctions between Important Distinctions between Clinical Practice and Basic ScienceClinical Practice and Basic Science

• Clinical practice: Clinical practice:

$ driven -- throughput!$ driven -- throughput!

Commercial medical imaging systems Commercial medical imaging systems

• Basic research: Basic research:

Can tolerate longer latenciesCan tolerate longer latencies

More interaction with acquisition More interaction with acquisition hardwarehardware


1010

Diverse Biomedical Imaging Diverse Biomedical Imaging Modalities Modalities

• CT: Tomographic Xray imagingCT: Tomographic Xray imaging

• MRI, fMRI: Magnetic Resonance ImagingMRI, fMRI: Magnetic Resonance Imaging

• PET: Positron Emission TomographyPET: Positron Emission Tomography

• UltrasoundUltrasound

• MicroscopyMicroscopy

• EEG, MEG: electro-encephalography, EEG, MEG: electro-encephalography,

magneto-encephalography magneto-encephalography

• ......


1111

Biomedical Image Analysis Biomedical Image Analysis

• From Bits to PicturesFrom Bits to Pictures

From raw image data to 2-d or 3-d imagesFrom raw image data to 2-d or 3-d images

• From Pictures to NumbersFrom Pictures to Numbers

Identification and extraction of featuresIdentification and extraction of features

• From Numbers to KnowledgeFrom Numbers to Knowledge

Comparison with comparable casesComparison with comparable cases

Comparison with prior historyComparison with prior history

• Archive both images and extracted parametersArchive both images and extracted parameters

Qualitative analysisQuantitative Analysis


1212

From Bits to ImagesFrom Bits to Images

• The different imaging techniques require The different imaging techniques require different image construction algorithmsdifferent image construction algorithms

CT scans use tomographic inversionCT scans use tomographic inversion

Positron imaging uses back-to-back photon Positron imaging uses back-to-back photon detection geometry to reconstruct source detection geometry to reconstruct source locationlocation

EEG and MEG images typically use forward EEG and MEG images typically use forward modelingmodeling

Raw data, and calibration parameters, are Raw data, and calibration parameters, are seldom stored. seldom stored.

• This is not a solved problem – algorithmic work This is not a solved problem – algorithmic work still needed!still needed!


1313

Converting from Raw Measurements to Converting from Raw Measurements to Images Images

• CT scans require conversion from integral CT scans require conversion from integral

transmission measurements vs. transmission measurements vs. to voxels to voxels

• Inverse problems such as these are often ill-posedInverse problems such as these are often ill-posed

(Swiss Federal Institute of Technology

animation)


1414

Forward Modeling Example: MEGForward Modeling Example: MEG

www.uke.uni-hamburg.de/institute/physiologie/neuro_physiologie/images/meg_bem_icon.gifwww.uke.uni-hamburg.de/institute/physiologie/neuro_physiologie/images/meg_bem_icon.gif

Goal is to measure electrical Goal is to measure electrical activity in the brain, from activity in the brain, from exterior B field exterior B field measurementsmeasurements

Simple models assume Simple models assume spherical head with scalar spherical head with scalar conductivity fieldconductivity field

More sophisticated models More sophisticated models use MRI data to determine use MRI data to determine both brain geometry and both brain geometry and conductivity tensorconductivity tensor

Exterior magnetic field Exterior magnetic field measurements are matched measurements are matched to current sources within this to current sources within this conductive volumeconductive volume


1515

How “hard” are the computational How “hard” are the computational challenges in generating images? challenges in generating images?

CPU cycles for image generationCPU cycles for image generation

Inverse problem: CAT scan of N integral transmission Inverse problem: CAT scan of N integral transmission measurements into i x j x k voxels measurements into i x j x k voxels

Near-real time turnaround from commercial scannersNear-real time turnaround from commercial scanners

Typical research image processing challenges are in Typical research image processing challenges are in the domain of few-CPU clusters, for typical image the domain of few-CPU clusters, for typical image acquisition rates. Typical research MRI imaging takes acquisition rates. Typical research MRI imaging takes ~ 1hr~ 1hr

Necessary CPU resources depend on tolerable Necessary CPU resources depend on tolerable latency and data rate, but for existing techniques latency and data rate, but for existing techniques this is not presently a limitation: images analyzed this is not presently a limitation: images analyzed essentially in real time. essentially in real time.

Moore’s Law rules.Moore’s Law rules.


1616


1717

How “hard” is data storage and How “hard” is data storage and transfer?transfer?

• Typical images are at most 256Typical images are at most 25633 for 3-d for 3-d

At 16 bits this corresponds to 33 MBytesAt 16 bits this corresponds to 33 MBytes

• Typical fMRI image archive is a few TBytesTypical fMRI image archive is a few TBytes

• Terabyte data sets are common today. 1 TB Terabyte data sets are common today. 1 TB of RAID disk costs ~4K$. Cheap!of RAID disk costs ~4K$. Cheap!

• Bottleneck is in image transfer – At 100 Bottleneck is in image transfer – At 100 Kbytes/sec, a 33 Mbyte image takes > 5 Kbytes/sec, a 33 Mbyte image takes > 5 minutes to transfer. minutes to transfer.


1818

From Images to Numbers: Quantitative From Images to Numbers: Quantitative AnalysisAnalysis

• Present clinical practice is uses expert Present clinical practice is uses expert judgment to obtain qualitative assessment, judgment to obtain qualitative assessment, often as a narrativeoften as a narrative

• Move to quantitative analysis has been slow Move to quantitative analysis has been slow

• Present research focus assumes pathology is Present research focus assumes pathology is manifested as morphological change – manifested as morphological change – identification of surfaces and boundaries, identification of surfaces and boundaries, determination of volumesdetermination of volumes

• Eventually, we’ll have clinical applications of Eventually, we’ll have clinical applications of “functional molecular imaging”“functional molecular imaging”


1919

Why Quantitative Image Analysis is Why Quantitative Image Analysis is HardHard

• Uncalibrated dataUncalibrated data

• Geometrical registrationGeometrical registration

• Things move over timeThings move over time

• Automated feature Automated feature recognitionrecognition

• Parameter Parameter fitting/extractionfitting/extraction

• UncertaintiesUncertainties

• Quantitative comparison Quantitative comparison with relevant group or past with relevant group or past historyhistory

S. Koonin (really)


2020

Mapping Macaque Brain Mapping Macaque Brain Differences with Geometric FlowDifferences with Geometric Flow

Faisal Beg, Johns Hopkins, from M. Miller talkFaisal Beg, Johns Hopkins, from M. Miller talk


2121

Geometrical “Flows” to Map Geometrical “Flows” to Map Changes Changes

(From M. Miller talk)

Distortion computed Distortion computed by defining a by defining a

continuous flow fieldcontinuous flow field


2222

(High-z Supernova Team)

Image SubtractionImage Subtraction


2323

Ideally....Ideally....

<xml id=“mricat" src=“mri_catalog.xml"></xml><xml id=“mricat" src=“mri_catalog.xml"></xml><table border="1" datasrc="#mricat"><table border="1" datasrc="#mricat"><tr><tr><td><span datafld=“PATIENT"></span></td><td><span datafld=“PATIENT"></span></td><td><span datafld=“IMAGE"></span></td><td><span datafld=“IMAGE"></span></td></tr></tr></table></table>......

12343 55 234

12456 22 45

125564 5 54

12667 300 56

12789 340 89

128546 44 66

129345 506 12

13045 234 78

13134 349 45

....

Raw data, with Raw data, with calibration info.calibration info.

Code bankCode bankwith version controlwith version control

Image Image generationgeneration

codecode

htt

p:/

/rad

iolo

gy.b

idm

c.h

arv

ard

.ed

u/p

rev/

htt

p:/

/rad

iolo

gy.b

idm

c.h

arv

ard

.ed

u/p

rev/

Head

ing

s/M

ed

Pro

fs/R

ese

arc

h/p

ag

et_

pap

er/

Head

ing

s/M

ed

Pro

fs/R

ese

arc

h/p

ag

et_

pap

er/

case

s/ca

ses.

htm

lca

ses/

case

s.h

tml

Reconstructed Reconstructed ImageImage

Image Image analysisanalysis

codecode

htt

p:/

/rad

iolo

gy.b

idm

c.h

arv

ard

.ed

u/p

rev/

htt

p:/

/rad

iolo

gy.b

idm

c.h

arv

ard

.ed

u/p

rev/

Head

ing

s/M

ed

Pro

fs/R

ese

arc

h/p

ag

et_

pap

er/

Head

ing

s/M

ed

Pro

fs/R

ese

arc

h/p

ag

et_

pap

er/

case

s/ca

ses.

htm

lca

ses/

case

s.h

tml

Sectioned Sectioned ImageImage

Feature Feature InterpretationInterpretation

codecode

\tarsal 4 \length 44 \width 20\tibia \width 34.2 \density 12\fibia \width 22 \density 5\fracture \x 12 \y 22 \signif

30...

Extracted Extracted ParameteParamete

rsrs


2424

From Numbers to Knowledge - From Numbers to Knowledge - DatabasesDatabases

• Current clinical approach assumes pathology is manifested Current clinical approach assumes pathology is manifested in morphologyin morphology

• Extraction of relevant image parameters is an ill-posed Extraction of relevant image parameters is an ill-posed problem – morphological analysis of variable blobsproblem – morphological analysis of variable blobs

• Feature parameters are only meaningful in the context of Feature parameters are only meaningful in the context of comparison populations and/or historycomparison populations and/or history

• Need (image features + clinical information + history) Need (image features + clinical information + history)

• Drives a need for merged data structuresDrives a need for merged data structures

Now starting in the context of patient recordsNow starting in the context of patient records

No ability to query across populationsNo ability to query across populations

• Need to include calibration data and uncertainties alsoNeed to include calibration data and uncertainties also


2525

Data Management IssuesData Management Issues

• Database issues – schema, indexing, transaction Database issues – schema, indexing, transaction modelmodel

• LongevityLongevity

• Confidentiality legalitiesConfidentiality legalities

• Resiliency/RedundancyResiliency/Redundancy

• Interoperability and federation across databasesInteroperability and federation across databases

• Metadata standardsMetadata standards

• Proprietary vs. Open standardsProprietary vs. Open standards

• Data Access!Data Access!


2626

Database IssuesDatabase Issues

• We see database challenges as a major We see database challenges as a major oncoming freight trainoncoming freight train

• Current databases do a poor job of dealing Current databases do a poor job of dealing with images as database objectswith images as database objects

• Present practice in other fields is to store a Present practice in other fields is to store a pointer to the image data within the pointer to the image data within the database structuredatabase structure

• Effectively exploiting image archives will Effectively exploiting image archives will require a queryable database that returns require a queryable database that returns an image stack an image stack


2727

The Database OpportunityThe Database Opportunity• Merge image archives with databases that Merge image archives with databases that

contain full data pedigree:contain full data pedigree:

Image header informationImage header information

Analysis pipeline version data and Analysis pipeline version data and parametersparameters

Calibration informationCalibration information

Extracted image feature parametersExtracted image feature parameters

Clinical presentation and annotationsClinical presentation and annotations

Effective user interfaceEffective user interface


2828

Some Goals Some Goals

SQL-compatible database of extracted feature SQL-compatible database of extracted feature parameters, that contains pointers to imagesparameters, that contains pointers to images

““Find me the set of images that contain identified Find me the set of images that contain identified features like this one..., with the following additional features like this one..., with the following additional criteria”criteria”

Pipelined on-the-fly execution of a new image Pipelined on-the-fly execution of a new image analysis algorithm, on a query-selected analysis algorithm, on a query-selected subset of images in archive subset of images in archive

““I have a new way to identify tumors, and want to run it I have a new way to identify tumors, and want to run it on all archived lung images”on all archived lung images”

Image-based query toolsImage-based query tools

““Return all images that have things that look like this....”Return all images that have things that look like this....”


2929

An Example of a Federated An Example of a Federated Distributed System: Biomedical Distributed System: Biomedical

Informatics Research Network (BIRN)Informatics Research Network (BIRN)

• Ambitious program attempting to link Ambitious program attempting to link

distributed data sets, across disparate distributed data sets, across disparate

length scaleslength scales

• Appears to us that the PI’s are well versed in Appears to us that the PI’s are well versed in

contemporary computer technologycontemporary computer technology

• Links across agencies and disciplinesLinks across agencies and disciplines


3030

• 3D or 4D Data Refinement

• Data Reduction

• Database Deposition

• Data Acquisition

Genome DB’s(M. Ellisman slide)(M. Ellisman slide)


3131

BIRN Project ObjectivesBIRN Project Objectives(from M. Ellisman talk)(from M. Ellisman talk)

• Establish a Establish a stable, high performance networkstable, high performance network linking key linking key Biotechnology Centers and Clinical Research CentersBiotechnology Centers and Clinical Research Centers

• Establish Establish distributed and linked data collectionsdistributed and linked data collections with partnering with partnering groups -groups -

• Facilitate the use of computational GRID infrastructure and integrate Facilitate the use of computational GRID infrastructure and integrate BIRN with other middleware projects -BIRN with other middleware projects -

• Enable Enable data miningdata mining from from multiple data collections or databasesmultiple data collections or databases on on neuroimaging and bioinformaticsneuroimaging and bioinformatics - -

• Build a Build a stable software and hardware infrastructurestable software and hardware infrastructure that will allow that will allow centers to coordinate efforts to centers to coordinate efforts to accumulate larger studiesaccumulate larger studies than can than can be carried out at one site.be carried out at one site.

BIRN ‘Test-Beds” have very clear technical and scientific BIRN ‘Test-Beds” have very clear technical and scientific goals!goals!


3232

Data Access IssuesData Access Issues

• Biomedical Imaging culture does not have a Biomedical Imaging culture does not have a heritage of sharing code or imagesheritage of sharing code or images

• This is in contrast to other communities, This is in contrast to other communities, even with in the life sciences (genetic data, even with in the life sciences (genetic data, for example)for example)

• A thoughtful examination of NASA data A thoughtful examination of NASA data management, with lessons learned, is management, with lessons learned, is available at available at http://adc.gsfc.nasa.gov/~gass/linsky/linsky.html http://adc.gsfc.nasa.gov/~gass/linsky/linsky.html


3333

NIH Data Access Policy (Feb 2003)NIH Data Access Policy (Feb 2003)

“ “Starting with the October 1, 2003 receipt date, Starting with the October 1, 2003 receipt date, investigators submitting an NIH application investigators submitting an NIH application seeking $500,000 or more in direct costs in any seeking $500,000 or more in direct costs in any single year are expected to include a plan for single year are expected to include a plan for data sharing or state why data sharing is not data sharing or state why data sharing is not possible.”possible.”

“ “Reviewers Reviewers will not factor the proposed data-will not factor the proposed data-sharing plan into the determination of scientific sharing plan into the determination of scientific merit or priority scoremerit or priority score. Program staff will be . Program staff will be responsible for overseeing the data sharing responsible for overseeing the data sharing policy and for assessing the appropriateness and policy and for assessing the appropriateness and adequacy of the proposed data-sharing plan.” adequacy of the proposed data-sharing plan.”

(http://grants1.nih.gov/grants/guide/notice-files/NOT-OD-03-(http://grants1.nih.gov/grants/guide/notice-files/NOT-OD-03-032.html)032.html)


3434

““Sharing Images”, Radiology, July 2003Sharing Images”, Radiology, July 2003Vannier and SummersVannier and Summers


3535

A Typical Biomedical Imaging A Typical Biomedical Imaging PaperPaper

My clever My clever algorithmalgorithm

Random image, beforeRandom image, before Random image, afterRandom image, after

No common set of test imagesNo common set of test imagesSource code usually unavailableSource code usually unavailableNo repository of before and after imagesNo repository of before and after imagesFew quantitative comparison metricsFew quantitative comparison metrics


3636

Examples of Image DatabasesExamples of Image DatabasesFrom Vannier et al, Proceedings of the IEEE, in pressFrom Vannier et al, Proceedings of the IEEE, in press


3737

A Middle Ground: Computer-Assisted Qualitative A Middle Ground: Computer-Assisted Qualitative AnalysisAnalysis

• Interactive visualization tools Interactive visualization tools

• Feature identification and highlightingFeature identification and highlighting

• Identification of relevant comparison imagesIdentification of relevant comparison images


3838

Interactive Visualization ExampleInteractive Visualization Example

Courtesy of Chris Johnson, University of UtahCourtesy of Chris Johnson, University of Utah


3939

How “hard” is uncertainty How “hard” is uncertainty visualization?visualization?

• Heritage of xray films is a strong influenceHeritage of xray films is a strong influence

• Interactive displays are useful, anecdotal evidence of Interactive displays are useful, anecdotal evidence of valuevalue

• There is a research community working on multi-There is a research community working on multi-dimensional visualization, including uncertaintiesdimensional visualization, including uncertainties

• We think the impediments to incorporating We think the impediments to incorporating uncertainties are twofolduncertainties are twofold

Intrinsic: imaging community seldom uses Intrinsic: imaging community seldom uses statistical/likelihood formulationstatistical/likelihood formulation

Cultural: physicians not accustomed to thisCultural: physicians not accustomed to this


4040

A summary: A summary: Present day and 5 year outlookPresent day and 5 year outlook

Computational ChallengeComputational Challenge TodayToday + 5 + 5 yrsyrs

Adequate CPU cyclesAdequate CPU cycles

Sufficient Data StorageSufficient Data Storage

Computer-assisted qualitative analysisComputer-assisted qualitative analysis

Full Quantitative AnalysisFull Quantitative Analysis

Fused/merged imagesFused/merged images

Visualization: Meaningful and usefulVisualization: Meaningful and useful

Connectivity: Access to remote dataConnectivity: Access to remote data

Merged DB structures: images + Merged DB structures: images + parametersparameters

Query/display tools, access to distr. Query/display tools, access to distr. datadata


4141

An Alternative Approach....?An Alternative Approach....?

• Our perspective thus far has been extending current Our perspective thus far has been extending current state-of-the-art.state-of-the-art.

• An alternative approach would be to use modeling & An alternative approach would be to use modeling & computational power to do what radiologists do, computational power to do what radiologists do, namely impose an informed filter on the imaging data namely impose an informed filter on the imaging data – using “laws” of physiology and prior experience – using “laws” of physiology and prior experience they collapse from many degrees of freedom to only they collapse from many degrees of freedom to only a few. a few.

• Can this be accomplished using physical laws Can this be accomplished using physical laws (mechanics, elastic properties, anatomic (mechanics, elastic properties, anatomic constraints...)? constraints...)?

• Full functional models at imaging resolution? Full functional models at imaging resolution?

• This is a This is a majormajor computational task, for each image. computational task, for each image.


4242

Recommendation #1: Recommendation #1: Implement the BISTI Implement the BISTI recommendationsrecommendations

• The Biomedical Information Science and The Biomedical Information Science and Technology Initiative (BISTI) report Technology Initiative (BISTI) report recommended investment to provide access recommended investment to provide access to a hierarchy of computer resources. to a hierarchy of computer resources.

• Availability of state-of-the-art computing Availability of state-of-the-art computing platforms is essential to the success of the platforms is essential to the success of the biomedical imaging community. biomedical imaging community.

• The only way to benefit from Moore’s Law is The only way to benefit from Moore’s Law is to periodically buy new hardware... to periodically buy new hardware...


4343

Recommendation #2: Recommendation #2: Calibrate!Calibrate!

• Tradition of qualitative analysis of images Tradition of qualitative analysis of images has not required procedures that has not required procedures that compensate for distortion and other compensate for distortion and other instrumental systematicsinstrumental systematics

• Calibration of each apparatus, e.g. MRI Calibration of each apparatus, e.g. MRI system, would allow for real registration of system, would allow for real registration of imagesimages

MRI scanner

Regular lattice Known and stable spacing


4444

Calibration Data not Typically Bundled with Calibration Data not Typically Bundled with ImagesImages

Typical geometrical Typical geometrical distortion from distortion from clinical MRI scannerclinical MRI scanner

Tradition of Tradition of qualitative analysis qualitative analysis has hindered has hindered quantitative quantitative calibrationcalibration


4545

Calibrations Can Aid in Fusing Imaging Calibrations Can Aid in Fusing Imaging DataData

Registration of images can use feature Registration of images can use feature recognition, or actual geometrical recognition, or actual geometrical

coordinatescoordinates

Incorporating calibration data into Incorporating calibration data into image structure allows post-image structure allows post-

processing processing

www.egi.comwww.egi.com


4646

Recommendation #3: Recommendation #3: Develop and distribute an open archive Develop and distribute an open archive

of standard test images.of standard test images.

• Typical medical image analysis paper uses Typical medical image analysis paper uses proprietary data set, qualitative proprietary data set, qualitative before/after comparisonbefore/after comparison

• NIBIB could promote apples-to-apples NIBIB could promote apples-to-apples comparison with “Bio-Lena” data set, both comparison with “Bio-Lena” data set, both raw and image dataraw and image data

New algorithms would use these New algorithms would use these test data, along with other test data, along with other images...images...

Ideally, upload results to an Ideally, upload results to an archive for community accessarchive for community access


4747

Standardized Test Image SetsStandardized Test Image Sets

• Use this as a testbed for Use this as a testbed for

metadata standardsmetadata standards

• National Library of Medicine National Library of Medicine

Visible Man is good starting Visible Man is good starting

pointpoint

• Include images from all Include images from all

modalities, model systemsmodalities, model systems

• Eventually, link to genetic data Eventually, link to genetic data

Mouse Atlas, Richard Leahy


4848

Recommendation #4: Recommendation #4: Cultivate an “open source” Cultivate an “open source”

approach to data sets and code approach to data sets and code

• Considerable spectrum across different scientific Considerable spectrum across different scientific

disciplines in their approach to proprietary vs. disciplines in their approach to proprietary vs.

open access data setsopen access data sets

NASA limits proprietary access to 12 monthsNASA limits proprietary access to 12 months

Gene sequences placed in archive upon Gene sequences placed in archive upon publicationpublication

Some federally funded projects have Some federally funded projects have immediate release of all dataimmediate release of all data

• Evolve to curating a CVS repository for both Evolve to curating a CVS repository for both

images and codes?images and codes?


4949

An NIH Example of Open Source An NIH Example of Open Source DevelopmentDevelopment

• Analysis of Functional NeuroImages (AFNI)Analysis of Functional NeuroImages (AFNI)

Robert Cox, NIMHRobert Cox, NIMH

• ModularModular

• Open SourceOpen Source

• ExtensibleExtensible


5050

Interactive Analysis with Interactive Analysis with AFNI (from L. AFNI (from L. Frank)Frank)

Graphing voxeltime series data

Displaying imagesfrom time series

ControlPanel


5151

Recommendation #5: Recommendation #5: Promote Computer-assisted Promote Computer-assisted

Qualitative Analysis in Clinical Qualitative Analysis in Clinical SettingSetting

• Near-term enhancement in clinical setting likely to Near-term enhancement in clinical setting likely to come from computer-assisted qualitative analysis come from computer-assisted qualitative analysis of biomedical imagesof biomedical images

• Interactive visualization tools: feature highlighting, Interactive visualization tools: feature highlighting, both automated and human-generatedboth automated and human-generated

• Computer-assisted differential diagnosis: local Computer-assisted differential diagnosis: local libraries of relevant comparison images?libraries of relevant comparison images?

• Essential to link extracted image information with Essential to link extracted image information with clinical records - natural language data objects?clinical records - natural language data objects?


5252

Recommendation #6:Recommendation #6:Develop Appropriate DatabasesDevelop Appropriate Databases

• Explore database schema that incorporate Explore database schema that incorporate imagesimages

Some relevant examples: FBI, NIMA, Some relevant examples: FBI, NIMA, astronomyastronomy

• Develop 20 queries that will guide index Develop 20 queries that will guide index structurestructure

• Design, deploy and iterate a simple end-to-end Design, deploy and iterate a simple end-to-end systemsystem

• Needs to be integrated with user interface and Needs to be integrated with user interface and visualization toolsvisualization tools


5353

Recommendation #7: Recommendation #7: Pose Grand Challenge Problems to Pose Grand Challenge Problems to

the Biomedical Imaging the Biomedical Imaging Community Community

Issue some Grand Challenges to foster developmentIssue some Grand Challenges to foster development

• Multi-mode image analysisMulti-mode image analysis

• Multi-scale integration Multi-scale integration

• ““Ground truth” imaging tests with blind Ground truth” imaging tests with blind phantomsphantoms

• Metadata/interoperabilityMetadata/interoperability

• Like protein-folding competitions, which have been Like protein-folding competitions, which have been successful and productive. successful and productive.

• Use this as tool to foster cross-pollination with Use this as tool to foster cross-pollination with other fields, and to accomplish social engineering.other fields, and to accomplish social engineering.

Documents

Computational Aspects of Medical Imaging