Clinical Prospects and Technological Challenges for Multimodality Imaging Applications in

Radiotherapy Treatment Planning

Clinical Prospects and Technological Challenges for Multimodality Imaging Applications in

Radiotherapy Treatment Planning

Issam El Naqa, PhD

Assistant ProfessorDepartment of Radiation Oncology

Washington University, School of Medicine, St. Louis, MO

SWAAPM Austin, TX, Spring 2008

Issam El Naqa, PhD

Assistant ProfessorDepartment of Radiation Oncology

Washington University, School of Medicine, St. Louis, MO

SWAAPM Austin, TX, Spring 2008

• Motivation– Increase usage of multimodality imaging

(CT,PET,MRI,MRS,US) in diagnostic, image-guided intervention, daily localization

– Complementary effect• Anatomical, physiological, soft tissue structures

– Improved target definition (Bradley et al ’04, Rasch et al ’05,

Zangheri et al ’05, Milker-Zabel et al ’06)

• Objectives– Integrate multiple information streams from different

imaging technologies to automatically define ‘biophysical’ target (normal structure) volume

• Motivation– Increase usage of multimodality imaging

(CT,PET,MRI,MRS,US) in diagnostic, image-guided intervention, daily localization

– Complementary effect• Anatomical, physiological, soft tissue structures

– Improved target definition (Bradley et al ’04, Rasch et al ’05,

Zangheri et al ’05, Milker-Zabel et al ’06)

• Objectives– Integrate multiple information streams from different

imaging technologies to automatically define ‘biophysical’ target (normal structure) volume

Why Multimodality Image Analysis?Why Multimodality Image Analysis?

Multimodality Image IntegrationMultimodality Image Integration

Anatomical Imaging Functional Imaging

MRI

CT

PETSPECT

MRS

PET/CT

Biophysical Target

Biophysical Target= ( , , ,...)f CT PET MRI

US

H&N Example: CT/MRI/PET H&N Example: CT/MRI/PET

Milker-Zabel et al., IJROBP ‘06

Prostate Example: CT/MRI/3D-TRUS Prostate Example: CT/MRI/3D-TRUS

Smith et al., IJROBP ‘07

Clinical Application ChallengesClinical Application Challenges• Increased acquisition time• Efficiency and automated delineation• Co-registration and fusion of different imaging data

– PET/CT, but how about other modalities?

• Increased acquisition time• Efficiency and automated delineation• Co-registration and fusion of different imaging data

– PET/CT, but how about other modalities?

Image RegistrationImage Registration

Image registrationImage registration

• Single modality deformable– In PET/CT registration of transmission images instead of

emission images

• Multimodality– Rigid

• PET to CT using normalized mutual information (NMI)

– Deformable Multimodality Registration• Feature based

• Volume Intensity based

• Single modality deformable– In PET/CT registration of transmission images instead of

emission images

• Multimodality– Rigid

• PET to CT using normalized mutual information (NMI)

– Deformable Multimodality Registration• Feature based

• Volume Intensity based

Deformable Registration (Level set)Deformable Registration (Level set)

Yang et al., SPIE ‘07

642

531

Improved Optical Flow Deformable RegistrationImproved Optical Flow Deformable Registration

Yang et al., ICCR 07

Multigrid

Multipass

Deformable Registration (Optical flow)Deformable Registration (Optical flow)

Yang et al., AAPM ‘07

Before registration After registration

Deformable Registration ToolDeformable Registration Tool

NMI Rigid Registration of Multimodality ImagesNMI Rigid Registration of Multimodality Images

MAX(NMI)

( ) ( )where

( , )

and ( ) is image entropy

H A H BNMI

H A B

H

+=

⋅

Example of NMI Registration (MR/CT)Example of NMI Registration (MR/CT)

Surface matching and FEM (Finite Element Method)

Surface matching and FEM (Finite Element Method)

FEM-based multi-organ deformable image registration (Brock et al., IJROBP ‘05)

Intensity RemappingIntensity Remapping

• Define the intensity mapping functionT(i) = f (s(i))+η(i)

• Finding function f through regressionf(s) = a0+a1*s+a2*s2+a3*s3+…+an*sn

• Bi-functional dependence: allow to remap one intensity value to two intensity values in the second image

• Define the intensity mapping functionT(i) = f (s(i))+η(i)

• Finding function f through regressionf(s) = a0+a1*s+a2*s2+a3*s3+…+an*sn

• Bi-functional dependence: allow to remap one intensity value to two intensity values in the second image

Multimodality optical flowMultimodality optical flow

• For any image registration:

• J(h) measure the distance (difference) between the moving image andthe fixed image. R(h) measure the variations of the motion field

• General solution:

• Similarity metrics– Mutual information– Cross-correlation– Correlation ratio

• For any image registration:

• J(h) measure the distance (difference) between the moving image andthe fixed image. R(h) measure the variations of the motion field

• General solution:

• Similarity metrics– Mutual information– Cross-correlation– Correlation ratio

Adaptive Radiotherapy Application: KVCT-MVCT Registration

Adaptive Radiotherapy Application: KVCT-MVCT Registration

Yang, Chaudhari, Goddu, Khullar,. Deasy, El Naqa

KVCTMVCT

Registered w/o correction

Registered w/ correction

Validation of Deformable Registrationusing a ‘biomechanical’ phantom

Validation of Deformable Registrationusing a ‘biomechanical’ phantom

Courtesy Deshan Yang (AAPM ‘07)

Image SegmentationImage Segmentation

ExamplesExamples

MR cardiac classification

(Unsupervised learning) (Zheng, El Naqa, ’05)

Microcalcification detection

(Supervised Machine learning)

(El Naqa et al. ’02)

PET/CT NSCLC delineation

(Active Contours)

Coronary stenosis detection (Edge detection and linking)

(El Naqa et al ’96)

Methods I: ClusteringMethods I: Clustering

Zheng et al., MRI ‘05

II. Active Contour Deformable ModelsII. Active Contour

Deformable Models

• Definition: Geometric representations for curves or surfaces that are defined explicitly or implicitly in the imaging domain. These models move (deform) under the influence of internal forces, which are defined within the curve or surface itself, and external forces, which are computed from the image data

• Pros– Boundary smoothness (continuity)– Subpixel accuracy – Prior information (atlas-based)– Mathematically tractable (PDE)– 2D curves are easily generalized to 3D surfaces

• Cons– PDE! (Numerical instability)

• Definition: Geometric representations for curves or surfaces that are defined explicitly or implicitly in the imaging domain. These models move (deform) under the influence of internal forces, which are defined within the curve or surface itself, and external forces, which are computed from the image data

• Pros– Boundary smoothness (continuity)– Subpixel accuracy – Prior information (atlas-based)– Mathematically tractable (PDE)– 2D curves are easily generalized to 3D surfaces

• Cons– PDE! (Numerical instability)

• Problems– non-convex optimization problem in (2)

– sensitivity to contour initialization

– dependency on parameterization

– inability to account for topological adaptation

• Problems– non-convex optimization problem in (2)

– sensitivity to contour initialization

– dependency on parameterization

– inability to account for topological adaptation

Parametric models--contParametric models--cont

Contour = cross section at L = 0 (i.e., {(x,y,z) | Φ (x,y,z;t) = 0})

L=0

L=+1

L=-1

L=+1

L=0

L=-1

Evolution in the normal direction

Geometric modelsGeometric models

PET Segmentation Examples

PET Segmentation Examples

Active Contour Segmentation (Synthetic Data)

Active Contour Segmentation (Synthetic Data)

Gradient-based Region-based

Active Contour Segmentation (Clinical Data)

Active Contour Segmentation (Clinical Data)

Gradient-based Region-based

El Naqa et al., ICCR ‘04

3D Active Contour Segmentation 3D Active Contour Segmentation

Multimodality Image AnalysisMultimodality Image Analysis

Algorithm to Apply to MultimodalityAlgorithm to Apply to Multimodality

Pre-processing: Motion-based Compensation in PETPre-processing: Motion-based Compensation in PET

Motion Blur

Deconvolution-corrected

45

67

8

2

4

6

8

100

2

4

6

8

10

12

Lateral (mm)Anterior-Posterior (mm)

Sup

erio

r-In

ferio

r (m

m)

El Naqa et al., Med Phys. ‘06

(b) (c) (d)

GTV-CTGTV-PETGTV-PET/CT

GTV-CTGTV-PETGTV-PET/CTInitializationMVLS

(a)CT

PET

Method II: Active Contours

(b) (c) (d)

GTV-PET (40% SUVmax)InitializationMVLS

CT

PET

(a)

(b)

(c)

(a) (b)

(c) (d) (e)

El Naqa et al., AAPM ‘06

(a)

Phantom Validation of Multimodality Concurrent

Segmentation I

Phantom Validation of Multimodality Concurrent

Segmentation I

Courtesy Sasa Mutic

CT

PET

MR

Phantom Validation of Multimodality Concurrent

Segmentation II

Phantom Validation of Multimodality Concurrent

Segmentation II

Phantom Validation of Multimodality Concurrent Segmentation III

Phantom Validation of Multimodality Concurrent Segmentation III

0

0.2

0.4

0.6

0.8

1

1 2 3 4

Balls

Ove

rlap

Ind

ex

PET/CT/MR CT only

0

2

4

6

8

1 2 3 4

Balls%

Err

or

in v

olu

me

esti

mat

e

PET/CT/MR CT only

El Naqa et al, ICIP ‘07

GUI Screen shot of the software tool. (1) image selector, (2) manual registration control, (3) window level control, (4) zoom control (5) 3D slice number control, (6) status information, (7) the working image panel, (8) ROI region contour, (9) not confirmed segmentation result, (10) right mouse click context menu, (11) menu, (12) the result display panel, zoomed in to ROI, (13) confirmed segmented regions, (14) separated 3D rendering window.

Multimodality Image Analysis ToolMultimodality Image Analysis Tool