Taking Imaging to Task in IG Interventions · Presented at the AAPM-PSOC "Horizons" Meeting (Nov. 7, 2013) (Bethesda MD) 1 Taking Imaging to Task in IG Interventions Jeff Siewerdsen,

J. H. Siewerdsen (Dept. of Biomedical Engineering, Johns Hopkins University)

11/8/2013

Presented at the AAPM-PSOC "Horizons" Meeting (Nov. 7, 2013) (Bethesda MD) 1

Taking Imaging to Taskin IG Interventions

Taking Imaging to Taskin IG Interventions

Jeff Siewerdsen, PhD FAAPMDepartment of Biomedical Engineering

Department of Computer ScienceRussell H. Morgan Department of Radiology

Armstrong Institute for Patient Safety and QualityJohns Hopkins University

Funding Support / DisclosuresNational Institutes of Health

Siemens HealthcareCarestream Health

Elekta Oncology

Guidance + InterventionPlanning Localization Verification

Image-Guided Interventions (IGI)

•T12•T11

Population Data

Interventional Device Models

Preoperative Images

AtlasInformation

Disease + Biomechanical

Models

PatientData


11/8/2013


Hybrid OR

Imaging in the ORMR‐Guided

AMIGO

(siemens.com) (imris.com)

BWH (ncigt.org)

Planned Planned

DeliveredDelivered

3D C-Arm Guidance3D Image Quality3D Image Quality Image ReconstructionImage Reconstruction

Video Overlay

PA

Target

Fissure

System IntegrationSystem IntegrationImage RegistrationImage Registration


11/8/2013


c

CBCT-Guided InterventionsHead & Neck / Skull Base Spine Orthopaedic Neurosurgery

Thoracic (VATS)Base of Tongue (TORS) Interventional Radiology

Patient‐specific priorsPreoperative images (CT, MR, PET)Planning information (trajectories and volume definitions)Intraoperative images (2D 3D multi‐modality)

Taking Imaging to Task

Intraoperative images (2D, 3D, multi‐modality)

Procedure‐specific priorsInterventional devices

Population, statistical, and model‐based priorsAtlases

Specification of the imaging taskSize, Spatial frequency contentContrast, LocationContrast, Location

fx

1 2F H x H x

Statistical Decision Theory (Hypothesis Testing)

Task FunctionWtask(f )Hypothesis 1

H1(x)Hypothesis 2H2(x)

fy


11/8/2013


Understand how a specification of imaging task can be leveraged to drive the imaging processTask-based detectability Design, optimizationTask-driven imaging

Learning Objectives

Task driven imaging

Understand how model-based image reconstruction can propel application in IGIImprove image quality, reduce doseUse of prior information

Understand how intraoperative imaging offers not only a means of high-precision guidance, y g p g ,but also a system for improved safety and quality assurance.Streamlined workflowLocalization, verificationIndependent checks, QA

Image Quality


11/8/2013


Image Quality… and Imaging Task

Prewhitening observer (PW)(PW)

2MTF f

Detectability Index (d’)

Spatial Resolution

222' taskNPS f

MTFdW

fd f

Image Noise Imaging Task

ICRU Report #54

DoseDetector Configuration

SamplingElectronic Noise

DetectionDiscriminationClassification

Image Quality… and Imaging TaskDetectability Index (d’)

Prewhitening observer (PW)Generalized (include anatomical background etc.)Non-prewhitening observer (NPW)

4

4

22 2

2

2 2int

'Q E

task

taskBS

EMTF f W

S f

dfd

MTF f W df

f

E f NfS f

p g ( )Anthropomorphic (e.g., eye filter, NPWEi)

Eye FilterInternal Noise

ICRU Report #54

Burgess et al., JOSA A (14) 1997

Wagner et al. Med Phys (4) 1977

also papers by Richard, Tward, Prakash, Gang (Med Phys 2005‐13)

Quantum NoiseElectronic Noise

AnatomicalBackground


11/8/2013


Detectability IndexExample Application: Task‐Based Technique Selection

Siewerdsen, NIM Phys. Rev. A (2012)

Soft-Tissue ImagingFresh cadaver (non-fixed)

Model-Based CBCT Reconstruction

PL + Huber Penalty (PLH) vs FBPRecon parameters for matched resolution

80 HU contrastESF width σ = 1 mm

Wang et al. PMB (in review)Wang et al. PMB (in review)


11/8/2013


6

8

10

12

14

16

CN

R

1.4× CNR

½ DoseLow Dose Comparison100 kVp (15 – 120 mAs)

PLH

FBP

Model-Based CBCT Reconstruction

ESF width = 1 mm

PLH

20 40 60 80 100 1200

2

4

mAs

PLH reconstructionMatched spatial resolution (ESF = 1 mm)Reduces noisePreserves high contrast structures

FBP

FBP

60 mAs30 mAs 120 mAs15 mAsWang et al. PMB (in review)Wang et al. PMB (in review)

Model-Based Image ReconstructionFor example: Penalized Likelihood (PL) Estimation

An objective function (L) with a roughness penalty (R) on noisy data:Improves low-dose image quality, robust to limited (sparse) dataN it ti l ith t l th bj tiNumerous iterative algorithms to solve the objective:

- EM, ordered subsets, paraboloidal surrogates, …

Log-Likelihood

Balances data fidelityversus the penalty

Measured Projections

Regularization / PenaltyDifference invoxel values

Penalty function(e.g., linear, quadratic, Huber, …)

Image Estimate


11/8/2013


ˆ arg max[lo ; )g ]( TL y R

Projection Data Image Estimate Reconstruction

Penalized Likelihood (PL)

Imaging Performancein Model-Based Reconstruction

Object dependence through its projections

T 1 TD{ ( )[A ] A}A+y RLLocal linearization of the reconstruction algorithm

arg max[lo ; )g ](L y R

Quadratic PenaltyPenaltyStrength

Log-Likelihood

Covariance and Noise‐Power Spectrum

Tˆ[cov{ }] cov{ }j jy L LNoise in

Reconstructions

,NPS FT[cov{ } ]j Nj,N

Local stationarity

Noise in Projections

Covariance and Noise‐Power Spectrum

jth VoxelLocation Dependence

G. Gang et al. SPIE Physics of Medical Imaging (2013)G. Gang et al. SPIE Physics of Medical Imaging (2013)J. J. FesslerFessler et al. IEEEet al. IEEE‐‐TMI (1996)TMI (1996)

Image Noise in Model-Based Reconstruction

2.0

x10-6

1.0

NPS(NPS(ffxx,f,fyy))PL ReconstructionPL Reconstruction24 cm Ellipse24 cm EllipseMonoenergeticMonoenergetic beambeamNo bowtie filterNo bowtie filter360 views, 360360 views, 360oo

11 mGymGy

0

1 1 mGymGy

G. Gang et al. SPIE Physics of Medical Imaging (2013)G. Gang et al. SPIE Physics of Medical Imaging (2013)G. Gang et al. Med Phys (under review)G. Gang et al. Med Phys (under review)


11/8/2013


Image Noise in Model-Based Reconstruction

2.0

x10-6

1.0

NPS(NPS(ffxx,f,fyy))

0.4

0.6d'(d'(x,yx,y))PL ReconstructionPL Reconstruction24 cm Ellipse24 cm EllipseMonoenergeticMonoenergetic beambeamNo bowtie filterNo bowtie filter360 views, 360360 views, 360oo

11 mGymGy

0

Imaging Task(e.g., Line‐Pair Discrimination)

0

0.2

1 1 mGymGy

G. Gang et al. SPIE Physics of Medical Imaging (2013)G. Gang et al. SPIE Physics of Medical Imaging (2013)G. Gang et al. Med Phys (under review)G. Gang et al. Med Phys (under review)

Detector

Task-Driven ImagingTask-Based Modeling

Task-Driven ImagingStart with a specification of task

Location in patientSpatial frequencies of interest

TraditionalCircular

Trajectory

X-raySource

TaskTask--DrivenDrivenTrajectoryTrajectory

p qTask-based image quality model

+ Model-based reconstruction Drive the image acquisition process

Exploit knowledge of the imaging system, patient, and reconstruction process

Define constraints of geometry, dose, Nproj

Acquire projections that maximize task performance

Preoperative CTPlanning Data

Task Definition

Task-DrivenAcquisition

StaymanStayman and and SiewerdsenSiewerdsen, Fully3D (2013), Fully3D (2013)


11/8/2013


Detectability Index (d’) and Trajectory ()

Task-Driven Imaging

22

22,

x y z

j

Taskj df df dWMTF fd

Local Predictor of NPS and MTF

2NPS

2,j

xj j kas yT zNPSd

df df fF W dMT

1 1, , , ,N N

0

x10-6

,NPS FT[cov{ } ]jN N

,MTF FT[PSF{ } ]jN N


Task-Driven ImagingDetectability Index (d’) and Trajectory ()

22

22,

x y z

j

Taskj df df dWMTF fd

Design Goal

1 1, , , ,N N “Standard”= 0° Orbit

Oblique0 Orbit

2,j

xj j kas yT zNPSd

df df fF W dMT

2ˆ arg max ,jd

Task-Driven

Orbit



11/8/2013


d’2(, |{}) 1st Best Projection (1,1)d’2(, |{1,1}) 2nd Best Projection (2,2)3rd Best Projection (3,3)d’2(, |{1,1},{2,2})d’2(, |{1,1},{2,2},{3,3}) 4th Best Projection (4,4)5th Best Projection (5,5)d’2(, |{1,1},{2,2},…,{4,4}) 6th Best Projection (6,6)d’2(, |{1,1},{2,2},…,{5,5}) 7th Best Projection (7,7)d’2(, |{1,1},{2,2},…,{6,6}) 8th Best Projection (8,8)d’2(, |{1,1},{2,2},…,{7,7})d’2(, |{1,1},{2,2},…,{8,8}) 9th Best Projection (9,9)10th Best Projection (10,10)d’2(, |{1,1},{2,2},…,{9,9}) 15th Best Projection (15,15)d’2(, |{1,1},{2,2},…,{14,14})d’2(, |{1,1},{2,2},…,{19,19}) 20th Best Projection (20,20)25th Best Projection (25,25)d’2(, |{1,1},{2,2},…,{24,24}) 30th Best Projection (30,30)d’2(, |{1,1},{2,2},…,{29,29}) 35th Best Projection (35,35)d’2(, |{1,1},{2,2},…,{34,34})-30

-20

-10

0

degrees)

Detectability Map

Task-Driven Imaging

0 50 100 150 200

10

20

30

(d

(degrees)

Ray Traversing Center of Nodule

21 1, arg max , |d

Objective Function

,

22 2 1 1

,, arg max , | ,d

23 3 1 1 2 2

,, arg max , | , , ,d


(deg)

-30

-10

“Bad” 220°Trajectory

d’2(, |{1,1},{2,2},…,{35,35})

Task-Driven Imaging

“Bad” 220° Trajectory Task-Based 220° Trajectory

50 100 150 200

(deg)

10

30

Task-Based 220°Trajectory

Lung Nodule

Surgical Tool



11/8/2013


“Bad” 220° Trajectory Task-Based 220° Trajectory

Task-Driven Imaging


-30

-20

-10

d’2(, |{1,1},{2,2},…,{35,35})

g)

Preoperative CTPlanning Data

Detectability Map

Task-Driven Imaging

0 50 100 150 200

0

10

20

30

(d

e

(deg)

Planning Data

Task Definition



11/8/2013


Standard Circular Short Scan Trajectory

Task-Driven Imaging

Task-Driven 215° Trajectory


Improve geometric precisionTarget ablationAvoidance of normal tissues

Promote minimally invasive techniques

From Precision… to SafetyThe Role of Intraoperative Imaging

o o e a y as e ec quesReduced co-morbidity, faster recoveryTreatment of otherwise “untreatable” disease

Support innovation in advanced proceduresAdvanced delivery systems (e.g., robotics, …)Integration of therapies (e.g., IGRT + IGS)

P ti t f t d OR lit (ORQA)

High‐PrecisionSurgery

Patient safety and OR quality assurance (ORQA)Integration of information independent checksStreamlined localization and guidance (w/o trackers)Detect complications, RFBs in the ORAssessment of the surgical productQuantitative measurement of the surgical product

Expose fundamental factors determining outcome

Broader Utilization


11/8/2013


CT (Prone)

3D-2D Registrationas a Safety Check vs. Wrong‐Level Surgery

CT (Supine)

OtakeOtake et al. Phys Med et al. Phys Med BiolBiol 57(17) (2012)57(17) (2012)

CC--arm Fluoroscopyarm FluoroscopyProne + SupineProne + SupineKyphosisKyphosis + + LordosisLordosisThorax, Abdomen, PelvisThorax, Abdomen, Pelvis

3D-2D Registrationas a Safety Check vs. Wrong‐Level Surgery

LevelCheck

T12

Failure criterion 5mm

L5

L4

L3

L5

L4

L3

L2

L1

T12

L5

L4

L3

L2

L1

L3

L2

L1

T12

T11

T10

T9

mPDE

Th

orax

Abd

ovi

s

2.4

2.0

1.7

2.4

2.0

1.7

2.4

OtakeOtake et al. Phys Med et al. Phys Med BiolBiol 57(17) (2012)57(17) (2012)

Before registration

After registration

L3

Clinical Studies at Johns Hopkins Hospital(mm)Deformation Case

Pel

v

2.0

1.7


11/8/2013


Joint Registration + ReconstructionKnown‐Component Reconstruction (KCR)Simultaneous registration and reconstructionSeparate the estimation process:

‐ Reconstruction of an unknown background A t

Unknown Background

Anatomy, ‐ Registration of a known component therein

Device models, I(n)

*

* *,

( )* ,ˆˆ , arg max ;n

IL y R

Known Component

I(n)

Registration Regularization

Data Fit (Likelihood)

Stayman et al IEEEStayman et al IEEE‐‐TMI (2012)TMI (2012)

Joint Registration + ReconstructionKnown‐Component Reconstruction (KCR)

True Volume FBP Penalized-Likelihood KCR

Reduced artifactsImproved image quality, reduced dose

Quality AssuranceQuantitative Assessment / Verification

f th S i l P d tp g q y,

Extension to deformable components

Stayman et al SPIE (2013)Stayman et al SPIE (2013)Stayman et al IEEEStayman et al IEEE‐‐TMI (2012)TMI (2012)

of the Surgical Product

( )nI


11/8/2013


Understand how a specification of imaging task can be leveraged to drive the imaging process Task-driven imaging

Learning Objectives

Task driven imaging

Understand how model-based image reconstruction can propel application in IGI Penalized likelihood estimation (PLH) Joint registration-reconstruction (KCR)

Understand how intraoperative imaging offers not only a means of high-precision guidance, y g p gbut also a system for improved safety and quality assurance. 3D2D registration (LevelCheck) Verification of surgical product (ORQA)

AcknowledgmentsI‐STAR LaboratoryImaging for Surgery, Therapy, and Radiology

www.jhu.edu/istar

Clinical CollaboratorsJay Khanna (Ortho Spine Surgery)Ziya Gokaslan (Neuro Spine Surgery)Gary Gallia (Neurosurgery)Gary Gallia (Neurosurgery)Doug Reh (Otolaryngology)Jeremy Richmon (Otolaryngology)Marc Sussman (Thoracic Surgery)John Carrino (MSK Radiology)Peter Pronovost (Armstrong Institute)

BMEJ Web Stayman, Wojciech ZbijewskiAdam Wang, Yoshito OtakeSebastian SchaferSajendra NithiananthanSajendra NithiananthanGrace Gang, Jen XuHao Dang, Steve Tilley

Comp SciRuss TaylorGreg HagerAli UneriDan MirotaSureerat Reaungamornrat

ECEJerry Prince

RadiologyKen TaguchiM Mahesh

Rad OncJohn WongJunghoon Lee

Documents

Taking Imaging to Task in IG Interventions · Presented at the AAPM-PSOC "Horizons" Meeting (Nov. 7, 2013) (Bethesda MD) 1 Taking Imaging to Task in IG Interventions Jeff Siewerdsen,