Challenges in Modern Medical Image Reconstruction · Challenges in Modern Medical Image Recon-struction Misha E. Kilmer Forward & Inverse Problems Reduced Order Image Models: Motivation

Challenges inModernMedical

Image Recon-struction

Misha E.Kilmer

Forward &InverseProblems

ReducedOrder ImageModels:Motivation

ParametricLevel Sets

ReducedOrderForwardModel

Trust RegionRegularizedGN

Results

Challenges in Modern Medical ImageReconstruction

Misha E. Kilmer

Department of MathematicsTufts University

SIAM ALA, June 2012

Thanks: NIH R01-CA154774

1 / 36



Misha E.Kilmer






Results

Outline

1 Forward & Inverse Problems

2 Reduced Order Image Models: Motivation

3 Parametric Level Sets

4 Reduced Order Forward Model

5 Trust Region Regularized GN

6 Results

2 / 36



Misha E.Kilmer






Results

Collaborators

PaLS and Diffuse Optical Tomography

• Prof. Eric Miller, ECE, Tufts

• Dr. Alireza Aghasi, ECE, GA Tech

• Mr. Fridrik Larusson, ECE, Tufts

• Prof. Sergio Fantini, BME, Tufts

ROM for Diffuse Optical Tomography

• Prof. Serkan Gugerin, Math, VA Tech

• Prof. Chris Beattie, Math, VA Tech

• Prof. Eric de Sturler, Math, VA Tech

• Dr. Saifon Chaturantabut, Math, VA Tech

Trust-region Regularized GN

• Prof. Eric de Sturler, Math, VA Tech

3 / 36



Misha E.Kilmer






Results

Forward Problem Description

The discrete forward problem for an observation vector m is

m = M(f) + η

where f is a discretized representation of a property, f(x) ofthe medium: e.g., f is a (vectorized) 2D or 3D “image” of

• electrical conductivity

• mass density

• optical absorption

• etc.

and M represents the appropriate (often nonlinear) model map.

4 / 36



Misha E.Kilmer






Results

Generic Inverse Problem Formulation

The inverse problem refers to the recovery of vectorized imagef ∈ Rm×n(×k) given the model M and noisy data m.

In many (medical) image reconstruction problems, surfacemeasurements ⇒ underdetermined, ill-posed.

minf∈Rm×n×k

‖m−M(f)‖2 + λ2Γ(f)

Regularization needed to to damp noise, force unique soln.

Question: Is a voxel-based reconstruction overkill and/or overlyambitious?

5 / 36



Misha E.Kilmer






Results

Motivation: Diffuse Optical Tomography

Find images of optical absorption/diffusion using measurementsof near infrared (frequency modulated) light on surface.

1

ν

∂

∂tη(x, t) = ∇ ·D(x)∇η(x, t)− µ(x)η(x, t) + bj(x)uj(t),

for x ∈ Ω

0 = η(x, t) + 2AD(x)∂

∂ξη(x, t), for x ∈ ∂Ω±

mi(t) =

∫∂Ωci(x)η(x, t) dx for i = 1, . . . , ndet

(see S. R. Arridge, Inverse Problems, 1999).

6 / 36



Misha E.Kilmer






Results

An Example: Breast Tissue Imaging

Breast tissue made up of adipose, fibroglandular, tumor. Each,a different optical contrast, but within each, fairly uniform.

Figure : Ben Brooksby, et. al Imaging breast adipose andfibroglandular tissue molecular signatures by using hybrid MRI-guidednear-infrared spectral tomography PNAS 2006 103 (23) 8828-8833 7 / 36



Misha E.Kilmer






Results

Shape-based Approach

Sufficient to model unknown f as piecewise continuous.

χD(x) =

1 x ∈ D0 x ∈ Ω\D.

In a continuous setting, the unknown property f(x) can bedefined over Ω

f(x) = fi(x)χD(x) + fo(x)(1− χD(x))

Goal: find ∂D (and parameters defining fi, fo).

8 / 36



Misha E.Kilmer






Results

Traditional Level Sets

Traditional level set (Santosa, ‘96), let φ : Ω→ R.• ∂D := (x, y) ∈ Ω|z = φ(x, y) = 0.• Topologically quite flexible (no. regions not spec. a priori)• Evolve the 3D function to pick up right no. connected

components by minimizing a cost function

Figure : Thanks, Wikipedia!9 / 36



Misha E.Kilmer






Results

Traditional Level Set Approach

If H(r) = 12(1 + sign(r)),

f(x) = fi(x)H(φ(x)− c) + fo(x)(1−H(φ(x)− c))

=

fi(x) φ(x) ≥ cfo(x) φ(x) < c

If fi, fo are known, the optimization problem is to recover φ.

Can be made to work for inverse problems but...

• Highly non-trivial implementation of evolution, speedfunction, initialization, etc.

• Rate of convergence

• Regularization

• Specialized optimization (e.g. van den Doel et al, Journalof Sci. Comp. 2010; van den Doel and Ascher, SISC 2012)

10 / 36



Misha E.Kilmer






Results

PaLS

Instead, φ function of m-length parameter vector p AND x:

φ(x,p) =

m0∑i=1

αiψi(x)

and p contains the expansion coefficients, and any parametersthat define the ith basis function.

Now, the property to be recovered is described

f(x,p) = fi(x)Hε(φ(x,p)− c) + fo(x) (1−Hε(φ(x,p)− c)) ,

where Hε differentiable surrogate for Heaviside function.

Solve for the parameter vector p that defines φ.

11 / 36



Misha E.Kilmer






Results

Choice of PaLS Function

φ(x,p) should be linear combination of “basis functions” orparametrically defined functions. Incomplete list:

• Polynomial basis [K., Miller, et al, SPIE Proceedings,‘04]

• Sinusiods, exponentials [Tarokh, NEU Ph.D. Thesis, ‘05]

• Multiquadric RBFs (topology optimization)[Wang & Wang, Int’l J. for Num. Mtds. Eng,‘06]

• CSRBFs (segmentation only) [Gelas et al, IEEE TIP,‘07]

• B-splines (model evolution application) [Bernard et al,IEEE TIP,‘09]

• CSRBFs for inverse problems [Aghasi, K., Miller,SIAM J. Imag. Sci, ‘11]

12 / 36



Misha E.Kilmer






Results

CSRBF from Wendland, Cambridge Univ. Press,‘05

A CSRBF: ψ(r) = (max(0, 1− r))2 (2r + 1); r =√x2 + y2.

13 / 36



Misha E.Kilmer






Results

Weighted CSRBFs

Let ψ : R+ → R denote a sufficiently smooth CSRBF.

φ(x,p) :=

m0∑j=1

αjψ(‖βj(x− χ(j))‖†),

where the χ(j) are the centers, βj are dilation factors, αj areexpansion coefficients and

‖x‖† :=√‖x‖22 + ν2

Desired parameter vector defining shape(s) p =

abχxχyχz

.

14 / 36



Misha E.Kilmer






Results

CSRBFs

Usingψj(x) := ψ(‖βj(x− χ(j))‖†)

we can think of our level set function φ(x,p) as a weightedsum of CSRBFs or “bumps”.

Advantages:

• Summing appropriately paired/inverted bumps can giveedges

• Compact support can imply sparse updates in context ofoptimization

15 / 36



Misha E.Kilmer






Results

Behavior - Geometry

Considering c-level sets with c ≈ 0 → representation of objectswith edges, complex geometries, with only a few CSRBF’s.

−1 1−1

1

16 / 36



Misha E.Kilmer






Results

Narrow Banding Illustration

Possible the supports of ∂∂φHε = δε(φ− c) and ∂φ

∂pjfor jth

parameter pj do not intersect, leading to efficiencies in theoptimization algorithm.

−1 1−1

1

supp(∂φ

∂µj)

supp(∂φ

∂µj0

)

supp(δ2,ε(φ− c))

17 / 36



Misha E.Kilmer






Results

Optimization Revisited

f(x,p) = fi(x)Hε(φ(x,p)− c) + f0(x)(1−Hε(φ(x,p)− c))

Consider fi(x) = fi and fo(x) = fo, (if unknown, append) andf(p) the discretization of f(x,p) over a grid.

minp‖m−M(f(p))‖2

Nonlinear LS problem of relatively small dimension, often noadditional regularization except stopping criterion.

18 / 36



Misha E.Kilmer






Results

Related Work

PaLS for

• Hyperspectral diffuse optical tomography, linear forwardmodel [see Larusson et al, Biomed. Optics Expr., ’2012]

• Dual energy X-ray Computed Tomography [see Semerciand Miller, IEEE TIP ’2011]

• 2D Limited angle CT & ERT, [see Aghasi, K., Miller,SIAM J. Imag. Sci., ’2011]

• 3D joint recon using electrical impedance tomography(EIT) & hydrology data [Aghasi, et al, in review]

19 / 36



Misha E.Kilmer






Results

When the Forward Model is Nonlinear ...

Remaining Concerns:

• When M is nonlinear, bottleneck still forward (adjoint)model evaluations.

• Ill-conditioned Jacobian, even if narrow banding exploitedto determine the search direction.

20 / 36



Misha E.Kilmer






Results

Reduced Order (Forward) Model for DOT

DOT-PaLS model, dynamical system notation1:

1

νE y(t; p) = −A(p)y(t; p)+Bu(t) with m(t; p) = Cy(t; p)

where E,A(p)Rn×n and B ∈ Rn×m,C ∈ Rn×`.

If uj(ω) is FT of jth source uj ,

mj(ω; p) = Ψ(ıω; p) uj(ω)

where

Ψ(s; p) = C( sν

E + A(p))−1

B

maps from sources (inputs) to measurements (outputs).

1Alternative approach, see Arridge et al, Inverse Problems, 200621 / 36



Misha E.Kilmer






Results

Reduced Order (Forward) Model

To solve minp ‖m−M(f(p))‖2, must evaluate M(f(p(k))) ⇒compute mj(ω`; p

(k)), for j = 1, . . . , nsrc, ` = 1, . . . , nω.Many large-scale PDE solves2 per optimization step!

Use a cheaper-to-evaluate approx. Ψr(s,p) such that

Ψ(s,p) ≈ Ψr(s,p) or equivalently mj(s,p) ≈ mr,j(s,p)

mj(s,p) = Ψ(s,p) uj(s) Ψ(s,p) = C (sE−A(p))−1 B

mr,j(s,p) = Ψr(s,p) uj(s) Ψr(s,p) = Cr (sEr −Ar(p))−1 Br

2Alternative to MRHS, ‘simultaneous source’ plus SSA/SA; see, e.g.Haber, 2011 presentation

22 / 36



Misha E.Kilmer






Results

Reduced Order Model

• Choose r-dimensional right modeling subspace (the trialsubspace) Ra(Vr), where Vr ∈ Rn×r

• Choose r-dimensional left modeling subspace (testsubspace) Ra(Wr), where Wr ∈ Rn×r

• Approximate y(t) ≈ Vryr(t) by forcing yr(t) to satisfy

WTr (EVryr −A(p)Vryr −B u) = 0 (Petrov-Galerkin)

• Leads to a reduced order model:

Er = WTr EVr, Br = WT

r B,

Ar(p) = WTr A(p)Vr, Cr = CVr,

• Here, use fact A(p) = A[0] + A[1](p).

23 / 36



Misha E.Kilmer






Results

Reduced Order Model

• Enforce Ψ(s,p) ≈ Ψr(s,p) via Rational Interpolation• Choose π1,π2, . . . ,πJ ∈ Rν and σ1, σ2, . . . , σK ∈ C

Ψ(σk,πj) = Ψr(σk,πj)

Ψ′(σk,πj) = Ψ′r(σk,πj)

∇pΨ(σk,πj) = ∇pΨr(σk,πj)

for k = 1, . . . ,K and j = 1, . . . , J .• Per opt. step: r × r system solves ∀ src, det, freq.; eval

WTr A(p(k))Vr. Up front cost to produce Vr,Wr, but

can be reused!

Serkan Gugercin, 12:15-12:40 in MS 8, “Interpolatory modelreduction strategies for nonlinear parametric inversion”

Eric de Sturler, 11:25-11:50 in MS 6, “Updatingpreconditioners for parameterized systems”

24 / 36



Misha E.Kilmer






Results

Trust Region Regularized GN for ParametricInversion3

The Gauss-Newton Model for solving the NLS problem:

mGN(p) =1

2rT r + rTJ(p− pc) +

1

2(p− pc)

TJTJ(p− pc),

where r = r(pc) and J = ∇r(pc).Let the reduced SVD of the m× n current Jacobian J withrank n be

J = UΣVT =

n∑i=1

σiuivTi ,

Then GN and LM have search steps that can be written

sΘ = −n∑i=1

viuTi r

σi· θi = −VΘΣ−1UT r,

where Θ = diag(θ1, . . . ,θn).3de Sturler and K., SISC, 2011.

25 / 36



Misha E.Kilmer






Results

SVD Analysis

R(sΘ) := mGN(pc)−mGN(pc+ sΘ) =1

2

n∑i=1

(uTi r)2θi(2−θi).

gives the estimated reduction of the objective function.Study this for (dM)GN and LM, we observe:

• Overdamping of directions that could have reducedfunction rapidly

• Emphasize large(r) singular values BUT this may ignoreimportant directions: Must take rhs into account

• Look at relative sizes of |uTi r|

• Avoid for which |uTi r|/σi could have undue influence;

others critical directions• Some (possibly damped) step in all critical directions

• Use (dual) trust region approach to limit step size

• If full GN step fits in TR, take it.26 / 36



Misha E.Kilmer






Results

TREGS in a Nutshell

Working with the reduced, compressed SVD

U = [UA,UB,UC ], V = [VA,VB,VC ],

Σ−1

= diag(Σ−1A ,Σ−1

B ,Σ−1C )

Θ = diag[θi], reordered θ’s

• group A: θi = 1

• group B: θi ∈ (0, 1)

• group C: θi = 0

Step is s = −VΘΣUT r.In combination with the GCV condition this leads to aguaranteed reduction proportional to‖JT r‖ ⇒ global convergence

27 / 36



Misha E.Kilmer






Results

Results Summary

• Lab results to show potential of PaLS with real data

• 2D Synthetic results to show the power of using ROMwith PaLS approach

• 3D Synthetic results to show the ability to recover edgesand illustrate superiority of TREGS in optimizing for PaLSparameters

28 / 36



Misha E.Kilmer






Results

Experimental Results, DOT Data4

• black rod(s), .5cm diameter

• emersed in milk/water solution

• 2D slice-by-slice PaLS reconstructions, 7cm×6.5cm over7cm length

• 7 CSRBFs, randomly chosen to start

• 72 data points per 2D image, image resolution 64 x 71

4Results thanks to F. Larusson, E. Miller, S. Fantini29 / 36



Misha E.Kilmer






Results

Experimental Results, DOT Data5

Single rod angled in the y-z plane. Excellent thicknessreconstruction, depth variation 5mm from truth




Misha E.Kilmer






Results

Experimental Results, DOT Dat6

Dual rods, different depths, appear overlapping from thisperspective. Truth, dashed lines.




Misha E.Kilmer






Results

ROM Example

32 sources & 32 detectors; heterogeneous noise + additiveOriginal System Size: n = 160,801Parametric Sample points: 5Up front cost: *) 10 n× n systems, 32 RHS per system

*) compressed SVD on n× 320ROM size: r = 250

No. Fun evals No. Jac evals blk systems sys sizeFOM 23 12 35 n× nROM 25 13 38 r × r

For other recons, reuse the V,W!

32 / 36



Misha E.Kilmer






Results

ROM Example, Con’t

50 100 150 200 250 300 350 400

50

100

150

200

250

300

350

400

0.02

0.04

0.06

0.08

0.1

0.12

0.14

0.16

0.18

50 100 150 200 250 300 350 400

50

100

150

200

250

300

350

400

0.02

0.04

0.06

0.08

0.1

0.12

0.14

50 100 150 200 250 300 350 400

50

100

150

200

250

300

350

400

0.02

0.04

0.06

0.08

0.1

0.12

0.14

50 100 150 200 250 300 350 400

50

100

150

200

250

300

350

400

0.02

0.04

0.06

0.08

0.1

0.12

0.14

33 / 36



Misha E.Kilmer






Results

3D TREGS vs. LM

15× 15 array src & dets 4cm × 4cm × 4cm, 32× 32× 21 gridm0 = 125 (CSRBFs in 5× 5× 5 grid) .05 percent Gaussiannoise; discrepancy stopping

34 / 36



Misha E.Kilmer






Results

TREGS: 44 Fev, 29 Jev; LM: 229 Fev, 52 Jev

6 7 8

9 10 11

12 13 14

6 7 8

9 10 11

12 13 14

6 7 8

9 10 11

12 13 14

35 / 36



Misha E.Kilmer






Results

Summary and Current, Future Work

• PaLS useful for medical imaging where pw cont. reducedmodeling makes sense. CSRBFs can capture shape, havenarrow-banding advantage

• Resulting nonlinear least squares problem solved efficientlyby TREGS

• For inverse problems involving nonlinear forward model,ROM-approach that exploits parameterized image model ispromising for reducing the computational bottleneck

• To Do: 3D nonlinear (hyperspectral) DOT. Merge withmeasurement sampling techniques [van den Doel &Ascher, 2011]?

• Other imaging modalities

36 / 36

Documents

Challenges in Modern Medical Image Reconstruction · Challenges in Modern Medical Image Recon-struction Misha E. Kilmer Forward & Inverse Problems Reduced Order Image Models: Motivation