A Computationally Efficient Approach for 2D-3D Image

Registration

Juri Minxha Medical Image AnalysisProfessor Benjamin Kimia

Spring 2011

Brown University

Review of Registration

Similarity Metric Optimization

1. Similarity Metric Mutual Information, Cross-Correlation, Correlation Ratio,

Cross Correlation Residual Entropy

2. Optimization, Non-gradient vs. GradientGauss-Newton, steepest descent, Levenberg-Larquardt, simplex method etc.

The main challenge is:Minimize computation time

Brown University

Review of the SoCV similarity metric

1. Similarity Measure: Sum of Conditional Variances

2. Optimization Algorithm: Gauss-Newton

1. Requires computation of gradient2. Fast convergence

Brown University

Similarity Metric: SoCV

I0 Ro

R0 =100·ln(256-I0)-300

1. Quantize images to 64 possible values2. Each pixel in the image on the left corresponds to

a bin in the histogram (64 x 64 bins)3. Notice the non-linear relationship between

I and R

Similarity Metric: SoCV

What happens if I0 is translated to the right?

For each value of R, we have a range of values in I’

Similarity Metric: SoCV

Compute the conditional expectation/mean of this distribution

Replace each value in R with the conditional mean

Similarity Metric: SoCV

Optimization: Gauss-NewtonGoal: Find values of 3D rigid-body transform that minimize S

Registration with CT, Fluoroscopy

I) Last time: registration of MRI-MRI

2) This time: registration of CT and fluoroscopy

1. CT volume (512x512x369) of alligator bone

2. A fluroroscopy video (30 seconds, 30 FPS)

3D-visualization of CT data (Slicer)

The big bone must be removed before projection!

Fluoroscopy video

1. ~30 seconds at 30 frames per second

2. The location of the bone is unknown throughout the video

3. Bone is being translated and Rotated

4. All artifacts must be removed before registration

Segmentation of smaller bone from CT

Individual frames Stacking the layers

Thresholding to remove background and onlyTaking the frames that correspond to top bone

Ray-Casting: Generating the DRR

50 100 150 200 250 300 350 400 450 500

50

100

150

200

250

300

350

400

450

500

20 40 60 80 100 120 140

50

100

150

200

250

300

350

400

450

500

20 40 60 80 100 120 140

50

100

150

200

250

300

350

400

450

500

Projecting the volume data onto the Three main axes.

Preparing the fluoroscopy image

Measure of Fit

50 100 150 200 250 300 350 400

50

100

150

200

250

300

350

400

50 100 150 200 250 300 350 400

50

100

150

200

250

300

350

400

Optimization over three parameters: Rotation in x Rotation in y Rotation in z

Manually adjusted three parameters: Translation in xTranslation in yTranslation in z

Shortcomings1. There is no ground truth2. Optimization was over a subset of the 6 parameters

required for rigid body motion3. The initial conditions are extremely important

1. The initial position has to be very close the final position

2. I manually translated the image, a procedure that should be automated

3. Speed optimization 4. Computational needs are excessive

References• A computationally efficient approach for 2D-3D image registration

Haque, M.N.; Pickering, M.R.; Biswas, M.; Frater, M.R.; Scarvell, J.M.; Smith, P.N.; 2010 Annual International Conference of the IEEEEngineering in Medicine and Biology Society (EMBC), Issue Date: Aug. 31 2010-Sept. 4 2010, On page(s): 6268 – 6271

• M. Pickering, A. Muhit, J. Scarvell, and P. Smith, "A new multimodalsimilarity measure for fast gradient-based 2D-3D imageregistration," in Proc. IEEE Int. Conf. on Engineering in Medicineand Biology (EMBC), Minneapolis, USA, 2009, pp. 5821-5824.

Brown University