Upload
brenda-wilkerson
View
228
Download
0
Tags:
Embed Size (px)
Citation preview
1
ITK Lecture 13:Registration
ITK Lecture 13:Registration
Damion Shelton
Methods in Image Analysis
CMU Robotics Institute 16-725
U. Pitt Bioengineering 2630
Spring Term, 2006
Damion Shelton
Methods in Image Analysis
CMU Robotics Institute 16-725
U. Pitt Bioengineering 2630
Spring Term, 2006
2
For more info/gory detail…For more info/gory detail…
Please see Chapter 8 in the Software Guide and Insight into Images for exhaustive detail
Please see Chapter 8 in the Software Guide and Insight into Images for exhaustive detail
3
What is registration?What is registration?
The process of aligning a target image to a source image
More generally, determining the transform that maps points in the target image to points in the source image
The process of aligning a target image to a source image
More generally, determining the transform that maps points in the target image to points in the source image
4
Transform typesTransform types
Rigid (rotate, translate) Affine (rigid + scale & shear) Deformable (affine + curvature) Many others
Rigid (rotate, translate) Affine (rigid + scale & shear) Deformable (affine + curvature) Many others
5
Registration in ITKRegistration in ITK
ITK uses an extensible registration framework
Various interchangeable classes exist Relatively easy to “twiddle” the part
you’re interested in while recycling prior work
ITK uses an extensible registration framework
Various interchangeable classes exist Relatively easy to “twiddle” the part
you’re interested in while recycling prior work
6
ITK terminologyITK terminology
Fixed image f(x) - stationary in space Moving image m(x) - the fixed image
with an unknown transform applied Goal: recover the transform T(x) which
maps points in f(x) to m(x)
Fixed image f(x) - stationary in space Moving image m(x) - the fixed image
with an unknown transform applied Goal: recover the transform T(x) which
maps points in f(x) to m(x)
7
Registration framework piecesRegistration framework pieces 2 input images, fixed and moving Metric - determines the “fitness” of the
current registration iteration Optimizer - adjusts the transform in an
attempt to improve the metric Interpolator - applies transform to image
and computes sub-pixel values
2 input images, fixed and moving Metric - determines the “fitness” of the
current registration iteration Optimizer - adjusts the transform in an
attempt to improve the metric Interpolator - applies transform to image
and computes sub-pixel values
8
ITK registration flowchartITK registration flowchart
9
ITK’s “Hello world” exampleITK’s “Hello world” example 2D floating point image inputs Please see the software guide (section
8.2) for code specifics I’m going to cover what each piece
does, not look at code per se
2D floating point image inputs Please see the software guide (section
8.2) for code specifics I’m going to cover what each piece
does, not look at code per se
10
Input imagesInput images
2D floating point Floating point avoids loss of precision
problems with integer pixel types
2D floating point Floating point avoids loss of precision
problems with integer pixel types
11
TransformTransform
TranslationTransform Permits translation only in 2D Documentation notes similarity to
AffineTransform, but improved speed (why…?)
TranslationTransform Permits translation only in 2D Documentation notes similarity to
AffineTransform, but improved speed (why…?)
12
MetricMetric
MeanSquaresImageToImageMetric Sum of squared differences between 2
images on a pixel-by-pixel basis A bit naïve Works for 2 images that were acquired
with the same imaging modality
MeanSquaresImageToImageMetric Sum of squared differences between 2
images on a pixel-by-pixel basis A bit naïve Works for 2 images that were acquired
with the same imaging modality
13
OptimizerOptimizer
RegularStepGradientDescent Follows the derivative of the metric Step size depends on rapid changes in
the gradient Step size eventually reaches a user-
defined value that determines convergence
RegularStepGradientDescent Follows the derivative of the metric Step size depends on rapid changes in
the gradient Step size eventually reaches a user-
defined value that determines convergence
14
InterpolatorInterpolator
LinearInterpolateImageFunction Fast and conceptually simple
LinearInterpolateImageFunction Fast and conceptually simple
15
WrapperWrapper
ImageRegistrationMethod Combines all of the previous classes
into a master class
registration->SetMetric( metric );registration->SetOptimizer( optimizer );registration->SetTransform( transform );registration->SetInterpolator( interpolator);
ImageRegistrationMethod Combines all of the previous classes
into a master class
registration->SetMetric( metric );registration->SetOptimizer( optimizer );registration->SetTransform( transform );registration->SetInterpolator( interpolator);
16
Other stepsOther steps
Set the region of the fixed image the registration will operate on (e.g. to ignore bad data)
Initialize the transform Twiddle the optimizer for best
performance*
Set the region of the fixed image the registration will operate on (e.g. to ignore bad data)
Initialize the transform Twiddle the optimizer for best
performance*
*may involve magic
17
Hello world inputHello world input
18
X & Y translation vs. timeX & Y translation vs. time
19
Metric vs. timeMetric vs. time
20
Registration resultsRegistration results
After registration converges/terminates you call GetLastTransformParamters to recover the final transform
For the Hello World example there are 2 parameters, X & Y translation
After registration converges/terminates you call GetLastTransformParamters to recover the final transform
For the Hello World example there are 2 parameters, X & Y translation
21
Double checking resultsDouble checking results
Use ResampleImageFilter to apply the transform for the fixed image
Take the output, compute a difference image with the moving image, examine the results
Good registration results in nothing much to see
Use ResampleImageFilter to apply the transform for the fixed image
Take the output, compute a difference image with the moving image, examine the results
Good registration results in nothing much to see
22
Image comparisonImage comparison
23
Keeping tabs on registrationKeeping tabs on registration Registration is often time consuming It’s nice to know that your algorithm isn’t
just spinning it’s wheels Use the observer mechanism in ITK to
monitor progress You’ve seen this before with filters
Registration is often time consuming It’s nice to know that your algorithm isn’t
just spinning it’s wheels Use the observer mechanism in ITK to
monitor progress You’ve seen this before with filters
24
Observer stepsObserver steps
Write an observer class that will process “iteration” events
Add the observer to the optimizer As a generic note, observers can observe
any class derived from itk::Object
Start registration as usual
Write an observer class that will process “iteration” events
Add the observer to the optimizer As a generic note, observers can observe
any class derived from itk::Object
Start registration as usual
25
Things observers can doThings observers can do
Print debugging info Update GUI Other small management functions Should not do anything too processor
intensive
Print debugging info Update GUI Other small management functions Should not do anything too processor
intensive
26
Multi-modality registrationMulti-modality registration
Remember how I said sum-of-squares difference is relatively naïve?
Mutual information helps overcome this problem
Section 8.4 shows how to implement a simple MI registration
Remember how I said sum-of-squares difference is relatively naïve?
Mutual information helps overcome this problem
Section 8.4 shows how to implement a simple MI registration
27
Notes about the MI exampleNotes about the MI example Significantly, largely the same piece of
code as Hello World Mutual Information is a metric, so we
can keep the optimizer, the interpolator, and so on
Majority of differences are in tweaking the metric, not in rewriting code
Significantly, largely the same piece of code as Hello World
Mutual Information is a metric, so we can keep the optimizer, the interpolator, and so on
Majority of differences are in tweaking the metric, not in rewriting code
28
MI InputsMI Inputs
T1 MRI Proton density MRI
29
MI OutputMI Output
Before After
30
Centered transformsCentered transforms
More natural (arguably) reference frame than having the origin at the corner of the image
Details are not appreciably different from other rigid registrations, see 8.5
More natural (arguably) reference frame than having the origin at the corner of the image
Details are not appreciably different from other rigid registrations, see 8.5
31
An aside: “Twiddling”An aside: “Twiddling”
A common criticism of many/most registration techniques is their number of parameters
A successful registration often depends on a very specific fine-tuning of the algorithm
“Generalized” registration is an open problem
A common criticism of many/most registration techniques is their number of parameters
A successful registration often depends on a very specific fine-tuning of the algorithm
“Generalized” registration is an open problem
32
Multi-Resolution registrationMulti-Resolution registration Useful to think of this as algorithmic
“squinting” Start with something simple and low-res Use low-res registration to seed the
next higher step Eventually run registration at high-res Also called “coarse to fine”
Useful to think of this as algorithmic “squinting”
Start with something simple and low-res Use low-res registration to seed the
next higher step Eventually run registration at high-res Also called “coarse to fine”
33
Multi-resolution schematicMulti-resolution schematic
34
Image pyramidsImage pyramids
35
OptimizationOptimization
Parameter dependency rears its ugly head
You often/usually need to adjust optimizer parameters as you move through the pyramid
You can do this using the Observer mechanism
Parameter dependency rears its ugly head
You often/usually need to adjust optimizer parameters as you move through the pyramid
You can do this using the Observer mechanism
36
Multi-resolution exampleMulti-resolution example
Again, mostly the same code as Hello World
Use MultiResolutionPyramidImage filter to build fixed and moving pyramids
MultiResolutionImageRegistrionMethod is now the overall framework
Again, mostly the same code as Hello World
Use MultiResolutionPyramidImage filter to build fixed and moving pyramids
MultiResolutionImageRegistrionMethod is now the overall framework
37
Benefits of multi-resolutionBenefits of multi-resolution Often faster More tolerant of noise (from “squinting”) Minimizes initialization problems to a
certain extent, though not perfect
Often faster More tolerant of noise (from “squinting”) Minimizes initialization problems to a
certain extent, though not perfect
38
See the software guide for…See the software guide for… Complete list of:
Transforms Optimizers Interpolation methods
You’re encouraged to mix and match!
Complete list of: Transforms Optimizers Interpolation methods
You’re encouraged to mix and match!
39
Deformable registrationDeformable registration
ITK has 2 techniques: Finite element: treat small image regions
as having physical properties that control deformation
Demons: images are assumed to have iso-intensity contours; image deformations occur by pushing on these contours
ITK has 2 techniques: Finite element: treat small image regions
as having physical properties that control deformation
Demons: images are assumed to have iso-intensity contours; image deformations occur by pushing on these contours
40
Model based registrationModel based registration
Build a simplified geometric model from a training set
Identify parameters that control the characteristics of the model
Register the model to a target image to adapt to a particular patient
Build a simplified geometric model from a training set
Identify parameters that control the characteristics of the model
Register the model to a target image to adapt to a particular patient
41
Model based, cont.Model based, cont.
Uses the Spatial Objects framework for representing geometry
Useful because it derives analytical data from the registration process, not just a pixel-to-pixel mapping
Uses the Spatial Objects framework for representing geometry
Useful because it derives analytical data from the registration process, not just a pixel-to-pixel mapping
42
Model-based reg. schematicModel-based reg. schematic
43
Model-based exampleModel-based example
44
Speed issuesSpeed issues
Execution time can vary wildly Optimizer (more naïve = faster) Image dimensionality (fewer = faster) Transform (fewer DOF = faster) Interpolator (less precise = faster)
Execution time can vary wildly Optimizer (more naïve = faster) Image dimensionality (fewer = faster) Transform (fewer DOF = faster) Interpolator (less precise = faster)
45
Take home messagesTake home messages
Exactly what parameters do what is not always obvious, even if you’re familiar with the code
Successful registrations can be something of an art form
Multi-resolution techniques can help Work within the framework!
Exactly what parameters do what is not always obvious, even if you’re familiar with the code
Successful registrations can be something of an art form
Multi-resolution techniques can help Work within the framework!