Medical Image Medical Image RegistrationRegistration
IntroductionIntroduction
Medical Images Applications:Medical Images Applications: Clinical DiagnosisClinical Diagnosis Treatment PlanningTreatment Planning Image Guided Surgery (Intervention)Image Guided Surgery (Intervention) Retrospective Studies (Chemical/Beam Retrospective Studies (Chemical/Beam Therapy)Therapy) PrePre--Surgery Simulation (VR environment)Surgery Simulation (VR environment) Growth StudyGrowth Study TrainingTraining
Medical Imaging ModalitiesMedical Imaging Modalities Visual description (wide variety) of organs Visual description (wide variety) of organs characteristics:characteristics: CT (XCT (X--Ray absorption)Ray absorption) MRI (TMRI (T11, T, T22, and H, and H++ distribution density)distribution density) US (Reflection coefficients, TOF, Doppler)US (Reflection coefficients, TOF, Doppler) Gamma Camera (Nuclear source radiation)Gamma Camera (Nuclear source radiation) SPECT (Nuclear source radiation, Tomography)SPECT (Nuclear source radiation, Tomography) PET (Nuclear source radiation, Tomography)PET (Nuclear source radiation, Tomography) Video (Laparoscopy, Video (Laparoscopy, EndoscopyEndoscopy , and , and LaryngoscopyLaryngoscopy))
Medical ImagesMedical Images
Two major categories:Two major categories: Anatomical: Anatomical: •• AnatomyAnatomy
•• MorphologyMorphology Functional:Functional:•• PhysiologyPhysiology
•• MetabolismMetabolism
Anatomical ModalitiesAnatomical Modalities
ExamplesExamples XX--Ray (DSA)Ray (DSA) CT (CTA)CT (CTA) MRI (MRA)MRI (MRA) US (Doppler)US (Doppler) Video (XVideo (X--ScopyScopy))
Functional ModalitiesFunctional Modalities
Examples:Examples: Gamma CameraGamma Camera SPECTSPECT PETPET Brain Mapping (EEG/MEG)Brain Mapping (EEG/MEG) fMRIfMRI fCTfCT
Medical Image Integration (1)Medical Image Integration (1)
Why:Why: Too many data:Too many data:•• Limitation of conventional Diagnosis (hardcopy)Limitation of conventional Diagnosis (hardcopy) Data Dependency:Data Dependency:•• Multiple information from a single sliceMultiple information from a single slice Physical Requirements:Physical Requirements:•• StereotacticStereotactic surgery (Physical frames)surgery (Physical frames)
Medical Image Integration (2)Medical Image Integration (2)
The goal is data integration:The goal is data integration: RegistrationRegistration•• Bring the modalities involved into spatial Bring the modalities involved into spatial alignmentalignment Fusion:Fusion:•• Integrated display of the data involvedIntegrated display of the data involved Vocabulary:Vocabulary:•• WarpingWarping, Co, Co--Registration, Matching, Alignment, Registration, Matching, Alignment,
Normalization, Morphing.Normalization, Morphing.
Registration DefinitionRegistration Definition General Definition:General Definition: A Transform (Mathematical mapping) which A Transform (Mathematical mapping) which relatesrelatesposition of position of correspondencecorrespondence structures in two images.structures in two images. Correspondence:Correspondence:
•• Exact: Point by point.Exact: Point by point.
•• NonNon--Exact: Structure by structureExact: Structure by structure Two Images:Two Images:
•• SourceSource--DestinationDestination
•• HeadHead--HatHat
•• SourceSource--TargetTarget
Mathematical mappingMathematical mapping
ExamplesExamples
A Mathematical MappingA Mathematical Mapping
Funny ExampleFunny Example
Correspondence (1)Correspondence (1)
Related Point
Related Structure
Landmark
Correspondence (2)Correspondence (2)
Registration ExamplesRegistration Examples
Registration ExampleRegistration ExampleDefinitionDefinition of local of local landmarkslandmarks
DefinitionDefinition of a of a deformationdeformation modelmodelBeforeBefore registrationregistrationAfterAfter deformabledeformable registrationregistration AveragingAveraging of 9 of 9 brainsbrains
Fusion ExampleFusion Example
CT-SPECT Fusion MR-SPECT Fusion
General ApplicationGeneral Application
Uses of image registrationUses of image registration•• Image segmentation/deformable atlasImage segmentation/deformable atlas
•• Characterization of normal vs. abnormal Characterization of normal vs. abnormal Shape/variationShape/variation
•• MultiMulti--modality fusionmodality fusion
•• Functional brain mapping/removing shape variationFunctional brain mapping/removing shape variation
•• Surgical planning and evaluationSurgical planning and evaluation
•• Image guided surgeryImage guided surgery
•• PrePre--surgical Simulationsurgical Simulation
•• Other field (GIS and etc.)Other field (GIS and etc.)
Mathematical DefinitionMathematical Definition
An optimization problem:An optimization problem: Source and Target Images:Source and Target Images:
Seek for a Mapping: Seek for a Mapping:
•• Map features ofMap features of ΣΣ region of region of SS to same region of to same region of DD Goal of optimization:Goal of optimization:
( ) ( ) 1 2 1 2
, ,
1 2 1 21, 1 1, 1, ,x Y x YN N N N
S Dx x x xI x x I x x
= = = =
1 2( , ) :x xT T T S D= →
( ) ( ) ( ) ( )( )2
1 21 2
2
1 1 2 1 1 2 2 1 2,1 1
, arg min , , , ,yx
NN
x x D ST T
x x
T T I x x I T x x T x x= =
= − ∑∑
Classification of MethodsClassification of Methods
ObjectObjectSubjectSubjectModalities Modalities involvedinvolved
Optimization Optimization procedureprocedureInteractionInteractionDomain of Domain of
transformationtransformation
Nature of Nature of transformationtransformation
Nature of Nature of Registration Registration basisbasis
DimensionalityDimensionality
DimensionalityDimensionality
Image are spatial or temporally relatedImage are spatial or temporally related Spatial Dimension:Spatial Dimension:•• 2D2D--2D2D
•• 2D2D--3D3D
•• 3D3D--3D3D Temporal (Time Series) with spatial dimension:Temporal (Time Series) with spatial dimension:•• 2D2D--2D2D
•• 2D2D--3D3D
•• 3D3D--3D3D
Spatial Registration MethodSpatial Registration Method
3D/3D registration of two images3D/3D registration of two images
2D/2D registration:2D/2D registration: Less complex by an order of magnitudeLess complex by an order of magnitude
2D/3D registration2D/3D registration Direct alignment of spatial data to projective Direct alignment of spatial data to projective data.data. Alignment of a single Alignment of a single tomographictomographic slice to slice to spatial data. (Atlas based segmentation)spatial data. (Atlas based segmentation)
Temporal RegistrationTemporal Registration
Time series of images application:Time series of images application: Monitoring of bone growth in children (long Monitoring of bone growth in children (long time interval)time interval) Monitoring of tumor growth (medium interval)Monitoring of tumor growth (medium interval) PostPost--operative monitoring of healing (short operative monitoring of healing (short interval)interval) Observing the passing of an injected bolus Observing the passing of an injected bolus through a vessel tree (ultrathrough a vessel tree (ultra--short interval)short interval)
Two images need to be compared.Two images need to be compared.
Nature of Registration BasisNature of Registration Basis
Image basedImage based ExtrinsicExtrinsic•• Based on foreign objects introduced into the Based on foreign objects introduced into the
imaged space.imaged space. IntrinsicIntrinsic•• Based on the image information as generated by Based on the image information as generated by
the patientthe patient
NonNon--image based (calibrated coordinate image based (calibrated coordinate systems)systems)
Extrinsic registration methodsExtrinsic registration methods
AdvantageAdvantage:: Registration is easy, fast, and can be automated.Registration is easy, fast, and can be automated. No need for complex optimization algorithms.No need for complex optimization algorithms.
DisadvantageDisadvantage Prospective character must be made in the preProspective character must be made in the pre--acquisition phase.acquisition phase. Often invasive character of the marker objects.Often invasive character of the marker objects. Patient Problems (Movement, detachment)Patient Problems (Movement, detachment) NonNon--invasive markers can be used, but less accurate.invasive markers can be used, but less accurate.
Extrinsic registration methodsExtrinsic registration methods
InvasiveInvasive StereotacticStereotactic frameframe FiducialsFiducials (screw markers)(screw markers)
NonNon--invasiveinvasive Mould,frame,dentalMould,frame,dental adapter,etcadapter,etc FiducialsFiducials (skin markers)(skin markers)
Extrinsic registration methodsExtrinsic registration methods
The registration transformation is often The registration transformation is often restricted to be rigid (translations and restricted to be rigid (translations and rotations only)rotations only)
Rigid transformation constraint, and Rigid transformation constraint, and various practical considerations, use of various practical considerations, use of extrinsic 3D/3D methods are limited to extrinsic 3D/3D methods are limited to brain and orthopedic imagingbrain and orthopedic imaging
Intrinsic registration methodsIntrinsic registration methods
Landmark basedLandmark based
Segmentation basedSegmentation based
VoxelVoxel property basedproperty based
Landmark based registrationLandmark based registration
AnatomicalAnatomical:: Salient and accurately locatable points of the Salient and accurately locatable points of the morphology of the visible anatomy, usually morphology of the visible anatomy, usually identified by the useridentified by the user
GeometricalGeometrical:: Points (structure) at the locus of the optimum Points (structure) at the locus of the optimum of some geometric property, e.g., local of some geometric property, e.g., local curvature curvature extremaextrema, corners, etc, generally , corners, etc, generally localized in an automatic fashion.localized in an automatic fashion.
Landmark Based Landmark Based RedistrationRedistration
T(y) S(x) T(h(x))
h(x)⇒
Landmark based registrationLandmark based registration The set of registration points is sparse:The set of registration points is sparse: Fast optimization proceduresFast optimization procedures Optimize Measures :Optimize Measures : Average distance between each landmark.Average distance between each landmark. Closest counterpart (Procrustean Metric).Closest counterpart (Procrustean Metric). Iterated minimal landmark distances.Iterated minimal landmark distances. AlgorithmAlgorithm Iterative closest point (ICP)Iterative closest point (ICP) Procrustean optimumProcrustean optimum QuasiQuasi--exhaustive searches, graph matching and exhaustive searches, graph matching and dynamic programming approaches.dynamic programming approaches.
Segmentation based RegistrationSegmentation based Registration
Rigid model based:Rigid model based: Anatomically the same structures (mostly Anatomically the same structures (mostly surfaces) are extracted from both images to surfaces) are extracted from both images to be registered, and used as the sole input for be registered, and used as the sole input for the alignment procedure.the alignment procedure.
Deformable model based:Deformable model based: An extracted structure (also mostly surfaces, An extracted structure (also mostly surfaces, and curves) from one image is elastically and curves) from one image is elastically deformed to fit the second image.deformed to fit the second image.
Rigid Model BasedRigid Model Based
““HeadHead--HatHat”” method:method: Rely on the segmentation of the skin surface Rely on the segmentation of the skin surface from CT,MR, and PET images of the head.from CT,MR, and PET images of the head.
Chamfer matching:Chamfer matching: Alignment of binary structures by means of a Alignment of binary structures by means of a distance transformdistance transform
Deformable model basedDeformable model based
Deformable curves:Deformable curves: Snakes, active contours,nets(3D)
Data structure:Data structure: Local functions, i.e., splines
Deformable model approach:Deformable model approach: Template model defined in one image template is deformed to match second image:
• Segmented structure• Un-segmented
VoxelVoxel property based registrationproperty based registration
Operate directly on the image grey valuesOperate directly on the image grey values
Two approaches:Two approaches: Immediately reduce the image gray value Immediately reduce the image gray value content to a representative set of scalars and content to a representative set of scalars and orientationsorientations Use the full image content throughout the Use the full image content throughout the registration process.registration process.
VoxelVoxel Property Based RegistrationProperty Based Registration
T(y) S(x) T(h(x))
h(x)⇒
Principal axes and moments basedPrincipal axes and moments based
Image center of gravity and its principal Image center of gravity and its principal orientations (principal axes) are computed orientations (principal axes) are computed from the image zero and first order from the image zero and first order momentmoment Align the center of gravity and the principal Align the center of gravity and the principal
orientationsorientations Principal axes :Easy implementation, no high Principal axes :Easy implementation, no high accuracyaccuracy Moment based: require preMoment based: require pre--segmentationsegmentation
Full image content basedFull image content based
Use all of the available information Use all of the available information throughout the registration process.throughout the registration process.
Automatic methods presentedAutomatic methods presented
Example of MethodsExample of Methods CrossCross--correlationcorrelation Fourier domain based.Fourier domain based. Minimization of variance of gray values within Minimization of variance of gray values within segmentationsegmentation Minimization of the histogram entropy of Minimization of the histogram entropy of difference images.difference images. Histogram clustering and minimization of Histogram clustering and minimization of histogram dispersionhistogram dispersion Maximization of Maximization of mutual informationmutual information Minimization of the absolute or squared intensity Minimization of the absolute or squared intensity differencesdifferences
NonNon--image based registrationimage based registration
Calibrated coordinate systemCalibrated coordinate system If the imaging coordinate systems of the two If the imaging coordinate systems of the two scanners involved are somehow calibrated to scanners involved are somehow calibrated to each other, which necessitates the scanners each other, which necessitates the scanners to be brought in to he same physical locationto be brought in to he same physical location
Registering the position of surgical tools Registering the position of surgical tools mounted on a robot arm to imagesmounted on a robot arm to images
Nature of TransformationNature of Transformation
Rigid (Translation and Rotation)Rigid (Translation and Rotation)
Affine (Parallel Lines to Parallel lines)Affine (Parallel Lines to Parallel lines)
Projective (Line to Line)Projective (Line to Line)
Curved (Elastic)Curved (Elastic)
Domain of transformationDomain of transformation
GlobalGlobal Apply to entire imageApply to entire image
LocalLocal Subsections have their ownSubsections have their own
ExamplesExamplesRigid
Affine
Projective
Nonlinear
Local Global
TransformationTransformation
Many methods require a preMany methods require a pre--registration registration (initialization) using a rigid or affine (initialization) using a rigid or affine transformationtransformation
Global rigid transformation is used most Global rigid transformation is used most frequently in registration applications frequently in registration applications ((FiducialFiducial markers).markers).
Application: Human headApplication: Human head
InteractionInteraction
InteractiveInteractive
SemiSemi--automaticautomatic
AutomaticAutomatic Present method not act fine.Present method not act fine.
Goal:Goal: Minimal interaction and speed, accuracy, or Minimal interaction and speed, accuracy, or robustnessrobustness
InteractionInteraction
Extrinsic methodsExtrinsic methods AutomatedAutomated SemiSemi--automaticautomatic Intrinsic methodsIntrinsic methods SemiSemi--automaticautomatic
•• Anatomical landmarkAnatomical landmark•• Segmentation basedSegmentation based AutomatedAutomated•• Geometrical landmarkGeometrical landmark•• VoxelVoxel property basedproperty based
Optimization ProcedureOptimization Procedure
Parameters for registration transformationParameters for registration transformation Parameters computed AnalyticallyParameters computed Analytically Parameters searched for reach an criteria.Parameters searched for reach an criteria.
Methods:Methods: Downhill simplex methodDownhill simplex method LevenbergLevenberg--Marquardt optimizationMarquardt optimization Simulated annealingSimulated annealing Genetic methodsGenetic methods QuasiQuasi--exhaustive searchingexhaustive searching
Modalities involvedModalities involved
MonomodalMonomodal
MultimodalMultimodal
Modality to modelModality to model
Patient to modalityPatient to modality
MonomodalMonomodal
One patient, One modalityOne patient, One modality CT, MRI, US, PET, SPECT, and etc.CT, MRI, US, PET, SPECT, and etc. Registration of temporal sequencesRegistration of temporal sequences::•• Temporal deformation of anatomical structures Temporal deformation of anatomical structures
(heart, chest, blood flow)(heart, chest, blood flow)
•• Growth, Pathology followGrowth, Pathology follow--upup
MultimodalMultimodal
One patient, Several modalitiesOne patient, Several modalities CTCT--MR, CTMR, CT--PET, MRPET, MR--PET, PETPET, PET--US, and etc.US, and etc. Correction of Correction of MRI/MRI/fMRIfMRI acquisitionsacquisitions Constraints to reconstruction/restoration algorithmsConstraints to reconstruction/restoration algorithms Interventional Imaging : registration between preInterventional Imaging : registration between pre-- and and intraintra--operative images (e.g. MRI and Ultrasound)operative images (e.g. MRI and Ultrasound)
Modality to ModelModality to Model
Several patients, One Several patients, One modalitmodalityy ModelModel--based (guided) segmentationbased (guided) segmentation Building of digital atlasesBuilding of digital atlases Registration/matching with an anatomical Registration/matching with an anatomical atlasatlas Spatial normalization, study of anatomical Spatial normalization, study of anatomical variabilityvariability
Patient to ModalityPatient to Modality
Several patients, Several Several patients, Several modalitmodalityy Human brain mappingHuman brain mapping AnatomoAnatomo--functional normalization functional normalization ((aaid for the id for the
study of functional variabilitystudy of functional variability))
SubjectSubject
IntrasubjectIntrasubject (data from one patient)(data from one patient)
IntersubjectIntersubject (data from multiple patient)(data from multiple patient)
Atlas (one image from patient and others Atlas (one image from patient and others from Atlas)from Atlas)
ObjectObject
Several organs are studied:Several organs are studied: BrainBrain HeartHeart SpineSpine Others.Others.
Theory of RegistrationTheory of Registration
Two Images:Two Images:
With Geometrical Mapping of:With Geometrical Mapping of: :1D Gray level mapping:1D Gray level mapping :2D Coordination mapping:2D Coordination mapping
( ) ( ), , ,s s s d d dI x y I x y
( ) ( )( )( ), ,d d d s s sI x y B I x y= ϕϕϕϕ( )B ⋅
( ),⋅ ⋅ϕϕϕϕ
( ) ( ) ( ) ( ) ( )( ), , , , , ,d d s s d d d s x s s y s sx y x y I x y I x y x yϕ ϕ= ⇒ =ϕϕϕϕ
Theory of RegistrationTheory of Registration
Registration approach:Registration approach: Forward:Forward:•• Map Map sourcesource coordination to coordination to destinationdestination and and
consider consider sourcesource gray level for mapped gray level for mapped coordination.coordination. Backward:Backward:
•• Find correspondences of destination pixels in Find correspondences of destination pixels in source image and estimate suitable gray level for source image and estimate suitable gray level for each one.each one.
ForwardForward--Reverse MappingReverse Mapping
Source Image Destination Image
ForwardForward--Reverse MappingReverse Mapping
Forward mapping drawbacks:Forward mapping drawbacks: Some pixel in destination has no value!Some pixel in destination has no value! HoleHole OverlapOverlap
ForwardForward--Reverse MappingReverse Mapping
Reverse mapping Problems:Reverse mapping Problems: Most pixel in destination has value Most pixel in destination has value HoleHole OverlapOverlap
Theory of RegistrationTheory of Registration
Need for gray level interpolation:Need for gray level interpolation: Mapping is integer to real, and we have gray Mapping is integer to real, and we have gray level at non integer coordination (level at non integer coordination (ForwardForward) or ) or need gray level at non integer coordination need gray level at non integer coordination ((ReverseReverse).).
Methods:Methods: Nearest NeighborhoodNearest Neighborhood SincSinc weighted windowsweighted windows SplineSpline
InterpolationInterpolation( ) [ ]( ) [ ]( ) ( )
[ ]( ) [ ]( ) ( )[ ]( ) [ ]( ) ( )
[ ]( ) [ ]( ) ( )
( ) ( ) ( ) ( )
( )
, 1 1 ,
1 1,
1 , 1
1, 1
, ,
1Weight: 1 cos
2 1
d d d s s s s s s s
s s s s s s s
s s s s s s s
s s s s s s s
d d d d i d i s i ii j
I x y x x y y I x y
x x y y I x y
x x y y I x y
x x y y I x y
I x y sinc x x sinc y y I x y
HW
π
• = − + − + +
− − + + +
− + − + +
− − + +
• = − −
⋅ • ⋅ = + +
∑∑
⋯
⋯
⋯
Geometrical MappingGeometrical Mapping
Rigid Body Transform:Rigid Body Transform: Rotation TranslationRotation Translation 33--6 freedom degree in 2D6 freedom degree in 2D--3D cases.3D cases. Application: Hard organs: Radiology, Spinal Application: Hard organs: Radiology, Spinal cord, Hip, skull, and femur.cord, Hip, skull, and femur.
( ) ( )( ) ( )
cos sin,
sin cos
d s
x
y
a
a
θ θθ θ
= +
= = −
X RX T
R T
Rigid Body TransformRigid Body Transform
Mathematical formulation:Mathematical formulation: 3 pixels to 33 pixels to 3--pixels (2D):pixels (2D):•• Exact solutionExact solution
•• Problems with landmark.Problems with landmark. N pixels to NN pixels to N--pixels (2D), N>>3:pixels (2D), N>>3:
1 1
2
,1
Data: ,
Model:
: min ,
N N
i ii i
NT
i i D Di
= =
×=
=
+ − = ∑R T
x y
y Rx + T
Goal Rx T y RR I
Rigid Body TransformRigid Body Transform
Reformulation:Reformulation:
Transform to center:Transform to center:
Need to maximize:Need to maximize:
[ ][ ] 21 2
,1 2
, , ,min ,
, , ,
D NN T
D DD NN
×
××
= ∈ℜ ⇒ == ∈ℜ R T
X x x xRX + T - Y RR I
Y y y y
……,= − = −X X X Y Y Yɶ ɶ
22
T T T TTr Tr Tr= + −Y - RX X X Y Y YX R
T TTr YX R
SVD
SimplifyT T→
R = WVYX = W V
T Y R X∑∑∑∑
TTTT= - *= - *= - *= - *
Procrustean TransformProcrustean Transform
RotationRotation
TranslationTranslation
Scaling in two axes.Scaling in two axes.
( ) ( )( ) ( )( ) ( )( ) ( )
cos sin,
sin cos
cos sin,
sin cos
d s
x
y
xx y
yx y
ak k
ak k
ak k
ak k
θ θθ θ
θ θθ θ
= +
= = −
= = −
X RX T
R T
R T
Procrustean TransformProcrustean Transform
Ordinary Version (Ordinary Version (KKxx==KKyy)) PhotogrammeteryPhotogrammetery and Medicaland Medical Four parameters and two points in2D spaceFour parameters and two points in2D space
Multiple points formulation:Multiple points formulation:[ ]( )
22
2
1 1
,
min , ,
2 2 0
1 2 D
TT
D×D
D NTji i
j ji ji ij
diag k ,k , k
kk
ρ
ρ= =
= ⇒ = =∂ = − + = ∂ ∑ ∑
R
K
RX - Y RR K RKX - Y RR I
R = R*K
YX R X
⋯
2
=
Procrustean TransformProcrustean Transform
Iterative Solution:Iterative Solution: Initialization:Initialization: Solve this problem:Solve this problem: Update Update KK(i+1)(i+1) UsingUsing
Check for convergenceCheck for convergence
[ ]1 1,1, ,1=( )K ⋯ 2
min ,T
D×D=R
RKX - Y RR I
2
1 1
2 2 0D NT
ji ij ji ji ij
kk
ρ= =
∂ = − + = ∂ ∑ ∑YX R X
Affine TransformAffine Transform
6/12 freedom degrees (2D/3D)6/12 freedom degrees (2D/3D)
Multiple points formulation:Multiple points formulation:
1 2 0
1 2 0
,x x x
y y y
a a a
a a a
= =
R T
( ) 1
2min
T− =⇒
TR XX XY
RX - YT Y R X= - *= - *= - *= - *
Projective/Perspective TransformProjective/Perspective Transform
NonlinearNonlinear
Observation correction (photo)Observation correction (photo)
Need 4 points.Need 4 points.
Multiple points Multiple points --> Iterative optimization> Iterative optimization
Less application in MedicalLess application in Medical
11 12 10 21 22 20
1 2 1 2
,1 1
s s s sd d
s s s s
a x a y a a x a y ax y
b x b y b x b y
+ + + += =+ + + +
Bilinear TransformBilinear Transform
NonlinearNonlinear
Quadrilateral to Quadrilateral with curved Quadrilateral to Quadrilateral with curved side.side.
10 10 01 11
10 10 01 11
d s s s s
d s s s s
x a a x a y a x y
y b b x b y b x y
= + + += + + +
Polynomial TransformPolynomial Transform
Nonlinear (polynomial of order P):Nonlinear (polynomial of order P):
Less application!Less application!
0 0 0 0
,p p i p p i
i i i id ij s s d ij s s
i j i j
x a x y y b x y− −
= = = =
= =∑∑ ∑∑
NonNon--Parametric MethodsParametric Methods
Correlation (Matched Filter)Correlation (Matched Filter)
( )( ) ( )
( )
( )( )( ) ( )( )
( )( ) ( )( )
1 2
1 2 1
22
1 2
1 2
1 2 1 1
2 22 2
1 2
, ,,
,
, ,,
I: Image and
, ,
T: Template
u v
u v
u vN
u v u v
T u v I u x v xC x x
I u x v x
T u v T I u x v x ICov x x
I u x v x I T u v T
− −=
− − − − − −
= − − − −
∑∑
∑∑∑∑
∑∑ ∑∑
NonNon--Parametric MethodsParametric Methods
Correlation (Matched Filter):Correlation (Matched Filter): Very Sensitive to NoiseVery Sensitive to Noise
A variation:A variation:
Summation will performed until reach a Summation will performed until reach a threshold.threshold.
( ) ( ) ( )
( ) ( ) ( ) ( )
1 2 1 2
1 2 1 2 1 2
, , ,
, , , ,
u v
Normalu v
E x x T u v I u x v x
E x x T u v T I u x v x I x x
= − − −
= − − − − +
∑∑∑∑
NonNon--Parametric MethodsParametric Methods
Frequency Domain:Frequency Domain: Fast MethodsFast Methods Rotation, Translation, Scaling has Rotation, Translation, Scaling has FrerquencyFrerquencydomain correspondence.domain correspondence. NonNon--robust to noise and restricted to rigid.robust to noise and restricted to rigid.
Frequency DomainFrequency Domain
TranslationTranslation
Search for peaks in space domainSearch for peaks in space domain
( ) ( )( ) ( )( ) ( )
( ) ( )1 1 2 2
2 1 2 1 1 1 2 2
*1 1 2 2 1 2
1 1 2 2*1 1 2 2 1 2
, ,
, ,,
, ,j x x F
I x x I x x x x
F Fe x x x x
F Fω ωω ω ω ω
δω ω ω ω
∆ + ∆
= − ∆ − ∆
= ←→ − ∆ − ∆
Frequency DomainFrequency Domain Rotation (Polar Coordination)Rotation (Polar Coordination)
Seek for Translation in angle axisSeek for Translation in angle axis
( ) ( ) ( ) ( ) ( )( )( ) ( )
( ) ( ) ( ) ( )( )( ) ( )
( ) ( ) ( )( ) ( )
1 1 2 2
2 1 2 1 1 2 1 1 2 2
2 1 2
1 1 2 1 2
1 2
*1 2
*1 2
, cos sin , sin cos
,
cos sin , sin cos
cos , sin
, ,, ;
, ,
j x x
I x x I x x x x x x
F e
F
e e
F r F rH r
F r F r
ω ω
ρ ρ
ϕ ϕ ϕ ϕ
ω ω
ω ϕ ω ϕ ω ϕ ω ϕ
ω θ ω θ
θ θ ϕθ ϕ
θ θ ϕ
− ∆ + ∆
= + − ∆ − + − ∆
=
∆ + ∆ − ∆ + ∆
= =
−=
−
⋯
Frequency DomainFrequency Domain
Rotation and Scaling (Polar Coordination)Rotation and Scaling (Polar Coordination)
( ) ( ) ( )( ) ( ) ( )( )( )( ) ( )( )
2 1 2 1 0 1 2 0 1 2 2
2 1 020
, cos sin , sin cos
1, ln ,
I x x I r x x r x x
F r F r rr
ϕ ϕ ϕ ϕ
θ θ ϕ
= + − +
= − + ∆
Radial Basis Function InterpolationRadial Basis Function Interpolation
Registration is an interpolation problemRegistration is an interpolation problem
RBFRBF--based in most used method.based in most used method.
RBF definition:RBF definition: Symmetric around a central pointsSymmetric around a central points For arbitrary For arbitrary the is RBF!the is RBF!
Goal: Find Goal: Find interpolantinterpolant function: function:
( )xφ ( )xφ µ−
( ) : D DT S= Ψ ℜ → ℜ
Radial Basis Function InterpolationRadial Basis Function Interpolation Problem Formulation:Problem Formulation: ΨΨ is RBF (same effect on all equidistance pointsis RBF (same effect on all equidistance points Mapping is error free:Mapping is error free: Landmarks points are more than space dimension Landmarks points are more than space dimension ((N>D+1N>D+1), ), N=D+1N=D+1 is affine transform. is affine transform. ΨΨ has linear components to cope with affine transform. A trade-off between Space warping and landmarks support. For erroneous interpolation, A tradeoff between error and smoothness is necessary.
( ) , 1, 2, ,i iT S i N= = ⋯ΨΨΨΨ
Radial Basis Function InterpolationRadial Basis Function Interpolation
A smoothness criteria:A smoothness criteria:
Optimization Problem:Optimization Problem:
2
22 2 22
2 2( ) ( ) 2
Bending Energy of a plate!
m m
f f fJ f f x dxdy
x x y yℜ
∂ ∂ ∂= = + + ∂ ∂ ∂ ∂ ∫
( )( )
( ) 1
1Suboptima
min , ( )
min ( ) , (l: ) 1, 2, ,k
D
i ikk
D
i ik k
J x T x
J x T x i N
Ψ =
=Ψ
Ψ = Ψ Ψ = Ψ =
∑⋯
Radial Basis Function InterpolationRadial Basis Function Interpolation
A Unique solution based on previous A Unique solution based on previous criteria is based on:criteria is based on:
For erroneous problem:For erroneous problem:
( )2 2logr r
( )2
1 1
min ( ) ( )D N
i ikk i
J x T xΨ = =
Ψ + − Ψ ∑ ∑ iiiiλλλλ
Radial Basis Function InterpolationRadial Basis Function Interpolation
Region support:Region support: GlobalGlobal LocalLocal
Limited Region Support:Limited Region Support: Specific RBFSpecific RBF Multiply RBF by a LimiterMultiply RBF by a Limiter Use Compact Support RBFUse Compact Support RBF
( )2 2exp 2r σ−
( ) ( ),f r g r σ
( )21 0
0
r r
r
σ σσ
− ≤ ≤>
Radial Basis Function InterpolationRadial Basis Function Interpolation
Features of RBFFeatures of RBF--based interpolation:based interpolation: No need to grid data (scattered data is No need to grid data (scattered data is possible)possible) No critical condition for data.No critical condition for data. Smoothness control (Bending Energy).Smoothness control (Bending Energy).
Radial Basis Function InterpolationRadial Basis Function Interpolation
Mathematical Formulation:Mathematical Formulation:
( ) ( )( )
1 1
1
1
1
11 12 1 1 1
21 22 2 2 2
1 2
Data: , and a RBF Kernel: :
Interpolant Function:
, 1, 2, ,
,
N Di i D
N
i i
N
iii
ii
N
Nij
N N NN N N
xg
y
R x g x x
y R x i N
g g g y
g g g yg
g g g y
ω
ωω
ω
=
=
=
∈ℜ ℜ → ℜ∈ℜ
= −
= = = ∑⋯⋯⋯⋮ ⋮ ⋮ ⋮ ⋮ ⋮⋯ ( ) , , 1, 2, , .i jg x x i j N= − = ⋯
Radial Basis Function InterpolationRadial Basis Function Interpolation
Matrix Formulation:Matrix Formulation:
Crucial Problem: (Inverse of G)Crucial Problem: (Inverse of G) ExistenceExistence UniquenessUniqueness
Condition on g:Condition on g: Positive Definite FunctionPositive Definite Function•• Positive Definite GPositive Definite G
GGGGΩ= YΩ = YΩ = YΩ = Y
1 1
( , ) 0N N
i ji ji j
g x xω ω= =
≥=∑∑TΩ GΩ
Radial Basis Function InterpolationRadial Basis Function Interpolation
Examples:Examples:
( )( ) ( )
( )2
2 2
1 2
2 2
20
: , 0
1 log : 0, 22
: 0, 0
log : 0
: ,2
r
r c m
Dr r m D
e m
r c m
r m
β
β β
β
β
β
β
β
ββ
+
−
>
+ ∈ − ≥
− = − > ∈
> ≥
+ ≥
∈ℜ − >
ℝ ℕ
ℕ
ℕ
Radial Basis Function InterpolationRadial Basis Function Interpolation
Need for additional terms for Need for additional terms for pdmpdm (Not (Not spdmspdm):):
Number of Unknown Parameters:Number of Unknown Parameters:
( )1
2 22 00 10 1 01 2 11 1 2 20 1 02 2
( ) ( ), ( )
( )
ND
ii m m mi
R x g x x p x p x
P x b b x b x b x x b x b x
ω π=
= − + ∈
= + + + + +
∑
1,
m DM N M
D
− + + =
Radial Basis Function InterpolationRadial Basis Function Interpolation
N Conditions:N Conditions: Exact interpolation:Exact interpolation:
( ) ( )1
11 12 13 14
21 22 23 24
, 1, 2, ,
Matrix Form:
Sample P: (N=4, m=2, D=2)
1 1 1 1
N
k i ii k mk
y g x x p x i N
x x x x
x x x x
ω=
= − + =
=
∑TY = GΩ+ P B
P
…
Radial Basis Function InterpolationRadial Basis Function Interpolation
It can be shown (Not easily)It can be shown (Not easily)
( ) ( )1
11
0 , 1
N
k i ik m ik
N
kkk
g x x p x y
x mα
ω
ω α
=
=
− + =
= ≤ −
∑∑
T
M×1M×M
YΩG P=
0BP 0
Radial Basis Function InterpolationRadial Basis Function Interpolation
Gaussian Minimization:Gaussian Minimization:
Thin Plate Thin Plate SplineSpline Minimization:Minimization:
22( )E F e dλλ λ= ∫ℝ
( )2
1 1
min ( ) ( )D N
i ikk i
J x T xλ
λ
Ψ = =
Ψ + − Ψ
∑ ∑T
N×N
M×1M×M
YΩG + I P=
0BP 0
Thin Plate Thin Plate SplineSpline in 2Din 2D
Formulation:Formulation:
( ) ( ) ( ) ( )
( )( )
22 22 2 2
2 2
2 2
00 01 02 01
2 2
10 11 12 11
log , 0,0
N
i i ii
N
i i ii
g r r r g x yx y
x a a x a y w g x x y y
y a a x a y w g x x y y
δ
=
=
∂ ∂= − ⇒ ∆ = + ≈ ∂ ∂ ′ = + + + − + −
′ = + + + − + −
∑∑
Thin Plate Thin Plate SplineSpline in 2Din 2D
Formulation for exact matching:Formulation for exact matching:
( )( )
2 2
00 01 02 01
2 2
10 11 12 11
0 11 1
0 01 1
1 11 1
0
0
0
N
k k k i k i k ii
N
k k k i k i k ii
N N
k kk k
N N
k k k kk k
N N
k k k kk k
x a a x a y w g x x y y
y a a x a y w g x x y y
w w
x w y w
x w y w
=
=
= =
= =
= =
′ = + + + − + −
′ = + + + − + −
= =
= =
= =
∑∑
∑ ∑∑ ∑∑ ∑
Thin Plate Thin Plate SplineSpline in 2Din 2D
Matrix FormulationMatrix Formulation
[ ] ( )
[ ]
1
2
2 2
01 02 0 00 01 02
1
2
11 11 1 10 11 12
,
1
1
k
k
k k kN ki k i k i
k
k
k
k
k k kN
k
k
G
G
x w w w a a a G G g x x y y
x
y
G
G
y w w w a a a G
x
y
′ = = − + − ′ =
⋮⋯⋮⋯
Thin Plate Thin Plate SplineSpline in 2Din 2D
Matrix FormMatrix Form1 2 01 02 0 00 01 02
1 2 11 12 1 10 11 12
12 1 1 1
21 2 2 2
1 2
1 2
1 2
0 0 0,
0 0 0
0 1
0 1
0 1
1 1 1 0 0 0
0 0 0
0 0 0
T
N N
N n
N
N
N N N N
N
N
x x x a a a
y y y a a a
G G x y
G G x y
G G x y
x x x
y y y
ω ω ωω ω ω
′ ′ ′ = = ′ ′ ′ = =
K
⋯ ⋯⋯ ⋯⋯ ⋯⋯ ⋯⋮ ⋮ ⋱ ⋯ ⋮ ⋮ ⋮ ⋮⋮ ⋮ ⋮ ⋱ ⋮ ⋮ ⋮ ⋮⋯ ⋯⋯ ⋯⋯ ⋯⋯ ⋯Y
L
ΩΩΩΩ
⇒ T T
3×3
PY = LΩ
P 0
Compact Support RBFCompact Support RBF
We need Positive Definite Function with We need Positive Definite Function with Compact Support Property.Compact Support Property.
How to generate:How to generate: Select one and recursive convolution!Select one and recursive convolution!
Practical problems:Practical problems:•• Convolution is hard or nonConvolution is hard or non--closed form.closed form.
( ) ( ) ( )2 2 2Dy
x y x y d xϕ ψ ψ∈ℜ
= −∫
Compact Support RBFCompact Support RBF
A general class is derived:A general class is derived:
( ) ( )
( ) ( )
( )( ) ( )( ) ( )
5 2
4
6 2
1 0 11
0 1
1 0 11
0 1
(1 ) 8 5 1
1 4 1
1 35 18 3
nn
nn
r rr
r
r rr
r
r r r
r r
r r r
+
+
+
+
+
− ≤ ≤− >
− ≤ ≤− = >
− + +
− +
− + +
≜
Compact Support RBFCompact Support RBF
CSRBF:CSRBF: No need to additional affine terms.No need to additional affine terms. Defined over [0,1]Defined over [0,1]•• Solve with scaling:Solve with scaling:
•• Unsolved problem: How to choose Unsolved problem: How to choose δδ
( ) 2
1
Ni
i ki i
x xR x ω ϕ
σ=
− = ∑
Principal Axes RegistrationPrincipal Axes Registration
A Primary phase of registrationA Primary phase of registration( ) ( )
( )( )
( )( )
( )( )
( )( )
( )( )
( )( )
, ,
, ,
, ,
, ,
, ,
, ,
1 , , Desired volume, ,
0 O.W.
, ,
, ,
, ,
, ,
, ,
, ,
x y z
g
x y z
x y z
g
x y z
x y z
g
x y z
x y zB x y z
xB x y z
xB x y z
yB x y z
yB x y z
zB x y z
zB x y z
∈=
=
=
=
∑∑∑∑∑∑
Principal Axes RegistrationPrincipal Axes Registration
Rotation AnglesRotation Angles
( ) ( ) ( )( )
( ) ( ) ( )( )
( ) ( ) ( )( )
( )( ) ( )( )
( ) ( ) ( )( )
( )
2 2
, ,
2 2
, ,
2 2
, ,
, ,
, ,
, ,
, ,
, ,
, ,
, ,
xx xy xz
yx yy yz
zx zy zz
xx g gx y z
yy g gx y z
zz g gx y z
xy g gx y z
yz g gx y z
zx g
I I I
I I I
I I I
I y y z z B x y z
I x x z z B x y z
I x x y y B x y z
I x x y y B x y z
I z z y y B x y z
I z z
− − = − − − − = − + − = − + − = − + − = − −
= − −
= −
∑∑∑∑∑
I
( ) ( )( ), ,
, ,gx y z
x x B x y z−∑
Landmark ExtractionLandmark Extraction
Landmark Definition:Landmark Definition:
A Principal biological DescriptorA Principal biological Descriptor
Examples:Examples: Lips, Eyes, Nasal, and etc.Lips, Eyes, Nasal, and etc.
Landmark is related to:Landmark is related to: Data GeometryData Geometry Mathematical DeformationMathematical Deformation Biology of organsBiology of organs
Landmark ExtractionLandmark Extraction
Landmark ExtractionLandmark Extraction Landmark Landmark vsvs Fourier Descriptor:Fourier Descriptor: Local InformationLocal Information Category:Category: Juxtaposition (Proximity)Juxtaposition (Proximity) Maximum CurvatureMaximum Curvature Marginal:Marginal:•• Limiter (Left, Right, Top, Bottom)Limiter (Left, Right, Top, Bottom)
•• Center of GravityCenter of Gravity
•• Point of maximum distance (Point of maximum distance (CoGCoG))
•• Center of circles and EllipsoidCenter of circles and Ellipsoid
Landmark ExtractionLandmark Extraction
Landmark ExtractionLandmark Extraction
A medical landmark:A medical landmark: Easy detection in all modalitiesEasy detection in all modalities Robustness (noise, interference, and etc.)Robustness (noise, interference, and etc.) NonNon--complex mathematical descriptioncomplex mathematical description Medical significance.Medical significance.
Landmark ExtractionLandmark Extraction
LandmarksLandmarks Internal:Internal:•• Hard to detection and recognition;Hard to detection and recognition;•• Time Consuming;Time Consuming;•• Task Dependent.Task Dependent. External:External:•• Geometrical/Image Processing;Geometrical/Image Processing;•• Robustness Problem against the noise.Robustness Problem against the noise.•• An open and unsolved problem!An open and unsolved problem!
Landmark ExtractionLandmark Extraction
A Simple approaches:A Simple approaches:
Correspondence problem!Correspondence problem!
( )
( )( )
( ) ( )( )
( ) ( )( )
( )( ) ( ) ( )
( ) ( )( )( )( )
2
, ,
2
, ,, ,
, , ,
, , ,
det ,,
,
x y x y
x y x yx y x y
I x y I x y I x y
x x yx, y
I x y I x y I x y
x y y
x yx y
trace x y
′ ′ ′ ′∈Ω ∈Ω
′ ′ ′ ′∈Ω ∈Ω′ ′ =
′ ′ ′ ′ ′ ′ ∂ ∂ ∂ ′ ′ ′∂ ∂ ∂ = ′ ′ ′ ′ ′ ′ ∂ ∂ ∂ ′ ′ ′∂ ∂ ∂
=
∑ ∑
∑ ∑D
DF
D
Landmark ExtractionLandmark Extraction
Image 2Image 2Image 1Image 1
SegmentationSegmentationSegmentationSegmentation
Contour ExtractionContour ExtractionContour ExtractionContour Extraction
Automatic Landmark Extraction and CorrespondenceAutomatic Landmark Extraction and Correspondence
Another method:Another method:
Landmark ExtractionLandmark Extraction
Landmark Extraction:Landmark Extraction: Dominant Point Detection;Dominant Point Detection; Contour Estimation with polygon;Contour Estimation with polygon; Clustering.Clustering.
Landmark ExtractionLandmark Extraction
Dominant Point Detection:Dominant Point Detection: Uses local curvature estimation.Uses local curvature estimation.
Landmark ExtractionLandmark Extraction
Contour Estimation with polygon:Contour Estimation with polygon: Principal Curves (Polygon):Principal Curves (Polygon):( ) ( ) ( ) sup : min
D Dτλ λ λ τ= − = −f X X f X f
Landmark ExtractionLandmark Extraction
Contour Estimation with polygon:Contour Estimation with polygon:
Landmark ExtractionLandmark Extraction
Clustering:Clustering: A self organizing Growing structures A self organizing Growing structures
Adaptation:
Growing
Landmark ExtractionLandmark Extraction
Clustering:Clustering: Sample are NOT simple point but edges.Sample are NOT simple point but edges.•• Need to new definition of membership, & etc.Need to new definition of membership, & etc.
Landmark ExtractionLandmark Extraction
Clustering:Clustering: Method:Method:•• Start from an consistence and suitable Start from an consistence and suitable
condition.condition.
•• Train to reach next match of current model.Train to reach next match of current model.
•• Insert/Delete Correspondence edges.Insert/Delete Correspondence edges.
Landmark ExtractionLandmark Extraction
Preprocessing:Preprocessing: SegmentationSegmentation Contour tracingContour tracing Contour correction (Contour correction (splinespline, Interpolation), Interpolation) SemiSemi--automated correspondenceautomated correspondence
Output:Output: Two sets of related contourTwo sets of related contour
Landmark ExtractionLandmark Extraction
Need for new definition:Need for new definition: Distance:Distance:•• Point to polygonPoint to polygon
•• Error Energy (point to side)Error Energy (point to side)
•• Error Energy (point to polygon)Error Energy (point to polygon)
Landmark ExtractionLandmark Extraction Clustering:Clustering: NG and GNGNG and GNG New data arrangement:New data arrangement: 2D to 3D conversion2D to 3D conversion Initial Condition:Initial Condition: Two triangles in two plane Two triangles in two plane •• Randomly But relatedRandomly But related•• Three lines with 120 degree distance (PCA)Three lines with 120 degree distance (PCA) Distance of incoming data and cell center.Distance of incoming data and cell center. Three statesThree states Membership:Membership: New definition:New definition:
( )1 1i
i
V
i pV
m x dsx v
+
=−∫
Landmark ExtractionLandmark Extraction
VoronoiVoronoi ImagesImages
Landmark ExtractionLandmark Extraction
Updates:Updates: Vertices and sidesVertices and sides Need for insert new side,Need for insert new side,•• Local Error (sides and vertices)Local Error (sides and vertices)
•• Estimation with a lineEstimation with a line2
0 2 2( )12
DP BP Epi i i
Ti ii
i irmsx P x P x Pi
a x aE x V x V
a+
∈ ∈ ∈
+= + − + −∑ ∑ ∑
Landmark ExtractionLandmark Extraction
Updates:Updates: Need for delete avoidable edge:Need for delete avoidable edge:•• Linearity errorLinearity error
•• Principal anglePrincipal angle
Stop:Stop: AkaikiAkaiki Criteria:Criteria:
( )( ) ( )log
2
S DG G
N
E ENρ α
+= +
Error and ValidationError and Validation
Actions after registration:Actions after registration: Accuracy of registration?Accuracy of registration? How to increase?How to increase?
The main PROBLEM:The main PROBLEM: No No GOLDGOLD standardstandard
Statistical Measure:Statistical Measure: Inconsistency in different experiments.Inconsistency in different experiments. Image Dependency;Image Dependency; All experiments has special protocol.All experiments has special protocol. Unclear description of Accuracy and Precision.Unclear description of Accuracy and Precision.
Error and ValidationError and Validation
Errors:Errors: FiducialFiducial Localization Error (FLE):Localization Error (FLE):•• Physical Landmark Uncertainty.Physical Landmark Uncertainty. FiducialFiducial Registration Error (FRE):Registration Error (FRE):•• Distance between correspondence Distance between correspondence LandmarkLandmark
after registration phase.after registration phase. Target Registration Error (TRE): Distance Target Registration Error (TRE): Distance between correspondence between correspondence pointspoints after after registration phase.registration phase.
Error and ValidationError and Validation
Errors:Errors:
Error and ValidationError and Validation
Errors:Errors: FLE:FLE:•• Operator and Algorithm.Operator and Algorithm. FRE:FRE:•• Rigid transform (large sets of points)Rigid transform (large sets of points) TRE:TRE:•• Hard to estimation: Training and test sets.Hard to estimation: Training and test sets.
Error and ValidationError and Validation
Errors (Rigid Transform):Errors (Rigid Transform): If FLE then FRE If FLE then FRE •• Expected value of FRE:Expected value of FRE: TRE (1998):TRE (1998):
( )20,N σ∝ 2 23 6N Nσ κ −∝
( ) 23 6N Nσ−
22 2
2 21
1 1
, : Matrices of eignvalues of landmarks
D Di
TRE FLEi j i i j
i j
x
N Dσ σ
= ≠
≅ + Λ + Λ
Λ Λ
∑∑
Error and ValidationError and Validation
Sensitivity Analysis of landmarks:Sensitivity Analysis of landmarks: Analytical relation.Analytical relation. Some nonSome non--closed form of mathematical closed form of mathematical equations.equations.( )( ) ( ) ( )
( )( ) ( ) ( ),,,,( , ) ( , ), , ( , ) ( , )( , ) ( , ), , ( , ) ( , )
s si id di ix xs si if x y s sx y i i y ys si ix xd di if x y d dx y y yi i d di if x y f x yx yf x y f x yx y f x y f x yx yf x y f x yx yf x y f x y f x y f x yx y x y
∂ ∂ ∂ ∂ ∂ ∂ = = ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ = = ∂ ∂∂ ∂ ∂ ∂S
S
Local RegistrationLocal Registration
Local Model:Local Model: Nearest NeighborNearest Neighbor•• Set pixel value to nearest landmark Set pixel value to nearest landmark Weighted AverageWeighted Average•• Set pixel value to weighted average (Distance)Set pixel value to weighted average (Distance) Local Weighted FittingLocal Weighted Fitting•• Fit a curve or surface based on NFit a curve or surface based on N--nearest nearest
landmark. landmark.
Local RegistrationLocal Registration
Example, Shepard Method:Example, Shepard Method:
( ) ( ) ( )
( )
( ) ( )
( ) ( )
( ) ( )( )
( )
1
1
2
1
;
Condition: ; 1 , 0 ; 1, 1, 2, ,
; 1, ; 0, , 1, 2, , ,
; 1Shepard: ; , ;
;
N
ii ii
ii i
N
i ii ii
i i j ii i
iii ii iN
ijj
j
f x x x f x
y f x
x x x x i N
x x x x i j N j i
x xx x x x
x xx xµ
α
α α
α α
σα σ
σ
=
=
=
=
= = ≤ ≤ = = = = ≠= =
−
∑
∑
∑
…
…
Local RegistrationLocal Registration
Another approach:Another approach:
( )
( ) ( ) ( )
( ) ( ) ( )
2
1,
1
Goal to Minimize: , 1, 2, ,
( ) , , 1, 2, ,
, 0 0
N
ji ij i ij j i
N
ii i i ii
i ii i i
E f x y i N
f x x f x f x y i N
f x x P x x P
α
α
= ≠
=
= − =
= = =
= + − =
∑∑
……
Local RegistrationLocal Registration
Problem of Region Support in RBF:Problem of Region Support in RBF: Fixed Fixed Nearest Neighbor (Large for isolated)Nearest Neighbor (Large for isolated) Global (Optimization)Global (Optimization)
Local RegistrationLocal Registration
Distance Effect:Distance Effect:
( ) ( ) ( )( ) ( ) ( )( ) ( )
( ) ( )( )1
,
, , ,
, ,
,
T
E
D E i
F E
pp
i ip i
d x a x a x a
d x a d Mx Ma M diag m
d x a d Mx Ma
d x a x a
= − −
= =
=
= −∑
Local RegistrationLocal Registration
Our idea:Our idea: VoronoiVoronoi and Tessellation Diagramand Tessellation Diagram
Local RegistrationLocal Registration
Our idea:Our idea: Distance point Distance point –– structurestructure Weighing function for merging multiple maps.Weighing function for merging multiple maps. Region SupportRegion Support Smoothness Smoothness CriteriCriteri
Local RegistrationLocal Registration
Structures:Structures: Single PointsSingle Points Single (Smooth) are of contourSingle (Smooth) are of contour Correspondence ContourCorrespondence Contour
Local RegistrationLocal Registration
Correspondence StructureCorrespondence Structure
Related Landmark ExtractionRelated Landmark Extraction
Split Structure to subSplit Structure to sub--structurestructure
Mapping Determination for each Mapping Determination for each subsub--Structure.Structure.
VoronoiVoronoi ImagesImages
Local Weighted MappingLocal Weighted Mapping
Local RegistrationLocal Registration
Alternative definition of distance:Alternative definition of distance: MinMin New definitionNew definition2 2
1 2( ) ( )1 ( ) 1( )
( ) ( )c
C p pt tc c
dx t dx td x tm x dt dt
dt dt dtx x t x x t
= = + − −∫ ∫
Local RegistrationLocal Registration
Local RegistrationLocal Registration
Local RegistrationLocal Registration
Local RegistrationLocal Registration
Region of support definition:Region of support definition: Use of GRBFUse of GRBF Use of EBFUse of EBF Use of QBFUse of QBF
( ) ( )
( ) ( ) ( ) ( )
( ) ( ) ( )
21
2
21 22
1
1
, ; , ( , , )
, ;
Mi
i mi i
DTj j
E Dj j
T
Q
xf x p x
x cN x c x c x c diag
N x c x c x c
µω ϕ
σ
σ σ σσ
=
−
=
−
− = + −
Σ = = − Σ − Σ =
Σ = − Σ −
∑
∑ …
Local RegistrationLocal Registration
Number of Center:Number of Center: GrowingGrowing--Pruning MethodsPruning Methods Cross ValidationCross Validation
Local RegistrationLocal Registration
Local Warping:Local Warping:
( ) ( )
( )
1
1
ln(2)
( ) 1Point to Curve Membership: ( ) , ( )
( )
( ) ( ) ( )
ln(3) 2, e , tanh ,
2
k
kC
k
i
C
Ck CN p
c CC
i
N
C k kk
m
m xx m x ds
x cm x
f x x m x x f x x
m m mm Arctg
m
σ
µ
λ µ
λσ σ π σ
∈
=
=
−
= =−
= + − = + ∫∑
∑