Geometric Registration for Deformable Shapes

2.2 Deformable RegistrationVariational Model· Deformable ICP

Variational ModelWhat is deformable shape matching?

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 33

Example

?

What are the Correspondences?

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 4

What are we looking for?

Problem Statement:

Given:• Two surfaces S1, S2 ⊆ ℝ3

We are looking for:• A reasonable deformation function f: S1 → ℝ3

that brings S1 close to S2

?

S1S2

f

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 55

Example

?

Correspondences? no shape match

too much deformation optimum

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 6

This is a Trade-Off

Deformable Shape Matching is a Trade-Off:• We can match any two shapes

using a weird deformation field

• We need to trade-off: Shape matching (close to data) Regularity of the deformation field (reasonable match)

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 77

Components:

Matching Distance:

Deformation / rigidity:

Variational Model

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 8

Variational Model

Variational Problem:• Formulate as an energy minimization problem:

)()()( )()( fEfEfE rregularizematch +=

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 9

Part 1: Shape Matching

Assume:• Objective Function:

• Example: least squares distance

• Other distance measures:Hausdorf distance, Lp-distances, etc.

• L2 measure is frequently used (models Gaussian noise)

S2f(S1)

( )212,1)( ),()( SSfdistfE match =

∫∈

=11

12

21)( ),()(

Sx

match dSdistfE xx

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 10

Point Cloud Matching

Implementation example: Scan matching• Given: S1, S2 as point clouds

S1 = {s1(1), …, sn(1)} S2 = {s1(2), …, sm(2)}

• Energy function:

• How to measure ?

Estimate distance to a point sampled surface

si(2)

fi(S1)( )∑

=

=m

ii

match SdistmSfE

1

2)2(1

1)( ,||)( s

( )x,1Sdist

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 11

Surface approximation

Solution #1: Closest point matching• “Point-to-point” energy

( )∑=

=m

iiSini

match sNNsdistmSfE

1

2)2()2(1)( )(,||)(1

si(2)

f(S1)

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 12

Surface approximation

Solution #2: Linear approximation• “Point-to-plane” energy• Fit plane to k-nearest neighbors• k proportional to noise level, typically k ≈ 6…20

si(2)

f(S1)

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 13

Surface approximation

Solution #3: Higher order approximation• Higher order fitting (e.g. quadratic)

Moving least squares

si(2)

f(S1)

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 14

Variational Model

Variational Problem:• Formulate as an energy minimization problem:

)()()( )()( fEfEfE rregularizematch +=

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 15

What is a “nice” deformation field?• Isometric “elastic” energies

Extrinsic (“volumetric deformation”) Intrinsic (“as-isometric-as

possible embedding”)

• Thin shell model Preserves shape (metric plus curvature)

• Thin-plate splines Allowing strong deformations, but keep shape

Part II: Deformation Model

)()( fE rregularize

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 16

Elastic Volume Model

Extrinsic Volumetric “As-Rigid-As Possible”• Embed source surface S1 in volume• f should preserve 3×3 metric tensor (least squares)

[ ]∫ −∇∇=1

2T)( )(V

rregularize dxfffE Ifirst fundamental form I (ℝ3×3)

S1

V1f

S2

∇f

f (V1)ambient space

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 17

Volume Model

Variant: Thin-Plate-Splines• Use regularizer that penalizes curved deformation

second derivative (ℝ3×3)

S1

V1f

S2

Hf =∇(∇f )

f (V1)ambient space

∫=1

2)( )()(V

frregularize dxxHfE

Eurographics 2010 Course – Geometric Registration for Deformable Shapes

How does the deformation look like?

original

as-rigid-aspossiblevolume

thinplate

splines

Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Intrinsic Matching (2-Manifold)• Target shape is given and complete• Isometric embedding

[ ]∫ −∇∇=1

2T)( )(S

rregularize dxfffE Ifirst fund. form (S1, intrinsic)

19

Isometric Regularizer

19

S1

f

S2

∇f

tangent space

Eurographics 2010 Course – Geometric Registration for Deformable Shapes

“Thin Shell” Energy• Differential geometry point of view

Preserve first fundamental form I Preserve second fundamental form II …in a least least squares sense

• Complicated to implement• Usually approximated

Volumetric shells (as shown before) Other approximation (next slide)

20

Elastic “Thin Shell” Regularizer

20

S1

S2

f

III

III

Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Example Implementation

Example: approximate thin shell model• Keep locally rigid

Will preserve metric & curvature implicitly

• Idea Associate local rigid transformation with surface points Keep as similar as possible Optimize simultaneously with deformed surface

• Transformation is implicitly defined by deformed surface (and vice versa)

21

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 22

Parameterization

Parameterization of S1• Surfel graph • This could be a mesh, but does not need to

edges encodetopology

surfel graph

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 23

Deformation

Ai

Orthonormal Matrix Aiper surfel (neighborhood),latent variable

Ai

predictionframe t frame t+1

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 24

Deformation

Ai

Orthonormal Matrix Aiper surfel (neighborhood),latent variable

Ai

prediction

error

frame t frame t+1

( ) ( )[ ]2)1()1()()()( ∑ ∑ ++ −−−=surfels neighbors

ti

ti

ti

ti

ti

rregularizejj

E ssssA

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 25

Unconstrained Optimization

Orthonormal matrices• Local, 1st order, non-degenerate parametrization:

• Optimize parameters α, β, γ, then recompute A0• Compute initial estimate using [Horn 87 ]

−−−=

00

0)(

γβγαβα

ti×C

)(

)exp()(

0

0

ti

ii

I ×

×

CA

CAA

+⋅=

=

Eurographics 2010 Course – Geometric Registration for Deformable Shapes 26

Variational Model

Variational Problem:• Formulate as an energy minimization problem:

)()()( )()( fEfEfE rregularizematch +=

Deformable ICP

Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Deformable ICP

How to build a deformable ICP algorithm• Pick a surface distance measure• Pick an deformation model / regularizer

2828

)()()( )()( fEfEfE rregularizematch +=

Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Deformable ICP

How to build a deformable ICP algorithm• Pick a surface distance measure• Pick an deformation model / regularizer• Initialize f(S1) with S1 (i.e., f = id)• Pick a non-linear optimization algorithm

Gradient decent (easy, but bad performance) Preconditioned conjugate gradients (better) Newton or Gauss Newton (recommended, but more work) Always use analytical derivatives!

• Run optimization

Eurographics 2010 Course – Geometric Registration for Deformable Shapes

Example

Example• Elastic model• Local rigid coordinate

frames• Align A→B, B→A

30

Geometric Registration for Deformable ShapesVariational Model�What is deformable shape matching?ExampleWhat are we looking for?ExampleThis is a Trade-OffVariational ModelVariational ModelPart 1: Shape MatchingPoint Cloud MatchingSurface approximationSurface approximationSurface approximationVariational ModelPart II: Deformation ModelElastic Volume ModelVolume ModelHow does the deformation look like?Isometric RegularizerElastic “Thin Shell” RegularizerExample ImplementationParameterizationDeformationDeformationUnconstrained OptimizationVariational ModelDeformable ICPDeformable ICPDeformable ICPExample