Iccv05 Matching

8/7/2019 Iccv05 Matching

1/83

Shape Matching for Model Alignment

3D Scan Matching and Registration, Part IICCV 2005 Short Course

Michael Kazhdan

Johns Hopkins University


2/83

Goal

Given two partially overlapping scans,

compute the transformation that merges

the two.

Partially Overlapping Scans Aligned Scans


3/83

Approach

1. (Find feature points on the two scans)

Partially Overlapping Scans


4/83

Approach


2. Establish correspondences

Partially Overlapping Scans


5/83

Approach


2. Establish correspondences

3. Compute the aligning transformation

Aligned ScansPartially Overlapping Scans


6/83


7/83

Correspondence

Goal:

Identify when two points on different scans

represent the same feature

?


8/83

Correspondence

Goal:

Identify when two points on different scans

represent the same feature:

Are the surrounding regions similar?

?


9/83

Global Correspondence

More Generally:

Given two models, determine if they

represent the same/similar shapes.

?


10/83


11/83


Approach:

1. Represent each model by a shape

descriptor:

A structured abstraction of a 3D model

That captures salient shape information


12/83


Approach:

1. Represent each model by a shape

descriptor.

2. Compare shapes

by comparing their

shape descriptors.

?


13/83

Shape Descriptors: Examples

Shape Histograms

Shape descriptor stores a histogram of how

much surface area resides withindifferent concentric shells in space.

Model

Shape Histogram

(Sectors + Shells)

Represents a 3D model by a 1D

(radial) array of values

[Ankerst et al. 1999]


14/83


Shape Histograms


much surface area resides withindifferent sectors in space.

Model

Shape Histogram

(Sectors + Shells)



(spherical) array of values


15/83


Shape Histograms


much surface area resides withindifferent shells and sectors in space.

Model

Shape Histogram

(Sectors + Shells)



(spherical x radial) array of values


16/83

Shape Descriptors: Challenge

The shape of a model does not change

when a rigid body transformation is

applied to the model.

Translation

Rotation


17/83


In order to compare two models,

we need to compare them

at their optimal alignment.

-


18/83


In order to compare two models,

we need to compare them

at their optimal alignment.

-

-

-

-

min- =


19/83


20/83

Shape Descriptors: Alignment

Exhaustive Search: Compare at all alignments

Exhaustive search for optimal rotation


21/83



Correspondence is determined by the

alignment at which models are closest

Exhaustive search for optimal rotation


22/83



Correspondence is determined by the

alignment at which models are closest

Properties: Gives the correct answer Even with fast signal processing tools, it is

hard to do efficiently


23/83


Normalization:

Put each model into a canonical frame

Translation: Rotation:

Translation

Rotation


24/83


Normalization:


Translation: Center of Mass Rotation:

Initial Models Translation-Aligned Models


25/83


Normalization:


Translation: Center of Mass Rotation: PCA Alignment

Translation-Aligned Models Fully Aligned Models

PCAAlignment


26/83


Normalization:


Translation: Center of Mass Rotation: PCA Alignment

Properties: Efficient

Not always robust

Cannot be used for feature matching


27/83


Invariance:

Represent a model by a shape descriptor

that is independent of the pose.

-


28/83


Example: Ankersts Shells

A histogram of the radial distribution of

surface area.

Shells Histogram

Radius

Area



29/83


Invariance:

Power spectrum representation

Fourier transform for translations Spherical harmonic transform for rotations

Frequency

Energy

Frequency

Energ

y

Circular Power Spectrum Spherical Power Spectrum


30/83

Translation Invariance

1D Function


31/83


+ + += +

Cosine/Sine Decomposition

1D Function


32/83


=

+ + +

Constant

= +

Frequency Decomposition

1D Function


33/83


34/83


=

+ + +

+ +

Constant 1st Order 2nd Order

= + +


1D Function


35/83


=

+ + +

+ + +

Constant 1st Order 2nd Order 3rd Order

= +

+

+


1D Function


36/83


Frequency subspaces are fixed by rotations:


37/83




38/83




39/83




40/83

+ + + + +


= + + +


+


=

Amplitudes

invariantto translation1D Function


41/83

Rotation Invariance

Represent each spherical function as a sum

of harmonic frequencies (orders)

+ += +

+ + +



42/83

Rotation Invariance



43/83

Rotation Invariance



44/83

Rotation Invariance



45/83

Rotation Invariance



46/83

+ ++

Rotation Invariance

Store how much (L2-norm) of the shape

resides in each frequency to get a rotation

invariant representation

= + + +



47/83


Invariance:

Represent a model by a shape descriptor

that is independent of the pose.

Properties:

Compact representation Not always discriminating


48/83

Outline

Global-Shape Correspondence Shape Descriptors Alignment

Partial-Shape/Point Correspondence From Global to Local Pose Normalization Partial Shape Descriptors

Registration Closed Form Solutions Branch and Bound

Random Sample Consensus


49/83

From Global to Local

To characterize the surface about a pointp,

take a global descriptor and:

center it aboutp (instead of the COM), and

restrict the extent to a small region aboutp.

Shape histograms as local shape descriptors

p


50/83


Given scans of a model:

Scan 1 Scan 2


51/83


Identify the features

Scan 1 Scan 2


52/83



Compute a local descriptor for each feature

Scan 1 Scan 2


53/83



Compute a local descriptor for each feature

Features correspond descriptors are similar

Scan 1 Scan 2


54/83

Pose Normalization


Translation: Accounted for by centering the

descriptor at the point of interest.

Rotation: We still need to be able to matchdescriptors across different rotations.


55/83

Pose Normalization

Challenge

Since only parts of the models are given, we

cannot use global normalization to align the

local descriptors


56/83

Pose Normalization

Challenge

Since only parts of the models are given, we

cannot use global normalization to align the

local descriptors

Solutions

Normalize using local information


57/83

Local Descriptors: Examples

Variations of Shape Histograms:For each feature, represent its local geometry in

cylindrical coordinates about the normal

nn

nn

height

radius

angle

Featurepoint


58/83




Spin Images: Store energyin each normal ring

n

nn


59/83




Spin Images: Store energyin each normal ring

Harmonic Shape Contexts:

Store power spectrum of

each normal ring

n

nn


60/83




Spin Images: Store energy

in each normal ring

Harmonic Shape Contexts:

Store power spectrum of

each normal ring

3D Shape Contexts: Search

over all rotations about the

normal for best match

n

nn


61/83

Outline

Global-Shape Correspondence Shape Descriptors

Alignment

Partial-Shape/Point Correspondence From Global to Local Limitations of Global Alignment

Partial Shape Descriptors

Registration Closed Form Solutions Branch and Bound



62/83

Registration

Ideal Case: Every feature point on one scan has a

(single) corresponding feature on the other.

piqi


63/83

Registration

Ideal Case: Every feature point on one scan has a

(single) corresponding feature on the other.

Solve for the optimal transformation T:

=

n

iii qTp

1

2

)(

piqi

[McLachlan, 1979]

[Arun et al., 1987]

[Horn, 1987]

[Horn et al., 1988]


64/83

Registration

Challenge: Even with good descriptors, symmetries in the

model and the locality of descriptors can result

in multiple and incorrect correspondences.


65/83

Registration

Exhaustive Search: Compute alignment error at each permutation

of correspondences and use the optimal one.

= =

n

iii

ET

pTp1

2

3

))((minargminargError

encecorrespondpossibleofSet=tionstransformabodyrigidofGroup=3E


66/83

Registration

Exhaustive Search: Compute alignment error at each permutation

of correspondences and use the optimal one.

Given points {p1,,p

n} on the

query, ifpimatches m

idifferent

target points:

= =

n

iii

ET

pTp1

2

3

))((minargminargError

i

n

i

m=

=1

| |=44=256 possible permutations

encecorrespondpossibleofSet=tionstransformabodyrigidofGroup=3E


67/83

Branch and Bound

Key Idea: Try all permutations but terminate early if the

alignment can be predicted to be bad.

By performing two comparisons, it

was possible to eliminate 16

different possibilities


68/83

Branch and Bound

Goal: Need to be able to determine if the alignment

will be a good one without knowing all of the

correspondences.


69/83

Branch and Bound



correspondences.

Observation:

Alignment needs topreserve the lengths

between points in a

single scan.


70/83

Branch and Bound



correspondences.

Observation:

Alignment needs topreserve the lengths

between points in a

single scan.

d1 d2

d1 d2


71/83


Observation: In 3D only three pairs of corresponding points

are needed to define a transformation.


72/83


Algorithm: Randomly choose three points on source

For all possible correspondences on target:

Compute the aligning transformation T For every other source pointp :

Find corresponding

point closest to T(p)

Compute alignment error


73/83





Find corresponding




74/83





Find corresponding




75/83





Find corresponding



p T(p)


76/83





Find corresponding


Compute alignment errorError = 0

p T(p)


77/83





Find corresponding




78/83





Find corresponding



pT(p)


79/83





Find corresponding




80/83





Find corresponding


Compute alignment errorError > 0


81/83

Summary

Global-Shape Correspondence Shape Descriptors

Shells (1D)

Sectors (2D) Sectors & Shells (3D)

Alignment Exhaustive Search

Normalization Invariance


82/83

Summary

Partial-Shape/Point Correspondence From Global to Local

Center at feature

Restrict extent Pose Normalization

Normal-based alignment

Partial Shape Descriptors

Normalization/invariance Normalization/exhaustive-search

S


83/83

Summary

Registration Closed Form Solutions

Global Symmetry

Local self similarity Branch and Bound

Inter-feature distances for early termination


Efficient transformation computation

Documents

Iccv05 Matching