Upload
barton
View
24
Download
3
Embed Size (px)
DESCRIPTION
CSE 554 Lecture 6: Deformation I. Fall 2011. Review. Source. Alignment Registering source to target by rotation and translation Rigid-body transformations Methods Aligning principle directions (PCA) Aligning corresponding points (SVD) Iterative improvement (ICP) Combines PCA and SVD. - PowerPoint PPT Presentation
Citation preview
CSE554 Deformation I Slide 1
CSE 554
Lecture 6: Deformation I
CSE 554
Lecture 6: Deformation I
Fall 2011
CSE554 Deformation I Slide 2
ReviewReview
• Alignment
– Registering source to target by rotation and translation
• Rigid-body transformations
• Methods
– Aligning principle directions (PCA)
– Aligning corresponding points (SVD)
– Iterative improvement (ICP)
• Combines PCA and SVD
Input
After PCA
After ICP
Source
Target
CSE554 Deformation I Slide 3
Non-rigid RegistrationNon-rigid Registration
• Rigid alignment cannot account for shape variance
• Non-rigid deformation can give a better fit
After non-rigid deformationRigid alignment
Source
Target
CSE554 Deformation I Slide 4
Non-rigid RegistrationNon-rigid Registration
• A minimization problem
– Minimizing the distance between the deformed source and the target
• “Fitting term”
– Minimizing the distortion to the source shape
• “Distortion term”
CSE554 Deformation I Slide 5
Intrinsic vs. ExtrinsicIntrinsic vs. Extrinsic
• Intrinsic methods
– Deforms points on the source curve/surface
– App: boundary curve or surface matching
• Extrinsic methods
– Deforms all points on and interior to the source curve/surface
– App: image or volume matching
CSE554 Deformation I Slide 6
Laplacian-based DeformationLaplacian-based Deformation
• An intrinsic method
– Simple to implement, produces reasonable results
• Preserving local shape features
– Widely used in graphics applications for interactive deformation
Reference: “Laplacian surface editing”, by Sorkine et al., 2004 (citation > 300)
CSE554 Deformation I Slide 7
• Input
– Source with n points: p1,…,pn
– Target with m points (m<=n): q1,…,qm
• Corresponding to first m source points
• Output
– Deformed locations of source points: p1’,…,pn’
SetupSetup
p1=q1
p3=q3
q2
p2
An example with 3 target points, two of which are stationary (red)
Source
Deformed
CSE554 Deformation I Slide 8
OverviewOverview
• Finding deformed locations pi’ that minimize:
– Ef: fitting term
• Measures how close are the deformed source to the target
– Ed: distortion term
• Measures how much the source shape is changed
E Ef Ed
CSE554 Deformation I Slide 9
Fitting TermFitting Term
• Sum of squared distances between corresponding points
– Over all pairs of source and target points
Ef i1
m pi' qi2q2
p2
CSE554 Deformation I Slide 10
Distortion TermDistortion Term
• Q: How to measure shape?
• A: By “bumpiness” at each vertex
– Laplacian: vector from the centroid of neighbors to the vertex
• Recall that in fairing, we reduced this vector to “smooth out” bumps
• A linear operator over point locations
Lpi pi 1Ni
jNi
pj
pi
pi1pi2
pi1 pi2
2
where Ni={i1,i2,…} are indices of neighboring vertices of pi
CSE554 Deformation I Slide 11
Distortion TermDistortion Term
• Minimizing changes in Laplacians during deformation
– Over all source points
Ed ni1
Lpi' i2pi
Lpi'pi'
i
i: Laplacian at pi before deformation
CSE554 Deformation I Slide 12
Putting TogetherPutting Together
• Finding deformed locations pi’ that minimize:
– A quadratic equation in terms of variables (pix’, piy’, piz’)
• qi, i are constants
• L[] is a linear operator
E Ef Ed
i1
m pi' qi2 ni1
Lpi' i2
CSE554 Deformation I Slide 13
Quadratic MinimizationQuadratic Minimization
• A general form of quadratic minimization:
– There are s variables: x=(x1,…,xs)T
– Each a1,…, ak is a length-s column vector (linear coefficients)
– Each b1,…, bk is a scalar (constant coefficients)
– k should be greater than s
min ki1
aiT x bi2
CSE554 Deformation I Slide 14
Quadratic MinimizationQuadratic Minimization
• Re-writing our minimization in the general form
– In 2D, there are 2n variables: x = (p1x’,…, pnx’, p1y’,…, pny’ )T
• In 3D, there are 3n variables
– We will next re-write each quadratic term in 2D as (aix-bi)2
• Can be extended easily to 3D
E Ef Ed
i1
m pi' qi2 ni1
Lpi' i2
CSE554 Deformation I Slide 15
Quadratic MinimizationQuadratic Minimization
• The ai and bi in the fitting term
– There are 2m quadratic terms
– In the first set of m terms:
• For i=1,…,m, bi=qix, ai contains all zero, except its (i)th entry is 1.
– In the second set of m terms:
• For i=1,…,m, bi+m=qiy, ai+m contains all zero, except its (i+n)th entry is 1
Ef i1
m pi' qi2 i1
m pix ' qix2 i1
m piy ' qiy22 mi1
aiT x bi2
CSE554 Deformation I Slide 16
Quadratic MinimizationQuadratic Minimization
• The ai and bi in the fitting term
– There are 2m quadratic terms
– Example with 3 vertices and 2 fitting constraints (n=3; m=2):
Ef i1
m pi' qi2 i1
m pix ' qix2 i1
m piy ' qiy22 mi1
aiT x bi2x
p1x 'p2x 'p3x 'p1y '
p2y '
p3y '
b1 q1xb2 q2xb3 q1yb4 q2y
a1T 1 0 0 0 0 0
a2T 0 1 0 0 0 0
a3T 0 0 0 1 0 0
a4T 0 0 0 0 1 0
CSE554 Deformation I Slide 17
Quadratic MinimizationQuadratic Minimization
• The ai and bi in the distortion term:
– There are 2n quadratic terms
– The first set of n terms:
• For i=1,…,n, ai is all zero except the (i)th entry is 1, the (j)th entries are -1/|Ni|
for all jNi, and bi=ix
– The second set of n terms:
• For i=1,…,n, ai+n is all zero except the (i+n)th entry is 1, the (j+n)th entries are
-1/|Ni| for all jNi, and bi+n=iy
Ed ni1
Lpi' i2 ni1
Lpix ' ix2 ni1
Lpiy ' iy2Lpi pi
1Ni jNi
pj
2 ni1
aiT x bi2
CSE554 Deformation I Slide 18
Quadratic MinimizationQuadratic Minimization
• The ai and bi in the distortion term:
– There are 2n quadratic terms
– Example with 3 vertices (n=3):
Ed ni1
Lpi' i2 ni1
Lpix ' ix2 ni1
Lpiy ' iy2Lpi pi
1Ni jNi
pj
x
p1x 'p2x 'p3x 'p1y '
p2y '
p3y '
b1 1xb2 2xb3 3xb4 1yb5 2yb6 3y
2 ni1
aiT x bi2
p1 p2
p3a1T 1 1
2 12
0 0 0 a2
T 12
1 12
0 0 0 a3
T 12
12
1 0 0 0 a4
T 0 0 0 1 12
12
a5T 0 0 0 1
21 1
2
a6T 0 0 0 1
2 12
1
CSE554 Deformation I Slide 19
Quadratic MinimizationQuadratic Minimization
• To solve:
• Re-write in matrix form:
min A x B2where A
a1T
akTis a k by s matrix
min ki1
aiT x bi2
B b1bk
is a length-k vector
CSE554 Deformation I Slide 20
Quadratic MinimizationQuadratic Minimization
• The minimizer is where the partial derivatives are all zero
– We have:
0 Ax B2
x
2 AT A x 2 AT B AT A x AT B
x AT A 1 AT B
CSE554 Deformation I Slide 22
ResultsResults
Deformed
A small deformation
CSE554 Deformation I Slide 23
ResultsResults
Deformed
A larger deformation
CSE554 Deformation I Slide 27
DiscussionDiscussion
• Limitations
– Local features are “skewed”, and they don’t scale with the model
• Reason: Laplacian changes with rotation or scale
– Two bumps that differ by rotation or scale have different Laplacians
• Which will be penalized by our distortion term
pipi'
Lpi Lpi'Lpi Lpi'
CSE554 Deformation I Slide 28
A Better Distortion TermA Better Distortion Term
• Not penalizing rotation and scaling of local features
– Transforming the original Laplacian vectors before comparing to the deformed Laplacians
– Ti is a matrix that describes how the local shape around pi is deformed
• Including translation, rotation and isotropic scaling
• Collectively called similarity transformations
– Entries of Ti are linear forms of variables pi’
• So that the minimization problem is still quadratic
Ed ni1
Lpi' Ti i2
CSE554 Deformation I Slide 29
Some QuestionsSome Questions
• How to represent similarity transformations as matrices?
• How to compute Ti so that it is linear in the deformed points?
• We will focus in the derivations of the 2D case
– 3D results will be briefly presented at the end
CSE554 Deformation I Slide 30
Similarity Transforms (2D)Similarity Transforms (2D)
• Homogeneous coordinates
– A 2D point: (x,y,1)
• A 2D vector: (x,y,0)
– A 3D point: (x,y,z,1)
• A 3D vector: (x,y,z,0)
CSE554 Deformation I Slide 31
Similarity Transforms (2D)Similarity Transforms (2D)
• Translation
– Cartesian coordinates: vector addition
– Homogeneous coordinates: matrix product
px'
py' vx
vy px
py
px'
py'
1
1 0 vx0 1 vy0 0 1
pxpy1
CSE554 Deformation I Slide 32
Similarity Transforms (2D)Similarity Transforms (2D)
• Isotropic scaling
– Cartesian coordinates: vector scaling
– Homogeneous coordinates: matrix product
px'
py' s px
py
px'
py'
1
s 0 00 s 00 0 1
pxpy1
CSE554 Deformation I Slide 33
Similarity Transforms (2D)Similarity Transforms (2D)
• Rotation
– Cartesian coordinates: matrix product
– Homogeneous coordinates: matrix product
px'
py' Cos Sin
Sin Cos pxpy
px'
py'
1
Cos Sin 0Sin Cos 0
0 0 1
pxpy1
CSE554 Deformation I Slide 34
Similarity Transforms (2D)Similarity Transforms (2D)
• Summary of elementary similarity transformations
– To combine transformations: take the product of these matrices
px'
py'
1
M pxpy1
Trsv 1 0 vx0 1 vy0 0 1
Rot Cos Sin 0Sin Cos 0
0 0 1
Translation by vector v
Scaling by scalar s
Rotation by angle
Scls s 0 00 s 00 0 1
CSE554 Deformation I Slide 35
Similarity Transforms (2D)Similarity Transforms (2D)
• General similarity transformations
– The product of any set of elementary matrices can be written this way
– Any choice of (a, w, tx, ty) can be written as a sequence of rotation,
isotropic scaling and translation
• Note that a and w can’t be both zero
T a w txw a ty0 0 1
a w txw a ty0 0 1
Trstx, ty.Scl a2 w2 .RotArcTan a
w
CSE554 Deformation I Slide 36
Computing Ti (2D)Computing Ti (2D)
• Suppose we know the deformed locations p i’
• Compute Ti as the similarity transform that best fits the
neighborhood of pi to that of pi’
• This is a quadratic minimization problem for entries of T i
– E.g., a, w, tx, ty
min Ti pi pi'2 jNi
Ti pj pj'2
CSE554 Deformation I Slide 37
Computing Ti (2D)Computing Ti (2D)
• The matrix form of the minimization is:
where is a 2|Ni|+2 by 4 matrix, and Ni={i1, i2,…} are indices of neighboring vertices of pi
C
pix piy 1 0
piy pix 0 1
pi1x pi1y 1 0
pi1y pi1x 0 1
min C
awtxty
pix 'piy '
pi1x 'pi1y '
2
CSE554 Deformation I Slide 38
Computing Ti (2D)Computing Ti (2D)
• By quadratic minimization:
– Linear expressions of variables (pix’ , piy’)
awtxty
CT C 1 CT
pix 'piy '
pi1x 'pi1y '
CSE554 Deformation I Slide 39
Distortion Term (2D)Distortion Term (2D)
• Two parts of each distortion term:
– Transformed Laplacian:
– Laplacian of the deformed locations:
Ti i D
awtxty
D CT C 1 CT
pix 'piy '
pi1x 'pi1y '
Lpi' L
pix 'piy '
pi1x 'pi1y '
L 1 0 1
Ni0 ...
0 1 0 1Ni
...
where
where is a 2 by 2|Ni|+2 matrix
Lpi' Ti i2D
ix iy 0 0
iy ix 0 0
CSE554 Deformation I Slide 40
Distortion Term (2D)Distortion Term (2D)
• Putting together:
– They form 2n quadratic terms (aix-bi)2 for x = (p1x’,…, pnx’, p1y’,…, pny’ )T
• All bi are zero
• Each ai can be extracted from H
H L D CT C 1 CT
H1, H2where
are its rows
Ed ni1
Lpi' Ti i2 n
i1
H1 pix 'piy '
pi1x 'pi1y '
2
ni1
H2 pix 'piy '
pi1x 'pi1y '
2
and
CSE554 Deformation I Slide 41
Results (2D)Results (2D)
New distortion term
Old distortion term
CSE554 Deformation I Slide 42
Results (2D)Results (2D)
New distortion termOld distortion term
CSE554 Deformation I Slide 43
Results (2D)Results (2D)
New distortion term
Old distortion term
CSE554 Deformation I Slide 44
Results (2D)Results (2D)
New distortion term
Old distortion term
CSE554 Deformation I Slide 45
RegistrationRegistration
• Use nearest neighbors as corresponding target locations
– Assuming the source is already close to the target
• Iterative closest point (ICP)
– 1. For each point on the source, assign its closest point on the target as its corresponding point. Compute Laplacian-based deformation.
• A threshold on the closest distance can be used to throw away unlikely correspondences
– 2. Repeat step (1) until a termination criteria is met.
• Maximum iteration or minimum RMSD improvement
CSE554 Deformation I Slide 46
ResultResult
After rigid alignment 1 iteration of Laplacian
7 iterations of Laplacian Overlaying all curves
CSE554 Deformation I Slide 47
ResultResult
• Weighting the distortion term
E Ef w Ed
small w
medium w
large w
CSE554 Deformation I Slide 48
Similarity Transforms (3D)Similarity Transforms (3D)
• Elementary transformation matrices
– To perform a sequence of transformations: take the product of these matrices
Translation by vector v
Scaling by scalar s
Rotation by angle around X axis
px'
py'
pz'
1
M
pxpypz1
Trsv
1 0 0 vx0 1 0 vy0 0 1 vz0 0 0 1
RotX,
1 0 0 00 Cos Sin 00 Sin Cos 00 0 0 1
Scls
s 0 0 00 s 0 00 0 s 00 0 0 1
CSE554 Deformation I Slide 49
Similarity Transforms (3D)Similarity Transforms (3D)
• General similarity transformations in 3D
– Approximates the product of a set of elementary matrices
• Up to a small rotation angle
• May introduce skewing for large rotations
T
s h3 h2 txh3 s h1 tyh2 h1 s tz0 0 0 1
CSE554 Deformation I Slide 50
Computing Ti (3D)Computing Ti (3D)
• Assuming known deformation, by quadratic minimization:
– Linear expressions of the deformed points pi’
• C is a 3|Ni|+3 by 7 matrix
sh1h2h3txtytz
CT C 1 CT
pix 'piy '
piz 'pi1x 'pi1y '
pi1z '
C
pix 0 piz piy 1 0 0
piy piz 0 pix 0 1 0
piz piy pix 0 0 0 1
pi1x 0 pi1z pi1y 1 0 0
pi1y pi1z 0 pi1x 0 1 0
pi1z pi1y pi1x 0 0 0 1
where
CSE554 Deformation I Slide 51
Distortion Term (3D)Distortion Term (3D)
• Constructing transformed Laplacian:
whereTi i D
sh1h2h3txtytz
D CT C 1 CT
pix 'piy '
piz 'pi1x 'pi1y '
pi1z '
where D
ix 0 iz iy 0 0 0
iy iz 0 ix 0 0 0
iz iy ix 0 0 0 0
CSE554 Deformation I Slide 52
Results (3D)Results (3D)
Stationary targetsMoving targets