MotivationMotivation
Easy modeling – generate new shapes byEasy modeling generate new shapes by deforming existing ones
3
MotivationMotivation
Easy modeling – generate new shapes byEasy modeling generate new shapes by deforming existing ones
4
ChallengesChallenges
User says as little as possible…User says as little as possible…
…algorithm deduces the rest6
ChallengesChallenges
“Intuitive deformation”Intuitive deformationglobal change + local detail preservation
7
ChallengesChallenges
“Intuitive deformation”Intuitive deformationglobal change + local detail preservation
8
Rules of the GameRules of the Game
Deformation Algorithm
ShapeDeformed
Shape
Position: Orientation/Scale: Other shape property:
Constraints
Position: “This point goes there”
Orientation/Scale: “The environment of this point should rotate/scale”
Other shape property: Curvature, perimeter,…
[ Parameterization is also “deformation”: constraints = curvature 0 everywhere ]10
ApproachesApproaches
• Surface deformationSurface deformation– Shape is empty shell
• Curve for 2D deformationCurve for 2D deformation
• Surface for 3D deformation
– Deformation only defined on shape
– Deformation coupled with shape representation
11
ApproachesApproaches
• Space deformationSpace deformation– Shape is volumetric
• Planar domain in 2DPlanar domain in 2D
• Polyhedral domain in 3D
– Deformation defined in neighborhood of shape
– Can be applied to any shape representation
12
ApproachesApproaches
• Surface deformationSurface deformation– Find alternative representation which is
“deformation invariant”deformation invariant
• Space deformation– Find a space map which has “nice properties”
13
Surface DeformationSurface Deformation
Setup:Setup:– Choose alternative representation f(S)
– Given S find S’ such that• Constraints(S’) are true
S
RR1
• Constraints(S ) are true
• f(S’) = f(S)
(or close)R2
R3(or close)
• An optimization problem
14
Shape RepresentationShape Representation
How good is the representation?o good s t e ep ese tat o ?– Representation should always be invariant to:
• Global translation• Global rotation• Global scale? Depends on application
– Shapes we want “reachable” should have similar representations
• Almost isometric deformation local translation + rotation
• Almost “conformal” deformationlocal translation + rotation + scale
15
Shape RepresentationShape Representation
RobustnessRobustness– How hard is it to solve the optimization problem?
Can we find the global minimum?– Can we find the global minimum?
– Small change in constraints similar shape?
Efficiency– Can it be solved at interactive rates?
16
Shape RepresentationsShape Representations
Rule of thumb:Rule of thumb:
If representation is a linear function of the coordinates deformation is:coordinates, deformation is:
Robust
Fast
But representation is not rotation invariant!
(for large rotations)(for large rotations)
17
Surface RepresentationsSurface Representations
• Laplacian coordinatesp
• Edge lengths + dihedral angles
• Pyramid coordinates
• Local frames
• ….
18
Laplacian Coordinates [Sorkine et al. 04]Laplacian Coordinates [Sorkine et al. 04]
• Control mechanismControl mechanism– Handles (vertices) moved by user
Region of influence (ROI)– Region of influence (ROI)
MovieMovie
19
Laplacian CoordinatesLaplacian Coordinates
δ = LV = (I-D-1A)VI = Identity matrix
D = Diagonal matrix [dii = deg(vi)]A = Adjacency matrix
V = Vertices in meshV = Vertices in mesh
Approximation to normals - unique up to translation
Reconstruct by solving LV = δ for V, with one constraint
Poisson equation 20
DeformationDeformation
• Pose modeling constraints for vertices C ⊂ VPose modeling constraints for vertices C ⊂ V– v’i = ui i ∈ C
• No exact solution, minimize error
Laplacian Laplacian User ConstraintsLaplaciancoordinates of original mesh
Laplaciancoordinates of
deformed mesh
User Constraints
21
DeformationDeformation
Laplacian Laplacian User ConstraintsLaplaciancoordinates of original mesh
Laplaciancoordinates of
deformed mesh
User Constraints
22
Linear Least Squares
• Ax = b with m equations, n unknowns
Linear Least Squares
q ,
• Normal equations: (ATA)x = ATb• Solution by pseudo inverse:
T 1 Tx = A+b = [(ATA)-1AT]b
If system under determined: x = arg min { ||x|| : Ax = b }If system over determined: x = arg min { ||Ax-b||2 }If system over determined: x arg min { ||Ax b|| }
23
Laplacian CoordinatesSanity Check
• Translation invariant?Translation invariant?
• Rotation/scale invariant?
δi
δiδi
24
“Rotation Invariant” CoordsRotation Invariant Coords
The representation should take into accountThe representation should take into account local rotations + scale
δ = L(v ) T δ = L(v’ )δi = L(vi) Tiδi = L(v i)
δi
vi v'i
δi δiTiδi
27
“Rotation Invariant” CoordsRotation Invariant Coords
The representation should take into accountThe representation should take into account local rotations + scale
δ = L(v ) T δ = L(v’ )δi = L(vi) Tiδi = L(v i)
Problem: Ti depends on deformed position v’i
28
Solution: Implicit TransformationsSolution: Implicit Transformations
Idea: solve for local transformation andIdea: solve for local transformation and deformed surface simultaneously
Transformationof the local frame
29
SimilaritiesSimilarities
Restrict Ti to “good” transformations = rotationRestrict Ti to good transformations = rotation + scale similarity transformation
Similarity Transformation
30
SimilaritiesSimilarities
• Conditions on Ti to be a similarity matrix?Conditions on Ti to be a similarity matrix?
i i 2• Linear in 2D:Auxiliary variables
Uniform Rotation +Uniform scale
Rotation + translation
31
Similarities – 3D caseSimilarities 3D case
• Not linear in 3D:ot ea 3 :
• Linearize by dropping the quadratic term– Effectively: only small rotations are handledEffectively: only small rotations are handled
33
Laplacian CoordinatesRealtime?
• Need to solve a linear system each frameNeed to solve a linear system each frame
(ATA)x = ATb( )
• Precompute sparse Cholesky factorization
• Only back substitution per frame
34
How to Find the Rotations?How to Find the Rotations?
• Laplacian coordinates – solve for themLaplacian coordinates solve for them– Problem: not linear
• Another approach: propagate rotations from h dlhandles
38
Rotation PropagationRotation Propagation
• Compute handle’s “deformation gradient”Compute handle s deformation gradient
• Extract rotation and scale/shear components
d d i O• Propagate damped rotations over ROI
39
Deformation GradientDeformation Gradient
• Handle has been transformed affinelyHandle has been transformed affinely
• Deformation gradient is:
• Extract rotation R and scale/shear S
40
Smooth PropagationSmooth Propagation
• Construct smooth scalar field [0,1]Construct smooth scalar field [0,1]– α(x)=1 Full deformation (handle)– α(x)=0 No deformation (fixed part)α( ) 0 ( p )– α(x)∈[0,1] Damp transformation (in between)
• Linearly damp scale/shear: S(x)= α(x)S(handle)( ) ( ) ( )
• Log scale damp rotation: R(x) = exp(α(x)log(R(handle))( ) p( ( ) g( ( ))
41
LimitationsLimitations
• Works well for rotationsWorks well for rotations
l i d ’ h d f i di• Translations don’t change deformation gradient– “Translation insensitivity”
42
The Curse of RotationsThe Curse of Rotations
• Can’t solve for them directly using a linearCan t solve for them directly using a linear system
• Can’t propagate if the handles don’t rotate• Can t propagate if the handles don t rotate
• Some linear methods work for rotations
• Some work for translations
• None work for both
43
The Curse of RotationsThe Curse of Rotations
• Non linearNon linearmethods workfor both large rotations andtranslation only
• No free lunch:much more expensive
44
Space DeformationSpace Deformation
• Deformation function on ambient spaceDeformation function on ambient space f : n → n
• Shape S deformed by applying f to points of SS’ f (S)S’ = f (S)
S S’f (x,y)=(2x,y)
45
MotivationMotivation
• Can be applied to any geometryCan be applied to any geometry– Meshes (= non-manifold, multiple components)
– Polygon soupsPolygon soups
– Point clouds
– Volumetric dataVolumetric data
• Complexity decoupled from geometry complexity• Complexity decoupled from geometry complexity– Can pick the best complexity for required deformation
46
Required PropertiesRequired Properties
• Invariant to global operatorsg p– Global translation– Global rotation
• Smooth
• Efficient to compute
• “Intuitive deformation” ?– Can pose constraints as in surface deformation
47
MLS Deformation [Schaeffer et al. ‘06]
pi pi
1. Handles pi 2. Target locations1. Handles pi 2. Target locations
3. Find best affine f i h
4 Deform f (v) = Mv+T
transformation that maps pi to
4. Deform f (v) Mv T
48
MLS Deformation [Schaeffer et al. ‘06]
pi pi
1. Handles pi 2. Target locations1. Handles pi 2. Target locations
3. Find best affine f i htransformation that
maps pi to
4 Deform f (v) = Mv+T Closed form solution4. Deform f (v) Mv T Closed form solution
49
Similarity TransformationsSimilarity Transformations
pi pi
1. Handles pi 2. Target locations1. Handles pi 2. Target locations
3. Find best similarityf i htransformation that
maps pi to
4 Deform f (v) = Mv+T Closed form solution4. Deform f (v) Mv T Closed form solution
51
Rigid TransformationsRigid Transformations
pi pi
1. Handles pi 2. Target locations1. Handles pi 2. Target locations
3. Find best rigidf i htransformation that
maps pi to
4 Deform f (v) = Mv+T Closed form solution4. Deform f (v) Mv T Closed form solutiongiven best similarity
53
LimitationsLimitations
Deforms all space - is not “shape aware”Deforms all space is not shape aware
57
Solution: CagesSolution: Cages
• Enclose the shape in a “cage” Ω ⊂ nEnclose the shape in a cage Ω ⊂• Deformation function defined only on cage
f : Ω → nf : Ω →Ω
• New problem: how to build the cage?
60
Deformation with a Cage
f
Deformation with a Cage
fi
xi xg(x)
F
i
S
x
S
S = {x x x } x fS = {x1,x2,...,xn} Source polygonSource polygon
xi fi
Target polygonTarget polygon
g(x) = ? Interior?61
Barycentric CoordinatesBarycentric Coordinates
ffi
xi x
F
i
S
x
( )iw x
S
( )i
x
wi(x) : Ω → n
BarycentricBarycentric CoordsCoords FunctionFunctionxxi
BarycentricBarycentric CoordsCoords FunctionFunction
62
Barycentric CoordinatesBarycentric Coordinates
ffi
xi xg(x)
F
i
S
x
( )iw x n
∑
S
( )i
x 1
( ) ( )F i ii
g x w x f=
= ∑x
xi1i
63
Barycentric Coordinates
Required properties
Barycentric Coordinates
Required properties• Translation invariance (constant precision)
xi x
fi
• Reproduction of identity (linear precision)g(x)
66
Barycentric CoordinatesBarycentric Coordinates
Constant + linear precision = affine invarianceConstant + linear precision = affine invariance
( ) ( )( )
( ) ( )i
Ax T i ii
g x w x Ax T
A w x x T w x
+ = +∑
+∑ ∑( ) ( )i i ii i
A w x x T w x= + =∑ ∑
x 1Ax T= +
67
Barycentric CoordinatesBarycentric Coordinates
Required propertiesRequired properties• Smoothness – at least C1
xi x
• Interpolation (Lagrange property) fi
g(x)
68
Example: Mean Value Coords[3D: Ju et al ’05]
1tan tan2 2( )
i i
k
α α−⎛ ⎞ ⎛ ⎞⎟ ⎟⎜ ⎜+⎟ ⎟⎜ ⎜⎟ ⎟⎜ ⎜⎝ ⎠ ⎝ ⎠
xi+1
( )ii
k xx x
⎝ ⎠ ⎝ ⎠=
−
( )k
x xi αi-1
αi
( )( ) ( )i
ii
i
k xw x k x=∑
xi-1
Closed form!
69
MV - LimitationsMV Limitations
Back to the pants problemBack to the pants problem MV negative on concave polygons
71
MV - LimitationsMV Limitations
Other leg moves in opposite (!) directionOther leg moves in opposite (!) direction
72
Barycentric CoordsBarycentric Coords
Additional property required:
( ) 0iw x ≥
Mean value coords only positive
i
Mean value coords only positive on convex polygons
73
Harmonic Coordinates[Joshi et al ’07]
Solve for wi(x):Solve for wi(x):
bj li h b d dsubject to: wi linear on the boundary and
74
Harmonic CoordinatesHarmonic Coordinates
Why does it work?Why does it work?
MVC use HC use
77
Euclidean distances resistance distances
Harmonic CoordinatesHarmonic Coordinates
Properties:Properties:– All required properties
• Smooth translation + rotation invariantSmooth, translation + rotation invariant
– Positive everywhere– Positive everywhere
No closed form need to solve a PDE– No closed form, need to solve a PDE
78
ReferencesReferences
• “On Linear Variational Surface Deformation Methods” [Botsch & Sorkine ‘08]
• Tutorial: “Interactive Shape Modeling and Deformation” [Sorkine & Botsch ’09]
• “Image deformation using moving least squares” [Schaefer et al ’06]
• “Mean Value Coordinates for Closed Triangular Meshes” [Ju et al ’05]
• “Harmonic coordinates for character articulation” [Joshi et al ’07]
• Excellent webpage on barycentric coordinates:• Excellent webpage on barycentric coordinates: http://www.inf.usi.ch/hormann/barycentric/
79