Upload
vankiet
View
256
Download
0
Embed Size (px)
Citation preview
Spectral geometry of shapes
Michael Bronstein
Institute of Computational Science, Faculty of InformaticsUniversity of Lugano, Switzerland
July 26, 2013
1 / 363
Dimensions of media
2 / 363
Dimensions of media
3 / 363
Dimensions of media
4 / 363
Dimensions of media
5 / 363
Ubiquitous 3D data
Google 3D Warehouse
6 / 363
Ubiquitous 3D data
Microsoft Kinect based on Primesense structured light range sensor
7 / 363
Ubiquitous 3D data
Intel presenting laptop-integratable range scanner prototype
8 / 363
Ubiquitous 3D data
3D printer
9 / 363
Ubiquitous 3D data
Point cloud from Kinect Mesh from Google 3DWarehouse
10 / 363
3D shapes vs. images
Array of pixels
11 / 363
3D shapes vs. images
Array of pixels
Implicit surface, Mesh, Point cloud
12 / 363
3D shapes vs. images
Affine, projective
13 / 363
3D shapes vs. images
Affine, projectiveWealth of nonrigid
deformations
14 / 363
3D shapes vs. images
12 +
12 =
12 +
12
= ?
15 / 363
3D shapes vs. images
12 +
12 =
12 +
12
= ?
16 / 363
Applications
Filtering Distances Correspondence
Symmetry Editing Retrieval
17 / 363
Spectral geometry
=
18 / 363
Why spectral geometry?
Representation independent
Intrinsic, hence deformation-invariant
Computationally efficient
Can be interpreted in terms of classical signal processing
19 / 363
Why spectral geometry?
Representation independent
Intrinsic, hence deformation-invariant
Computationally efficient
Can be interpreted in terms of classical signal processing
20 / 363
Why spectral geometry?
Representation independent
Intrinsic, hence deformation-invariant
Computationally efficient
Can be interpreted in terms of classical signal processing
21 / 363
Why spectral geometry?
Representation independent
Intrinsic, hence deformation-invariant
Computationally efficient
Can be interpreted in terms of classical signal processing
22 / 363
Agenda
Euclidean heat equation
Non-Euclidean heat equation
Laplace-Beltrami operator, its eigenvalues and eigenfunctions
Global stuctures: Diffusion distances
Local stuctures: Heat kernel and other spectral descriptors
Glocal stuctures: Stable region detectors
Applications: Shape retrieval, Correspondence, Symmetry
23 / 363
Heat equation
Heat equation on a circle S1 2
u(, t) +tu(, t) = 0
Initial condition u(, 0) = v()
Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t) and T are eigenfunctions of the operators and tBasic solutions are of the form
un(, t) = en2tein
Sum of solutions is also a solution
u(, t) =n0
anen2tein
24 / 363
Heat equation
Heat equation on a circle S1{u(, t) + tu(, t) = 0
Initial condition u(, 0) = v()
Laplacian = 22
Solution u : S1 [0,) R is the amount of heat at point at time t
Solution is separable: u(, t) = ()T (t) and T are eigenfunctions of the operators and tBasic solutions are of the form
un(, t) = en2tein
Sum of solutions is also a solution
u(, t) =n0
anen2tein
25 / 363
heat_circ.mp4Media File (video/mp4)
Heat equation
Heat equation on a circle S1{u(, t) + tu(, t) = 0
Initial condition u(, 0) = v()
Laplacian = 22
Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t)
and T are eigenfunctions of the operators and tBasic solutions are of the form
un(, t) = en2tein
Sum of solutions is also a solution
u(, t) =n0
anen2tein
26 / 363
heat_circ.mp4Media File (video/mp4)
Heat equation
Heat equation on a circle S1{u(, t) + tu(, t) = 0
Initial condition u(, 0) = v()
Laplacian = 22
Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t)
()T (t) + ()T (t) = 0
and T are eigenfunctions of the operators and tBasic solutions are of the form
un(, t) = en2tein
Sum of solutions is also a solution
u(, t) =n0
anen2tein
27 / 363
heat_circ.mp4Media File (video/mp4)
Heat equation
Heat equation on a circle S1{u(, t) + tu(, t) = 0
Initial condition u(, 0) = v()
Laplacian = 22
Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t)
()
()=T (t)
T (t)=
and T are eigenfunctions of the operators and tBasic solutions are of the form
un(, t) = en2tein
Sum of solutions is also a solution
u(, t) =n0
anen2tein
28 / 363
heat_circ.mp4Media File (video/mp4)
Heat equation
Heat equation on a circle S1{u(, t) + tu(, t) = 0
Initial condition u(, 0) = v()
Laplacian = 22
Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t) and T are eigenfunctions of the operators and t
() = ()
tT (t) = T (t)
Basic solutions are of the form
un(, t) = en2tein
Sum of solutions is also a solution
u(, t) =n0
anen2tein
29 / 363
heat_circ.mp4Media File (video/mp4)
Heat equation
Heat equation on a circle S1{u(, t) + tu(, t) = 0
Initial condition u(, 0) = v()
Laplacian = 22
Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t) and T are eigenfunctions of the operators and t
ein = n2ein
ten2t = n2en
2t
Basic solutions are of the form
un(, t) = en2tein
Sum of solutions is also a solution
u(, t) =n0
anen2tein
30 / 363
heat_circ.mp4Media File (video/mp4)
Heat equation
Heat equation on a circle S1{u(, t) + tu(, t) = 0
Initial condition u(, 0) = v()
Laplacian = 22
Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t) and T are eigenfunctions of the operators and tBasic solutions are of the form
un(, t) = en2tein
Sum of solutions is also a solution
u(, t) =n0
anen2tein
31 / 363
heat_circ.mp4Media File (video/mp4)
Heat equation
Heat equation on a circle S1{u(, t) + tu(, t) = 0
Initial condition u(, 0) = v()
Laplacian = 22
Solution u : S1 [0,) R is the amount of heat at point at time tSolution is separable: u(, t) = ()T (t) and T are eigenfunctions of the operators and tBasic solutions are of the form
un(, t) = en2tein
Sum of solutions is also a solution
u(, t) =n0
anen2tein
32 / 363
heat_circ.mp4Media File (video/mp4)
Heat equation
Solution must satisfy the initial condition
u(, 0) = v()
Solution is given as convolution u(, t) = (v ? ht)()
ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).
Heat operator Ht = et is compact and self-adjoint
33 / 363
Heat equation
Solution must satisfy the initial condition expressed as Fourierseries in the orthonormal basis {ein}n0
n0ane
ine0 =n0v, ein
=an
ein
Solution is given as convolution u(, t) = (v ? ht)()
ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).
Heat operator Ht = et is compact and self-adjoint
34 / 363
Heat equation
Solution must satisfy the initial condition expressed as Fourierseries in the orthonormal basis {ein}n0
u(, t) =n0v, ein
an
einen2t
=n
1
2
20
v()eind einen2t
Solution is given as convolution u(, t) = (v ? ht)()
ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).
Heat operator Ht = et is compact and self-adjoint
35 / 363
Heat equation
Solution must satisfy the initial condition expressed as Fourierseries in the orthonormal basis {ein}n0
u(, t) =n0v, eineinen2t
=1
2
20
v()n
en2tein()d
Solution is given as convolution u(, t) = (v ? ht)()
ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).
Heat operator Ht = et is compact and self-adjoint
36 / 363
Heat equation
Solution must satisfy the initial condition expressed as Fourierseries in the orthonormal basis {ein}n0
u(, t) =n0v, eineinen2t
=1
2
20
v()n
en2tein()
ht()
d
Solution is given as convolution u(, t) = (v ? ht)()
ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).
Heat operator Ht = et is compact and self-adjoint
37 / 363
Heat equation
Solution must satisfy the initial condition expressed as Fourierseries in the orthonormal basis {ein}n0
u(, t) =n0v, eineinen2t
=1
2
20
v()n
en2tein()
ht()
d
= (v ? ht)()
Solution is given as convolution u(, t) = (v ? ht)()
ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).
Heat operator Ht = et is compact and self-adjoint
38 / 363
Heat equation
Solution is given as convolution u(, t) = (v ? ht)()
ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).
Heat operator Ht = et is compact and self-adjoint
39 / 363
Heat equation
Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)
ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).
Heat operator Ht = et is compact and self-adjoint
40 / 363
Heat equation
Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).
Heat operator Ht = et is compact and self-adjoint
41 / 363
Heat equation
Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).ht( ) = how much heat is transferred from point topoint in time t
Heat operator Ht = et is compact and self-adjoint
42 / 363
Heat equation
Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).ht( ) = how much heat is transferred from point topoint in time t
Heat operator Ht : v() 7 (v ? ht)() has the sameeigenfunctions as
and eigenvalues en2t
Heat operator Ht = et is compact and self-adjoint
43 / 363
Heat equation
Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).ht( ) = how much heat is transferred from point topoint in time t
Heat operator Ht : v() 7 (v ? ht)() has the sameeigenfunctions as
Hteim = eim ? ht() =1
2
20
eimn0
en2tein()d
and eigenvalues en2t
Heat operator Ht = et is compact and self-adjoint
44 / 363
Heat equation
Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).ht( ) = how much heat is transferred from point topoint in time t
Heat operator Ht : v() 7 (v ? ht)() has the sameeigenfunctions as
Hteim = eim ? ht() =n0
en2tein
1
2
20
ei(nm)d
and eigenvalues en2t
Heat operator Ht = et is compact and self-adjoint
45 / 363
Heat equation
Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).ht( ) = how much heat is transferred from point topoint in time t
Heat operator Ht : v() 7 (v ? ht)() has the sameeigenfunctions as
Hteim = eim ? ht() =n0
en2tein
1
2
20
ei(nm)d mn
and eigenvalues en2t
Heat operator Ht = et is compact and self-adjoint
46 / 363
Heat equation
Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).ht( ) = how much heat is transferred from point topoint in time tHeat operator Ht : v() 7 (v ? ht)() has the sameeigenfunctions as
Hteim = eim ? ht() =n0
en2tein
1
2
20
ei(nm)d mn
= en2teim
and eigenvalues en2t
Heat operator Ht = et is compact and self-adjoint
47 / 363
Heat equation
Solution is given as convolution u(, t) = (v ? ht)() and isshift-invariant: if u(, t) is the solution of the heat equationwith initial condition v(), then u( 0, t) is the solutionwith initial condition v( 0)ht() is called the heat kernel of the equation, and is theresponse to initial condition () (impulse response).ht( ) = how much heat is transferred from point topoint in time t
Heat operator Ht = et is compact and self-adjoint
48 / 363
Signal processing intuition
LSI
h
49 / 363
Signal processing intuition
LSI
v u = v ? h
50 / 363
Euclidean vs Non-Euclidean heat equation
Euclidean
Heat equation on S1{u(, t) + tu(, t) = 0
u(, 0) = v()
Non-Euclidean
Heat equation on manifold X{Xu(x, t) +
tu(x, t) = 0
u(x, 0) = v(x)51 / 363
heat_man.mp4Media File (video/mp4)
Riemannian geometry
Manifold X is locallyhomeomorphic to the tangentplane TxX
Riemannian metric
, TxX : TxX TxX Rdepending smoothly on x
Isometry is a shape deformationnot changing the metric
Exponential map
expx : TxX X
x1
x2
xTxX
?We assume manifolds without boundary for simplicity52 / 363
Riemannian geometry
Manifold X is locallyhomeomorphic to the tangentplane TxX, spanned by tangentvectors x1(x), x2(x)
Riemannian metric
, TxX : TxX TxX Rdepending smoothly on x
Isometry is a shape deformationnot changing the metric
Exponential map
expx : TxX X
x1
x2
xTxX
?We assume manifolds without boundary for simplicity53 / 363
Riemannian geometry
Manifold X is locallyhomeomorphic to the tangentplane TxX, spanned by tangentvectors x1(x), x2(x)
Riemannian metric
, TxX : TxX TxX Rdepending smoothly on x
Isometry is a shape deformationnot changing the metric
Exponential map
expx : TxX X
x1
x2
xTxX
?We assume manifolds without boundary for simplicity54 / 363
Riemannian geometry
Manifold X is locallyhomeomorphic to the tangentplane TxX, spanned by tangentvectors x1(x), x2(x)
Riemannian metric
, TxX : TxX TxX Rdepending smoothly on x
Isometry is a shape deformationnot changing the metric
Exponential map
expx : TxX X
x1
x2
xTxX
?We assume manifolds without boundary for simplicity55 / 363
Riemannian geometry
Manifold X is locallyhomeomorphic to the tangentplane TxX, spanned by tangentvectors x1(x), x2(x)
Riemannian metric
, TxX : TxX TxX Rdepending smoothly on x
Isometry is a shape deformationnot changing the metric
Exponential map
expx : TxX X
x1
x2
xTxX v
expx(v)
?We assume manifolds without boundary for simplicity56 / 363
Laplace-Beltrami operator
Given a smooth function f : X R, define f expx : TxX R
Gradient
Xf(v) = (f expx)(0) =((f expx)(0)
x1,(f expx)(0)
x2
)
Taylor expansion (f expx)(v) f(x) + Xf(x), vTxXLaplace-Beltrami operator
Xf(x) = (f expx)(0) = i
2(f expx)(0)x2i
Intrinsic (expressed solely in terms of the Riemannian metric),and thus isometry-invariant
Stokes theorem:
57 / 363
Laplace-Beltrami operator
Given a smooth function f : X R, define f expx : TxX R
Gradient
Xf(v) = (f expx)(0) =((f expx)(0)
x1,(f expx)(0)
x2
)Taylor expansion (f expx)(v) f(x) + Xf(x), vTxX
Laplace-Beltrami operator
Xf(x) = (f expx)(0) = i
2(f expx)(0)x2i
Intrinsic (expressed solely in terms of the Riemannian metric),and thus isometry-invariant
Stokes theorem:
58 / 363
Laplace-Beltrami operator
Given a smooth function f : X R, define f expx : TxX R
Gradient
Xf(v) = (f expx)(0) =((f expx)(0)
x1,(f expx)(0)
x2
)Taylor expansion (f expx)(v) f(x) + Xf(x), vTxXLaplace-Beltrami operator
Xf(x) = (f expx)(0) = i
2(f expx)(0)x2i
Intrinsic (expressed solely in terms of the Riemannian metric),and thus isometry-invariant
Stokes theorem:
59 / 363
Laplace-Beltrami operator
Given a smooth function f : X R, define f expx : TxX R
Gradient
Xf(v) = (f expx)(0) =((f expx)(0)
x1,(f expx)(0)
x2
)Taylor expansion (f expx)(v) f(x) + Xf(x), vTxXLaplace-Beltrami operator
Xf(x) = (f expx)(0) = i
2(f expx)(0)x2i
Intrinsic (expressed solely in terms of the Riemannian metric),and thus isometry-invariant
Stokes theorem:
60 / 363
Laplace-Beltrami operator
Given a smooth function f : X R, define f expx : TxX RGradient
Xf(v) = (f expx)(0) =((f expx)(0)
x1,(f expx)(0)
x2
)Taylor expansion (f expx)(v) f(x) + Xf(x), vTxXLaplace-Beltrami operator
Xf(x) = (f expx)(0) = i
2(f expx)(0)x2i
Intrinsic (expressed solely in terms of the Riemannian metric),and thus isometry-invariant
Stokes theorem:X
Xf(x)g(x)dx =
XXf(x),Xg(x)TxXdx
61 / 363
Laplace-Beltrami operator
Given a smooth function f : X R, define f expx : TxX RGradient
Xf(v) = (f expx)(0) =((f expx)(0)
x1,(f expx)(0)
x2
)Taylor expansion (f expx)(v) f(x) + Xf(x), vTxXLaplace-Beltrami operator
Xf(x) = (f expx)(0) = i
2(f expx)(0)x2i
Intrinsic (expressed solely in terms of the Riemannian metric),and thus isometry-invariantStokes theorem:
XXf(x)g(x)dx Xf,gL2(X)
=
XXf(x),Xg(x)TxXdx
f,gH10(X)
62 / 363
Properties of the Laplace-Beltrami operator
Constant eigenfunction: Xf = 0 if f = const
Symmetry: Xf, gL2(X) = Xg, fL2(X)Locality: (Xf)(x) does not depend on f(x
) at any x 6= xLinear precision: if X is a plane and f = ax+ by + c, thenXf = 0
Positive semi-definiteness: Xf, fL2(X) 0Maximum/minimum principle: functions satisfying Xf = 0(harmonic) have no maxima/minima in the interior of X
63 / 363
Properties of the Laplace-Beltrami operator
Constant eigenfunction: Xf = 0 if f = const
Symmetry: Xf, gL2(X) = Xg, fL2(X)
Locality: (Xf)(x) does not depend on f(x) at any x 6= x
Linear precision: if X is a plane and f = ax+ by + c, thenXf = 0
Positive semi-definiteness: Xf, fL2(X) 0Maximum/minimum principle: functions satisfying Xf = 0(harmonic) have no maxima/minima in the interior of X
64 / 363
Properties of the Laplace-Beltrami operator
Constant eigenfunction: Xf = 0 if f = const
Symmetry: Xf, gL2(X) = Xg, fL2(X)Locality: (Xf)(x) does not depend on f(x
) at any x 6= x
Linear precision: if X is a plane and f = ax+ by + c, thenXf = 0
Positive semi-definiteness: Xf, fL2(X) 0Maximum/minimum principle: functions satisfying Xf = 0(harmonic) have no maxima/minima in the interior of X
65 / 363
Properties of the Laplace-Beltrami operator
Constant eigenfunction: Xf = 0 if f = const
Symmetry: Xf, gL2(X) = Xg, fL2(X)Locality: (Xf)(x) does not depend on f(x
) at any x 6= xLinear precision: if X is a plane and f = ax+ by + c, thenXf = 0
Positive semi-definiteness: Xf, fL2(X) 0Maximum/minimum principle: functions satisfying Xf = 0(harmonic) have no maxima/minima in the interior of X
66 / 363
Properties of the Laplace-Beltrami operator
Constant eigenfunction: Xf = 0 if f = const
Symmetry: Xf, gL2(X) = Xg, fL2(X)Locality: (Xf)(x) does not depend on f(x
) at any x 6= xLinear precision: if X is a plane and f = ax+ by + c, thenXf = 0
Positive semi-definiteness: Xf, fL2(X) 0
Maximum/minimum principle: functions satisfying Xf = 0(harmonic) have no maxima/minima in the interior of X
67 / 363
Properties of the Laplace-Beltrami operator
Constant eigenfunction: Xf = 0 if f = const
Symmetry: Xf, gL2(X) = Xg, fL2(X)Locality: (Xf)(x) does not depend on f(x
) at any x 6= xLinear precision: if X is a plane and f = ax+ by + c, thenXf = 0
Positive semi-definiteness: Xf, fL2(X) 0Maximum/minimum principle: functions satisfying Xf = 0(harmonic) have no maxima/minima in the interior of X
68 / 363
Discretization of the Laplace-Beltrami operator
Manifold X is sampled at n points {x1, . . . , xn}.
Function f : X R represented as an n-dimensional vectorf = (f(x1), . . . , f(xn))
Discrete version of the Laplacian
(Xf)(xi) nj=1
wij(fi fj)
In matrix notation
Lf = (DW)f
where D is a diagonal matrix with elements dii =
k 6=iwik
69 / 363
Discretization of the Laplace-Beltrami operator
Manifold X is sampled at n points {x1, . . . , xn}.Function f : X R represented as an n-dimensional vectorf = (f(x1), . . . , f(xn))
Discrete version of the Laplacian
(Xf)(xi) nj=1
wij(fi fj)
In matrix notation
Lf = (DW)f
where D is a diagonal matrix with elements dii =
k 6=iwik
70 / 363
Discretization of the Laplace-Beltrami operator
Manifold X is sampled at n points {x1, . . . , xn}.Function f : X R represented as an n-dimensional vectorf = (f(x1), . . . , f(xn))
Discrete version of the Laplacian
(Xf)(xi) nj=1
wij(fi fj)
In matrix notation
Lf = (DW)f
where D is a diagonal matrix with elements dii =
k 6=iwik
71 / 363
Discretization of the Laplace-Beltrami operator
Manifold X is sampled at n points {x1, . . . , xn}.Function f : X R represented as an n-dimensional vectorf = (f(x1), . . . , f(xn))
Discrete version of the Laplacian
(Xf)(xi) nj=1
wij(fi fj)
In matrix notation
Lf = (DW)f
where D is a diagonal matrix with elements dii =
k 6=iwik
72 / 363
Discretization of the Laplace-Beltrami operator
xj
xi
xj
xi
ij
ij
Point cloud
Combinatorial Laplacian
wij =
{1 i, j E0 else
Triangular mesh
Cotangent weight scheme
wij =
{ cotij+cotij2 i, j E0 else
Tutte 1963 Pinkall, Polthier 199373 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Xf = 0 if f = const
Symmetry:
Locality:
Linear precision:
Positive semi-definiteness:
Positive weights: wij 0 and each vertex i has at least onewij > 0
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
74 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry:
Locality:
Linear precision:
Positive semi-definiteness:
Positive weights: wij 0 and each vertex i has at least onewij > 0
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
75 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry: Xf, g = Xg, f
Locality:
Linear precision:
Positive semi-definiteness:
Positive weights: wij 0 and each vertex i has at least onewij > 0
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
76 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry: L = LT
Locality:
Linear precision:
Positive semi-definiteness:
Positive weights: wij 0 and each vertex i has at least onewij > 0
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
77 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors
Locality:
Linear precision:
Positive semi-definiteness:
Positive weights: wij 0 and each vertex i has at least onewij > 0
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
78 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors
Locality: (Xf)(x) does not depend on f(x) at any x 6= x
Linear precision:
Positive semi-definiteness:
Positive weights: wij 0 and each vertex i has at least onewij > 0
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
79 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors
Locality: wij = 0 if i, j / E
Linear precision:
Positive semi-definiteness:
Positive weights: wij 0 and each vertex i has at least onewij > 0
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
80 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors
Locality: wij = 0 if i, j / ELinear precision: if X is a plane and f = ax+ by + c, thenXf = 0
Positive semi-definiteness:
Positive weights: wij 0 and each vertex i has at least onewij > 0
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
81 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors
Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then
(Lx)i =j
wij(xi xj) = 0
Positive semi-definiteness:
Positive weights: wij 0 and each vertex i has at least onewij > 0
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
82 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors
Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then
(Lx)i =j
wij(xi xj) = 0
(a flat plate must have zero bending energy)
Positive semi-definiteness:
Positive weights: wij 0 and each vertex i has at least onewij > 0
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
83 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors
Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then
(Lx)i =j
wij(xi xj) = 0
(a flat plate must have zero bending energy)
Positive semi-definiteness: Xf, f 0
Positive weights: wij 0 and each vertex i has at least onewij > 0
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
84 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors
Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then
(Lx)i =j
wij(xi xj) = 0
(a flat plate must have zero bending energy)
Positive semi-definiteness: fTLf 0, i.e., L 0
Positive weights: wij 0 and each vertex i has at least onewij > 0
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
85 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors
Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then
(Lx)i =j
wij(xi xj) = 0
(a flat plate must have zero bending energy)
Positive semi-definiteness: fTLf 0, i.e., L 0Positive weights: wij 0 and each vertex i has at least onewij > 0
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
86 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors
Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then
(Lx)i =j
wij(xi xj) = 0
(a flat plate must have zero bending energy)
Positive semi-definiteness: fTLf 0, i.e., L 0Positive weights: wij 0 and each vertex i has at least onewij > 0, sufficient to satisfy discrete maximum principle
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
87 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors
Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then
(Lx)i =j
wij(xi xj) = 0
(a flat plate must have zero bending energy)
Positive semi-definiteness: fTLf 0, i.e., L 0Positive weights: wij 0 and each vertex i has at least onewij > 0, sufficient to satisfy discrete maximum principlePositive weights + Symmetry PSD
Convergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
88 / 363
Desired properties of discrete Laplacians
Constant eigenfunction: Satisfied by construction of L
Symmetry: L = LT with real eigenvalues and orthogonaleigenvectors
Locality: wij = 0 if i, j / ELinear precision: equivalently, if X is a plane, then
(Lx)i =j
wij(xi xj) = 0
(a flat plate must have zero bending energy)
Positive semi-definiteness: fTLf 0, i.e., L 0Positive weights: wij 0 and each vertex i has at least onewij > 0, sufficient to satisfy discrete maximum principlePositive weights + Symmetry PSDConvergence: solution of discrete PDE with L converges tosolution of continuous PDE with X as n
89 / 363
No free lunch
SYM LOC LIN POS PSD CONCombinatorial1 Cotangent2 Mean value3 Intrinsic Delaunay4
No free lunch theorem: all these properties cannot be satisfiedsimultaneously
1Tutte 63; 2Pinkal, Polthier 1993; 3Floater, Hormann 2006; 4Bobenko,
Springborn 200790 / 363
No free lunch
SYM LOC LIN POS PSD CONCombinatorial1 Cotangent2 Mean value3 Intrinsic Delaunay4
No free lunch theorem: all these properties cannot be satisfiedsimultaneously
1Tutte 63; 2Pinkal, Polthier 1993; 3Floater, Hormann 2006; 4Bobenko,
Springborn 2007; Wardetzky et al. 200791 / 363
Agenda
Euclidean heat equation
Non-Euclidean heat equation
Laplace-Beltrami operator, its eigenvalues and eigenfunctions
Global stuctures: Diffusion distances
Local stuctures: Heat kernel and other spectral descriptors
Glocal stuctures: Stable region detectors
Applications: Shape retrieval, Correspondence, Symmetry
92 / 363
Laplace-Beltrami eigenvalues
For compact manifold X the Laplace-Beltrami operator iscompact
Spectral theorem: eigenvalues Xi = ii are countable,and {i}i0 form an orthonormal basis on L2(X)Eigenvalues are real 0 0 1 , and is the onlyaccumulating point
Weyls law for 2-manifold: n 4narea(X) for n
93 / 363
Laplace-Beltrami eigenvalues
For compact manifold X the Laplace-Beltrami operator iscompact
Spectral theorem: eigenvalues Xi = ii are countable,and {i}i0 form an orthonormal basis on L2(X)
Eigenvalues are real 0 0 1 , and is the onlyaccumulating point
Weyls law for 2-manifold: n 4narea(X) for n
94 / 363
Laplace-Beltrami eigenvalues
For compact manifold X the Laplace-Beltrami operator iscompact
Spectral theorem: eigenvalues Xi = ii are countable,and {i}i0 form an orthonormal basis on L2(X)Eigenvalues are real 0 0 1 , and is the onlyaccumulating point
Weyls law for 2-manifold: n 4narea(X) for n
95 / 363
Laplace-Beltrami eigenvalues
For compact manifold X the Laplace-Beltrami operator iscompact
Spectral theorem: eigenvalues Xi = ii are countable,and {i}i0 form an orthonormal basis on L2(X)Eigenvalues are real 0 0 1 , and is the onlyaccumulating point
Weyls law for 2-manifold: n 4narea(X) for n
Weyl 191196 / 363
Discrete Laplace-Beltrami eigenvalues
Weak form eigendecomposition
X, L2(X) = , L2(X)
Choose hat functions 1, . . . , n that are piece-wise linear onthe triangles s.t. i(xi) = 1 and i(xj) = 0 for j 6= iRepresent (x) = 11(x) + + nn(x)Rewrite the weak form eigenproblem as
ni=1
iXi, jL2(X) sij stifness
=
ni=1
i i, jL2(X) mij mass
In matrix notation:S = M
Lumped mass matrix A = diag(a1, . . . , an) M, where ai isthe area element at vertex i
97 / 363
Discrete Laplace-Beltrami eigenvalues
Weak form eigendecomposition
X, L2(X) = , L2(X)
Choose hat functions 1, . . . , n that are piece-wise linear onthe triangles s.t. i(xi) = 1 and i(xj) = 0 for j 6= i
Represent (x) = 11(x) + + nn(x)Rewrite the weak form eigenproblem as
ni=1
iXi, jL2(X) sij stifness
=
ni=1
i i, jL2(X) mij mass
In matrix notation:S = M
Lumped mass matrix A = diag(a1, . . . , an) M, where ai isthe area element at vertex i
98 / 363
Discrete Laplace-Beltrami eigenvalues
Weak form eigendecomposition
X, L2(X) = , L2(X)
Choose hat functions 1, . . . , n that are piece-wise linear onthe triangles s.t. i(xi) = 1 and i(xj) = 0 for j 6= iRepresent (x) = 11(x) + + nn(x)
Rewrite the weak form eigenproblem asni=1
iXi, jL2(X) sij stifness
=
ni=1
i i, jL2(X) mij mass
In matrix notation:S = M
Lumped mass matrix A = diag(a1, . . . , an) M, where ai isthe area element at vertex i
99 / 363
Discrete Laplace-Beltrami eigenvalues
Weak form eigendecomposition
X, L2(X) = , L2(X)
Choose hat functions 1, . . . , n that are piece-wise linear onthe triangles s.t. i(xi) = 1 and i(xj) = 0 for j 6= iRepresent (x) = 11(x) + + nn(x)Rewrite the weak form eigenproblem as
ni=1
iXi, jL2(X) sij stifness
=
ni=1
i i, jL2(X) mij mass
In matrix notation:S = M
Lumped mass matrix A = diag(a1, . . . , an) M, where ai isthe area element at vertex i
100 / 363
Discrete Laplace-Beltrami eigenvalues
Weak form eigendecomposition
X, L2(X) = , L2(X)
Choose hat functions 1, . . . , n that are piece-wise linear onthe triangles s.t. i(xi) = 1 and i(xj) = 0 for j 6= iRepresent (x) = 11(x) + + nn(x)Rewrite the weak form eigenproblem as
ni=1
iXi, jL2(X) sij stifness
=
ni=1
i i, jL2(X) mij mass
In matrix notation:S = M
Lumped mass matrix A = diag(a1, . . . , an) M, where ai isthe area element at vertex i
101 / 363
Discrete Laplace-Beltrami eigenvalues
Weak form eigendecomposition
X, L2(X) = , L2(X)
Choose hat functions 1, . . . , n that are piece-wise linear onthe triangles s.t. i(xi) = 1 and i(xj) = 0 for j 6= iRepresent (x) = 11(x) + + nn(x)Rewrite the weak form eigenproblem as
ni=1
iXi, jL2(X) sij stifness
=
ni=1
i i, jL2(X) mij mass
In matrix notation:S = M
Lumped mass matrix A = diag(a1, . . . , an) M, where ai isthe area element at vertex i
102 / 363
Discrete Laplace-Beltrami eigenvalues
Weak form eigendecomposition
X, L2(X) = , L2(X)
Choose hat functions 1, . . . , n that are piece-wise linear onthe triangles s.t. i(xi) = 1 and i(xj) = 0 for j 6= iRepresent (x) = 11(x) + + nn(x)Rewrite the weak form eigenproblem as
ni=1
iXi, jL2(X) sij stifness
=
ni=1
i i, jL2(X) mij mass
In matrix notation:S = M
Lumped mass matrix A = diag(a1, . . . , an) M, where ai isthe area element at vertex i , S is cotangent Laplacian L
103 / 363
Diagonalization of the Laplacian
Eigendecomposition can be posed as the minimization problem
min
off(TL) s.t. TA = I,
with off-diagonality penalty off(X) =
i6=j x2ij .
Jacobi iteration: compose = R3R2R1 as a sequence ofGivens rotations, where each new rotation tries to reduce theoff-diagonal terms
Analytic expression for optimal rotation for given pivot
Rotation applied in place no matrix multiplication
Guaranteed decrease of the off-diagonal terms
Orthonormality guaranteed by construction
Jacobi 1846
104 / 363
Diagonalization of the Laplacian
Eigendecomposition can be posed as the minimization problem
min
off(TA1/2LA1/2) s.t. T = I,
with off-diagonality penalty off(X) =
i6=j x2ij .
Jacobi iteration: compose = R3R2R1 as a sequence ofGivens rotations, where each new rotation tries to reduce theoff-diagonal terms
Analytic expression for optimal rotation for given pivot
Rotation applied in place no matrix multiplication
Guaranteed decrease of the off-diagonal terms
Orthonormality guaranteed by construction
Jacobi 1846105 / 363
Diagonalization of the Laplacian
Eigendecomposition can be posed as the minimization problem
min
off(TA1/2LA1/2) s.t. T = I,
with off-diagonality penalty off(X) =
i6=j x2ij .
Jacobi iteration: compose = R3R2R1 as a sequence ofGivens rotations, where each new rotation tries to reduce theoff-diagonal terms
Analytic expression for optimal rotation for given pivot
Rotation applied in place no matrix multiplication
Guaranteed decrease of the off-diagonal terms
Orthonormality guaranteed by construction
Jacobi 1846106 / 363
Diagonalization of the Laplacian
Eigendecomposition can be posed as the minimization problem
min
off(TA1/2LA1/2) s.t. T = I,
with off-diagonality penalty off(X) =
i6=j x2ij .
Jacobi iteration: compose = R3R2R1 as a sequence ofGivens rotations, where each new rotation tries to reduce theoff-diagonal terms
Analytic expression for optimal rotation for given pivot
Rotation applied in place no matrix multiplication
Guaranteed decrease of the off-diagonal terms
Orthonormality guaranteed by construction
Jacobi 1846107 / 363
Diagonalization of the Laplacian
Eigendecomposition can be posed as the minimization problem
min
off(TA1/2LA1/2) s.t. T = I,
with off-diagonality penalty off(X) =
i6=j x2ij .
Jacobi iteration: compose = R3R2R1 as a sequence ofGivens rotations, where each new rotation tries to reduce theoff-diagonal terms
Analytic expression for optimal rotation for given pivot
Rotation applied in place no matrix multiplication
Guaranteed decrease of the off-diagonal terms
Orthonormality guaranteed by construction
Jacobi 1846108 / 363
Diagonalization of the Laplacian
Eigendecomposition can be posed as the minimization problem
min
off(TA1/2LA1/2) s.t. T = I,
with off-diagonality penalty off(X) =
i6=j x2ij .
Jacobi iteration: compose = R3R2R1 as a sequence ofGivens rotations, where each new rotation tries to reduce theoff-diagonal terms
Analytic expression for optimal rotation for given pivot
Rotation applied in place no matrix multiplication
Guaranteed decrease of the off-diagonal terms
Orthonormality guaranteed by construction
Jacobi 1846109 / 363
To see the sound
Chladni 1787 110 / 363
chladni.movMedia File (video/quicktime)
Chladni plates
Vibration modes (Laplace-Beltrami eigenfunctions) aresolutions of the Helmholtz equation
X =
Eigenvalues are vibration resonance frequencies
111 / 363
Chladni plates
Vibration modes (Laplace-Beltrami eigenfunctions) aresolutions of the Helmholtz equation
X =
Eigenvalues are vibration resonance frequencies
112 / 363
ShapeDNA
Laplace-Beltrami eigenvalues can be used as anisometry-invariant shape descriptor (shape DNA)
10 20 30 40 50 60 70 80 90 1000
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04
Reuter, Wolter, Peinecke 2005113 / 363
Can we hear the shape of the drum?
isometric = isospectral
Can one hear the shape of the drum?
Kac 1966
114 / 363
Can we hear the shape of the drum?
isometric?== isospectral
Can one hear the shape of the drum?
Kac 1966
115 / 363
Can we hear the shape of the drum?
isometric?== isospectral
Can one hear the shape of the drum?
Kac 1966116 / 363
Can we hear the shape of the drum?
By Minakshisundaram-Pleijel expansion of the heat trace
tr(Ht) =n0
etn =i0
antn
(for a 2-manifold without boundary) the following properties canbe recovered (heard) from the spectrum of the Laplacian:
Area: a0 = area(X)
Total Gaussian curvature: a1 =13
X K(x)dx
Can we hear the metric?
Minakshisundaram, Pleijel 1949117 / 363
Can we hear the shape of the drum?
By Minakshisundaram-Pleijel expansion of the heat trace
tr(Ht) =n0
etn =i0
antn
(for a 2-manifold without boundary) the following properties canbe recovered (heard) from the spectrum of the Laplacian:
Area: a0 = area(X)
Total Gaussian curvature: a1 =13
X K(x)dx
Can we hear the metric?
Minakshisundaram, Pleijel 1949118 / 363
Can we hear the shape of the drum?
By Minakshisundaram-Pleijel expansion of the heat trace
tr(Ht) =n0
etn =i0
antn
(for a 2-manifold without boundary) the following properties canbe recovered (heard) from the spectrum of the Laplacian:
Area: a0 = area(X)
Total Gaussian curvature: a1 =13
X K(x)dx
Can we hear the metric?
Minakshisundaram, Pleijel 1949119 / 363
Can we hear the shape of the drum?
By Minakshisundaram-Pleijel expansion of the heat trace
tr(Ht) =n0
etn =i0
antn
(for a 2-manifold without boundary) the following properties canbe recovered (heard) from the spectrum of the Laplacian:
Area: a0 = area(X)
Total Gaussian curvature: a1 =13
X K(x)dx, and by
Gauss-Bonnet theorem, the Euler characteristic: a1 =23 (X)
Can we hear the metric?
Minakshisundaram, Pleijel 1949120 / 363
Can we hear the shape of the drum?
By Minakshisundaram-Pleijel expansion of the heat trace
tr(Ht) =n0
etn =i0
antn
(for a 2-manifold without boundary) the following properties canbe recovered (heard) from the spectrum of the Laplacian:
Area: a0 = area(X)
Total Gaussian curvature: a1 =13
X K(x)dx, and by
Gauss-Bonnet theorem, the Euler characteristic: a1 =23 (X)
Can we hear the metric?
Minakshisundaram, Pleijel 1949121 / 363
One cannot hear the shape of the drum!
Counter example of isospectral non-isometric shapes
Gordon, Web, Wolpert 1992122 / 363
Laplace-Beltrami eigenfunctions
0 1 2 3
Eigenfunctions are invariant to isometric deformations?
0 1 2 3
?Up to some ambiguities, as we will see later
123 / 363
Laplace-Beltrami eigenfunctions
0 1 2 3
Eigenfunctions are invariant to isometric deformations?
0 1 2 3
?Up to some ambiguities, as we will see later124 / 363
Fourier analysis on manifolds
A function f L2(X) can be represented as the Fourier series inthe orthonormal basis {i}i0
f(x) =i0f, iL2(X)
ai
i(x)
= a0 + a1 + a2 + . . .
125 / 363
Fourier analysis on manifolds
A function f L2(X) can be represented as the Fourier series inthe orthonormal basis {i}i0
f(x) =i0f, iL2(X)
ai
i(x)
= a0 + a1 + a2 + . . .
126 / 363
Shape filtering
Filtering by applying a transfer function F () to the Fouriercoefficients
f(x) =i0
F (i)f, iL2(X)i(x)
Filter shape embedding coordinates
Fourier basis {i}i0 depends on the shape - must beupdated iteratively (fractional filtering)
0.5
1
0.5
1
Original Lowpass Bandpass
Levy 2006
; Kovnatsky, Bronstein2 2012
127 / 363
Shape filtering
Filtering by applying a transfer function F () to the Fouriercoefficients
f(x) =i0
F (i)f, iL2(X)i(x)
Filter shape embedding coordinates
Fourier basis {i}i0 depends on the shape - must beupdated iteratively (fractional filtering)
0.5
1
0.5
1
Original Lowpass Bandpass
Levy 2006
; Kovnatsky, Bronstein2 2012
128 / 363
Shape filtering
Filtering by applying a transfer function F () to the Fouriercoefficients
f(x) =i0
F (i)f, iL2(X)i(x)
Filter shape embedding coordinates
Fourier basis {i}i0 depends on the shape - must beupdated iteratively (fractional filtering)
0.5
1
0.5
1
Original Lowpass Bandpass
Levy 2006; Kovnatsky, Bronstein2 2012129 / 363
Heat kernel
Heat kernel of the heat equation on X
ht(x, x) = Ht(x, x)
Not shift-invariant anymore!
For initial condition u(x, t) = v(x), the solution is given by
Htv(x) =i0v, iL2(X)Hti(x)
=
Xv(x)ht(x, x
)dx
Non shift-invariant convolution
130 / 363
Heat kernel
Heat kernel of the heat equation on X
ht(x, x) = Ht(x, x) =
i0(x, x), iL2(X)Hti(x)
Not shift-invariant anymore!
For initial condition u(x, t) = v(x), the solution is given by
Htv(x) =i0v, iL2(X)Hti(x)
=
Xv(x)ht(x, x
)dx
Non shift-invariant convolution
131 / 363
Heat kernel
Heat kernel of the heat equation on X
ht(x, x) = Ht(x, x) =
i0
eiti(x)i(x)
Not shift-invariant anymore!
For initial condition u(x, t) = v(x), the solution is given by
Htv(x) =i0v, iL2(X)Hti(x)
=
Xv(x)ht(x, x
)dx
Non shift-invariant convolution
132 / 363
Heat kernel
Heat kernel of the heat equation on X
ht(x, x) = Ht(x, x) =
i0
eiti(x)i(x)
Not shift-invariant anymore!
For initial condition u(x, t) = v(x), the solution is given by
Htv(x) =i0v, iL2(X)Hti(x)
=
Xv(x)ht(x, x
)dx
Non shift-invariant convolution
133 / 363
Heat kernel
Heat kernel of the heat equation on X
ht(x, x) = Ht(x, x) =
i0
eiti(x)i(x)
Not shift-invariant anymore!
For initial condition u(x, t) = v(x), the solution is given by
Htv(x) =i0v, iL2(X)Hti(x) =
i0
Xv(x)i(x
)dxeiti(x)
=
Xv(x)ht(x, x
)dx
Non shift-invariant convolution
134 / 363
Heat kernel
Heat kernel of the heat equation on X
ht(x, x) = Ht(x, x) =
i0
eiti(x)i(x)
Not shift-invariant anymore!
For initial condition u(x, t) = v(x), the solution is given by
Htv(x) =i0v, iL2(X)Hti(x) =
Xv(x)
i0
i(x)eiti(x)dx
=
Xv(x)ht(x, x
)dx
Non shift-invariant convolution
135 / 363
Heat kernel
Heat kernel of the heat equation on X
ht(x, x) = Ht(x, x) =
i0
eiti(x)i(x)
Not shift-invariant anymore!
For initial condition u(x, t) = v(x), the solution is given by
Htv(x) =i0v, iL2(X)Hti(x) =
Xv(x)
i0
i(x)eiti(x)dx
=
Xv(x)ht(x, x
)dx
Non shift-invariant convolution
136 / 363
Heat kernel
Heat kernel h100(x0, x) for different positions x0
137 / 363
Euclidean vs Non-Euclidean heat equation
Euclidean{u(, t) + tu(, t) = 0
u(, 0) = v()
Laplacian Laplacian eigenfunction
ein = n2ein
Heat kernel (shift-invariant)
ht( ) =n0
en2tein()
Solution
u(, t) = (v ? ht)()
Heat operator Ht = et
Non-Euclidean{Xu(x, t) +
tu(x, t) = 0
u(x, 0) = v(x)
Laplace-Beltrami operator XLaplace-Beltrami eigenfunctions
Xi(x) = ii(x)
Heat kernel (not shift-invariant)
ht(x, x) =
n0
eiti(x)i(x)
Solution
u(x, t) =
Xv(x)ht(x, x
)dx
Heat operator Ht = etX138 / 363
Agenda
Euclidean heat equation
Non-Euclidean heat equation
Laplace-Beltrami operator, its eigenvalues and eigenfunctions
Global stuctures: Diffusion distances
Local stuctures: Heat kernel and other spectral descriptors
Glocal stuctures: Stable region detectors
Applications: Shape retrieval, Correspondence, Symmetry
139 / 363
Diffusion maps
Diffusion kernel is a kernel of the form
k(x, x) =i0
K(i)i(x)i(x)
satisfying the following properties
i0K
2(i)
Diffusion maps
Diffusion kernel is a kernel of the form
k(x, x) =i0
K(i)i(x)i(x)
satisfying the following propertiesi0K
2(i)
Diffusion maps
Diffusion kernel is a kernel of the form
k(x, x) =i0
K(i)i(x)i(x)
satisfying the following propertiesi0K
2(i)
Diffusion maps
Diffusion kernel is a kernel of the form
k(x, x) =i0
K(i)i(x)i(x)
satisfying the following propertiesi0K
2(i)
Diffusion maps
Diffusion kernel is a kernel of the form
k(x, x) =i0
K(i)i(x)i(x)
satisfying the following propertiesi0K
2(i)
Diffusion maps
Diffusion map is a map from point x to a sequence
: x 7 {K(i)i(x)}i0 `2
Depending on the choice of the kernel K(), different maps areobtained
Laplacian eigenmap2: K() = 1
Diffusion map1,3: heat kernel K() = et
Global point signature4: K() = 1
1Berard, Besson, Gallot 1994; 2Belkin, Niyogi 2001; 3Coifman, Lafon 2006;4Rustamov 2007
145 / 363
Diffusion maps
Diffusion map is a map from point x to a sequence
: x 7 {K(i)i(x)}i0 `2
Depending on the choice of the kernel K(), different maps areobtained
Laplacian eigenmap2: K() = 1
Diffusion map1,3: heat kernel K() = et
Global point signature4: K() = 1
1Berard, Besson, Gallot 1994; 2Belkin, Niyogi 2001; 3Coifman, Lafon 2006;4Rustamov 2007
146 / 363
Diffusion maps
Diffusion map is a map from point x to a sequence
: x 7 {K(i)i(x)}i0 `2
Depending on the choice of the kernel K(), different maps areobtained
Laplacian eigenmap2: K() = 1
Diffusion map1,3: heat kernel K() = et
Global point signature4: K() = 1
Defined up to isometry in the diffusion map space (for simpleeigenvalues, sign flip of the corresponding eigenfunction)
1Berard, Besson, Gallot 1994; 2Belkin, Niyogi 2001; 3Coifman, Lafon 2006;4Rustamov 2007
147 / 363
Diffusion maps
Diffusion embedding of different poses of the human shape
Belkin, Niyogi 2001; Coifman, Lafon 2006148 / 363
Diffusion maps and distances
By Parsevals theorem,i0
K2(i)(i(x) i(x))2 =X
(k(x, z) k(x, z))2dz
Diffusion map is an isometric embedding of the diffusionmetric in the (infinite dimensional) Euclidean space
Since the kernel K() decays, diffusion distance can beapproximated using the first n terms
d(x, x)
(ni=1
K2(i)(i(x) i(x))2)1/2
149 / 363
Diffusion maps and distances
By Parsevals theorem,i0
K2(i)(i(x) i(x))2 (x)(x)2
`2
=
X
(k(x, z) k(x, z))2dz d2(x,x)
Diffusion map is an isometric embedding of the diffusionmetric in the (infinite dimensional) Euclidean space
Since the kernel K() decays, diffusion distance can beapproximated using the first n terms
d(x, x)
(ni=1
K2(i)(i(x) i(x))2)1/2
150 / 363
Diffusion maps and distances
By Parsevals theorem,i0
K2(i)(i(x) i(x))2 (x)(x)2
`2
=
X
(k(x, z) k(x, z))2dz d2(x,x)
Diffusion map is an isometric embedding of the diffusionmetric in the (infinite dimensional) Euclidean space
Since the kernel K() decays, diffusion distance can beapproximated using the first n terms
d(x, x)
(ni=1
K2(i)(i(x) i(x))2)1/2
151 / 363
Diffusion distances
K() = et
t = 100 t = 500 t = 1000
Coifman, Lafon 2006152 / 363
Diffusion distances
K() = 1
K() = 1
Commute time Biharmonic
Lipman, Rustamov, Funkhouser 2010153 / 363
Agenda
Euclidean heat equation
Non-Euclidean heat equation
Laplace-Beltrami operator, its eigenvalues and eigenfunctions
Global stuctures: Diffusion distances
Local stuctures: Heat kernel and other spectral descriptors
Glocal stuctures: Stable region detectors
Applications: Shape retrieval, Correspondence, Symmetry
154 / 363
Diffusion kernel descriptors
Associate each point x with a heat kernel signature (HKS)
p(x) : x 7 (kt1(x, x), . . . , ktn(x, x))ht(x, x) =
i0
eit2i (x)
Multi-scale, isometry-invariant, dense point-wise descriptor
Sun, Ovsjanikov, Guibas 2009; Gebal et al. 2009155 / 363
Diffusion kernel descriptors
Associate each point x with a heat kernel signature (HKS)
p(x) : x 7 (kt1(x, x), . . . , ktn(x, x))ht(x, x) =
i0
eit2i (x)
Multi-scale, isometry-invariant, dense point-wise descriptor
Sun, Ovsjanikov, Guibas 2009; Gebal et al. 2009156 / 363
Heat kernel signature
For small t, heat kernel diagonal can be expanded as
ht(x, x) 1
4t
(1 +
1
3K(x)t+O(t2)
)
HKS can be interpreted as a multi-scale Gaussian curvature
Heat kernel diagonal (auto diffusion) ht(x, x) = how muchheat remans at point x after time t
Sun, Ovsjanikov, Guibas 2009157 / 363
Heat kernel signature
For small t, heat kernel diagonal can be expanded as
ht(x, x) 1
4t
(1 +
1
3K(x)t+O(t2)
)HKS can be interpreted as a multi-scale Gaussian curvature
Heat kernel diagonal (auto diffusion) ht(x, x) = how muchheat remans at point x after time t
Sun, Ovsjanikov, Guibas 2009158 / 363
Heat kernel signature
For small t, heat kernel diagonal can be expanded as
ht(x, x) 1
4t
(1 +
1
3K(x)t+O(t2)
)HKS can be interpreted as a multi-scale Gaussian curvature
Heat kernel diagonal (auto diffusion) ht(x, x) = how muchheat remans at point x after time t
Sun, Ovsjanikov, Guibas 2009159 / 363
Scale invariance
Heat kernel signature is not scale invariant!
0 10 20 30 40 50 60 70 80 90 1000
0.005
0.01
0.015
0.02
0.025
0.03
0.035
0.04
Original shape: i, i, ht(x, x)
Shape scaled by :12i,
1i,
12h2t(x, x)
Bronstein, Kokkinos 2010160 / 363
Scale invariant heat kernel signature
Let h() = h (x, x) be logarithmically-sampled HKS
HKS of a shape scaled by is h() = 2h( 2 log )Applying log and derivative
d
dlog h() =
d
d
(2 log + log h( 2 log )
)
=dd h( 2 log )h( 2 log ) g(2 log )
a scale-covariant HKS is obtained
g() =dd h()
h()= log
i0 ie
i2i (x)i0 e
i2i (x)
In the Fourier domain
g()F G() g() F G()ei2 log
Take |G()| to obtain scale-invariant HKS (SI-HKS)
Bronstein, Kokkinos 2010
161 / 363
Scale invariant heat kernel signature
Let h() = h (x, x) be logarithmically-sampled HKSHKS of a shape scaled by is h() = 2h( 2 log )
Applying log and derivative
d
dlog h() =
d
d
(2 log + log h( 2 log )
)
=dd h( 2 log )h( 2 log ) g(2 log )
a scale-covariant HKS is obtained
g() =dd h()
h()= log
i0 ie
i2i (x)i0 e
i2i (x)
In the Fourier domain
g()F G() g() F G()ei2 log
Take |G()| to obtain scale-invariant HKS (SI-HKS)
Bronstein, Kokkinos 2010
162 / 363
Scale invariant heat kernel signature
Let h() = h (x, x) be logarithmically-sampled HKSHKS of a shape scaled by is h() = 2h( 2 log )Applying log and derivative
d
dlog h() =
d
d
(2 log + log h( 2 log )
)
=dd h( 2 log )h( 2 log ) g(2 log )
a scale-covariant HKS is obtained
g() =dd h()
h()= log
i0 ie
i2i (x)i0 e
i2i (x)
In the Fourier domain
g()F G() g() F G()ei2 log
Take |G()| to obtain scale-invariant HKS (SI-HKS)
Bronstein, Kokkinos 2010
163 / 363
Scale invariant heat kernel signature
Let h() = h (x, x) be logarithmically-sampled HKSHKS of a shape scaled by is h() = 2h( 2 log )Applying log and derivative
d
dlog h() =
d
d
(2 log + log h( 2 log )
)=
dd h( 2 log )h( 2 log ) g(2 log )
a scale-covariant HKS is obtained
g() =dd h()
h()= log
i0 ie
i2i (x)i0 e
i2i (x)
In the Fourier domain
g()F G() g() F G()ei2 log
Take |G()| to obtain scale-invariant HKS (SI-HKS)
Bronstein, Kokkinos 2010
164 / 363
Scale invariant heat kernel signature
Let h() = h (x, x) be logarithmically-sampled HKSHKS of a shape scaled by is h() = 2h( 2 log )Applying log and derivative
d
dlog h() =
d
d
(2 log + log h( 2 log )
)=
dd h( 2 log )h( 2 log ) g(2 log )
a scale-covariant HKS is obtained
g() =dd h()
h()= log
i0 ie
i2i (x)i0 e
i2i (x)
In the Fourier domain
g()F G() g() F G()ei2 log
Take |G()| to obtain scale-invariant HKS (SI-HKS)
Bronstein, Kokkinos 2010
165 / 363
Scale invariant heat kernel signature
Let h() = h (x, x) be logarithmically-sampled HKSHKS of a shape scaled by is h() = 2h( 2 log )Applying log and derivative
d
dlog h() =
d
d
(2 log + log h( 2 log )
)=
dd h( 2 log )h( 2 log ) g(2 log )
a scale-covariant HKS is obtained
g() =dd h()
h()= log
i0 ie
i2i (x)i0 e
i2i (x)
In the Fourier domain
g()F G() g() F G()ei2 log
Take |G()| to obtain scale-invariant HKS (SI-HKS)
Bronstein, Kokkinos 2010
166 / 363
Scale invariant heat kernel signature
Let h() = h (x, x) be logarithmically-sampled HKSHKS of a shape scaled by is h() = 2h( 2 log )Applying log and derivative
d
dlog h() =
d
d
(2 log + log h( 2 log )
)=
dd h( 2 log )h( 2 log ) g(2 log )
a scale-covariant HKS is obtained
g() =dd h()
h()= log
i0 ie
i2i (x)i0 e
i2i (x)
In the Fourier domain
g()F G() g() F G()ei2 log
Take |G()| to obtain scale-invariant HKS (SI-HKS)Bronstein, Kokkinos 2010
167 / 363
Scale invariant heat kernel signature
HKS h()
Bronstein, Kokkinos 2010168 / 363
Scale invariant heat kernel signature
HKS h() Scale-covariant HKS h()
Bronstein, Kokkinos 2010169 / 363
Scale invariant heat kernel signature
HKS h() Scale-covariant HKS h() Scale-invariant HKS |G()|
Bronstein, Kokkinos 2010170 / 363
Scale invariant heat kernel signature
Three components of descriptors shown as RGB colors
HKS SI-HKS
Bronstein, Kokkinos 2010171 / 363
Wave kernel signature
Different physical model: a quantum particle with initialenergy distribution f().
Described by the Schrodinger equation(iX +
t
)(x, t) = 0
Solution (x, t) complex wave function oscillatory behavior
|(x, t)|2 = probability to find particle at point x at time t.Solution in spectral domain
(x, t) =k0
eiktf(k)k(x)
Aubry, Schlickewei, Cremers 2011172 / 363
Wave kernel signature
Different physical model: a quantum particle with initialenergy distribution f(). (Frequency = h Energy)
Described by the Schrodinger equation(iX +
t
)(x, t) = 0
Solution (x, t) complex wave function oscillatory behavior
|(x, t)|2 = probability to find particle at point x at time t.Solution in spectral domain
(x, t) =k0
eiktf(k)k(x)
Aubry, Schlickewei, Cremers 2011173 / 363
Wave kernel signature
Different physical model: a quantum particle with initialenergy distribution f(). (Frequency = h Energy)
Described by the Schrodinger equation(iX +
t
)(x, t) = 0
Solution (x, t) complex wave function oscillatory behavior
|(x, t)|2 = probability to find particle at point x at time t.Solution in spectral domain
(x, t) =k0
eiktf(k)k(x)
Aubry, Schlickewei, Cremers 2011174 / 363
Wave kernel signature
Different physical model: a quantum particle with initialenergy distribution f(). (Frequency = h Energy)
Described by the Schrodinger equation(iX +
t
)(x, t) = 0
Solution (x, t) complex wave function oscillatory behavior
|(x, t)|2 = probability to find particle at point x at time t.Solution in spectral domain
(x, t) =k0
eiktf(k)k(x)
Aubry, Schlickewei, Cremers 2011175 / 363
Wave kernel signature
Different physical model: a quantum particle with initialenergy distribution f(). (Frequency = h Energy)
Described by the Schrodinger equation(iX +
t
)(x, t) = 0
Solution (x, t) complex wave function oscillatory behavior
|(x, t)|2 = probability to find particle at point x at time t.
Solution in spectral domain
(x, t) =k0
eiktf(k)k(x)
Aubry, Schlickewei, Cremers 2011176 / 363
Wave kernel signature
Different physical model: a quantum particle with initialenergy distribution f(). (Frequency = h Energy)
Described by the Schrodinger equation(iX +
t
)(x, t) = 0
Solution (x, t) complex wave function oscillatory behavior
|(x, t)|2 = probability to find particle at point x at time t.Solution in spectral domain
(x, t) =k0
eiktf(k)k(x)
Aubry, Schlickewei, Cremers 2011177 / 363
Wave kernel signature
Family of log-normal initial energy distributions
fe() exp((log e log )
2
22
)
Probability to find particle at point x
pe(x) = limT
1
T
T0|(x, t)|2dt =
k1
f2e (k)2k(x)
Each point is associated with the wave kernel signature(WKS)
p(x) : x 7 (pe1(x), . . . , pen(x))
Aubry, Schlickewei, Cremers 2011178 / 363
Wave kernel signature
Family of log-normal initial energy distributions
fe() exp((log e log )
2
22
)Probability to find particle at point x
pe(x) = limT
1
T
T0|(x, t)|2dt =
k1
f2e (k)2k(x)
Each point is associated with the wave kernel signature(WKS)
p(x) : x 7 (pe1(x), . . . , pen(x))
Aubry, Schlickewei, Cremers 2011179 / 363
Wave kernel signature
Family of log-normal initial energy distributions
fe() exp((log e log )
2
22
)Probability to find particle at point x
pe(x) = limT
1
T
T0|(x, t)|2dt =
k1
f2e (k)2k(x)
Each point is associated with the wave kernel signature(WKS)
p(x) : x 7 (pe1(x), . . . , pen(x))
Aubry, Schlickewei, Cremers 2011180 / 363
A signal processing view of HKS
0
0.2
0.4
0.6
0.8
1
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05
Collection of low pass filters
p(x) =k0
p1(k)...pn(k)
2k(x)pi() = exp(ti)
Sun, Ovsjanikov, Guibas 2009181 / 363
A signal processing view of WKS
0
0.2
0.4
0.6
0.8
1
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05
Collection of band-pass filters
p(x) =k0
p1(k)...pn(k)
2k(x)pi() = exp
((log ei log )
2
2
)Aubry, Schlickewei, Cremers 2011
182 / 363
Waves vs Heat
Distance in HKS descriptor space
Poor spatial feature localization!
Aubry, Schlickewei, Cremers 2011183 / 363
Waves vs Heat
Distance in WKS descriptor space
Better spatial feature localization
Lower discriminativity
Aubry, Schlickewei, Cremers 2011184 / 363
Waves vs Heat
103
102
101
100
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False positive rate
True
pos
itive
rate
HKSWKS
103
102
101
100
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False negative rate
True
neg
ativ
e ra
te
WKS has higher sensitivity than HKS (better at low FPR)
Yet has lower specificity (worse at low FNR)
WKS is better for correspondence problems
HKS is better for shape retrieval problems
Bronstein 2011185 / 363
Waves vs Heat
103
102
101
100
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False positive rate
True
pos
itive
rate
HKSWKS
103
102
101
100
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False negative rate
True
neg
ativ
e ra
te
WKS has higher sensitivity than HKS (better at low FPR)
Yet has lower specificity (worse at low FNR)
WKS is better for correspondence problems
HKS is better for shape retrieval problems
Bronstein 2011186 / 363
Waves vs Heat
103
102
101
100
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False positive rate
True
pos
itive
rate
HKSWKS
103
102
101
100
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False negative rate
True
neg
ativ
e ra
te
WKS has higher sensitivity than HKS (better at low FPR)
Yet has lower specificity (worse at low FNR)
WKS is better for correspondence problems
HKS is better for shape retrieval problems
Bronstein 2011187 / 363
Waves vs Heat
103
102
101
100
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False positive rate
True
pos
itive
rate
HKSWKS
103
102
101
100
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False negative rate
True
neg
ativ
e ra
te
WKS has higher sensitivity than HKS (better at low FPR)
Yet has lower specificity (worse at low FNR)
WKS is better for correspondence problems
HKS is better for shape retrieval problems
Bronstein 2011188 / 363
Optimal spectral descriptors
Build optimal spectral descriptor of the form
pf (x) =k0
f1(k)...fn(k)
2k(x)
Parametrized by family of frequency responsesf = (f1(), . . . , fn()).
Similar in spirit to Wiener filter:
attenuate frequencies with large noise content (deformation)pass frequencies with large signal content (discriminativegeometric features)
Hard to model axiomatically
...yet easy to learn from examples!
Bronstein 2011189 / 363
Optimal spectral descriptors
Build optimal spectral descriptor of the form
pf (x) =k0
f1(k)...fn(k)
2k(x)Parametrized by family of frequency responsesf = (f1(), . . . , fn()).
Similar in spirit to Wiener filter:
attenuate frequencies with large noise content (deformation)pass frequencies with large signal content (discriminativegeometric features)
Hard to model axiomatically
...yet easy to learn from examples!
Bronstein 2011190 / 363
Optimal spectral descriptors
Build optimal spectral descriptor of the form
pf (x) =k0
f1(k)...fn(k)
2k(x)Parametrized by family of frequency responsesf = (f1(), . . . , fn()).
Similar in spirit to Wiener filter:
attenuate frequencies with large noise content (deformation)pass frequencies with large signal content (discriminativegeometric features)
Hard to model axiomatically
...yet easy to learn from examples!
Bronstein 2011191 / 363
Optimal spectral descriptors
Build optimal spectral descriptor of the form
pf (x) =k0
f1(k)...fn(k)
2k(x)Parametrized by family of frequency responsesf = (f1(), . . . , fn()).
Similar in spirit to Wiener filter:
attenuate frequencies with large noise content (deformation)pass frequencies with large signal content (discriminativegeometric features)
Hard to model axiomatically
...yet easy to learn from examples!
Bronstein 2011192 / 363
Optimal spectral descriptors
Build optimal spectral descriptor of the form
pf (x) =k0
f1(k)...fn(k)
2k(x)Parametrized by family of frequency responsesf = (f1(), . . . , fn()).
Similar in spirit to Wiener filter:
attenuate frequencies with large noise content (deformation)pass frequencies with large signal content (discriminativegeometric features)
Hard to model axiomatically
...yet easy to learn from examples!
Bronstein 2011193 / 363
Optimal spectral descriptor
Given a set of points x, knowingly similar points (positive)x+, and knowingly dissimilar points (negative) x
Find frequency responses f by solving
minfpf (x) pf (x+) pf (x) pf (x)
Boils down to Mahalanobis metric learning problem
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05
0.5
0
0.5
1
Optimal filters learned from positive and negative examples
Bronstein 2011194 / 363
Optimal spectral descriptor
Given a set of points x, knowingly similar points (positive)x+, and knowingly dissimilar points (negative) xFind frequency responses f by solving
minfpf (x) pf (x+) pf (x) pf (x)
Boils down to Mahalanobis metric learning problem
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05
0.5
0
0.5
1
Optimal filters learned from positive and negative examples
Bronstein 2011195 / 363
Optimal spectral descriptor
Given a set of points x, knowingly similar points (positive)x+, and knowingly dissimilar points (negative) xFind frequency responses f by solving
minfpf (x) pf (x+) pf (x) pf (x)
Boils down to Mahalanobis metric learning problem
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05
0.5
0
0.5
1
Optimal filters learned from positive and negative examples
Bronstein 2011196 / 363
Optimal spectral descriptor
Given a set of points x, knowingly similar points (positive)x+, and knowingly dissimilar points (negative) xFind frequency responses f by solving
minfpf (x) pf (x+) pf (x) pf (x)
Boils down to Mahalanobis metric learning problem
0 0.005 0.01 0.015 0.02 0.025 0.03 0.035 0.04 0.045 0.05
0.5
0
0.5
1
Optimal filters learned from positive and negative examples
Bronstein 2011197 / 363
Optimal spectral descriptor
103
102
101
100
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False positive rate
True
pos
itive
rate
HKSWKSOpt (0.03)Opt (0.09)
103
102
101
100
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1
False negative rate
True
neg
ativ
e ra
te
HKSWKSOpt (0.03)Opt (0.09)
Better specificity and sensitivity than HKS and WKS
Bronstein 2011198 / 363
Optimal spectral descriptor
Better spatial feature localization than WKS
Better discriminativity than HKS
Bronstein 2011199 / 363
Optimal spectral descriptor
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 120
30
40
50
60
70
80
90
100
Percentage of best matches
Hit
rate
in %
HKSWKSOpt
Better performance in correspondence problems
Bronstein 2011200 / 363
ShapeGoogle
HKS descriptors Geometric words Bag of words
Given point-wise descriptor p(x)
Quantize each p in a fixed vocabulary V = {v1, . . . ,vn}
Ovsjanikov, Bronstein2, Guibas 2009, 2011201 / 363
ShapeGoogle
HKS descriptors Geometric words Bag of words
Given point-wise descriptor p(x)
Quantize each p in a fixed vocabulary V = {v1, . . . ,vn}
Ovsjanikov, Bronstein2, Guibas 2009, 2011202 / 363
ShapeGoogle
HKS descriptors Geometric words Bag of words
Given point-wise descriptor p(x)
Quantize each p in a fixed vocabulary V = {v1, . . . ,vn}Bag-of-words shape descriptor
P =
X
v(x)dx
Ovsjanikov, Bronstein2, Guibas 2009, 2011203 / 363
ShapeGoogle
HKS descriptors Geometricexpressions
Spatially-sensitivebag of words
Given point-wise descriptor p(x)
Quantize each p in a fixed vocabulary V = {v1, . . . ,vn}Spatially-sensitive bags of pairs of words
P =
XX
v(x)v(x)ht(x, x)dxdx
Ovsjanikov, Bronstein2, Guibas 2009, 2011204 / 363
ShapeGoogle
Example of bags of words
Ovsjanikov, Bronstein2, Guibas 2009, 2011205 / 363
ShapeGoogle
Shape retrieval using HKS bag-of-feature on SHREC benchmark
Ovsjanikov, Bronstein2, Guibas 2009, 2011206 / 363
Partial matching
Given a bag-of-features descriptor P of a part Y Y
Find part X X with equal area such that the localized bagof features
P(X ) =
X
vda
best matches P
Regularize part boundary length a la Mumford-Shah
minX
P(X )P2 + L(X ) s.t. area(X ) = area(Y )Ambrosio-Tortorelli approximation
minu,
P(u)P2 + 1 X2u2da+ 2
X2da
+24
X
(1 )2da s.t.Xuda = area(Y )
Pokrass, Bronstein2 2011, 2013
; Mumford, Shah 1989
207 / 363
Partial matching
Given a bag-of-features descriptor P of a part Y YFind part X X with equal area such that the localized bagof features
P(X ) =
X
vda
best matches P
Regularize part boundary length a la Mumford-Shah
minX
P(X )P2 + L(X ) s.t. area(X ) = area(Y )Ambrosio-Tortorelli approximation
minu,
P(u)P2 + 1 X2u2da+ 2
X2da
+24
X
(1 )2da s.t.Xuda = area(Y )
Pokrass, Bronstein2 2011, 2013
; Mumford, Shah 1989
208 / 363
Partial matching
Given a bag-of-features descriptor P of a part Y YFind part X X with equal area such that the localized bagof features
P(X ) =
X
vda
best matches P
Regularize part boundary length a la Mumford-Shah
minX
P(X )P2 + L(X ) s.t. area(X ) = area(Y )
Ambrosio-Tortorelli approximation
minu,
P(u)P2 + 1 X2u2da+ 2
X2da
+24
X
(1 )2da s.t.Xuda = area(Y )
Pokrass, Bronstein2 2011, 2013; Mumford, Shah 1989209 / 363
Partial matching
Given a bag-of-features descriptor P of a part Y YFind part X X with equal area such that the localized bagof features
P(X ) =
X
vda
best matches P
Regularize part boundary length a la Mumford-Shah
minX
P(X )P2 + L(X ) s.t. area(X ) = area(Y )Relaxation: represent X by an indicator functionu : X [0, 1]
P(u) =
X
vuda
Ambrosio-Tortorelli approximation
minu,
P(u)P2 + 1 X2u2da+ 2
X2da
+24
X
(1 )2da s.t.Xuda = area(Y )
Pokrass, Bronstein2 2011, 2013; Mumford, Shah 1989
210 / 363
Partial matching
Given a bag-of-features descriptor P of a part Y YFind part X X with equal area such that the localized bagof features
P(X ) =
X
vda
best matches P
Regularize part boundary length a la Mumford-Shah
minX
P(X )P2 + L(X ) s.t. area(X ) = area(Y )Ambrosio-Tortorelli approximation
minu,
P(u)P2 + 1 X2u2da+ 2
X2da
+24
X
(1 )2da s.t.Xuda = area(Y )
Pokrass, Bronstein2 2011, 2013; Mumford, Shah 1989211 / 363
Partial matching
Pokrass, Bronstein2 2011, 2013212 / 363
Agenda
Euclidean heat equation
Non-Euclidean heat equation
Laplace-Beltrami operator, its eigenvalues and eigenfunctions
Global stuctures: Diffusion distances
Local stuctures: Heat kernel and other spectral descriptors
Glocal stuctures: Stable region detectors
Applications: Shape retrieval, Correspondence, Symmetry
213 / 363
Component trees
Create a hierarchy of shape points dependent on time parameter t
t = 0: start with single points forming disjoint clustersCi = {xi}
For increasing t merge clusters Ci and Cj if they are closeenough in the sense of some intrinsic (e.g. diffusion) distance
d(Ci, Cj) = minxCixCj
d(x, x) t
Single-linkage agglomerative clustering
Hierarchy described as component tree
Litman, Bronstein2 2011214 / 363
Component trees
Create a hierarchy of shape points dependent on time parameter t
t = 0: start with single points forming disjoint clustersCi = {xi}For increasing t merge clusters Ci and Cj if they are closeenough in the sense of some intrinsic (e.g. diffusion) distance
d(Ci, Cj) = minxCixCj
d(x, x) t
Single-linkage agglomerative clustering
Hierarchy described as component tree
Litman, Bronstein2 2011215 / 363
Component trees
Create a hierarchy of shape points dependent on time parameter t
t = 0: start with single points forming disjoint clustersCi = {xi}For increasing t merge clusters Ci and Cj if they are closeenough in the sense of some intrinsic (e.g. diffusion) distance
d(Ci, Cj) = minxCixCj
d(x, x) t
Single-linkage agglomerative clustering
Hierarchy described as component tree
Litman, Bronstein2 2011216 / 363
Component trees
Create a hierarchy of shape points dependent on time parameter t
t = 0: start with single points forming disjoint clustersCi = {xi}For increasing t merge clusters Ci and Cj if they are closeenough in the sense of some intrinsic (e.g. diffusion) distance
d(Ci, Cj) = minxCixCj
d(x, x) t
Single-linkage agglomerative clustering
Hierarchy described as component tree
Litman, Bronstein2 2011217 / 363
Component trees
Litman, Bronstein2 2011218 / 363
Maximally stable components
Observation: different components grow at different speed
Stability of a component
(Ci(t)) =A(Ci(t))ddtA(Ci(t))
Measures relative change of area as functionof change of threshold
(Better stability functions are available)
Maximally stable components: localmaximizers of
Ci(t 1)
Ci(t)
Ci(t+ 1)
Litman, Bronstein2 2011219 / 363
Maximally stable components
Observation: different components grow at different speed
Stability of a component
(Ci(t)) =A(Ci(t))ddtA(Ci(t))
Measures relative change of area as functionof change of threshold
(Better stability functions are available)
Maximally stable components: localmaximizers of
Ci(t 1)
Ci(t)
Ci(t+ 1)
Litman, Bronstein2 2011220 / 363
Maximally stable components
Litman, Bronstein2 2011221 / 363
Agenda
Euclidean heat equation
Non-Euclidean heat equation
Laplace-Beltrami operator, its eigenvalues and eigenfunctions
Global stuctures: Diffusion distances
Local stuctures: Heat kernel and other spectral descriptors
Glocal stuctures: Stable region detectors
Applications: Shape retrieval, Correspondence, Symmetry
222 / 363
Point-wise correspondence
x yt
X Y
Correspondence is a bijection t : X Y
Isometry dX(x, x) = dY (t(x), t(x
)) w.r.t to intrinsic metrics(e.g. diffusion distances) dX on X and dY on Y
Minimum-distortion correspondence
NP-hard problem!
Memoli, Sapiro 2005; Bronstein2, Kimmel 2006
; Wang et al. 2011
223 / 363
Point-wise correspondence
x y
x
dX dY
y
t
X Y
Correspondence is a bijection t : X YIsometry dX(x, x
) = dY (t(x), t(x)) w.r.t to intrinsic metrics
(e.g. diffusion distances) dX on X and dY on Y
Minimum-distortion correspondence
NP-hard problem!
Memoli, Sapiro 2005; Bronstein2, Kimmel 2006
; Wang et al. 2011
224 / 363
Point-wise correspondence
x y
x
dX dY
y
t
X Y
Correspondence is a bijection t : X YIsometry dX(x, x
) = dY (t(x), t(x)) w.r.t to intrinsic metrics
(e.g. diffusion distances) dX on X and dY on YMinimum-distortion correspondence
mintdX(x, x)dY (t(x), t(x))
Gromov-Hausdorff distance
NP-hard problem!
Memoli, Sapiro 2005; Bronstein2, Kimmel 2006
; Wang et al. 2011
225 / 363
Point-wise correspondence
x y
x
dX dY
y
t
X Y
Correspondence is a bijection t : X YIsometry dX(x, x
) = dY (t(x), t(x)) w.r.t to intrinsic metrics
(e.g. diffusion distances) dX on X and dY on YMinimum-distortion correspondence
mintdX(x, x)dY (t(x), t(x)) Gromov-Hausdorff distance
NP-hard problem!
Memoli, Sapiro 2005; Bronstein2, Kimmel 2006
; Wang et al. 2011
226 / 363
Point-wise correspondence
x y
x
dX dY
y
t
X Y
Correspondence is a bijection t : X YIsometry dX(x, x
) = dY (t(x), t(x)) w.r.t to intrinsic metrics
(e.g. diffusion distances) dX on X and dY on Y
Minimum-distortion correspondence
mintdX(x, x) dY (t(x), t(x))+ pX(x) pY (t(x))
NP-hard problem!
Memoli, Sapiro 2005; Bronstein2, Kimmel 2006; Wang et al. 2011227 / 363
Point-wise correspondence
x y
x
dX dY
y
t
X Y
Correspondence is a bijection t : X YIsometry dX(x, x
) = dY (t(x), t(x)) w.r.t to intrinsic metrics
(e.g. diffusion distances) dX on X and dY on Y
Minimum-distortion correspondence
mintdX(x, x) dY (t(x), t(x))+ pX(x) pY (t(x))
NP-hard problem!
Memoli, Sapiro 2005; Bronstein2, Kimmel 2006; Wang et al. 2011228 / 363
Functional correspondence
F(X) F(Y )
Given f : X R, corresponding function on Y is g = f t1
Correspondence T : F(X) F(Y ) between spaces offunctions on X and Y is a linear map
t can be recovered from T
Ovsjanikov et al. 2012229 / 363
Functional correspondence
F(X) F(Y )
Given f : X R, corresponding function on Y is g = f t1
Correspondence T : F(X) F(Y ) between spaces offunctions on X and Y is a linear map
t can be recovered from T
Ovsjanikov et al. 2012230 / 363
Functional correspondence
F(X) F(Y )
Given f : X R, corresponding function on Y is g = f t1
Correspondence T : F(X) F(Y ) between spaces offunctions on X and Y is a linear