Upload
sydney-watson
View
220
Download
2
Tags:
Embed Size (px)
Citation preview
Diffusion-geometricmaximally stable component detection in
deformable shapes
Roee Litman, Alexander Bronstein, Michael Bronstein
Diffusion-geometricmaximally stable component detection in
deformable shapes
In a nutshell…
MSERMaximally Stable Extremal Region
Diffusion Geometry
ShapeMSER
The Feature Approachfor Images
Deformable Shape
Analysis
Feature Approach in Images
Feature based methods are the infrastructure laid in the base of many computer vision algorithms:
– Content-based image retrieval– Video tracking– Panorama alignment– 3D reconstruction form stereo
3
Problem formulation
• Find a semi-local feature detector– High repeatability– Invariance to isometric deformation– Robustness to noise, sampling, etc.
• Add discriminative descriptor
RESULTSThe “what”
Visual Example
6
Visual Example
7
More Results
8
(Taken from the TOSCA dataset)
Horse regions + Human regions
Region Matching
Query 1st, 2nd, 4th, 10th, and 15th matches9
3D Human Scans
10Taken from the SCAPE dataset
Scanned Region Matching
Query 1st, 2nd, 4th, 10th, and 15th matches11
Volume vs. Surface
12
Volume & surface isometry Boundary isometryOriginal
Volumetric Shapes
• Usually shapes are modeled as 2D boundary of a 3D shape.
• Volumetric shape model better captures "natural" behavior of non-rigid deformations.(Raviv et-al)
• Diffusion geometry terms can easily be applied to volumes
• 2D Meshes can be voxelized
13
Volumetric Regions
14Taken from the SCAPE dataset
METHODOLOGY(The “how)”
Original MSER (Matas et-al)
MSER
• Popular image blob detector• Near-linear complexity:• High repeatability [Mikolajczyk et al. 05]• Robust to affine transformation and
illumination changes
nnO loglog
17
MSER – In a nutshell
1. Threshold image at consecutive gray-levels2. Search regions whose area stay nearly the
same through a wide range of thresholds
• Efficient detection of maximally stable regions requires construction of a component tree
18
MSER – In a nutshell
19
Algorithm overview
Algorithm overview
Represent as weighted
graph
Component tree
Stable component detection
Represent as
weighted graph
Component tree
Stable component detection
Algorithm overview
Represent as weighted
graph
Component tree
Stable component detection
Represent as
weighted graph
Image as weighted graph
• An undirected graph can be created from an image, where:– Vertices are pixels– Edges by adjacency rule, e.g. 4-neiborhood
23
Weighting the graph
In images• Gray-scale as vertex-weight• Color as edge-weight [Forssen]
In Shapes• Curvature (not deformation invariant)• Diffusion Geometry
Weighting Option
• For every point on the shape:• Calculate the prob. of a random walk to return
to the same point.– Similar to Gaussian curvature– Intrinsic, i.e. – deformation invariant
Weight example
Color-mapped Level-set animation
26
Diffusion Geometry
• Analysis of diffusion (random walk) processes• Governed by the heat equation
• Solution is heat distributionat point at time
27
txf ,
tx
Heat-Kernel
• Given– Initial condition – Boundary condition, if these’s a boundary
• Solve using:
• i.e. - find the “heat-kernel”
28
0,0 xfxf
X t ydayfyxhtxf 0,,
yxht ,
The probability density for a transitionby random walk of length ,from to
Probabilistic Interpretation
yxht ,
x
t
y
x y29
Spectral Interpretation
• How to calculate ?• Heat kernel can be calculated directly from
eigen-decomposition of the Laplacain
• By spectral decomposition theorem:
xx iiiX
i
iit
t yxeyxh i ,
30
yxht ,
Laplace-Beltrami Eigenfunctions
Deformation Invariance
Computational aspects
• Shapes are discretized as triangular meshes– Can be expressed as undirected graph– Heat kernel & eigenfunctions are vectors
• Discrete Laplace-Beltrami operator
• Several weight schemes for • is usually discrete area elements
j
jiiji
iX ffwa
f1
ijw
ia
33
Computational aspects
• In matrix notation
• Solve eigendecomposition problem
iii AW
WfAfX1
j
jiiji
iX ffwa
f1
34
Auto-diffusivity
• Special case - • The chance of returning to after time • Related to Gaussian curvature by
• Now we can attach scalar value to shapes!
x t
xxht ,
2
3
11
4
1, xOxK
txxht
xK
36
Weight example
Color-mapped Level-set animation
37
Algorithm overview
Represent as weighted
graph
Component tree
Stable component detection
Represent as
weighted graph
Component tree
Stable component detection
The Component Tree
• Tree construction is a pre-process of stable region detection
• Contains level-set hierarchy,i.e. nesting relations.
• Constructed based on a weighted graph (vertex- or edge-weight)
• Tree’s nodes are level-sets(of the graph’s cross-sections)
39
Tree Example
“Graphic” Example
• A graph• Edge-weighted• 7 Cross-Section• 5 Cross-section• Two 5 level-sets
(with altitude 4)• Every level-set has
– Size (area)– Altitude (maximal weight)
1
4
78
9
8
4
144
1
4
78
9
8
4
1
Tree Construction
46
1
4
78
9
8
4
1
1 4
7
8
4
Algorithm overview
Represent as weighted
graph
Component tree
Stable component detection
Represent as
weighted graph
Component tree
Stable component detection
Detection Process
• For every leaf component in the tree:– “Climb” the tree to its root, creating the sequence:– Calculate component stability
– Local maxima of the sequenceare “Maximally stable components”
11
11
ii
iii
i
iii CACA
CwCwCA
CA
CwCACs
KCCC ...21
132 ,...,, KCsCsCs
48
PERFORMANCEThe Detils
Benchmarking The Method
• Method was tested on SHREC 2010 data-set:– 3 basic shapes (human, dog & horse)– 9 transformations, applied in 5 different strengths– 138 shapes in total
50
Original Deformation
Noise
Scale
Holes
Results
51
Quantitative Results
• Vertex-wise correspondences were given• Regions were projected onto another shape,
and overlap ratio was measured• Overlap ratio between a region and its
projected counterpart is
• Repeatability is the percent of regions with overlap above a threshold
R
'R
'
'',
RRA
RRARRO
52
0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10
10
20
30
40
50
60
70
80
90
100
X: 0.7586Y: 64.06
overlap
repe
atab
ility
(%
)Repeatability
53
65% at 0.75
Conclusion
• Stable region detector for deformable shapes• Generic detection framework:
– Vertex- and edge-weighted graph representation– Works on surface and/or volume data
• Partial matching & retrieval potential• Tested quantitatively (on SHREC10)
Thank You
Any Questions?