Geometry-based Shape Matching - Utrecht University...Geometry-based Shape Matching S. Biasotti, D. Giorgi, S. Marini, F. Robbiano and M. Spagnuolo CNR-IMATI-GE - Italy 22/06/2007 2

1

USCS 2007: Multimedia Retrieval

GeometryGeometry--based Shape based Shape MatchingMatching

S. S. BiasottiBiasotti, , D.D. GiorgiGiorgi, , S. MariniS. Marini, , F.F. RobbianoRobbiano and and M. M. SpagnuoloSpagnuolo

CNRCNR--IMATIIMATI--GE GE -- ItalyItaly

22/06/2007 2

GeometryGeometry--Based shape descriptorsBased shape descriptors

These descriptors are based on measurements of angles, curvature, area, volume, distances, and normals of the object to be described

The geometry-based shape descriptors– are concise – can be independent of the coordinate system

of the object – capture salient shape features– some of them can be based on a random

sampling of object surface

2

22/06/2007 3

Shape distributions (Osada et al. 2002)Shape distributions (Osada et al. 2002)

Describes the overall shape of a 3D object by encoding its spatial distribution

The signature is based on a function defined on a random sampling of the model surface

The function measures different geometric properties of the object

Shape Distributions, ACM Transactions on Graphics, Vol. 21, No. 4, October 2002, Pages 807–832.

22/06/2007 4

Shape distributionsShape distributions (Osada et al. 2002)(Osada et al. 2002)

A3: angle between three random points on the surfaceD1: distance between a fixed point and one random point on the surfaceD2: distance between two random points on the surfaceD3: square root of the area of the triangle between three random points on the surfaceD4: cube root of the volume of the tetrahedron between four random points on the surface.

3

22/06/2007 5

Shape distributionsShape distributions (Osada et al. 2002)(Osada et al. 2002)

22/06/2007 6


Eight similarity measures have been experimented for the comparison of the shape distribution:

–

– Bhattacharyya:

– Probability Density Function (PDF), Minkowski :

– Cumulative Density Function (CDF), Minkowski :LN

LN

4

22/06/2007 7


Performance evaluated on 133 shape model grouped into 25 classes

22/06/2007 8


Matching between– Watertight or polygon-soup models– Range images

Shape Descriptor – Invariant w.r.t. rotation– Need to be scaled– Not unique (based on a random sampling)

Matching Approach– Global matching

5

PosePose--oblivious shape signature (Gal et al. 2007)oblivious shape signature (Gal et al. 2007)

The Pose oblivious is a 2D histogram that combines two scalar functions defined on the boundary surface of the 3D shape.– the local diameter function: this function measures the diameter of

the 3D shape in the neighbourhood of each vertex

– the centricity function: this function measures the average geodesic distance from a vertex to all other vertices on the mesh

IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, 2007


Local Diamiter:– Given a point on the object surface, the local diameter is

based on the measure of the diameters in a cone around the direction opposite to the normal of the point

Centricity Function:– The centricity of each vertex is defined as the average

geodesic distance to all other vertices

6


The shape signature is an histogram that combines both CF and DF

The signature is represented as 2D array of scalar values between [0,0] and [1,1]

Each array bin with values (x,y) contains the approximated probability of a point on the boundary of the mesh to have a DF value of x and a CF value of y


Matching between almost watertight models– gaps and surface boundaries may cause problems to the

computation of the centricity function

Shape descriptor– Rotation invariant


• correlation coefficient

•

– Best results are obtained on database of articulated figures of animals and humans

7

LightLight--field (Chen et al. 2003) field (Chen et al. 2003)

Two objects are similar if their views are similarthe similarity between two 3D models can be measured by summing up the similarity from all corresponding views

EUROGRAPHICS 2003Volume 22 (2003), Number 3

LightLight--field (Chen et al. 2003)field (Chen et al. 2003)

The cameras can be put on the 20 vertices of a regular dodecahedronA silhouettes is computed from each viewThe dissimilarity DA, between two 3D models is defined as:

– i denotes different rotations between camera positions– I1k and I2k are corresponding images under i-th rotation– d denotes the dissimilarity between two images

8


The comparison between two silhouettes is based on the centroid distance function– distance to boundary points from the centroid of

the shape.


Matching between– Watertight

• polygon-sup models may cause problems when silhouette are computed

Shape descriptor– A lot of views are needed to make the algorithm robust

w.r.t. model orientation


9

The 3D model is voxelized and decomposed into a collection of functions defined on concentric spheresfor each function spherical harmonics decomposition is used to produce a 1D descriptorby analyzing spheres at different radii, a 2D descriptor is obtained by combining the 1D descriptors,

Spherical Harmonic representation (Kazdhan et al. Spherical Harmonic representation (Kazdhan et al. 2003)2003)

Eurographics Symposium on Geometry Processing (2003)

Spherical Harmonic representation (Kazdhan et al. Spherical Harmonic representation (Kazdhan et al. 2003)2003)

2D descriptors are compared by using the L2 norm

Matching between– Watertight or polygon soup models

Shape Descriptor– (almost) rotation invariant


10

22/06/2007 19

SpinSpin--Images (Jonson&Hebert 98)Images (Jonson&Hebert 98)

The object surface is represented by a dense collection of 3D points and surface normals (oriented points)A local description is computed for each oriented point The whole shape descriptor is the set of the sampled points and the set of local shape descriptions

IEEE Transactions on Pattern Analysis and Machine Intelligence 1998

22/06/2007 20


The local descriptor (spin-image) is obtained from a local base associated to the point. The surface around the oriented point is

described by two parameters:– the radial coordinate α

(perpendicular distance to the line through the surface normal)

– The elevation coordinate β, (the signed perpendicular distance to the tangent plane defined by vertex normal and position.

11

22/06/2007 21


The coordinates (α, β) are computed for each vertex in the surface mesh within the local area around the oriented point The bins indexed by (α, β) represent the spin-image:– dark areas in the image correspond to bins that contain

many projected points.

22/06/2007 22


All of the spin-images from one surface (the model) are constructed and stored in a spin-image stackA vertex is selected at random from the other surface (the scene) and its spin-image is computedPoint correspondences are established between the selected point (scene) and the points with best matching spin-images on the other surface (model)

Object comparison by using the Spin-Images:

12

22/06/2007 23


Matching between:– Watertight or polygon-soup models– Range images– 3D scenes

Shape descriptor– Rotation and scale invariant– Not unique (based on a random sampling)

Matching Approach– Global matching– Partial matching between similar models or occluded

models– 3D model registration

Partial Matching with priority driven search (Funkhouser Partial Matching with priority driven search (Funkhouser et al. 2006)et al. 2006)

Spherical harmonics have been proposed for global matching, but they can be used as local descriptors for partial matching

correspondences between local descriptors of the query and the target model are computed A priority queue is used to compute the similarity between models

Eurographics Symposium on Geometry Processing (2006)

13


Initially, a priority queue, Q, is created to store partial matchesAll pairwise correspondences between the features of the query and target objects are created and loaded onto the priority queueThe best partial match, m, is popped off the priority queueThe match is extended by one feature correspondenceThe process stops when c target objects, each one characterized by k feature correspondences are found



Shape Descriptor– rotation invariant

Matching Approach– Partial matching

14

Salient geometric features (Gal&CohenSalient geometric features (Gal&Cohen--Or, 2006)Or, 2006)

The surface of the object is analyzed and a set of regions is identified as salientSalient regions are complex sub-parts of the surfaceEach feature is encoded as a vectorComparison between two objects is obtained by comparing the two set of salient features

In the example a sub-part correspondence is shown by providing the same colour for the matched sub-parts

ACM Transactions on Graphics (TOG), Volume 25 Issue 1, 2006


The local shape descriptor is a point p on a surface and its associated quadric patch that approximate the surface in a local neighbourhood of p.Salient geometric features are obtained by clustering together a set of descriptors such that they have a high curvature relative to their surroundings, and a high variance of curvaturevalues

15


Each salient feature is associated with a vector index (a signature) and inserted into a geometric hash tableGiven a query object, its salient feature are extracted and usedto to query the database for a list of matching features. The returned features identify the models having larger number of matches.



Shape Descriptor– rotation invariant

Matching Approach– self-similarity– Alignments – Partial matching.

16


Questions?Questions?


Retrieval of 3D objectsRetrieval of 3D objects

StructureStructure--based shape based shape machingmaching

S. S. BiasottiBiasotti, , D.D. Giorgi, S. Marini, Giorgi, S. Marini, F.F. RobbianoRobbiano and and M. SpagnuoloM. Spagnuolo

CNRCNR--IMATIIMATI--GE GE -- ItalyItaly

17

OutlineOutline

Methods based on structural descriptions– 2D shapes

• Medial axis• Shock graphs

– 3D shapes• Skeletonization based on volumetric thinning• Graphs from surface decompositions• Reeb graphs

StructuralStructural shapeshape descriptorsdescriptors

Take into account structural properties (adjacency of parts, branching,..)Describe a shape in a way that it is easily understood by peopleRepresent the way basic components connect to form a whole…

18

Medial axisMedial axis

Defined by Blum in 1967 in three ways:– Locus of points of the centre of the maximal discs

included inside the boundary– Grassfire analogy– Ridges in the distance

transform

Each arc of the graph is medial and acts as an axis of symmetryIn 2D defines a linear graph

MedialMedial axisaxis

It decomposes shapes into a graph based on local symmetriesBranches can be prioritized, pruned, and attributedInvariant to rigid shape transformationsExact computation is expensiveMA may yield complex (non linear) structures

19

Shock Shock graphgraph

Provides a “dynamic” view of the MA, orienting arcs according to the radius of the maximal discsIt is defined as the locus of singularities (shocks) generated during the grassfire propagation from the shape boundaryCharacterizes regions of the MA as first to fourth order shocks (protrusion, neck, bend and seed points)

http://www.lems.brown.edu/vision/researchAreas/ShockMatching/shock-matching.html

shocks

Shock Shock graphgraph and and shapeshape matchingmatching ((SiddiqiSiddiqi etet al 1998, al 1998, 1999)1999)

Shock graph is a directed and acyclic graph (DAG)It may be coded in a tree (shock tree)Idea: find the largest common sub-graph, in this case, a sub-tree.Starting at the root of the tree, best matches between the sub-trees are recursively found using a depth-first approach

The matching algorithm is based on spectral coding, based on the adjacency matrix, of the tree

20

Shock Shock graphgraph and and shapeshape matchingmatching

All nodes are labelled with a vector of the eigenvalue sums of its sub-trees sorted by valueCloser vectors indicate closer isometries

Shock Graph and Shape Matching, Siddiqi et al. IJCV 35(1):13-32, 1999

Shock Shock graphgraph and and shapeshape matchingmatching

The complexity of matching two trees is

where n, L respectively denote the maximum number of nodes and leaves of the treesShock graphs have been combined with aspect graphs to compare 3D objects from sets of views (Cyr&Kimia 2004)The shock scaffold (Leymarie&Kimia 2001) for 3D shapes is a more complex structure

L))n),O(nnn(O(n loglogmax 2

21

3D 3D objectsobjects

Medial axis is more complex in 3D than 2D– it has 1D curves and 2D sheets in 3D

For 3D objects, curve skeletons are often used instead of medial axis

Skeletonization Skeletonization basedbased on on volumetricvolumetric thinningthinning

Mesh voxelizationThinning of the voxelsCentreline generationGraph simplification (optional)Graph labelling with attributes

22

CurveCurve--skeletonskeleton forfor 3D 3D objectobject similaritysimilarity ((SundarSundar etet al. al. 2003)2003)

3D skeletonization– Volumetric thinning– Voxel clustering– Skeletal graph

Graph matching– spectral matching technique

used for shock graphsTime complexity is better than O(n3)

Skeleton Based Shape Matching and Retrieval, Sundar et al., Proc. of SMI 2003

ManyMany--toto--many matching of curve skeletons (Cornea many matching of curve skeletons (Cornea et al. 2005)et al. 2005)

Algorithm overview:– Extract curve-skeleton from every object– Many-to-many matching of curve-skeletons

• Similarity score• Part correspondence

Curve-skeletons are matched using the so-called Earth Mover’s Distance (EMD)– EMD is defined as the minimum amount of work necessary

to transform one point set into the other– The method has been successfully applied to partial

matching problem

3D object retrieval using Many-to-many Matching of Curve Skeletons, Cornea et al., Proc. of SMI 2005, pp. 368-373

23

ManyMany--toto--many matching of curve skeletonsmany matching of curve skeletons

EMD provides:• direct correspondence between object parts

i.e., registration• similarity score

1 2

ResultsResults

Classification rate (using 1-NN rule): 71,1%– 74,3% if also parent classes are considered

First tier: 17,2%Second tier: 22,7%

These results havebeen obtained on the PSB

3D object retrieval using Many-to-many Matching of Curve Skeletons, Cornea et al., Proc. of SMI 2005, pp. 368-373

24

GraphsGraphs fromfrom surfacesurface decompositionsdecompositions

Code the relationship among shape segments made of face clusters (segments)Segments usually correspond to graph nodesProperties depend on the segmentation criterionThe hierarchy between segments may be explicitly coded in the graphSome examples:– Hierarchical Mesh Decomposition using Fuzzy Clustering and Cuts,

Katz&Tal, 2003– Polyhedral Surface Decomposition with Applications,

Zuckerberger et al, 2002– Scale-Space Representation of 3D Models and Topological

Matching, Bespalov et al., 2003– Local feature extraction and matching partial objects, Bespalov

et al., 2006

HierarchicalHierarchical meshmesh decompositiondecomposition

Example: Katz et. al, SIGGRAPH 2003

25

ScaleScale--space space representationrepresentation ((BespalovBespalov etet al. 2003)al. 2003)

Shape is coded in a binary tree The surface is recursively decomposed according to the distance on the surface between pointsThe distance between two points P and Q depends on the angle variation that occur along the shortest path between P and QA geometric descriptor (real vector) is associated to each node of the treeThe matching is performed using a sub-graph isomorphism technique between trees

Local feature extraction Local feature extraction ((BespalovBespalov et al. 2006)et al. 2006)

In addition to the geodesic angle variation, a max-angle distance has been consideredThe method is tested for partial data matchingSub-part correspondence is shown by providing the same colour for each corresponding sub-part

Local feature extraction and matching partial objects. Bespalov et al., CAD 38(9): 1020-1037, 2006

26

M

Reeb graphReeb graph

Reeb graphs store the evolution of the level sets of a real function

f

Reeb graph definitionReeb graph definition

given f: S→R defined on the manifold M, the Reeb graphof M wrt f is the quotient space defined by “~”:

(X1, f(X1)) ~ (X2, f(X2)) ⇔ f(X1) = f(X2) and X1 and X2 are in the same connected component of f -1(f(X1))

M

f

G.Reeb. Sur le points singuliers d’une forme de Pfaff completement integrable ou d’unefonctionn numerique. Comptes Rendus Hebd. Acad. Science, Paris, 1946, 222: 847-849

27

PPropertiesroperties

It provides a 1D structure of the shapeIt describes the shape of an object under the lens of the function fThe flexibility of the choice of the function f makes it adaptable to different tasksWhen dealing with shape matching, f is usually chosen to be translation and rotation invariant

f

min

max

OverviewOverview of of RGsRGs whenwhen the the functionfunction ff variesvaries

height bounding spherecenter

integralgeodesic

curvature extrema

barycenter

f

min

max

28

Reeb Reeb graphgraph basedbased representationsrepresentations

Several variations of the Reeb graph have been considered for graph matchingSome examples:– Multiresolution Reeb graph (MRG), (Hilaga et al.

2001, Bespalov et al. 2003)– augmented Multiresolution Reeb graph (aMRG),

(Tung&Schmitt 2004, 2005)– Extended Reeb graph (ERG), (Biasotti et al. 2003,

2006)

MultiresolutionMultiresolution Reeb Reeb graphgraph

It is defined on the basis of the function:

where g represents the geodesic distance(M. Hilaga, Y. Shinagawa, T. Komura, T. L. Kunii, “Topology Matching for Fully Automatic Similarity

Estimation of 3D Shapes”, Siggraph 2001, 2001)

Surface protrusions are maxima of the function f

f

min

max

29

MultiresolutionMultiresolution Reeb Reeb graphgraph

Provides a hierarchical graph encoding

The graph is extracted inserting contours in a progressive manner

The area A of a region and the relative size L of the interval of f are associated as attributes to nodes

GraphGraph matchingmatching

Similarity between two nodes P,Q is weighted on their attributes:

Nodes with maximal similarity are paired if:– Share the same range of f– Parent nodes are matched– Belong to graph paths

already matched

The distance between two MRGs is the sum of all node similarities

10|,)()(|)1(|)()(|),( <<−−+−= ααα QLPLQAPAQPsim

30

MultiresolutionMultiresolution Reeb graphReeb graph

Independent of object position in spaceThe measure is stable to object deformations, independent of the object position and the postureThe computational cost of the function evaluation is O(n2log n) due to the Dijkstra’salgorithm, approximations are providedGood computational cost of the graph extraction: O(n+k) (where k is the number of added vertices)

ResultsResults

Experiments performed over a test set of 230 models

31

AugmentedAugmented MultiresolutionMultiresolution Reeb Reeb graphgraph

Attributes of the nodes are enriched with geometric measures related to the spatial extent of the region– Relative volume– Statistic measure of the chords– Koenderink shape index– Statistic orientation of the triangle normals– Statistic on the texture (when available)

Graph matching: an additional rule– Two nodes are matched if the parents of their

neighbours have been matched at the previous level

ResultsResults on the SHREC track on on the SHREC track on watertightwatertight modelsmodels

More results are available athttp://watertight.ge.imati.cnr.it/

32

EExtendedxtended Reeb GraphReeb Graph

Founds on an extended Reeb equivalence– let f:M→R be a real valued function;– let I={(fmin, f1),(fh,fmax),(fi,fi+1),i=1…h-1} ∪ {fmin,f1,…fh,

fmax} be a partition of [fmin, fmax];• an extended Reeb equivalence between P, Q∈M is

given by:• f(P), f(Q) belong to the same element of I;• P, Q belong to the same

connected component of f -1(f(t)), t∈I.

f

fmax

fmin

fh

f1

fi…

…

Computationa topology techniques for shapemodelling applications, S. Biasotti, PhD thesis, 2004

ExtendedExtended Reeb Reeb graphgraph propertiesproperties

It preserves the topology of the manifoldThe associated graph is directed and acyclicIt can be extended to manifolds with boundariesNodes of the graph correspond to surface segments that– may describe a single region or– summarize the node sub-graph

33

Geometric embedding of the ERG (Geometric embedding of the ERG (BiasottiBiasotti et al 2006)et al 2006)

Each arc can be oriented using the growing direction of the mapping function: the ERG is a direct acyclic graph

Store with each ERG node n a representation of the sub-graph associated to n (e.g., using spherical harmonics, Kazhdan et al. 2003)

For each ERG arc e, compute the number of slices traversed by the arc (arc length)

Sub-part correspondence by structural descriptors of 3D shapes, S. Biasotti et al., CAD, 38(9):1002-1019, 2006

ERG matching strategy ERG matching strategy

Two ERGs are compared using a graph-matching approach based on the “best common subgraph” detectionAlso sub-part correspondences are recognizedHeuristics are used to improve– Quality of the results– Computational time

34

Extended Reeb graphsExtended Reeb graphs

Given G1 and G2, two direct, acyclic and attributed graphs:– the distance d between two nodes v1∈G1 and

v2∈G2 is

– denoting S a common subgraph of G1 and G2, the distance D(G1,G2) is:

3321

21SSS Szw+Stw+Gw=)v,d(v

][wi 0,1∈

∑ 1=wi

RetrievalRetrieval performance performance usingusing ERGERG

Experiments on a set of 280 models(Reeb graphs for shape analysis and applications, Biasotti et al., to appear in Theoretical

Computer Science)

35

Partial correspondence using ERGPartial correspondence using ERG

Models with similar appearance

Partial correspondence using ERGPartial correspondence using ERG

Objects with dissimilar global appearance

36

PartialPartial matchingmatching performanceperformance

Results of the track of SHREC on partial matching: http://partial.ge.imati.cnr.it

MethodsMethods basedbased on on structuralstructural descriptionsdescriptions

Structural decompositions are a means to represent the most relevant sub-parts of an object in a given contextMatching techniques are based on approximations of the sub-graph isomorphism problemShape matching methods based on structural descriptors generally are:– Suitable for articulated objects– Extensible to partial matching

37





IntrinsicIntrinsic and and embeddingembedding--basedbased techniquestechniques

S. S. BiasottiBiasotti, , D.D. Giorgi, S. Marini, Giorgi, S. Marini, F.F. RobbianoRobbiano and and M. SpagnuoloM. Spagnuolo

CNR-IMATI-GE - Italy

38

OutlineOutline

Method relying on intrinsic shape information and embedding techniques: • Shape DNA [Reuter et al. 2006]• Bending Invariant Surface Signatures [Elad and

Kimmel 2003]• Spectral Embedding [Jain and Zhang 2007]

OutlineOutline

Method relying on intrinsic shape information and embedding techniques: • Shape DNA [Reuter, Wolter, Peinecke: Laplace-

Beltrami spectra as ‘Shape-DNA’ of surfaces and solids. CAD 38 (2006)]

IDEA: computing a descriptor from the intrinsicgeometry of a shape, namely from the Riemannian

structure of the manifold representing the object

• Bending Invariant Surface Signatures [Elad and Kimmel 2003]

• Spectral Embedding [Jain and Zhang 2007]

39

ShapeShape DNA [DNA [ReuterReuter etet al. 2006]al. 2006]

The shape DNA is the beginning of the spectrum of the Laplace – Beltrami operator, defined for realvalued functions on Riemannian manifolds:

Given a Riemannian n-manifold M and f:M ℜ the Laplace – Beltrami operator is

(different from the discrete Laplacian on graphs)

Shape DNA =

with eigenvalues of the Helmholtz equation

)(: fgraddivf =Δ

{ } 010 ... ≥ℜ∈≤≤≤ mmλλλ

ff λ−=Δiλ

ShapeShape DNA DNA [[ReuterReuter etet al. 2006]al. 2006]

Courtesy of Martin Reuter

40

ShapeShape DNA: DNA: MatchingMatching [[ReuterReuter etet al. 2006]al. 2006]

Shape DNA signatures are m-dimensionalfeature vectors, that can be comparedusing e.g. the Euclidean p-norm:

as well as the Hausdorff distance, the Pearson correlation distance…According to empirical evidence, d2 yieldsgood results while being easy to compute

( )pm

i

piip vuvud

1

1

, ⎟⎠

⎞⎜⎝

⎛−= ∑

=

ShapeShape DNA: DNA: MatchingMatching [[ReuterReuter etet al. 2006]al. 2006]

Matching results on a small database of meshes, includingdifferent classes of deformed models, show a nice clusteringof objects

Other experiments on collections of grey-scale and colourimages [RWP07]Recent research: medical applications on brain surfaces, using statistical methods to distinguish populations; extentionto 3D brain data

41

OutlineOutline

Method relying on intrinsic shape information: • Shape DNA [Reuter et al. 2006]

Methods based on shape embedding:• Bending Invariant Surface Signatures [Elad and

Kimmel: On bending invariant signatures for surfaces. IEEE Trans. PAMI 25(10), 2003]

• Spectral Embedding [Jain and Zhang: A spectralapproach to shape-based retrieval of articulated 3D models, CAD 39(5), 2007]IDEA: providing an embedding of surfaces in a

small Euclidean space, to build a descriptor thatguarantees invariance to bending and is suitable to

deal with articulated objects

BendingBending InvariantInvariant SurfaceSurface SignaturesSignatures [[EladElad and and KimmelKimmel 03]03]

Geodesic distances between surface points are invariant to surface bendingIdea: use geodesic distances to define anisometrical embedding of a surface in a smalldimensional Euclidean space, in whichgeodesic distances are approximated byEuclidean onesMethod: apply a MultiDimensional Scaling(MDS) procedure on a geodesic distancematrix, with geodesics computed via the Fast Marching on Triangulated Domains (FMTD) algorithm

42


Sample with n vertices a given triangulated surface, via iterative Voronoi sampling, and build an n x ndissimilarity matrix D

with the geodesic distance between verticescomputed following the FMTD algorithm

Define a dimension m for the Euclidean embeddingspace and apply MDS on the matrix D, yelding ann x m matrix whose rows define the coordinates in of the points of the signature surface

( )2ijijD δ=ijδ ji,

mℜ


These two steps define a bending invariant descriptor, that allows to translate the problem of matching non-rigid objects in various posture into a simpler problem of matching rigid objects

43


Drawback: embedding in the Euclidean space mayintroduce metric distortions

Extension to non-Euclidean embeddings (such asembedding on the sphere [Bronstein et al. 2005]) and introduction of Generalized MDS [Bronstein etal. 07]

In [Bronstein et al. 2006] partial surface matching isalso addressed, introducing the Partial Embeddingdistance

BendingBending InvariantInvariant SurfaceSurface SignaturesSignatures: : MatchingMatching [[EladElad and and KimmelKimmel 03]03]

Given the surface signatures, any algorithm toevaluate the similarity of rigid objects can beinvolved in the comparison stepExample: Compute the vectors of the first few moments of the surfaces and compute theirEuclidean distanceClustering results on classes of a small number of deformed objects confirm that the faster the MDS method, the worse its performance in terms of separation between classesApplication to face recognition

44

SpectralSpectral EmbeddingEmbedding [[JainJain and and ZhangZhang 2007]2007]

Ideas similar to [Elad and Kimmel 2003] are developed, introducing a descriptor suitable tocompare articulated objectsThe matrix is an affinity matrix involving a Gaussian of width

with geodesic distances approximatedthrough an heuristicThe embedding in is given by the first eigenvectors of the matrix, computed via Nyström approximation

σD

σδ 2

,

ij

eD ji

−=

mℜ m

SpectralSpectral EmbeddingEmbedding [[JainJain and and ZhangZhang 2007]2007]

The descriptor is given by the embeddedsurface or by the matrix first eigenvalues

45

SpectralSpectral EmbeddingEmbedding: : MatchingMatching [[JainJain and and ZhangZhang2007]2007]

Compare shapes by computing existing shapedescriptors (Light Field, Spherical Harmonics) on spectral embeddingsUse the vectors of normalized eigenvalues and define

Compute a correspondence cost derived from the correspondence between the vertices of the twoshapes (possibly after a first filter using EVD)

∑= +

⎥⎦

⎤⎢⎣

⎡−

=m

i Si

Qi

Si

Qi

EVD SQD1 2

121

2

21

21

21),(

λλ

λλ

( ) ( ) ( )( )∑∈

−=Qp

SQCCD pmatchVpVSQD ,

SpectralSpectral EmbeddingEmbedding: : MatchingMatching [[JainJain and and ZhangZhang2007]2007]

PrecisionPrecision--Recall plot for McGill databaseRecall plot for McGill database

46





TechniquesTechniques basedbased on on algebraicalgebraic--topologytopology toolstools

S. Biasotti, D. Giorgi, S. Marini, F. Robbiano and M. Spagnuolo

CNR-IMATI-GE - Italy

47

OutlineOutline

Techniques based on algebraic-topologytools• Size theory and size functions [Frosini 1991, d’Amico

et al. 2006, Biasotti et al. 2007]• Persistent homology, barcodes and intervals

[Edelsbrunner et al 2002, Carlsson et al. 2005, Cohen-Steiner et al. 2007 ]

OutlineOutline

Techniques based on algebraic-topologytools• Size theory and size functions [Frosini and Landi:

Size Theory as a topological tool for computer vision. Pattern Recognition and Image Analysis 9, 1999]

IDEA: Shapes are toplogical spaces endowed withreal functions describing their properties; preserving shapes means preserving such

properties

• Persistent homology, barcodes and intervals[Edelsbrunner et al 2002, Carlsson et al. 2005, Cohen-Steiner et al. 2007 ]

48

SizeSize TheoryTheory and and SizeSize FunctionsFunctions [Frosini 1991][Frosini 1991]

Size Theory proposes an approach wherecomparing shapes means comparing propertiesexpressed by real functions; if two shapes are similar, a homeomorphism between the shapespreserving the function values must exist

How can we measure how well a homeomorphismcan preserve the values taken by the consideredfunction?

In Size Theory preserving shapes means preservingthe natural pseudo-distance


The size function of the size pair , withis the function

that takes each (x,y) to the number of components of the lower level set , that contain at least a point of

( )fM ,

( ) ( ){ } N→<ℜ∈ yxyxfM :,: 2,l

yMxM

ℜ→Mf :

49



Size functions can be represented as countablecollections of points and lines of the plane withmultiplicities (named cornerpoints and cornerlines); eachsize function is completely determined by its formal series

r+a+b+c

50

MultidimensionalMultidimensional SizeSize FunctionsFunctions [[BiasottiBiasotti etet al. al. 2007]2007]

Recent extension to the multidimensional case, where the measuring function

It has been demonstrated that there exists a foliation in half-planes of s.t. on each leafof the foliation the multidimensional size functioncoincides with a particular 1-dimensional sizefunctionA multidimensional matching distance can bedefined, based on the 1D matching distance on each leaf of the foliation, which is stable w.r.t. smallchanges of the measuring functions and provides a lower bound for the natural pseudo-distance

kMf ℜ→: ),...,,( 21 kffff =

kk ℜ×ℜ

MultidimensionalMultidimensional SizeSize FunctionsFunctions [[BiasottiBiasotti etet al. al. 2007]2007]

51

OutlineOutline

Techniques based on algebraic-topologytools• Size theory and size functions [Frosini 1991, d’Amico

et al. 2006, Biasotti et al. 2007]

• Persistent homology, barcodes and intervals[Carlsson, Zomorodian, Collins, Guibas: Persistencebarcodes for shapes. International Journal of ShapeModeling 11, 2005]

IDEA: Furnishing a scale to assess the relevance of topological events occurring in a growing space

PersistentPersistent HomologyHomology [Edelsbrunner [Edelsbrunner etet al. 2002]al. 2002]

The idea of Persistent Homology is to control the placement of topological events in a growing space and assess their relevance according to their life-time

Given a growing complex K, represented by a filtration

the j-persistent k-th homology group of Ki is a groupisomorphic to the image of the homomorphism

induced by the inclusion of Ki into Ki+j

Persistence represents the life-time of cycles in the growing filtration

{ } of subcomplex,, 1,...,

+= = iin

nii KKKKK

( ) ( )jik

ik

kij KHKH +→:η

52

PersistentPersistent HomologyHomology [Edelsbrunner [Edelsbrunner etet al. 2002]al. 2002]

The persistent homology of a growing complex can berepresented by a set of intervals, called persistence intervals: a

P – interval is a pair

such that there exists a cycle that is completed at the level i of the filtration and remains non-bounding until the level j

jijiji <≤+∞∪Ζ∈ 0,,),,(

PersistencePersistence HomologyHomology and and BarcodesBarcodes [Carlsson [Carlsson etet al. 2005]al. 2005]

The shape of a complex K can be described byfiltering the complex by the increasing values of a

real function

Idea: construct a new complex strictly related to K, namely the tangent complex T(K) (closure of the space of all tangents to all points in K), and filter it

with the function computing the curvature at a point along a tangent direction

The barcode of the shape is the set of P – intervalsfor the filtered tangent complex

53

BarcodesBarcodes: : MatchingMatching [Carlsson [Carlsson etet al. 2005]al. 2005]

Barcode pseudo-metric:

JIJIJIJI IU −=),(barcode, ain intervals, δset theis , barcodesbetween matchingA 21 SS

( ){ }212121 ,..,),( SJSItsJISSSSM ∈∈=×⊆),(pair onemost at in occurs ,in intervalany s.t. 21 JISS

torelative ,between Distance 21 MSS( ) ( )∑ ∑

∈ ∈

+=MJI NL

M LJISSD),(

21 ,, δ

intervals matchednon ofset thewith N

( ) ( )2121 ,min, SSDSSD MM=

BarcodesBarcodes: : MatchingMatching [Carlsson [Carlsson etet al. 2005]al. 2005]

Examples on mathematical surfacesClassification results on a set of 80 hand-drawn copies of letters

54

ProteinProtein DockingDocking via via PersistencePersistence [[WangWang etet al. al. 2004]2004]

Rigid protein dockinganalyzing shapecomplementarityDescribe protrusions and cavities on molecularsurfaces using a succintset of point pairscomputed from the elevation functionAlign such pairs and evaluate the resultingconfigurations using a simple and rapid scoringfunction


QuestionsQuestions??

Documents

Geometry-based Shape Matching - Utrecht University...Geometry-based Shape Matching S. Biasotti, D. Giorgi, S. Marini, F. Robbiano and M. Spagnuolo CNR-IMATI-GE - Italy 22/06/2007 2