Partial Shape Matching

Partial Shape MatchingPartial Shape Matching

Hu JianweiHu Jianwei2006-10-112006-10-11

Shape RetrievalShape Retrieval

3D Query

3D Database

Best Match

Global Shape MatchingGlobal Shape Matching

Skeleton Based SimilaritySkeleton Based Similarity Reeb Graph Based SimilarityReeb Graph Based Similarity

Shape HistogramsShape Histograms Light Field Descriptor Light Field Descriptor Extended Gaussian Image Extended Gaussian Image …………

MergingMerging

Aligned ScansPartially Overlapping Scans

The same class with different overall shapesThe same class with different overall shapes


Partial matching is a much harder problem than global matching, since it needs to search for and define the subparts prior to measuring similarities.

Exhaustive Search?


SegmentationSegmentation

Salient Geometric Features for Partial Salient Geometric Features for Partial Shape Matching and SimilarityShape Matching and Similarity

ACM Transactions on Graphics, Vol. 25, No. 1, January 2006, Pages 130–ACM Transactions on Graphics, Vol. 25, No. 1, January 2006, Pages 130–150.150.

Ran Gal & Daniel Cohen-OrRan Gal & Daniel Cohen-Or

Tel-Aviv UniversityTel-Aviv University

OutlineOutline

Local Surface DescriptorsLocal Surface Descriptors Salient Geometric FeaturesSalient Geometric Features Indexing and Geometric HashingIndexing and Geometric Hashing

Quadric Fitting and Curvature EstimationQuadric Fitting and Curvature Estimation

Quadric FittingQuadric FittingDouros and Buxton [2002]Douros and Buxton [2002]

Curvature EstimationCurvature EstimationOhtake et al [2004]Ohtake et al [2004]

Compact RepresentationCompact Representation

Defining the Local PatchesDefining the Local Patches Error Measuring: Squared Algebraic Error Measuring: Squared Algebraic

DistancesDistances Threshold: of the model bounding box Threshold: of the model bounding box

diagonal lengthdiagonal length

410

Local Surface DescriptorsLocal Surface Descriptors

A Patch (Points)A Patch (Points) A PointA Point A CurvatureA Curvature

Gaussian Curvature vs Max CurvatureGaussian Curvature vs Max Curvature

Sample the Surface RandomlySample the Surface Randomly

Osada et al [2001]Osada et al [2001] Elad et al [2001]Elad et al [2001]

OutlineOutline


Salient Geometric FeaturesSalient Geometric Features

A set of descriptors that have:A set of descriptors that have: A high curvature relative to their surroundingsA high curvature relative to their surroundings A high variance of curvature valuesA high variance of curvature values

A function of one free parameter: the scaleA function of one free parameter: the scale

Saliency GradeSaliency Grade

31 2( ) ( ) ( ) ( )S Area d Curv d N F Var F The area of the patch associated with d relative to the sphere sizeThe curvature associated with dThe number of local minimums or

maximums curvatures in the cluster

The curvature variance in the cluster


31 2( ) ( ) ( ) ( )S Area d Curv d N F Var F

The saliency of the regionThe degree of interestingness of the cluster



OutlineOutline


Indexing and Geometric HashingIndexing and Geometric Hashing

Each salient feature is associated with a Each salient feature is associated with a vector indexvector index

The vector index is defined by the four The vector index is defined by the four terms: terms:

Geometric hash table contains the vector Geometric hash table contains the vector indicesindices

( ), ( ), ( ), ( )Area d Curv d N F Var F

Best TransformationBest Transformation

Compute the transformationCompute the transformation Triplet of pointsTriplet of points Voting system Voting system [Lamdan and Wolfson 1988][Lamdan and Wolfson 1988]

Time and StorageTime and Storage

Self-SimilaritySelf-Similarity

Shape AlignmentShape Alignment

Partial Shape RetrievalPartial Shape Retrieval

Retrieval ResultRetrieval Result

Spin Image vs SGFsSpin Image vs SGFs

Registration of Synthetic ScansRegistration of Synthetic Scans

ReconstructionReconstruction

Local Feature Extraction and Local Feature Extraction and Matching Partial ObjectsMatching Partial Objects

Computer-Aided Design 38 (2006) 1020–1037Computer-Aided Design 38 (2006) 1020–1037

Dmitriy Bespalov, William C. Regli & Ali ShokoufandehDmitriy Bespalov, William C. Regli & Ali Shokoufandeh

Drexel UniversityDrexel University

Traditional CAD RepresentationTraditional CAD Representation

Watertight boundary-Watertight boundary-representation solidrepresentation solid Implicit surfacesImplicit surfaces Analytic surfacesAnalytic surfaces NURBS, etcNURBS, etc

Topologically and Topologically and geometrically consistentgeometrically consistent

Produced by kernel Produced by kernel modelers and CAD modelers and CAD systemssystems

Traditional CAD RepresentationTraditional CAD Representation

Usually a mesh or point Usually a mesh or point cloudcloud

Usually an approximate Usually an approximate representationrepresentation

Sometimes error proneSometimes error prone Produced by CAD Produced by CAD

systems, animation tools, systems, animation tools, laser scanners, etclaser scanners, etc

What is a Scale-Space What is a Scale-Space Representation?Representation?

Commonly used for Coarse-to-Commonly used for Coarse-to-Fine representations of an objectFine representations of an object

Very popular in Computer VisionVery popular in Computer Vision Basic Idea:Basic Idea:

At each At each scalescale, topologically relevant , topologically relevant components will components will decomposedecompose the the object into so called object into so called salient partssalient parts

RecursiveRecursive application of this application of this paradigm will create the object’s paradigm will create the object’s scale space hierarchyscale space hierarchy

Why Scale-Space Representation?Why Scale-Space Representation?

A unified framework for matchingA unified framework for matching Different features can be parameterized as Different features can be parameterized as

different scale-space decompositionsdifferent scale-space decompositions Robust & consistent across noisy and diverse Robust & consistent across noisy and diverse

data setsdata sets

MethodMethod

Start with CAD modelStart with CAD model Perform geometry-based decompositionPerform geometry-based decomposition Construct hierarchical “feature” graphConstruct hierarchical “feature” graph Use hierarchical matching to compare graphsUse hierarchical matching to compare graphs

Algorithm Overview (I)Algorithm Overview (I)

1. Given model P, compute mesh representation M

2. Define measurement function: Our d is the maximum angle on an angular shortest path distance function between every two faces on M will be captured in a pair-wise distance matrix D.

d(t1,t2)

Algorithm Overview (II)Algorithm Overview (II)

3. 3. DecomposeDecompose MM into components relevant using a into components relevant using a sisingular value decompositionngular value decomposition of distance matrix of distance matrix DD

Compute the SVD decompositionCompute the SVD decomposition

with with

Compute the order-k compression matrixCompute the order-k compression matrix

Let denote the jLet denote the jthth column of , column of ,

Form sub-feature as the union of facesForm sub-feature as the union of faces

withwith

TD U V

1 2( , , , )nDiag

( )1( , , ,0, ,0)k T

kD UDiag V

jC ( )kD

1{ , , }kS c c

jM

it M ( , ) ( , )i i jd t S d t c

Algorithm Overview (III)Algorithm Overview (III)

4. Recursive 4. Recursive feature decompositionfeature decomposition using two using two principle components creates binary principle components creates binary feature feature treestrees

feature tree for swivelfeature tree for simple_bracket

Algorithm Overview (IV)Algorithm Overview (IV)

5. Compare feature trees (bottom up dynamic programming) using The Largest Common Subgraph Algorithm [Ullmann JR. 1976]

swivelsimple_bracket

When to Stop?When to Stop?

2 3,i jt M t M m l i jt t t t

. .s t

2 3 ( , ) ( , )m l m l i jt M t M t t D t t

The feature is decomposed into sub-features and if the angular distance between components of and is large.

1M

2M 3M

2M3M

Example DecompositionExample Decomposition

Feature Decomposition on Noisy DataFeature Decomposition on Noisy Data


Retrieved Relevant models

Relevant models

andrecall

Retrieved Relevant models

Retrieved models

andprecision

A precision–recall graph for retrieval experiment using: 1. A Reeb Graph technique; 2. a Scale–Space technique with the max-angle distance function and simple sub-graph isomorphism for matching; 3. the original Scale–Space technique with a geodesic distance function; 4. a random retrieval technique.

Fidelity ExperimentsFidelity Experiments

Retrieval experimentRetrieval experiment

Partial Shape Matching of 3D Partial Shape Matching of 3D Shapes with Priority-Driven SearchShapes with Priority-Driven Search

Eurographics Symposium on Geometry Processing (2006)Eurographics Symposium on Geometry Processing (2006)

T.Funkhouser & P.ShilaneT.Funkhouser & P.Shilane

Princeton UniversityPrinceton University

System ExecutionSystem Execution

Preprocessing phasePreprocessing phase Constructing RegionsConstructing Regions Computing Shape DescriptorsComputing Shape Descriptors Selecting Distinctive FeaturesSelecting Distinctive Features

Query phaseQuery phase Creating Pairwise Feature CorrespondencesCreating Pairwise Feature Correspondences Searching for the Optimal Multi-Feature MatchSearching for the Optimal Multi-Feature Match

Constructing RegionsConstructing Regions

Computing Shape DescriptorsComputing Shape Descriptors

Generate and store a representation of the

shape for each spherical region Spin image Spin image [Johnson A. & Hebert M. 1999][Johnson A. & Hebert M. 1999]

Extended Gaussian Image Extended Gaussian Image [Horn, 1984][Horn, 1984]

Spherical Harmonics Spherical Harmonics [Frome A. et al. 2004][Frome A. et al. 2004]

…………

Selecting Distinctive FeaturesSelecting Distinctive Features

Compute the difference between every two Compute the difference between every two descriptors in the data basedescriptors in the data base

Build a rank-to-difference mapping (RTD)Build a rank-to-difference mapping (RTD) Distinctive Feature: both similar to features in Distinctive Feature: both similar to features in

few other objects (ones in the same class) and few other objects (ones in the same class) and different from the rest (objects in other classes)different from the rest (objects in other classes)

2L

Discounted Cumulative GainDiscounted Cumulative Gain

1

12

, 1

,log ( )

i ii

G i

DCG GDCG otherwise

i

22

11

log ( )

M

C

j

DCGDCG

j

Selecting Distinctive FeaturesSelecting Distinctive Features

The selection algorithm iteratively chooses The selection algorithm iteratively chooses the feature with highest DCG whose positithe feature with highest DCG whose position is not closer than a Euclidean distance on is not closer than a Euclidean distance threshold to the position of any previously threshold to the position of any previously selected featureselected feature

Result of Result of Preprocessing PhasePreprocessing Phase

A small set of featuresA small set of features Each feature associated with Each feature associated with

A positionA position A normalA normal A radiusA radius A shape descriptorA shape descriptor A rank-to-difference mapping (RTD)A rank-to-difference mapping (RTD) A retrieval performance score (DCG)A retrieval performance score (DCG)

The Cost FunctionThe Cost Function

1 2( , ) ( , )rankC Rank RTD D Rank RTD D

1 2

1 2radius

R RC

RAVG RAVG

1 1 2 2normalC r n r n ��

rank radius normalcorrespondence rank rank radius radius normal normalC C C C

The Cost FunctionThe Cost Function1 2

1 2

1 2

1 2

max( , )length

L LLAVG LAVG

CL L

LAVG LAVG

1 1 2 2 1 1 2 2 1 1 2 2orient a b a b a a b bC n n n n v n v n v n v n ��

length orientchord length length orient orientC C C

, ,

( ) ( , )match correspondence chordi k i j k i j

C C i C i j

Query PhaseQuery Phase

Comparison to Previous MethodsComparison to Previous Methods

Documents

Partial Shape Matching