Bag-of-Feature-Graphs: A New Paradigm for Non-rigid Shape Retrieval

ICPR 2012Department of Computer Science Center for Visual Computing

Bag-of-Feature-Graphs: A New Paradigm for Non-rigid Shape Retrieval

Tingbo HOU, Xiaohua HOU, Ming ZHONG and Hong QIN

Department of Computer ScienceStony Brook University (SUNY SB)

Department of Computer Science Center for Visual Computing ICPR 2012Department of Computer Science Center for Visual Computing

Nonrigid Shape Retrieval

……

Shape Query

Shape Database

Retrieved Shapes


Overview of BoFG Inspired by the ideas from Bag-of-Words (BoW) and Spatial-

Sensitive Bag-of-Words (SS-BoW)

Feature-driven

Concise and fast to compute

Spatially informative


Previous Works Relevant to This Project

Bag-of-Words1. Y. Liu, H. Zha, and H. Qin. CVPR, 2006.2. H. Tabia, M. Daoudi, J. P. Vandeborre, and O. Colot. 3DOR, 2010.3. R. Toldo, U. Castellani, and A. Fusiello. VC, 2010. 4. G. Lavoué. 3DOR, 2011.

Shape Google (Spatially-Sensitive Bag-of-Words)1. M. Ovsjanikov, A. M. Bronstein, L. J. Guibas and M. M. Bronstein. NORDIA,

2009.2. (SI-HKS) M. M. Bronstein and I. Kokkinos. CVPR, 2010.3. A. M. Bronstein, M. M. Bronstein, L. J. Guibas, and M. Ovsjanikov. ACM

TOG, 2011.


Background (1) Heat Kernel on surface

Amount of heat transferred from a point to in time

: -th eigenvalue and eigenfunction of the Laplace-Beltrami operator Heat Kernel Signature (HKS):

HKS descriptor A vector of HKS probed at different values of

Properties of Heat Kernel Intrinsic (Invariant to rigid and isometric deformation) Informative (locally and globally shape aware) Stable


Background (2) Geometric words

A representative HKS vector Clustered in the HKS descriptor space by the k-means algorithm

Vocabulary

Similarity of point and word


Shape-Google Revisit (1)

Bag-of-Words Word distribution of each point

BoW descriptor: vector

Measure the frequencies of words appearing on the shape


Shape-Google Revisit (2)

Spatially-Sensitive Bag-of-Words SS-BOW descriptor: matrix

Measure the frequencies of word pairs


New Paradigm: Bag-of-Feature Graphs (1)

Motivation: Reduce computation complexity

Considering all points on shape -> only considering feature points

Vector/matrix of word frequencies -> feature graphs associated with words


Formulation (1) Feature set:

Feature graph associated with the -th geometric word represented as matrix

: Heat Kernel

Bag-of-Feature-Graphs representation of shape

…𝐺1 𝐺2 𝐺3


Formulation (2) BoFG descriptor

Multi-dimensional scaling (MDS): Choosing the 6 largest eigenvalues of each graph matrix denoted by

vector

Shape distance

Retrieval by approximate nearest neighbor (ANN) search


Nonrigid Shapes and Their BoFG Descriptors


Time Complexity of BoW, SS-BOW and BoFG

: Number of vertices

: Time complexity for computing HKS descriptor of a vertex


Experiments Test dataset: TOSCA1

12 classes of 148 non-rigid shapes Each shape has 3K 30K vertices

Evaluated methods: BoW, FSS-BoW, SI-HKS,

Vocabulary 48 words for BoW and SS-BoW (clustered from all shape points) 4 words for BoFG (clustered only from feature points)

Feature numbers in BoFG: for each shape

1http://toca.cs.technion.ac.il/book/shrec.html



Experiments

Time performance (in seconds) of three descriptors on two shapes with 3K and 30k vertices


Experiments

Precision-recall curves of evaluated methods, with categories of (1) null, (2) scale changes and (3) holes.

(1) (2) (3)


Partial shape retrieval Query shape is only a part of a complete model

Online feature alignment is required to extract corresponding sub-graphs


Summary Bag-of-Feature-Graphs (BoFG) is a new paradigm for shape

representation

This representation is feature-driven, concise, and spatially-aware

The key idea is to construct graphs of features associated with geometric words

BoFG has much improved time-performance and competitive retrieval results in comparison with other state-of-the-art methods


Future Work Investigate graph comparison with heavy outliers

Improve the performance on partial shape retrieval

Acknowledgements: Research Grants from National Science Foundation


Thank You!

Questions?

Documents

Bag-of-Feature-Graphs: A New Paradigm for Non-rigid Shape Retrieval