TEMPLATE BASED SHAPE DESCRIPTORRaif RustamovDepartment of Mathematics and Computer ScienceDrew University, Madison, NJ, USA

Components of descriptors in general

Selection of surface feature Mapping Signal Processing

Need this discussion to set up the context for our approach

Selection of surface feature

A function on the surface that captures a property relevant to shape description: constant function (restriction of the

surface's characteristic function to the surface itself)

distance to the center of mass curvature components of the normal vector

We refer to the selected function as the feature function.

Mapping

The feature function is used to construct a new function defined on some predetermined domain The new domain called the “mapping domain” the new function the “mapped feature function”

Common mapping domains: Spheres Planes the 3D space (surface's bounding volume) surface itself

Mapping procedures: projection Identity, if mapping domain = surface itself

Signal Processing

Extract concise noise-robust numerical descriptor from the mapped feature function.

Depends on the mapping domain: Sphere – Spherical Harmonic Transform Plane or box volume – 2D or 3D Fourier

transform Ball volume – 3D Zernike Transform.

Mapped feature function is expanded in a series in terms of the relevant basis

Expansion coefficients are used as the shape descriptor

Example I: Saupe, Vranic 2001

Shoot rays from the origin (center of mass), determine the distance to the farthest intersection point with the bounding mesh

Parameterize the rays by the unit sphere to obtain a function on the sphere

Use spherical harmonic transform on this function to extract the numerical shape descriptor.

Example I: Saupe, Vranic 2001

Surface feature Distance to the origin

Mapping Mapping domain: the unit sphere Mapping procedure: project onto the

sphere, resolve collisions by selecting the larger function value

Signal processing Spherical harmonic transform

Example II: Depth Buffer

Heczko, Keim, Saupe, Vranic 2002 Place a normalized mesh into a unit cube Generate six gray-scale images on each

face of the cube by parallel projection The grayness value is the distance from the

cube face to the model Apply 2D Fourier transform to each of

the six gray-scale images

Example II: Depth Buffer

For each cube face: Surface feature

distance from mesh point to the face Mapping

Mapping domain: cube face Mapping procedure: project onto the face,

resolve collisions by selecting the smaller function value

Signal processing 2D Fourier transform

Generality

More examples easily generated Compare to classification in Bustos et al.

survey Mapping ≈ object abstraction Signal processing ≈ numerical

transformation

Observations

Mapping: Mapping domain:

≠ original surface a primitive geometry: sphere, plane etc

Mapping procedure: Projection

Signal processing well established: Fourier, Zernike, Spherical

Harmonics limits possible mapping domains

Contributions

Mapping: Mapping domain:

any fixed surface – template Mapping procedure:

interpolation: mean-value coordinates, Shepard Signal processing

via manifold harmonics – eigenfunctions of Laplace-Beltrami operator

Why templates?

Mademlis, Daras, Tzovaras, Strintzis 2008: Since ellipses approximate elongated

shapes better than spheres: Mapping domain: ellipsoid Signal processing: ellipsoidal harmonics

Showed experimentally better retrieval results than sphere + spherical harmonics

We take this idea further: Mapping domain: any fixed surface

Why template ≠surface itself? Expand the feature function in terms of

the manifold harmonics of the original surface?

Problem: notoriously difficult to match the harmonics coming from different surfaces Sign flipping Eigenfunction switching Linear combinations

Fixed template: extracted expansion coefficients are in direct correspondence

Why interpolation?

Projection Mapped feature function can be discontinuous at

overlaps Gibbs effect may render low-frequency expansion

coefficients used as the shape descriptor inadequate for representing the function

Feature function is distance to the originJump discontinuity

Why interpolation?

Projection Redundancy

the value sets of the mapped feature functions on various templates will be almost the same

limits the gains of concatenating descriptors obtained from different templates

Why interpolation?

Interpolation No Gibbs effect

mapped feature function is smooth Less redundancy

the value sets of the mapped feature functions on various templates depend on relative positions

mean-value coordinates can inject more shape information into the mapped feature function

a mesh can be reconstructed given the mean-value coordinates

Construction of the descriptor

Selection of surface feature Mapping Signal Processing

Now discuss details

Selection of surface feature

All models are normalized using shift, continuous PCA, isotropic scaling

Many possibilities, but not: the characteristic function nor linear function of coordinates To focus discussion f = distance from a mesh point to the origin Similar to Saupe, Vranic 2001

Mapping

Model surface S, Template surface T Given Construct Shepard interpolation Mean-value interpolation

Mapping: Shepard

Model surface S, Template surface T Given Construct

0th order precision: constant functions reproduced

Mapping: Barycentric

Model surface S, Template surface T Given Construct

are barycentric coordinates of point p with respect to vertex

1st order precision: linear functions reproduced

Mapping: Barycentric

A few different kinds of barycentric coordinates Mean-value, positive mean-value Harmonic Maximum Entropy Green coordinates, Complex in 2D

We use mean-value coordinates Closed formula Fastest to evaluate

Signal processing

We have a function Need a compact representation

Expand the function into series Use low-frequency coefficients

Need a function basis on template surface T

Manifold harmonics = Laplace-Beltrami eigenfunctions

Signal Processing

Manifold harmonics generalize Fourier basis to Riemannian manifolds

Spherical harmonics = manifold harmonics on the sphere

Have similar properties Orthogonal Concept of frequency

low-frequency coefficients are noise-robust convey essential information about function

Signal Processing

Egenvalues, eigenfunctions solve Evaluation procedures well known Solve symmetric eigenvalue problem for a matrix We use cotangent Laplacian with voronoi point-

areas Pre-compute for the given templates and store Our templates have about 500 vertices, the

process takes less than 3 seconds The storage for each template = 10,500 floats=

=20*500+500 = #eigs * #vertices + #vertices

to store eigenvectors to store point areas

Resulting shape descriptor

Feature function Mapped feature function The template’s feature function Quotient function Expand into series

Resulting shape descriptor

Feature vector, N=20

For template surface T,

Normalization: scale to get a = 1 Use L2 distance

Experiments: Benchmark

Models: watertight benchmark 400 closed surface models 20 equal object classes

Experiments: Implementation

Implemented in MATLAB Use C++ for mean-value coordinate

computation Timing

About 1 minute per model when mean-value coordinates used

Could make faster if simplified the models

Experiments: Templates

Templates: randomly chosen models Simplified using Qslim Makes mean-value computation faster

Compare Mapping Methods

Template = sphere Projection vs. Shepard (a=1,2,3) vs.

Mean-value At long distance behavior of mean-value

interpolant is similar to that of Shepard with a=2

Compare templates

Mapping via mean-value interpolation

M

Compare templates

Beneficial to combine – relative independence

All templates are normalized as objects in the benchmark – span similar spatial regions

The descriptors could have been made even more independent if the templates were differently posed.

Future work

Investigate dependency between the nature of the template and the produced retrieval results

No “ideal" template for all kinds of shapes Flexibility of our approach – choose optimal

templates based on the shape database at hand

How to choose? Can we design a rotationally invariant

descriptor?