A Diffusion Wavelet Approach For 3 D Model Matching

A DIFFUSION WAVELET APPROACH FOR 3-D MODEL

MATCHINGAuthors: K.P. Zhu, Y.S. Wong, W.F. Lu, J.Y.H. Fuh

Presented by: Raphael Steinberg

2

SCHEDULE

IntroductionDiffusion MapsWavelets and Diffusion WaveletsFisher’s Discriminant Ratio (FDR)Retrieval ProcedureResultsConclusions

3

INTRODUCTION

Currently - A larger than ever number of 3D Models in CAD, computer games, multimedia, molecular biology, computer vision and more

There is a need for 3D Retrieval

4

INTRODUCTION (2)

Tagging are not always available or sufficient to describe the model we require

Combine topological information with multi-scale properties

5

Model Reusability (CAD/Animation)

Model MatchingVideo Retrieval (2.5D/Virtual

environments)EcommerceCorrecting defectsEfficient RepresentationMany other uses…

MOTIVATION FOR 3D RETRIEVAL

6

OBSTACLES IN RETRIEVAL

Partial retrieval - Non-transitive

Functional description

How to match text tags with vertices and texture?

Orthonormal coordinate system

7

8

3D MODEL MATCHING – PRIOR ART

Feature vectors using wavelets to mesh vertices – localized in both space & frequency – Paquet et. al. 2000

Random sampling for comparison – Osada et. al. 2001

9

SPHERICAL HARMONICS (SH)

Global method in Euclidean space

lacks multi-scale analysis

Legendre polynomials solve the Laplace equation in Spherical coordinatesVranic et. al. 2001

10

SPHERICAL WAVELETS (SW)

Multi-scale in Euclidean space

Lacks connectivity on the manifold

Tannenbaum et. al. 2007

11

SCHEDULE

IntroductionDiffusion MapsWavelets and Diffusion WaveletsFisher’s Discriminant Ratio (FDR)Retrieval ProcedureResultsConclusions

12

DIFFUSION MAPS INTRODUCTION

Originally suggested by Stephan Lafon and R.R. Coifman from Yale Math, circa 2005

Many other manifold learning techniques exist

Data analysis based on geometric properties of the data set

13

MANIFOLD LEARNING ALGORITHMS

MANI - Manifold learning Matlab tool

14

DIFFUSION MAPS

X

Y

Z

• vi is a feature vector

• Contains descriptive information about the 3D model

Kiv

,Div D K

Coifman - 2005

15

DIFFUSION MAPS

Assumptions• Points are sampled uniformly on

the manifold• Smooth manifold (no fractals in

our case)• Fixed boundary conditions• Enough points = feature vectors

(N→∞)

16

1) Use RBF Gaussian Kernel to choose ε

2) Normalize W to create a Stochastic Matrix

DIFFUSION MAPS ALGORITHM

2/2

, , e i jx x

i j i jw x x

1M D W

,1

N

i jj

D w

Lu et. al. 2009

17

DIFFUSION MAPS ALGORITHM (2)

3) Diffuse by taking higher powers of t “The diffusion distance is equal to the Euclidean distance in the diffusion map space” , Nadler et. al. 2005

4) Cut manifold according to dominant eigenvalues

18

DIFFUSION MAPS CODE EXAMPLEfunction checker();

close all;

tetha=2*pi*rand(1,500);

z=[cos(tetha);sin(tetha)];

figure(1);scatter(z(1,:),z(2,:),'b*');hold on;

N=size(z,2);

epsilon=linspace(0.01,.3,10);

%epsilon=.3;

W=nan(N);

summer=nan(1,length(epsilon));

for k=1:length(epsilon)

for i=1:N

parfor j=1:N

W(i,j)=exp(-sum((z(:,j)-z(:,i)).^2)/2/epsilon(k));

end

end

summer(k)=sum(sum(W));

end

figure;scatter(log(epsilon),log(summer));title('Epsilon - linear region')

p=polyfit(log(epsilon),log(summer),1);

d=2*p(1);%manifold dimension

M=W*diag(1./sum(W,2));

[U V]=svds(M);

sync=max(U(:,2));

figure(1);scatter(U(:,2)./sync,U(:,3)./sync,'rd')

title('Original manifold as stars and reconstructed manifold as diamonds')

end

19

SCHEDULE

IntroductionDiffusion MapsWavelets and Diffusion WaveletsFisher’s Discriminant Ratio (FDR)Retrieval ProcedureResultsConclusions

20

PROBLEMS WITH MESH SIMPLIFICATION

21

WAVELETS

22

NOVELTY – DIFFUSION WAVELETS

Combination of Diffusion Maps and Wavelets

Used for non-linear dimensionality reduction

Extension of wavelets to the unit circle (just as diffusion maps extends the Fourier transform)

23

DIFFUSION WAVELETS INTUITION

fine c o a rs e

o rigina l m e s h

glo b a l

lo c a l

glo b a l

lo c a l

24

EXAMPLE OF DIFFUSION WAVELETS

Wavelet basis

ψ(2,2,3)

Scaling basis φ(1,1,1)

Wavelet basis

ψ(4,2,5)

Waveletbasis

ψ(3,2,3)

25

DIFFUSION WAVELETS

Use an optimization scheme to construct the scaling functions

Each scaling function should deal with a single dimension and be orthogonal to the other scaling functions

Extension of wavelets to the sphere (or to any other manifold)

26

DIFFUSION WAVELETS (2)

Better than LOD (Level of Detail - simplifies meshes)

Involved algorithm – very few implementations exist

0 1 1j

1 2 1j

0M 1M 1jM jM

0G 1G 1jG jG2

1A 2 j

jA0A

... ...

...

27

WAVELET DECOMPOSITION EXAMPLE

f ine

c o ars e

s c ale j= 0

s c ale j= 1

s c ale j= 4

x 0,0

x 1,1 x 1,2

x 2,1 x 2,2

x 4,1 x 4,2

28

WAVELET COEFFICIENTS

0 500 1000 1500-0.1

0

0.1

0.2

0.3

0 500 1000 1500-0.05

0

0.05

0.1

0.15

0.2

0 500 1000 1500-0.4

-0.2

0

0.2

0.4

0 500 1000 1500-0.4

-0.2

0

0.2

0.4

Scale 3 Scale 4

Scale 1 Scale 2

29

SCHEDULE

IntroductionDiffusion MapsWavelets and Diffusion WaveletsFisher’s Discriminant Ratio (FDR)Retrieval ProcedureResultsConclusions

30

FINDING SHAPE FEATURE VECTORS (X)

, , , , , 0, 4j j j j jX j a c j

2 3 4 5 6 7 8 9 10

0.65

0.7

0.75

0.8

0.85

0.9

Number of decomposition level

Cla

ssifi

catio

n ra

te

• Take 1,450 coefficients (wavelet + scaling) at each decomposition level• Increase wavelet coefficient number from 0 to 450 and decrease scaling coefficient number from 1,450 to 1,000

31

FISHER’S DISCRIMINANT RATIO

b – between classesw – within class (after wavelet decomposition) j – scale

.

.b

w

Interclass distR

Intraclass distSS

-10 -5 0 50

0.2

0.4

0.6

0.8

1

1.2

1

2

Sw 1

Sw 2

Sb

32

FISHER’S DISCRIMINANT RATIO

'

4

, , , 4,3, ,0j

i jFDR X FDR X j

'2

1j j

w w i

j j

iw w

XFDR i

RX

C C

33

IRPR CURVE

Measure performance – use Princeton University 3D database

IRPR – Information Retrieval Precision-Recall

34

IRPR CURVE

m = relevant matches r = # of retrieved models

1) Precision =

2) Recall =

mr

1i

mn

i in Class size

1 2, , ,i

i i ii n

35

SCHEDULE

IntroductionDiffusion MapsWavelets and Diffusion WaveletsFisher’s Discriminant Ratio (FDR)Retrieval ProcedureResultsConclusions

36

3D MODEL RETRIEVAL PROCEDURE

Compute the diffusion wavelet for each 3D model

Obtain the model representing vector X

Compute the 2nd order statistics of X for each scale

37

1) Start with a coarsest scale comparison

2)Advance up to the finest scale

3) Stop on threshold or when finest scale reached

* Use a threshold to determine if a model is from a certain class

MODEL MATCHING PROCEDURE

38

SCHEDULE

IntroductionDiffusion MapsWavelets and Diffusion WaveletsFisher’s Discriminant Ratio (FDR)Retrieval ProcedureResultsConclusions

39

EXPERIMENTAL RESULTS

0 10 20 30 40 50 60 70 80 90 1000

10

20

30

40

50

60

70

Recall (%)

Pre

cis

ion(%

)

refine to scale 0

refine to scale 4

refine to scale 8

0 20 40 60 80 1000

20

40

60

80

100

Recall (%)

Pre

cisi

on(%

)

DW

SW

SH

DW gives better results than SH and SW

Differences in scaling levels

40

VISUAL RESULTS

41

SCHEDULE

IntroductionDiffusion MapsWavelets and Diffusion WaveletsFisher’s Discriminant Ratio (FDR)Retrieval ProcedureResultsConclusions

42

AUTHORS’ CONCLUSIONS

Surfaces with sharp peaks, grooves or holes contain high-frequency information which is not addressed by the wavelet multi-resolution (use diffusion wavelet packets instead?)

Possible to extend to partial matching

DW presents better results than SH and SW

43

MY CONCLUSIONS

Paper presents a novel solutionDiffusion Wavelets was never used before for 3D Retrieval

Less novel solutions:IRPR is a common measure in

database retrievalFischer Discriminant Ratio is a

common statistical measure

44

MY CONCLUSIONS (2)

Technically sound, feasibleTaking available code it seems

possible to reconstruct the resultsSeems like a reasonable solution to

the problem of 3D object retrieval

45

MY CONCLUSIONS (3)

The diffusion wavelet part could be explained in more detail

Not clear in which way the wavelets are constructed

How are the wavelet functions affected when a new model is inserted?

46

MY CONCLUSIONS (4)

Not self-containing but reference papers are exceptionally goodMissing explanations about

diffusion wavelets

47

“WOULD LIKE TO HAVE” (TECHNICAL/1)

Non-rigid extensionsHow would retrieval change if we

know the 3D model is non-rigid?Can we have an extension of

Diffusion Wavelets for non-rigid manifolds?

48

“WOULD LIKE TO HAVE” (TECHNICAL/2)

How to automatically choose the level of decomposition

49

“WOULD LIKE TO HAVE” (TECHNICAL/3)

An intuitive explanation - why prefer Diffusion Wavelets over Diffusion Wavelet Packets?

Wavelet Packets seem to give more information especially in high frequencies…

50

“WOULD LIKE TO HAVE” (TECHNICAL/4)

Numerical problems of overflow of the FDR - use logarithm instead of inverse?

51

“WOULD LIKE TO HAVE” (PRESENTATION/1)

Block diagram of the algorithm

52

“WOULD LIKE TO HAVE” (PRESENTATION/2)

Web-based Graphical User Interface

53

“WOULD LIKE TO HAVE” (PRESENTATION/3)

Error analysis

54

“WOULD LIKE TO HAVE” (PRESENTATION/4)

More explanations on Diffusion Wavelets

55

CONCLUSIONS

Shape retrieval requires multi-scale analysis

3D models, like most real-life objects, are embedded in a low dimension manifold

Results are robust to noise and to mesh simplifications

56

CONCLUSIONS (2)

Diffusion Wavelets give good retrieval results for 3D objects

Possible to extend the proposed method to include texture, sound, smell, elasticity and any other possibly given attribute of the 3D model

57

THE END

58

REFERENCES[1] K.P. Zhu, Y.S. Wong, W.F. Lu, J.Y.H. Fuh. , Department of Mechanical Engineering, National University of

Singapore “A diffusion wavelet approach for 3-D model matching” Computer Aided Design, Elsevier, Nov. 2008

[2] Presentation by R.R. Coifman et. al.

[3] J. Lu et. al. “Dominant Texture and Diffusion Distance Manifolds“, Eurographics, Volume 28 ,

Issue 2, Pages 667 - 676, Mar. 2009

[4] Diffusion wavelets Matlab code:

http://www.math.duke.edu/~mauro/diffusionwavelets.html#Code|outline

[5] The Princeton Shape Benchmark:

http://shape.cs.princeton.edu/benchmark/

[6] Nadler, B., Lafon, S., Coifman, R., Kevrekidis, I. “Diffusion maps, spectral clustering and eigenfunctions of Fokker-Planck operators”.

[7] Ulrike von Luxburg, “A tutorial on spectral clustering”. Statistical Journal 2007

[8] Personal communications with K.P. Zhu

[9] MANI - Manifold learning Matlab tool

http://www.math.umn.edu/~wittman/mani/

[10] Vranic D, Saupe D, Richter J. Tools for 3D-object retrieval: Karhunen-Loeve transform and spherical harmonics. In: Proc. IEEE workshop on multimedia signal processing; 2001. p. 29398.

59

REFERENCES[11] Osada R, Funkhouser T, Chazelle B, Dobkin D. Matching 3D models with shape

distributions, In: Proc. shape modeling international. 2001. p. 15466.

[12] Laga H, Nakajima M. Statistical spherical wavelet moments for content-based 3D model Retrieval. In: Computer graphics international 2007, CGI. 2007; 2007. p.1-8.

[13] Nain D, Haker S, Bobick A, Tannenbaum A. Multiscale 3-D shape representation and segmentation using spherical wavelets. IEEE Transactions on Medical Imaging 2007;26(4), pages 598-618.