Sketch-Based Shape Retrieval M. Eitz, R. Richter, K. Hildebrand, M. Alexa, TU Berlin; T. Boubekeur,...

Sketch-Based Shape RetrievalM. Eitz, R. Richter, K. Hildebrand, M. Alexa, TU Berlin;

T. Boubekeur, Tele ParisTech – CNRS;

Outline

• What is sketch based shape retrieval?• Sketch data base• Bag-of-features shape retrieval• GALIF: Gabor local line-based feature• Conclusions & Results

What is sketch based shape retrieval?

• sketch 3D model

Sketch data base

• Based on the Princeton Shape Benchmark (PSB), authors gather a lot of sketches.

• Analysis result: users mostly sketch objects from a simple side or frontal view.

• The sketches are free to download.

Sketch data base

Bag-of-features shape retrieval

• Assuming there are two documents:1. Bob likes to play basketball, Jim likes too2. Bob also likes to play football games.

• Construct a Dictionary: – Dictionary = {1:”Bob”, 2. “like”, 3. “to”, 4. “play”, 5.

“basketball”, 6. “also”, 7. “football”, 8. “games”, 9. “Jim”, 10. “too”}

Bag-of-features shape retrieval

• The two documents can be encoded by:① [1, 2, 1, 1, 1, 0, 0, 0, 1, 1]② [1, 1, 1, 1 ,0, 1, 1, 1, 0, 0]

counts

Bag-of-features shape retrieval

Best-view selection

• Uniformly distributed views:1. Select d seeds on a unit sphere,2. Lloyd relaxations iteratively,3. d Voronoi cell centers as d view directions.4. d ={22; 52; 102; 202}

Perceptually best views

Training set: manually define best and worst viewpoints in PSB

Learn a “best view classifier” from the training set using SVM.

Learn some best viewpoints based on the uniform viewpoints.

• For each view direction vi , predict its probability pi = p(vi) of being a best view.

• The probability is a smooth scalar field over a sphere and best views are local maxima.

GALIF: Gabor local line-based feature

• Gabor filter

: rotate an image by angle

Orientation-selective filter bank

Given k different orientations, we can compute k different images:

• (i)dft is the (inverse) discrete Fourier transformation

• I --- input sketch• * --- point-wise multiplication

Local GALIF feature definition

• I is divided into nxn regions • S, t <= n• i = 1, 2, ..., k. ------ orientataions

Conclusions & Results

• Main differences with our paper:1. Best view selection2. Feature representation

Results

Q&A