Pyramids of FeaturesFor Categorization

Greg Griffin and Will Coulter(see Lazebnik et al., CVPR 2006, too)

Intuition: Approximates optimal partial matching

U

Adapted from http://people.csail.mit.edu/kgrauman/slides/pyr_match_iccv2005.ppt

Intuition [cont’d]: Combine bags of features with spatial information

0 5 10 15 20 25 30 35 40 45 500

1

2

3

4

5

6

7

8

9

10

0 5 10 15 20 25 30 35 40 45 500

1

2

3

4

5

6

7

8

9

10

category

Disadvantages

• Same objects in different quadrants

• Objects sliced by bins

Possible Solutions

• Flipping / rotating image

• Sliding / shuffling histogram bins

Possible Solutions [cont’d]

• Split histogram in powers of three

Implementation Overview

Image and Feature Extraction (SIFT on regular grid)

↓

Vocabulary Translation (200 words (k-means))

↓

Histogram Generation (flips, slides, arbitrary mixing)

↓

Matching (full or partial pyramid)

↓

Decision (best match, voting, SVM)

Sanity Check 1Graphical mini-confusion matrix (and rot. invariance)

Sanity Check 2

Scene Database

Scene Database

81.1% vs 67.4%

(100)

Caltech 101 64.6% vs. 33.1% (30 , 16)

Caltech 101

minaret windsor chair joshua tree okapi

cougar body beaver crocodile ant

97.6% , 86.7% 94.6% , 80.0% 87.9% , 40.0% 87.8% , 33.3%

27.6% , 6.7% 27.5% , 13.3% 25.0% , 13.3% 25.0% , 0.0%

(30 , 16)

Caltech 101

Work in Progress

• 256 Performance– 64 times more work scene database– 6.4 times more work than 101

• SVM– one-vs-all weighting issues– speed it up?– improve performance

• Improvements– Flip, Slide, Arbitrary

• Powers-of-3 histogram bins

Open Questions

• Performance of arbitrary match bins– Try random sampling?– Allow multiple best matches?

• Chess/pattern example

• Grid example

• Optimal kernel level weights

Implementation Details

• [block diagram]• Images (288^2,b&w,squished) and feature

extraction (-weak,-pca,+sift)• Vocab generation (200 words, 20,000-

small)• Histogram(fliplr,flipud,slide,arbitrary,bag of

features)• match (full&partial pyramids)• Decision (best match,voting,SVM)