C OMPUTATION M ODEL FOR V ISUAL C ATEGORIZATION Bhuwan Dhingra

COMPUTATION MODEL FOR VISUAL CATEGORIZATIONBhuwan Dhingra

OVERVIEW

Objective: To study the hierarchy of object categorization using a computational model for vision.

Three levels of categorization – super-ordinate, basic and subordinate.

Basic level categories – maximize cue validities, and dominate any taxonomy.

Categorization implemented in unsupervised manner in the current model.

HYPOTHESES

Rosch et al, [1], claim that basic level categories accessed first.

Marc and Joubert, [2], claim that in a purely visual task super-ordinate categories accessed first.

Role of expertise emphasized several times in the literature, [3].

THE MODEL

Bag-of-Features:

THE MODEL

Extracted histograms clustered in an unsupervised manner using k-means algorithm.

Distance metric used – (1-correlation(h1,h2)), where h1 and h2 are two histograms.

DATASET

30 images for each subordinate category using Google image search of the keywords.

DATASET

FurnitureAnimal

TableChairBirdDog

Coffee Table

Picnic Table

Rocking Chair

Bar-stool

Crow

Pigeon

Foxhound

Dalmation

Super-ordinate classes

Basic classes

Sub-ordinate classes

TESTS

Test 1: Study which type of categorization dominates as the number of detected key-points is varied.

Test 2: Study how the performance of the categorization changes with the number of images.

Test 3: Study the effect of increasing the number of images of one basic category compared to others

Different categorizations were implemented by setting k = 2,4,8.

PERFORMANCE INDICES

Rand Index:

TP, TN, FP, FN are true positive and negatives, and false positives and negatives.

Purity: Percentage of correctly assigned points, assuming majority class for each cluster.

Normalized Mutual Information: Information theoretic mutual information between clusters and classes (normalized to 1).

Silhouette Index: Based on the ratio of the within class scatter to between class scatter.

RESULTS Variation of the performance metrics with

Peak Threshold or the number of key-points detected.

-0.01 0 0.01 0.02 0.03 0.04 0.05 0.06 0.070.35

0.4

0.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

Peak Threshold

Pur

ity

Purity vs Peak Threshold

Super-ordinateBasicSub-ordinate

-0.01 0 0.01 0.02 0.03 0.04 0.05 0.06 0.070.02

0.04

0.06

0.08

0.1

0.12

0.14

Peak Threshold

Silh

oue

tte

In

dex

Silhouette Index vs Peak Threshold

Super-ordinateBasicSub-ordinate

-0.01 0 0.01 0.02 0.03 0.04 0.05 0.06 0.070.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Peak Threshold

Ran

d I

nd

ex

Rand Index vs Peak Threshold

Super-ordinateBasicSub-ordinate

-0.01 0 0.01 0.02 0.03 0.04 0.05 0.06 0.070

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Peak Threshold

Nor

mal

ize

d M

utu

al I

nfo

rmat

ion

NMI vs Peak Threshold

Super-ordinateBasicSub-ordinate

RESULTS Variation of the performance metrics with

Peak Threshold or the number of key-points detected.

-0.01 0 0.01 0.02 0.03 0.04 0.05 0.06 0.070.35

0.4

0.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

Peak Threshold

Pur

ity

Purity vs Peak Threshold

Super-ordinateBasicSub-ordinate

-0.01 0 0.01 0.02 0.03 0.04 0.05 0.06 0.070.02

0.04

0.06

0.08

0.1

0.12

0.14

Peak Threshold

Silh

oue

tte

In

dex

Silhouette Index vs Peak Threshold

Super-ordinateBasicSub-ordinate

-0.01 0 0.01 0.02 0.03 0.04 0.05 0.06 0.070.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Peak Threshold

Ran

d I

nd

ex

Rand Index vs Peak Threshold

Super-ordinateBasicSub-ordinate

-0.01 0 0.01 0.02 0.03 0.04 0.05 0.06 0.070

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Peak Threshold

Nor

mal

ize

d M

utu

al I

nfo

rmat

ion

NMI vs Peak Threshold

Super-ordinateBasicSub-ordinate

RESULTS Variation of the performance metrics with

Peak Threshold or the number of key-points detected.

-0.01 0 0.01 0.02 0.03 0.04 0.05 0.06 0.070.35

0.4

0.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

Peak Threshold

Pur

ity

Purity vs Peak Threshold

Super-ordinateBasicSub-ordinate

-0.01 0 0.01 0.02 0.03 0.04 0.05 0.06 0.070.02

0.04

0.06

0.08

0.1

0.12

0.14

Peak Threshold

Silh

oue

tte

In

dex

Silhouette Index vs Peak Threshold

Super-ordinateBasicSub-ordinate

-0.01 0 0.01 0.02 0.03 0.04 0.05 0.06 0.070.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Peak Threshold

Ran

d I

nd

ex

Rand Index vs Peak Threshold

Super-ordinateBasicSub-ordinate

-0.01 0 0.01 0.02 0.03 0.04 0.05 0.06 0.070

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Peak Threshold

Nor

mal

ize

d M

utu

al I

nfo

rmat

ion

NMI vs Peak Threshold

Super-ordinateBasicSub-ordinate

RESULTS Variation of performance metrics with

number of images:

10 15 20 25 30

0.3

0.32

0.34

0.36

0.38

0.4

0.42

0.44

0.46

Images per sub-ordinate category

Nor

mal

ize

d M

utu

al I

nfo

rmat

ion

NMI vs Number of Images

Super-ordinateBasicSub-ordinate

10 15 20 25 300.4

0.45

0.5

0.55

0.6

0.65

0.7

0.75

0.8

0.85

0.9

Images per sub-ordinate category

Pur

ity

Purity vs Number of Images

Super-ordinateBasicSub-ordinate

10 15 20 25 30

0.2

0.25

0.3

0.35

0.4

Images per sub-ordinate category

Ran

d I

nd

ex

Rand Index vs Number of Images

Super-ordinateBasicSub-ordinate

10 15 20 25 300.065

0.07

0.075

0.08

0.085

0.09

0.095

0.1

0.105

0.11

Images per sub-ordinate category

Silh

oue

tte

In

dex

Silhoutte Index vs Number of Images

Super-ordinateBasicSub-ordinate

RESULTS Effect of expertise Two subordinate and one basic level categories

taken together, ex: {{dalmation, foxhound}, bird} Trial 1: Training samples of subordinate categories

half of basic categoryTrial 2: Training samples of subordinate category equal to basic category

30 600

0.05

0.1

0.15

0.2

0.25

0.3

0.35

Number of images in Basic Category

Ran

d I

nd

ex

Effect of Expertise

dogbirdchairtable

SOME PROBLEMS

White background images sometimes classified separate from cluttered background. Solution: Foreground extraction

High variability in Normalized Mutual Information (NMI)

Effect of expertise not clear Solution: Test for exponential increase in

images

REFERENCES

[1] Rosch, E., Mervis, C., Gray, W., Johnson, D., & Boyes-Braem, P. (1976). Basic objects in natural categories. Cognitive Psychology.

[2] Marc, J.M.M., Joubert, O.R., Nespoulous, J.L. & Fabre-Thorpe, M (2009). The time-course of visual categorizations: you spot the animal faster than the bird. PLoS one.

[3] Johnson, K.E., Mervis, C.B. (1997). Effects of varying levels of expertise on the basic level of categorization. Journal of Expert Psychology.