Embed Size (px)
Citation preview
Categorization: Scenes & Objects
(P)Lavanya Sharan
March 16th, 2011
Last time
• What is category?
• Functional vs. communicational
• Basic-level categories (Rosch)
• Entry-level categories and prototypicality
Visual perception of categoriesObjects (e.g., animal vs. non-animal, cars vs. houses, German shepherds vs. other dogs etc.)
Kirchner & Thorpe, 2005 Grill-Spector & Kanwisher, 2005
Visual perception of categoriesScenes (e.g., desert vs. canyon, low openness vs. high openness etc.)
Oliva & Schyns, 2000
Greene & Oliva, 2009
Why does the choice of category matter?
Object Non-object
Detection task: Should be easiest and fastest.
Why does the choice of category matter?
House Dog
Categorization task: Should be harder and slower?Image sources: dogbreedinfo.com,
cambridge2000.com
Why does the choice of category matter?
Danish Farm Dog Old Danish Chicken Dog
Categorization task: This one should be hardest and slowest?
Image source: dogbreedinfo.com,
Classic results in object recognition
Mostly from psychophysical experiments.
We can tell object from non-object in a brief glance.
We can tell scene categories in a brief glance.
Beyond categories, details about scenes and objects can be inferred in a brief glance.
Short reaction times and ERP data suggests we can do these tasks quickly.
Objects: Thorpe et al. 1996, Grill-Spector & Kanwisher 2005, Kanwisher et al. 1997, Potter 1975, Rosch 1978, Nakayama et al. 1995, Biederman 1987, Intraub 1981, Peterson & Gibson 1993,…
Scenes: Biederman 1972, Potter 1975, 1976, Intraub 1981, Oliva and Schyns 2000, Oliva & Torralba 2001, Rousselet et al. 2005, Evans & Treisman 2005, Fei-fei et al. 2007, Greene & Oliva 2009,...
Today: Object recognition & fMRI
Slide source: Jody Culham
Today: Object recognition & fMRI
Slide source: Jody Culham
Why care about what fMRI has to say?
Slide source: Jody Culham
Category-specific regions
Image source: Grill-Spector 2008
Blue = Object > Scrambled objectsLateral Occipital Complex (LOC)
Red = Faces > Non-face objectsFusiform Face Area (FFA) + few others
Green = Places > ObjectsParahippocampal place area (PPA) + few others
Magenta = Faces + ObjectsDark Green = Places + Objects
Also regions for body parts, letter vs. textures, tools vs. animals etc.
LOC: Basics
Two regions: LO + pFus/OTSShape, surfaces, contours. Not low-level features (e.g., colors, textures).Global shape, not local contours.
Image source: Grill-Spector 2008
LOC: Basics
Image source: Grill-Spector 2008
Objection recognition implies invariances
Size
Position
Rotation
Illumination
...
Can fMRI activations tell us how these invariances are achieved?
Unlikely. But, let’s study how invariant LOC responses are. Is it truly correlated with successful object recognition?
Measuring position invariance in LO
Image source: Grill-Spector 2008
Position effects larger than category effects in LO!
Image source: Grill-Spector 2008
For pFus/OTS greater position invariance than LO (Schwarzlose et al. 2008)
Rotation sensitivity in LOC
Image source: Grill-Spector 2008
Andreson et al. 2009 used fMRI adaptation.
If a region is sensitive to rotation, then repeating the same (or similar view) will cause a change in response.
Mixed story. Rotation sensitivity depends on categories and brain regions.
Implications for object recognition theories/models
View dependence.What is the representation for object recognition? View-sensitive neurons (low-level representations) and population coding? Or view-invariant neurons outside regions measured?
Do we really need view-invariant representations for object recognition?
Implications for object recognition theories/models
Domain specialization.Given that some object categories seem special, what does that imply for object recognition theories? General-purpose computations vs. specialized features and computation?
Alternative interpretation of fMRI data: These regions are really about expertise with a visual category rather than the category itself (e.g., faces).
Implications for object recognition theories/models
Distributed processing.It is possible to predict (using machine learning techniques on fMRI data) which object category a person is looking at (even when FFA etc. are not considered).
Advantage of distributed code, recognize more objects?
Implications for object recognition theories/models
Effect of experience.Size of category-specific regions changes as children mature to become adults. These changes are not simply geometric scaling.
Learned representations vs. innate modules?
