Professor Aditi Majumder - Donald Bren School of ...majumder/vispercep/student08/functions.pdf · Professor Aditi Majumder 1 Perceiving Function and Category yVisually perceive yFunctionality

Ish Rishabh and Vivek SinghCS213: Visual Perception

Professor Aditi Majumder

1

Perceiving Function and Category

Visually perceiveFunctionality of objects that we seeCategorization: Classify into known types

2

AgendaPart 1: Perception of functionPart 2: Categorization: Various phenomenaPart 3: Theories of CategorizationPart 4: Recognizing letters and words

PART 1Perception Phenomena

PART 2Categorization

PART 3Theories of Categorization

PART 4Recognizing letter & word

3

Perception of functionPerception of

Function

Affordances

Direct

Categorization

Indirect

Flat surfaceHorizontalStrong

= SittableSittable

J. J. Gibson All others

PART 1Perception PhenomenaDirect PerceptionIndirect perception




4

Direct Perception of Function

Traditional approach was categorizationGestalt psychologists argued direct perceptionAffordances: properties that prompt user interactionJ. J. Gibson (1979) claimed that:

Objects can be grasped upon, sat uponNo standard categories for such affordances


Direct PerceptionIndirect perception




5

AffordancesTwo important considerations:1. Functional form: Function must follow from

formRound wheels: rollingTriangular wheel?

2. Observer relativity: Affordances perceived depends upon the observer.






6

Affordances (continued)Neisser (1989)Functional properties that conform to both conditions are called physical affordancesThese are necessary, but not sufficient for direct perception (Gibson)

Similar affordances

Send Letters Dumb garbageDifferent perceptions






7

Affordances (continued)Neisser suggested:

Affordances and categorization are fundamentally different modes of perceptionAccomplished by different neural systems

Evident in patient with damaged ventral system

Affordances (dorsal)

Categorization (ventral)






8

Affordances (shortcomings)

Cannot account for all functional information that we perceive

Example: CDsHence categorization approach is important






9

Indirect Perception of Function by Categorization

≈

Meant for flying

Meant for flying





10

Boils down to categorizing object

Categorization:Four Components1. Object representation2. Category representation3. Comparison processes4. Decision processes


Object representationCategory representationComparison processesDecision processes




Category 1 Category 2 Category n

Object

11

Categorization:1. Object representation

Shape is the most importantTemplatesFourier spectraFeature listsStructural descriptions

Other informationTextureColorSizeOrientation

PART 1Perception PhenomenaDirect PerceptionIndirect perception> Object representation

Category representationComparison processesDecision processes




12

Categorization:2. Category representation

Shape is the most importantTemplatesFourier spectraFeature listsStructural descriptions

Other informationTextureColorSizeOrientation


Object representation> Category representation

Comparison processesDecision processes




13

Categorization:3. Comparison processes

Object and category representations should be of the same typeComparison: Serial or parallel?

Comparing across categories: parallelVery large number of known categories

Comparing elements within representation: Not obvious.


Object representationCategory representation

> Comparison processesDecision processes




14

Categorization:4. Decision processes

Which category does the object belong to?Should support:1. Novelty2. Uniqueness

For mutually exclusive classesA thing cannot simultaneously be cat and dog

Get fit value for each categoryThree approaches of decision making:

Threshold ruleMaximum (best-fit) ruleMaximum-over-threshold ruleMost appropriate


Object representationCategory representationComparison processes

> Decision processes




15

Categorization:4. Decision processes (cont.)

Problem with threshold: Not unique







16


Problem with best fit: No novelty







17


Maximum over threshold: Preserves uniqueness







18


Maximum over threshold: Preserves novelty







19

Perceptual Categorization:Phenomena1. Defining categories and their structure2. Effects of perspective viewing conditions on

categorization3. Does part structure help in categorization4. Contextual effects on categorization5. Visual agnosia

PART 1 √Perception Phenomena

PART 2CategorizationCategoriesPerspective viewingPart structureContextVisual agnosia



20

Perceptual Categorization:Defining categories

Categorical structure is largely hierarchicalDog < Mammal < Animal < Living thing …

Two ways of representing:1. Hierarchical trees2. Venn diagrams

Animals

Dogs Birds Fishes

Beagles

Collies

Dachshunds

Robins

Eagles

Ostriches

Trout

Salmon

Sharks

AnimalsDogs

FishesBirds

DachshundsBeagles

Collies

Robins

Eagles

Ostriches Trout

SalmonSharks



CategoriesPerspective viewingPart structureContextVisual agnosia



21

Perceptual Categorization:Defining categories (cont.)

Variations within each categoryNot all dogs look alike, nor all birds, nor carsWhat is the basis of categorizing objects in a category?

Classical approach: AristotleCategory was designated by a set of rules

Necessary and sufficient conditions for membershipConditions: List of properties that object must haveExample: Triangle (closed polygon, three lines)



CategoriesAristotle ViewPrototype

Perspective viewingPart structureContextVisual agnosia



22

Perceptual Categorization:Aristotelian view

Binary category membershipEither in category or not

Is it good at explaining natural perceptual categories?Ludwig Wittgenstein (1953) said “No”.

Name features common to all gamesFamily members resemblance, but no necessary or sufficient condition definition



Categories> Aristotle View

PrototypePerspective viewingPart structureContextVisual agnosia



23

Perceptual Categorization:Prototype

Eleanor Rosch, UC Berkeley (1970s)All natural categories might be structured in a similar way in terms of a central or ideal exampleThis is called PrototypePrototype is an average member

‘Doggiest’ possible dog



CategoriesAristotle View

> PrototypePerspective viewingPart structureContextVisual agnosia



24

Perceptual Categorization:Aristotelian vs Prototype

Rule-based vs instance-based representationBinary versus graded membership

How doggy a dog is?Prototypes are used naturally

Chihuahua rated poorly as dogs than beagles




> PrototypePerspective viewingPart structureContextVisual agnosia



25

Perceptual Categorization:Levels of categories

At which hierarchical level do we categorize an object?

Lassie < Dog < Mammal < Animal < … ?Most people identify object at an intermediary levelRosch defined it as basic-level categorySuperordinate categories: above basicSubordinate categories: below basic




> PrototypeBasic-level categoriesEntry-level categories




26

Perceptual categorization:Basic-level categories

Highest level category such that:1. Similar shape2. Similar motor interactions

Piano, guitar3. Common attributes




> Prototype> Basic-level categories

Entry-level categories




27

Perceptual Categorization:Entry-level categories

Example:Category: Bird

Robin, sparrow: identified as ‘birds’Ostrich: identified as ‘ostrich’

For some basic-level categories with wide variety:

Typical objects are classified at basic levelAtypical objects are classified at subordinatelevel

Jolicoeur (1984) called them entry-level




> PrototypeBasic-level categories

> Entry-level categories




28

Categorization:Perspective Viewing

Perspective views influence speed and accuracy of recognitionSome views of objects are easier to recognize than other





29

Categorization:Perspective Viewing

Experiment: Palmer, Rosch and Chase (1981)Subjects rated many views of the same object

Canonical PerspectiveOther subjects named entry-level category of many objects

Latency was noted


PART 2CategorizationCategoriesPerspective viewing

> Canonical viewPriming effectOrientation effect

Part structureContextVisual agnosia



30

Categorization:Perspective Viewing PART 1 √

Perception Phenomena






31

Categorization:Perspective Viewing PART 1 √







32

Perspective Viewing:Canonical view hypotheses

Hypotheses:1. Frequency hypothesis

Frequently seen views are more canonicalBut cups seen from above are not

2. Maximal information hypothesisViews that provide more information about shape and use of object are more canonicalBest views tend to show multiple sides

Both hypotheses are true to some extent







33

Perspective Viewing: Relation to Priming effect

Priming effect (Bartram, 1974)Two sets of images shown. Latency noted.Categorizing is faster and more accurate if the object is presented a second timeHeightened state of readinessRepetitions need not be exact replica

Different perspective view may be presented

Irving Biederman used this to study the effect on perspective viewing on categorization



Canonical view> Priming effect

Orientation effectPart structureContextVisual agnosia



34

Perspective Viewing:Priming effect

Modification in position, reflection or size does not affect priming effectChanges in perspective doesHowever, if same parts are visible in different perspective, then no effect







35

Perspective Viewing:Priming effect PART 1 √







750 ms

≈600 ms≈600 ms ≈600 ms ≈600 ms 800 ms

≈600 ms

Position Orientation Size PerspectiveParts visible

PerspectiveParts hidden

36

Categorization:Orientation effect

Rotation of object along line of sightPierre Jolicoeur (1985)

Faster categorization of objects in their normal, upright orientation

Orientation effects diminish with practicePeople may store multiple representations of the same object at different orientations



Canonical viewPriming effect

> Orientation effectPart structureContextVisual agnosia



37

Categorization:Effects of Part structure

Biederman and Cooper (1991)2 experiments

Based on priming effectUsed line drawings of objects

Experiment 1In first image, half contours were deletedCompliment image had only those lines

Experiment 2First image, some parts deletedCompliment image has only those parts





38

Part Structure:Experiments

Identity primingSame set was used in two trialsJust for baseline

Compliment primingComplement sets were used in two trials

Different exemplar primingA totally different perspective view was used in the second trial





39

Part Structures:Results of experiments PART 1 √





40

Categorization:Contextual effects

Depends upon prior knowledgeDepends on surroundings in the view





41

Categorization:Visual Agnosia phenomenon

Brain damaged patientsInability to categorize previously known objectsApperceptive agnosia

Sensory processing damagedAssociative agnosia

Perceptual part intact, but association lostProsopagnosia

Cannot recognize faces visually





42

Visual agnosia PART 1 √Perception Phenomena




43

RecapPerception of function1. Direct (affordances)2. Indirect (categorization)

CategorizationCategories (basic-level, entry-level)Effects of perspective on categorizationEffects of part structureEffects of surrounding contextVisual agnosia is related to categorization


PART 2 √Categorization



44

Theories for Object Categorization

How objects might be perceived in the visual human systemMost Prominent:

Recognition by Components (RBC) Theory Irving Biederman (1985,1987)Also called Geon theory



PART 3Theory of categorization

Recog.by componentsExplaining phenomenaOther theories

PART 4Identifying letter & words

Recognition By Components Theory

Objects can be specified as spatial arrangements of primitive volumetric components called geons.Geons

geometric ionsA set of generalized cylinders which are easily distinguishable from each other.Letters: Words :: Geons: Objects






Geons Cross sectional curvature1.Straight2.Curved

Symmetry1.Reflectional2.Rotational3.None

Axis curvature1.Straight2.Curved






Geons Cross sectional curvature1.Straight2.Curved

Symmetry1.Reflectional2.Rotational3.None

Axis curvature1.Straight2.Curved

Cross-sectional Size variation1.Expanding2.Expanding & Contracting3.Fixed

Aspect ratio1. Equal2. Axis greater3. Cross section greater

#of geons (qualitatively)= 2*3*2*3=36#of geons (quantitatively)= 36*3= 108






Nonaccidental featuresProperties to identify geons.Not dependent on ‘accidents’ of viewpoint.






Geon relationsSpatial relation of geons:: Order of alphabets in words

e.g. SIDE-CONNECTED, LARGER-THAN108 different relations #of 2 geon objects > 1,000,000






A Neural Network implementation

Edges

Features

Geon attributes

Relations

Feature Assembly

[Hummel and Biederman, 1992]






Explaining empirical Phenomena

Prototypes /typicalityBasic level/entry level categoriesPerceptual viewing conditionsPart structureContextual effectsVisual Agnosia



PART 3Theory of categorizationRecog.by componentsExplaining phenomenaOther theories


Prototypes/typicalityCategorization is function of geon matching

‘Rough’ i.e. qualitative descriptionsSubordinate category (fine grained changes ) can not be explained.





Basic level/entry level categories

Typical members closely match the geons for basic level descriptionsAtypical members do not. Hence not normally classified in Basic level. But, how are they classified?

Not clear





Viewing conditionsCanonical perspective

Some geons get occludedPriming effect

Not studied





Part structureComponent geons trigger activation, not lines and edges. Half the lines are enough for geon activation.





Contextual effectsCannot be explained as RBC looks only at parts of the object.But the idea can be extended to ‘scenes’ whose components are ‘objects’.





Visual AgnosiaAll views are ‘unusual’ to patients. Not much known !





WeaknessesLack of representation power

108 cylinders, 108 relations

Finer discrimination required

Dog Vs CatFace recognition

Implementation?





Viewpoint specific theoriesMultiple Views are required

1 view cannot capture the 3D modelMultiple ‘good’ views with low latency

Hence:Aspect GraphsAlignment with 3-D modelsAlignment with 2-D view combinations





Aspect graphsMany views of the same object are actually very similarAspect: Multiple views matched to a common abstract representation

Aspect1 Aspect2

View 1 View2 View 3





Aspect graphsAspect: Multiple views matched to a common abstract representation

3 different top views

1 common representation i.e. aspect





Aspect GraphsDifferent aspects connected if continuous change takes viewer from one aspect to the other.

32 4

1





Aspect Graphs: IssuesScalabilityInnate 3-D ability of the brainDiscrimination (capture topology, not geometry)





Alignment with 3-D Models

3-D Models stored in brain. Mapped to 2-D images.

Still top-down

1. Image

2. Edge features

3. Matching with 3-d model

4. Object recognition





Alignment with 2-D View combinations

Use a ‘few’ 2-D views in brain rather than 3-D model. A method that can derive new 2-D views from a few stored ones.





WeaknessesView point theories don’t explain:

Innate 3-D abilityNovel objectsNon rigid objectsPart structureExemplar variation





Identifying Letters and words

Perceiving letters as well as understanding words. Easier than object categorization:

Two-DimensionalityCombinatorial structure

Study:Identifying LettersIdentifying within wordsInteractive activation model




PART 4Identifying letter & wordsIdentifying lettersIdentifying letters in wordsInteractive activation modelAlternative explanations

Identifying LettersCan be identified using:

TemplatesStructural descriptionsFeatures

[McCelelland & Rumelhart, 1981]




PART 4Identifying letter &

wordsIdentifying lettersIdentifying letters in wordsInteractive activation modelAlternative explanations

Identifying LettersFuzzy Logic Model of Perception (FLMP)

[Massaro & Hary,1986]






Identifying letters within words

Letters are not detected independently of words.

HWO NMYA RSETELTE NCA OYU RPTERO WNOHOW MANY LETTERS CAN YOU REPORT NOW?






EffectsWord superiority effectWord-nonword effectWord-letter effect

83%

70%

70%

[McCelelland & Rumelhart, 1981]






Interactive Activation Model

Proposes a multilayer neural network like model.

Feature levelLetter levelWord Level






Interactive Activation Model PART 1






Alternative Explanations

Word shapeSerial Letter recognitionParallel recognition

Moving window effectBoundary effect






Word ShapeWe recognize a word is the pattern of ascending, descending, and neutral characters [James Cattell, 1886]

test Error rates Explanationtesf 13% Consistent word

shapetesc 7% Inconsistent word

shape






Word Shape Vs. Letter Shape

Letter shape more important than Word shape [Monk & Hulme, 1983]

than Same Word shape Different Word shape

Same letter shape

tban15% missed

tnan19% missed

Differentletter shape

tdan8% missed

tman10% missed






Serial letter recognitionAnalogy to dictionary [Gough, 1972].

Start with 1st letter, then 2nd and so on. Search for a letter in random strings.

3rd letter 30 ms, 4th letter 40 ms. Bigger words take longer to recognize. Effects

NOUTHSORTH

Cannot explain word superiority effect






Parallel letter recognition

Moving window effect [McConkie & Rayner 1975]

Fixate on words (200-300ms), then saccadic movement (20-35ms).

Fovea (3 or 4 letters)Neighboring (8 or 9 letters)Parafovea (15 to 20 characters)






Moving window effectWindow size

Sentence Reading rate

3 letters An experimxxx xxx xxxxxxxxx xx 207 wpm9 letters An experiment wax xxxxxxxxx xx 308 wpm15 letters An experiment was condxxxxx xx 340 wpm






Invisible boundary effect

Word shown Properties Speed chart Identical word (control) 210mschovt Similar word shape

Some letters in common240ms

chyft Dissimilar word shapeSome letters in common

280ms

ebovf Similar word shapeNo letters in common

300ms

[Rayner, 1975]






SummaryPart 1: Perception of function phenomena

Direct Vs. IndirectPart 2: Categorization phenomena

Parts, categories, viewpoints, agnosiaPart 3: Theory of categorization

Recognition By ComponentsPart 4: Recognizing letters and words

Interactive Activation model

Documents

Professor Aditi Majumder - Donald Bren School of ...majumder/vispercep/student08/functions.pdf · Professor Aditi Majumder 1 Perceiving Function and Category yVisually perceive yFunctionality