Chapter 4: Object Chapter 4: Object RecognitionRecognition• What do various disorders of shape recognition What do various disorders of shape recognition

tell us about object recognition?tell us about object recognition?- Apperceptive visual agnosia (Ch. 2)- Apperceptive visual agnosia (Ch. 2)- Associative visual agnosia- Associative visual agnosia- Perceptual categorization deficit- Perceptual categorization deficit

• What do neuroimaging studies tell us about What do neuroimaging studies tell us about object recognition?object recognition?

• The computational interpretation: What does The computational interpretation: What does cognitive evidence suggest about constraints on cognitive evidence suggest about constraints on the nature of the neural representations the nature of the neural representations underlying object recognition?underlying object recognition?

Object RecognitionObject Recognition

• Dr. Harley has Dr. Harley has filled you in on the filled you in on the neuroscience side neuroscience side of the miracle…of the miracle…

Farah (2000) – Fig. 4.1

Visual AgnosiasVisual Agnosias• Visual AgnosiaVisual Agnosia: A blanket : A blanket

term for impaired visual term for impaired visual object recognition object recognition following brain damage following brain damage when elementary visual when elementary visual functions (acuity, visual functions (acuity, visual fields) are adequate. fields) are adequate.

• Apperceptive visual Apperceptive visual agnosiaagnosia: Inability to group : Inability to group the local features into a the local features into a coherent perceptual coherent perceptual representation (Ch 2)representation (Ch 2)

• Associative visual agnosiaAssociative visual agnosia: : Inability to recognize Inability to recognize visually presented objects, visually presented objects, despite having a coherent despite having a coherent perceptual representation perceptual representation

Visual Agnosias: Copying and Visual Agnosias: Copying and drawing from memory.drawing from memory.Apperceptive agnosia Associative agnosia

Copying

Memory

Copying

Associative visual agnosia: Associative visual agnosia: CriteriaCriteria• Patients can form percepts (do perceptual Patients can form percepts (do perceptual

grouping) unlike apperceptive agnosia grouping) unlike apperceptive agnosia patients.patients.

• They can see an object well enough to They can see an object well enough to describe its appearance, to draw it, or to describe its appearance, to draw it, or to succeed in a same/different test of succeed in a same/different test of appearances.appearances.

• They have difficulty recognizing visually They have difficulty recognizing visually presented objects (can’t name or sort presented objects (can’t name or sort objects by category)objects by category)

• They can demonstrate knowledge of objects They can demonstrate knowledge of objects from other sensory modalitiesfrom other sensory modalities

Associative agnosia behavior: Associative agnosia behavior: An impairment of An impairment of shape shape perceptionperception• They can copy complex objects. They can copy complex objects.

BUT their copying behavior is BUT their copying behavior is abnormal – HJA took 6 hours to abnormal – HJA took 6 hours to do the cathedral.do the cathedral.

• They are very sensitive to They are very sensitive to visual quality of stimuli. They visual quality of stimuli. They recognize real objects better recognize real objects better than line drawings; face than line drawings; face recognition of unfamiliar people recognition of unfamiliar people is impaired by changing lighting is impaired by changing lighting conditions.conditions.

• They make visual shape They make visual shape recognition errors; they might recognition errors; they might call a baseball bat, a paddle, call a baseball bat, a paddle, knife, baster, etc. knife, baster, etc.

Associative Visual Agnosia Associative Visual Agnosia

• The human analog of the IT-lesioned The human analog of the IT-lesioned monkeys.monkeys.– They fail to recognize objects because They fail to recognize objects because

they fail to represent shape normally.they fail to represent shape normally.•Slow, slavish copyingSlow, slavish copying

•Sensitivity to visual quality of the stimuliSensitivity to visual quality of the stimuli

•Visual shape errorsVisual shape errors

Regions of brain damage Regions of brain damage associated with different types of associated with different types of visual agnosiavisual agnosia• Apperceptive Apperceptive

AgnosiaAgnosia: Diffuse : Diffuse damage to the damage to the occipital lobe and occipital lobe and surrounding areassurrounding areas

• Associative AgnosiaAssociative Agnosia: : Occipitotemporal Occipitotemporal regions of both regions of both hemisphereshemispheres

From Banich (2004)

Perceptual categorization Perceptual categorization deficitdeficit• Difficulty recognizing Difficulty recognizing

objects viewed from objects viewed from unusual perspectives unusual perspectives or uneven illumination or uneven illumination conditions (Warrington conditions (Warrington & Taylor, 1973)& Taylor, 1973)

• Initially assumed to Initially assumed to reflect an impairment reflect an impairment of viewpoint-invariant of viewpoint-invariant object recognitionobject recognition

BUT: Is Perceptual BUT: Is Perceptual Categorization Deficit really Categorization Deficit really about loss of shape about loss of shape representations?representations?• Not impaired in the real world.Not impaired in the real world.

• Not impaired under all viewing conditions Not impaired under all viewing conditions per se.per se.

• Mainly impaired when matching an USUAL to Mainly impaired when matching an USUAL to an UNUSUAL view; normal people have an UNUSUAL view; normal people have similar, less serious, type of difficulty.similar, less serious, type of difficulty.

• Associated with unilateral RH lesions, usually Associated with unilateral RH lesions, usually in parietal cortex, not inferotemporal cortexin parietal cortex, not inferotemporal cortex

Perceptual categorization Perceptual categorization deficitdeficit

• Demonstrates value of examining Demonstrates value of examining evidence carefully and trying to link evidence carefully and trying to link it with what is known to see whether it with what is known to see whether patterns fit or not.patterns fit or not.

• In this case, the pattern differs In this case, the pattern differs greatly from other visual agnosias. greatly from other visual agnosias.

Functional Neuroimaging Functional Neuroimaging Studies: Studies:

• Goal of PET, fMRI studiesGoal of PET, fMRI studies: To localize the : To localize the psychological process(es) of interest psychological process(es) of interest

• Research designResearch design: Measure brain activity in : Measure brain activity in at least two conditions: a control at least two conditions: a control (baseline) condition and an experimental (baseline) condition and an experimental condition.condition.– Psychological process localized by subtracting Psychological process localized by subtracting

brain activity in the control (baseline) condition brain activity in the control (baseline) condition from brain activity in the experimental from brain activity in the experimental condition.condition.

PET Imaging (PET Imaging (Posner & Raichle, Posner & Raichle,

19941994))Upper row: Control PET scan (resting while looking at static fixation point) is subtracted from looking at a flickering checkerboard stimulus positioned 5.5° from fixation point.Middle row: Subtraction procedure produces a somewhat different image for each of 5 subjects.Bottom row: The 5 images are averaged to eliminate noise, producing the image at the bottom.

Where in the brain does visual Where in the brain does visual recognition occur? recognition occur?

• Visual recognition Visual recognition localized to the localized to the posterior half of posterior half of the brain – really?the brain – really?

• Whoops!!! Why Whoops!!! Why did these studies did these studies fail to localize fail to localize visual recognition?visual recognition?

• Poor methodology!Poor methodology!

Neuroimaging methods: The Neuroimaging methods: The subtraction techniquesubtraction technique

• Using the subtraction technique effectively Using the subtraction technique effectively requires that the baseline and experimental requires that the baseline and experimental conditions differ only in the process of conditions differ only in the process of interest.interest.– Requires careful logical analysis of the task to Requires careful logical analysis of the task to

determine the best comparison conditions.determine the best comparison conditions.

• If there are many differences between the If there are many differences between the baseline and experimental conditions, it is baseline and experimental conditions, it is difficult to interpret the results of the study.difficult to interpret the results of the study.

Neuroimagining Studies of Object Neuroimagining Studies of Object Recognition: Comparing baseline Recognition: Comparing baseline and experimental conditionsand experimental conditions

Sequence of trial eventsSequence of trial events– Fixation pointFixation point– Stimulus presentationStimulus presentation– Task (two types)Task (two types)

Passive Viewing TaskPassive Viewing Task::

Are stimuli comparable Are stimuli comparable (e.g., size, complexity)(e.g., size, complexity)

Mind is not a vacuumMind is not a vacuum

Active Viewing TaskActive Viewing Task::

What is task?What is task?

What is response?What is response?

Sergent et al (1992a): Sergent et al (1992a): Active viewing task.Active viewing task.

• Baseline Baseline conditioncondition

- View fixation - View fixation pointpoint

• Experimental Experimental conditioncondition- View fixation - View fixation pointpoint- See line drawing - See line drawing of objectof object- Decide whether - Decide whether the object is living the object is living or non-livingor non-living- Make Yes/No - Make Yes/No responseresponse

Localizing Visual Localizing Visual RecognitionRecognition

• Is human visual recognition Is human visual recognition supported by one general purpose supported by one general purpose recognition system or are there recognition system or are there specialized modules for recognizing specialized modules for recognizing objects, faces, printed words?objects, faces, printed words?

• Good functional neuroimaging studies Good functional neuroimaging studies do exist to test these possibilities and do exist to test these possibilities and will be discussed in Chapters 5 & 6.will be discussed in Chapters 5 & 6.

Constraints on the nature of shape Constraints on the nature of shape

representations in ITrepresentations in IT: : Coordinate Coordinate systemssystems• Cannot be simple viewer-centered or Cannot be simple viewer-centered or

environmentally-centeredenvironmentally-centered– IT cells respond to a given shape over IT cells respond to a given shape over

changes in the position, size, and changes in the position, size, and picture plane orientation of objectpicture plane orientation of object

– Impairments of IT lesioned monkeys: Impairments of IT lesioned monkeys: suggest they have lost abstract shape suggest they have lost abstract shape representation representation

Constraints: Coordinate systems Constraints: Coordinate systems cont.cont.

• Two possibilities:Two possibilities:1. Object-centered 1. Object-centered

coordinate systemcoordinate system

2. A cluster of multiple 2. A cluster of multiple viewer-centered viewer-centered object representations object representations plus the ability to plus the ability to transform one transform one representation to representation to another as necessary another as necessary (a la Multiple Views (a la Multiple Views model of Tarr, 1995).model of Tarr, 1995).

Constraints: Coordinate systems Constraints: Coordinate systems cont.cont.• Farah prefers the Multiple Viewer-Centered Farah prefers the Multiple Viewer-Centered

representation option because:representation option because:– Position, size, and orientation invariance shown Position, size, and orientation invariance shown

by IT cells is not perfect; consistent with some by IT cells is not perfect; consistent with some views being better learned than others.views being better learned than others.

– IT neurons can selectively learn arbitrary IT neurons can selectively learn arbitrary associations between pairs of stimuli, a associations between pairs of stimuli, a prerequisite for deriving invariance from viewer-prerequisite for deriving invariance from viewer-centered representations.centered representations.

– Consequently, because cell activity takes time to Consequently, because cell activity takes time to decay and because seeing different views of an decay and because seeing different views of an object tends to be clustered in time, the object tends to be clustered in time, the correlation (association) between several correlation (association) between several different retinotopic views of the same object different retinotopic views of the same object could be learned. could be learned.

Constraints: Constraints: Primitives and Primitives and OrganizationOrganization

• PrimitivesPrimitives– Cannot be contoursCannot be contours– Could be either surface-based (2D) or Could be either surface-based (2D) or

volume-based (e.g., 3D geon-like parts).volume-based (e.g., 3D geon-like parts).

• OrganizationOrganization– Hierarchical? We really don’t know Hierarchical? We really don’t know

much about how multipart objects are much about how multipart objects are representedrepresented

Constraints: Constraints: ImplementationImplementation

Symbolic Model vs Neural netSymbolic Model vs Neural net• Is a perceptual representation created and Is a perceptual representation created and

then compared to a memory representation? then compared to a memory representation? (Implies that the memory representation can (Implies that the memory representation can be destroyed, yet the perceptual be destroyed, yet the perceptual representation remains intact). Unlikely, representation remains intact). Unlikely, since no such evidence exists. since no such evidence exists.

• Does the input get coded and recoded in a Does the input get coded and recoded in a succession of neural nets as it is processed succession of neural nets as it is processed through the visual system? (Implies that through the visual system? (Implies that impairment in test of object memory is impairment in test of object memory is associated with perceptual impairment). Yes, associated with perceptual impairment). Yes, this seems true in associative agnosia.this seems true in associative agnosia.

Constraints: Constraints: ImplementationImplementation

• Distributed Distributed representation representation more likely than more likely than local local representationrepresentation– Single unit Single unit

recording recording – Graceful Graceful

degradation degradation following damage following damage to ITto IT