1

Psy280: Psy280: PerceptionPerception

Prof. AndersonProf. Anderson

Department of PsychologyDepartment of Psychology

Week 5Week 5

2

What’s the meaning of life? Not What’s the meaning of life? Not exactlyexactly

Computational approach to visionComputational approach to vision David Marr (MIT)David Marr (MIT)

What is the purpose of vision?What is the purpose of vision? What are the problems it must solve?What are the problems it must solve?

Visual philosophersVisual philosophers

3

Computational problems Computational problems in object recognitionin object recognition

What is it?What is it? Object constancy: Variability in sensory Object constancy: Variability in sensory

informationinformation Retinal positionRetinal position Viewing positionViewing position OcclusionOcclusion Lighting, colourLighting, colour

4

Computational problems Computational problems in object recognitionin object recognition

Where is it? Where’s Waldo?Where is it? Where’s Waldo?

?

5

Two visual cortical Two visual cortical pathwayspathways

These problems are reflected in the These problems are reflected in the organization of the visual systemorganization of the visual system

Ventral “What” pathwayVentral “What” pathway Inferior longitudinal fasciculusInferior longitudinal fasciculus

Dorsal “Where” pathwayDorsal “Where” pathway Superior longitudinal Superior longitudinal

fasciculusfasciculus

6

Dissociation of what and Dissociation of what and where in the monkeywhere in the monkey

Landmark and object discrimination Landmark and object discrimination task (Pohl, 1973)task (Pohl, 1973)

Parietal lobeParietal lobe WhereWhere

Temporal lobeTemporal lobe WhatWhat

Landmark Object discrimination

7

Neuroimaging evidence Neuroimaging evidence for “what” and “where:for “what” and “where:

Attend to change in objects or Attend to change in objects or locationslocations

ObjectsObjects Occipito-temporal Occipito-temporal

LocationsLocations Posterior parietalPosterior parietal

Same objectsDifferent location

8

Ventral “what” pathwayVentral “what” pathway

V4: Isoluminant colorV4: Isoluminant color Lingual, fusiform gyrusLingual, fusiform gyrus

MT (V5): MotionMT (V5): Motion Middle temporal gyrusMiddle temporal gyrus

9

Neuropsychological Neuropsychological evidence:evidence:

Color (V4) and Motion Color (V4) and Motion (MT)(MT) Fractionation of perception following Fractionation of perception following

cortical lesionscortical lesions AchromatopsiaAchromatopsia

Loss of colour visionLoss of colour vision AkinetopsiaAkinetopsia

Loss of motionLoss of motion

10

Higher-order “what” Higher-order “what” pathway characteristicspathway characteristics

Complex response profileComplex response profile Dissimilar to V1Dissimilar to V1 Not simple orientation, colour, motionNot simple orientation, colour, motion SelectivitySelectivity

Hands, Hands,

faces etcfaces etc

11

Disorders of higher-order Disorders of higher-order ventral visual pathwayventral visual pathway

Agnosia: “without knowledge”Agnosia: “without knowledge” Visual agnosia: vision w/out Visual agnosia: vision w/out

knowledgeknowledge Modality specific: Restricted to visionModality specific: Restricted to vision Not a memory disorderNot a memory disorder

Item can be recognized Item can be recognized through other modalitiesthrough other modalities Touch, sound, smellTouch, sound, smell

12

Higher-order cortex is Higher-order cortex is highly specialized: highly specialized:

ProsopagnosiaProsopagnosia Largely specific to facesLargely specific to faces Can distinguish between faces and Can distinguish between faces and

objectsobjects Difficulty in distinguishing between Difficulty in distinguishing between

facesfaces Facial identificationFacial identification

Within categoryAcross category

13

Is there a region of the Is there a region of the brain devoted to faces? brain devoted to faces?

Fusiform face area (FFA)Fusiform face area (FFA) Right middle fusiform gyrus Right middle fusiform gyrus

especially responsive to faces especially responsive to faces relative to other objectsrelative to other objects

FFA

14

Neural selectivity: Neural selectivity: Evolution vs experienceEvolution vs experience

FFA and other objectsFFA and other objects ““Greebles”Greebles” Train to recognize Train to recognize individualsindividuals Experts but not novices Experts but not novices activate FFAactivate FFA

Potentially not face Potentially not face specificspecific

Reflects both evolution Reflects both evolution and experience?and experience?

15

Object recognition: Object recognition: InvarianceInvariance

Recognize object Recognize object despite differences in:despite differences in: Size, orientation, Size, orientation,

viewpoint, lighting, viewpoint, lighting, colour, locationcolour, location

Inferotemporal cortex Inferotemporal cortex (IT)(IT) Anterior ventral streamAnterior ventral stream Invariance in neural Invariance in neural

responseresponse

16

Evidence for constancy: Evidence for constancy: Lateral occipital complex Lateral occipital complex

(LOC)(LOC) Likely locus of object constancyLikely locus of object constancy Reduction in fMRI response w/ repetitionReduction in fMRI response w/ repetition InvarianceInvariance

Size, location,viewpoint, illumination, Size, location,viewpoint, illumination, occlusionocclusion No effect of occlusion

17

How does cortex How does cortex represent all objects?represent all objects?

Specificity codingSpecificity coding Grandmother cellsGrandmother cells Every orientationEvery orientation Every colorEvery color

Distributed codingDistributed coding ““Coarse” coding across Coarse” coding across

neuronsneurons CombinatorialCombinatorial

ColorColor 3 cone types3 cone types

FormForm Shape columns in ITShape columns in IT GeonsGeons

Geons

18

Ventral stream neurons Ventral stream neurons and consciousnessand consciousness

Binocular rivalryBinocular rivalry

fMRI evidence: FFA fMRI evidence: FFA and awarenessand awareness FFA turns on when FFA turns on when

aware of facesaware of faces FFA turns off when FFA turns off when

unaware of facesunaware of faces

Ventral stream Ventral stream representations representations support support consciousnessconsciousness

FFA PPA

19

Dorsal pathway: ActionDorsal pathway: Action Double dissociationDouble dissociation

Agnosia vs. optic ataxiaAgnosia vs. optic ataxia Apperceptive AgnosiaApperceptive Agnosia

Ventral stream damageVentral stream damage Impaired perceptionImpaired perception Intact actionIntact action

Appropriate reachingAppropriate reaching graspinggrasping

Optic ataxiaOptic ataxia Dorsal stream damageDorsal stream damage Intact perceptionIntact perception Impaired actionImpaired action

20

Break! Break! 10 minutes please10 minutes please

21

How does it all come How does it all come together? The binding together? The binding

problemproblem Division of labour: Division of labour: Parallel processingParallel processing Colour, shape, motion, Colour, shape, motion,

depth, locationdepth, location All in separate regionsAll in separate regions

How bound together?How bound together? Unified perceptionUnified perception Not separate featuresNot separate features

22

Integration (binding) Integration (binding) across feature mapsacross feature maps

Synthesis requires attention—allows Synthesis requires attention—allows coherence across feature maps: coherence across feature maps: ObjectsObjects

23

Attention is mental glueAttention is mental glue

Allows features to Allows features to stick togetherstick together

Without which, Without which, perception falls perception falls apartapart

No coherent No coherent perception of worldperception of world

24

What is attention?What is attention? Everybody knows what attention is. It is Everybody knows what attention is. It is

taking possession by the mindtaking possession by the mind, in clear and , in clear and vivid form, of vivid form, of one out of what seem several one out of what seem several simultaneously possible objectssimultaneously possible objects or trains of or trains of thought. Focalization, concentration of thought. Focalization, concentration of consciousness are of its essence. consciousness are of its essence. It implies It implies withdrawal from somethings in order to withdrawal from somethings in order to deal effectively with othersdeal effectively with others, and is a , and is a condition which condition which has a real opposite in the has a real opposite in the confused, dazed scatterbrain stateconfused, dazed scatterbrain state …. ….

William James (1890)William James (1890)

25

What is attention?What is attention? ““taking possession of the mind”taking possession of the mind”

Control of the focus of attentionControl of the focus of attention ““one out of what seem several simultaneously one out of what seem several simultaneously

possible objects”possible objects” Inability attending to multiple things at onceInability attending to multiple things at once

““It implies withdrawal from somethings in It implies withdrawal from somethings in order to deal effectively with othersorder to deal effectively with others”” ““Paying” attention comes with a costPaying” attention comes with a cost It has limited capacities that must be sharedIt has limited capacities that must be shared

““has a real opposite in the confused, dazed has a real opposite in the confused, dazed scatterbrain state”scatterbrain state” Attention is the glue that keeps perception togetherAttention is the glue that keeps perception together

26

Pay attention!Pay attention!

QuickTime™ and aCinepak decompressor

are needed to see this picture.

27

Visual experience: A Visual experience: A grand perceptual illusiongrand perceptual illusion Rich and complex?Rich and complex? Sorry folks, its an illusionSorry folks, its an illusion We don’t “see” as much as we believeWe don’t “see” as much as we believe

Don’t notice big changes in our environmentDon’t notice big changes in our environment Change blindnessChange blindness

We fill-in our experience to make it coherentWe fill-in our experience to make it coherent Little persistence of experience from one Little persistence of experience from one

moment to the nextmoment to the next Seems coherentSeems coherent

28

Selective attentionSelective attention DefinitionDefinition

Process relevant and ignore irrelevantProcess relevant and ignore irrelevant Why do we need selective Why do we need selective

attention?attention? Can’t remember/processes everythingCan’t remember/processes everything

Can be independent from eye Can be independent from eye movementsmovements Helmholtz (1894)Helmholtz (1894)

Fixate eyesFixate eyes Brief flashBrief flash Can pick what to perceiveCan pick what to perceive

Perception is not fixedPerception is not fixed Not just what your eyes doNot just what your eyes do Perceptual free willPerceptual free will

Can choose what to perceiveCan choose what to perceive

29

Attention: Feature Integration Attention: Feature Integration TheoryTheory

Attention needed to glue Attention needed to glue features togetherfeatures together

Feature vs. conjunction Feature vs. conjunction visual search visual search (Treisman)(Treisman) Parallel (“Pop out”)Parallel (“Pop out”)

E.g., Color, orientationE.g., Color, orientation Requires little attention Requires little attention

Serial Serial Color and orientationColor and orientation Requires focal attentionRequires focal attention Need to move attention aroundNeed to move attention around

Parallel Cereal

30

Attention is required for Attention is required for binding across feature binding across feature

mapsmaps Need attentional Need attentional beambeam

Without attention?Without attention? Illusory conjunctionsIllusory conjunctions Unbound featuresUnbound features

RecombinedRecombined Illusory perceptionIllusory perception

Need attention to glue Need attention to glue features togetherfeatures together

Otherwise fall apartOtherwise fall apart

Report seeing Yellow Square and purple triangle

31

Perceptual primitives:Perceptual primitives:What makes a “feature”?What makes a “feature”?

Perceptual primitivesPerceptual primitives Building blocks of perceptionBuilding blocks of perception Cortical feature mapsCortical feature maps

Luminance, orientation, color,Luminance, orientation, color,

Motion, depthMotion, depth

Higher-order objectsHigher-order objects Synthesis of primitivesSynthesis of primitives Objects defined by Objects defined by

conjunctions of primitivesconjunctions of primitives Need attention to bond into Need attention to bond into

compoundscompounds

Share primitives

Unique primitives

Molecules of perception?

Pop-out No pop-out

32

Constructivism vs Gestalt Constructivism vs Gestalt approaches to perceptionapproaches to perception StructuralismStructuralism

Perception is created by adding elements togetherPerception is created by adding elements together Perception from the bottom upPerception from the bottom up Like building a house Like building a house

Gestalt approachesGestalt approaches Top-down perceptual organizationTop-down perceptual organization Perceptual inferences: Best GuessesPerceptual inferences: Best Guesses ““Different than sum of its parts”Different than sum of its parts” Not like building a houseNot like building a house

Don’t want best guess, want correct answerDon’t want best guess, want correct answer Don’t want illusory foundation or beams!Don’t want illusory foundation or beams!

33

Constructivism vs Gestalt Constructivism vs Gestalt approaches to perceptionapproaches to perception Perception is Perception is

much more than much more than what is projected what is projected on retinaon retina

StructuralismStructuralism inputs determine inputs determine

perceptionperception GestaltGestalt

Active role of Active role of perceiverperceiver

Same input, many different kinds of perceptionSame input, many different kinds of perception

34

Gestalt principles of Gestalt principles of perceptual organizationperceptual organization

Best guesses about Best guesses about worldworld HeuristicsHeuristics ““Rules of thumb”Rules of thumb”

Visual system is problem Visual system is problem solving for ussolving for us Visual intelligenceVisual intelligence

Takes into account Takes into account probability of occurrenceprobability of occurrence

What is likely vs notWhat is likely vs not

Likely

Unlikely

35

Some of the Some of the principles/heuristicsprinciples/heuristics

SimilaritySimilarity Good figure/simplicityGood figure/simplicity

Good continuationGood continuationAll have multiple interpretations but we all tend to agree on one

36

Visual “guessing”: Visual “guessing”: Shape from shadingShape from shading

Sensitivity to perceptual invarianceSensitivity to perceptual invariance Light from above, not from belowLight from above, not from below

Square recessed

Square raised

37

Figure-ground Figure-ground segregationsegregation

What’s an object?What’s an object? What’s What’s

background?background?

Mental imposition Mental imposition of depth of depth OcclusionOcclusion Infer continuation of Infer continuation of

background background underneath objectunderneath object

Best guessBest guess Unlikely share same Unlikely share same

contourscontours

38

Figure-ground Figure-ground segregationsegregation

Factors that influence itFactors that influence it SymmetrySymmetry

DevelopmentDevelopment BeautyBeauty

MeaningMeaning See arrows?See arrows?

EmotionEmotion Perceptual autismPerceptual autism Pair faces w/ shockPair faces w/ shock Bias perception toward Bias perception toward

vasevase

Black White

39

Chicken or the egg?Chicken or the egg? Meaning processed before or after figure-Meaning processed before or after figure-

ground segregation?ground segregation?

Top-down perceptual organization over-Top-down perceptual organization over-rides initial processing rides initial processing Only aware of the final products of this processOnly aware of the final products of this process

40

Substantial feedback Substantial feedback onto early visual onto early visual

processingprocessing Higher-order cortex Higher-order cortex influences lower-influences lower-order cortexorder cortex

More feedback than More feedback than feedforward feedforward connectionsconnections

Has delayed influenceHas delayed influence Neural processing is Neural processing is

dynamicdynamic Need to revise classic Need to revise classic

definition of receptive definition of receptive fieldfield

Greater # of feedback connections applies to early cortex as well

41

Meaning and perceptual Meaning and perceptual primitivesprimitives

Common fateCommon fate Can perceive objects Can perceive objects

based on motionbased on motion

Higher-order Higher-order meaning influences meaning influences lower-order lower-order perceptual primitivesperceptual primitives E.g. motion E.g. motion

42

Meaning influences Meaning influences perceptual primitivesperceptual primitives

Structure from motionStructure from motion Can perceive objects Can perceive objects

based on motionbased on motion

Higher-order meaning Higher-order meaning influences lower-order influences lower-order perceptual primitivesperceptual primitives E.g. motion E.g. motion

43

The EndThe End