1

Psy280: PerceptionPsy280: Perception

Prof. AndersonProf. AndersonDepartment of PsychologyDepartment of Psychology

Week 5Week 5

2

nn WhatWhat’’s the meaning of life? Not exactlys the meaning of life? Not exactly

nn Computational approach to visionComputational approach to visionnn David Marr (MIT)David Marr (MIT)

nn What is the purpose of vision?What is the purpose of vision?nn What are the problems it must solve?What are the problems it must solve?

Visual philosophersVisual philosophers

3

Computational problems inComputational problems inobject recognitionobject recognition

nn What is it?What is it?nn Object constancy: Variability in sensoryObject constancy: Variability in sensory

informationinformationnn Retinal positionRetinal positionnn Viewing positionViewing positionnn OcclusionOcclusionnn Lighting, colourLighting, colour

4

Computational problems inComputational problems inobject recognitionobject recognition

nn Where is it? WhereWhere is it? Where’’s Waldo?s Waldo?

?

5

Two visual cortical pathwaysTwo visual cortical pathwaysnn These problems are reflected in theThese problems are reflected in the

organization of the visual systemorganization of the visual systemnn Ventral Ventral ““WhatWhat”” pathway pathwaynn Inferior longitudinal fasciculusInferior longitudinal fasciculus

nn Dorsal Dorsal ““WhereWhere”” pathway pathwaynn Superior longitudinalSuperior longitudinalfasciculusfasciculus

6

Dissociation of what andDissociation of what andwhere in the monkeywhere in the monkey

nn Landmark and object discrimination taskLandmark and object discrimination task(Pohl, 1973)(Pohl, 1973)

nn Parietal lobeParietal lobenn WhereWhere

nn Temporal lobeTemporal lobenn WhatWhat

Landmark Object discrimination

7

Neuroimaging evidence forNeuroimaging evidence for““whatwhat”” and and ““where:where:

nn Attend to change in objects or locationsAttend to change in objects or locationsnn ObjectsObjectsnn Occipito-temporalOccipito-temporal

nn LocationsLocationsnn Posterior parietalPosterior parietal

Same objectsDifferent location

8

Ventral Ventral ““whatwhat”” pathway pathway

nn V4: Isoluminant colorV4: Isoluminant colornn Lingual, fusiform gyrusLingual, fusiform gyrus

nn MT (V5): MotionMT (V5): Motionnn Middle temporal gyrusMiddle temporal gyrus

9

Neuropsychological evidence:Neuropsychological evidence:Color (V4) and Motion (MT)Color (V4) and Motion (MT)

nn Fractionation of perception followingFractionation of perception followingcortical lesionscortical lesions

nn AchromatopsiaAchromatopsiann Loss of colour visionLoss of colour vision

nn AkinetopsiaAkinetopsiann Loss of motionLoss of motion

10

Higher-order Higher-order ““whatwhat”” pathway pathwaycharacteristicscharacteristics

nn Complex response profileComplex response profilenn Dissimilar to V1Dissimilar to V1nn Not simple orientation, colour, motionNot simple orientation, colour, motionnn SelectivitySelectivity

nn Hands,Hands,faces etcfaces etc

11

Disorders of higher-orderDisorders of higher-orderventral visual pathwayventral visual pathway

nn Agnosia: Agnosia: ““without knowledgewithout knowledge””nn Visual agnosia: vision w/out knowledgeVisual agnosia: vision w/out knowledgenn Modality specific: Restricted to visionModality specific: Restricted to visionnn Not a memory disorderNot a memory disorder

nn Item can be recognizedItem can be recognizedthrough other modalitiesthrough other modalitiesnn Touch, sound, smellTouch, sound, smell

12

Higher-order cortex is highlyHigher-order cortex is highlyspecialized: Prosopagnosiaspecialized: Prosopagnosia

nn Largely specific to facesLargely specific to facesnn Can distinguish between faces and objectsCan distinguish between faces and objectsnn Difficulty in distinguishing between facesDifficulty in distinguishing between facesnn Facial identificationFacial identification

Within categoryAcross category

13

Is there a region of the brainIs there a region of the braindevoted to faces?devoted to faces?

nn Fusiform face area (FFA)Fusiform face area (FFA)nn Right middle fusiform gyrus especiallyRight middle fusiform gyrus especially

responsive to faces relative to other objectsresponsive to faces relative to other objects

FFA

14

Neural selectivity:Neural selectivity:Evolution vs experienceEvolution vs experience

nn FFA and other objectsFFA and other objectsnn ““GreeblesGreebles””nn Train to recognizeTrain to recognizeindividualsindividualsnn Experts but not novicesExperts but not novicesactivate FFAactivate FFA

nn Potentially not facePotentially not facespecificspecific

nn Reflects both evolutionReflects both evolutionand experience?and experience?

15

Object recognition: InvarianceObject recognition: Invariancenn Recognize object despiteRecognize object despite

differences in:differences in:nn Size, orientation, viewpoint,Size, orientation, viewpoint,

lighting, colour, locationlighting, colour, locationnn Inferotemporal cortex (IT)Inferotemporal cortex (IT)

nn Anterior ventral streamAnterior ventral streamnn Invariance in neuralInvariance in neural

responseresponse

16

Evidence for constancy: LateralEvidence for constancy: Lateraloccipital complex (LOC)occipital complex (LOC)

nn Likely locus of object constancyLikely locus of object constancynn Reduction in fMRI response w/ repetitionReduction in fMRI response w/ repetitionnn InvarianceInvariance

nn Size, location,viewpoint, illumination, occlusionSize, location,viewpoint, illumination, occlusionNo effect of occlusion

17

How does cortex represent allHow does cortex represent allobjects?objects?

nn Specificity codingSpecificity codingnn Grandmother cellsGrandmother cellsnn Every orientationEvery orientationnn Every colorEvery color

nn Distributed codingDistributed codingnn ““CoarseCoarse”” coding across coding across

neuronsneuronsnn CombinatorialCombinatorial

nn ColorColornn 3 cone types3 cone types

nn FormFormnn Shape columns in ITShape columns in ITnn GeonsGeons

Geons

18

Ventral stream neurons andVentral stream neurons andconsciousnessconsciousness

nn Binocular rivalryBinocular rivalry

nn fMRI evidence: FFAfMRI evidence: FFAand awarenessand awarenessnn FFA turns on whenFFA turns on when

aware of facesaware of facesnn FFA turns off whenFFA turns off when

unaware of facesunaware of faces

nn Ventral streamVentral streamrepresentationsrepresentationssupport consciousnesssupport consciousness

FFA PPA

19

Dorsal pathway: ActionDorsal pathway: Actionnn Double dissociationDouble dissociation

nn Agnosia vs. optic ataxiaAgnosia vs. optic ataxiann Apperceptive AgnosiaApperceptive Agnosia

nn Ventral stream damageVentral stream damagenn Impaired perceptionImpaired perceptionnn Intact actionIntact action

nn Appropriate reachingAppropriate reachingnn graspinggrasping

nn Optic ataxiaOptic ataxiann Dorsal stream damageDorsal stream damagenn Intact perceptionIntact perceptionnn Impaired actionImpaired action

20

Break!Break!10 minutes please10 minutes please

21

How does it all come together?How does it all come together?The binding problemThe binding problem

nn Division of labour:Division of labour:Parallel processingParallel processingnn Colour, shape, motion,Colour, shape, motion,

depth, locationdepth, locationnn All in separate regionsAll in separate regions

nn How boundHow boundtogether?together?nn Unified perceptionUnified perceptionnn Not separate featuresNot separate features

22

Integration (binding) acrossIntegration (binding) acrossfeature mapsfeature maps

nn Synthesis requires attentionSynthesis requires attention——allowsallowscoherence across feature maps: Objectscoherence across feature maps: Objects

23

Attention is mental glueAttention is mental glue

nn Allows features to stickAllows features to sticktogethertogether

nn Without which,Without which,perception falls apartperception falls apart

nn No coherentNo coherentperception of worldperception of world

24

What is attention?What is attention?nn Everybody knows what attention is. It is Everybody knows what attention is. It is takingtaking

possession by the mindpossession by the mind, in clear and vivid form,, in clear and vivid form,of of one out of what seem several simultaneouslyone out of what seem several simultaneouslypossible objectspossible objects or trains of thought.or trains of thought.Focalization, concentration of consciousness areFocalization, concentration of consciousness areof its essence. of its essence. It implies withdrawal fromIt implies withdrawal fromsomethings in order to deal effectively withsomethings in order to deal effectively withothersothers, and is a condition which , and is a condition which has a realhas a realopposite in the confused, dazed scatterbrainopposite in the confused, dazed scatterbrainstatestate ……..

William James (1890)William James (1890)

25

What is attention?What is attention?nn ““taking possession of the mindtaking possession of the mind””

nn Control of the focus of attentionControl of the focus of attentionnn ““one out of what seem several simultaneouslyone out of what seem several simultaneously

possible objectspossible objects””nn Inability attending to multiple things at onceInability attending to multiple things at once

nn ““It implies withdrawal from somethings in orderIt implies withdrawal from somethings in orderto deal effectively with othersto deal effectively with others””nn ““PayingPaying”” attention comes with a cost attention comes with a costnn It has limited capacities that must be sharedIt has limited capacities that must be shared

nn ““has a real opposite in the confused, dazedhas a real opposite in the confused, dazedscatterbrain statescatterbrain state””nn Attention is the glue that keeps perception togetherAttention is the glue that keeps perception together

26

Pay attention!Pay attention!

27

Visual experience: A grandVisual experience: A grandperceptual illusionperceptual illusion

nn Rich and complex?Rich and complex?nn Sorry folks, its an illusionSorry folks, its an illusionnn We donWe don’’t t ““seesee”” as much as we believe as much as we believenn DonDon’’t notice big changes in our environmentt notice big changes in our environment

nn Change blindnessChange blindnessnn We fill-in our experience to make it coherentWe fill-in our experience to make it coherentnn Little persistence of experience from oneLittle persistence of experience from one

moment to the nextmoment to the nextnn Seems coherentSeems coherent

28

Selective attentionSelective attentionnn DefinitionDefinition

nn Process relevant and ignore irrelevantProcess relevant and ignore irrelevantnn Why do we need selectiveWhy do we need selective

attention?attention?nn CanCan’’t remember/processest remember/processes

everythingeverythingnn Can be independent from eyeCan be independent from eye

movementsmovementsnn Helmholtz (1894)Helmholtz (1894)

nn Fixate eyesFixate eyesnn Brief flashBrief flashnn Can pick what to perceiveCan pick what to perceive

nn Perception is not fixedPerception is not fixednn Not just what your eyes doNot just what your eyes donn Perceptual free willPerceptual free will

nn Can choose what to perceiveCan choose what to perceive

29

Attention: Feature Integration TheoryAttention: Feature Integration Theory

nn Attention needed to glueAttention needed to gluefeatures togetherfeatures together

nn Feature vs. conjunction visualFeature vs. conjunction visualsearch search (Treisman)(Treisman)nn Parallel (Parallel (““Pop outPop out””))

nn E.g., Color, orientationE.g., Color, orientationnn Requires little attentionRequires little attention

nn SerialSerialnn Color and orientationColor and orientationnn Requires focal attentionRequires focal attentionnn Need to move attention aroundNeed to move attention around

Parallel Cereal

30

Attention is required forAttention is required forbinding across feature mapsbinding across feature maps

nn Need attentional beamNeed attentional beamnn Without attention?Without attention?

nn Illusory conjunctionsIllusory conjunctionsnn Unbound featuresUnbound features

nn RecombinedRecombinednn Illusory perceptionIllusory perception

nn Need attention to glueNeed attention to gluefeatures togetherfeatures together

nn Otherwise fall apartOtherwise fall apart

Report seeing Yellow Squareand purple triangle

31

Perceptual primitives:Perceptual primitives:What makes a What makes a ““featurefeature””??

nn Perceptual primitivesPerceptual primitivesnn Building blocks of perceptionBuilding blocks of perceptionnn Cortical feature mapsCortical feature maps

nn Luminance, orientation, color,Luminance, orientation, color,Motion, depthMotion, depth

nn Higher-order objectsHigher-order objectsnn Synthesis of primitivesSynthesis of primitivesnn Objects defined byObjects defined byconjunctions of primitivesconjunctions of primitivesnn Need attention to bond intoNeed attention to bond into

compoundscompoundsShareprimitives

Uniqueprimitives

Molecules of perception?

Pop-out No pop-out

32

Constructivism vs GestaltConstructivism vs Gestaltapproaches to perceptionapproaches to perception

nn StructuralismStructuralismnn Perception is created by adding elements togetherPerception is created by adding elements togethernn Perception from the bottom upPerception from the bottom upnn Like building a houseLike building a house

nn Gestalt approachesGestalt approachesnn Top-down perceptual organizationTop-down perceptual organizationnn Perceptual inferences: Best GuessesPerceptual inferences: Best Guessesnn ““Different than sum of its partsDifferent than sum of its parts””nn Not like building a houseNot like building a house

nn DonDon’’t want best guess, want correct answert want best guess, want correct answernn DonDon’’t want illusory foundation or beams!t want illusory foundation or beams!

33

Constructivism vs GestaltConstructivism vs Gestaltapproaches to perceptionapproaches to perception

nn Perception is muchPerception is muchmore than what ismore than what isprojected on retinaprojected on retina

nn StructuralismStructuralismnn inputs determineinputs determine

perceptionperceptionnn GestaltGestaltnn Active role ofActive role of

perceiverperceivernn Same input, many different kinds of perceptionSame input, many different kinds of perception

34

Gestalt principles ofGestalt principles ofperceptual organizationperceptual organization

nn Best guesses about worldBest guesses about worldnn HeuristicsHeuristicsnn ““Rules of thumbRules of thumb””

nn Visual system is problemVisual system is problemsolving for ussolving for usnn Visual intelligenceVisual intelligence

nn Takes into accountTakes into accountprobability of occurrenceprobability of occurrence

nn What is likely vs notWhat is likely vs not

Likely

Unlikely

35

Some of theSome of theprinciples/heuristicsprinciples/heuristics

nn SimilaritySimilaritynn Good figure/simplicityGood figure/simplicity

nn Good continuationGood continuationAll have multipleinterpretations butwe all tend to agreeon one

36

Visual Visual ““guessingguessing””::Shape from shadingShape from shading

nn Sensitivity to perceptual invarianceSensitivity to perceptual invariancenn Light from above, not from belowLight from above, not from below

Squarerecessed

Squareraised

37

Figure-ground segregationFigure-ground segregationnn WhatWhat’’s an object?s an object?nn WhatWhat’’s background?s background?

nn Mental imposition ofMental imposition ofdepthdepthnn OcclusionOcclusionnn Infer continuation ofInfer continuation of

backgroundbackgroundunderneath objectunderneath object

nn Best guessBest guessnn Unlikely share sameUnlikely share same

contourscontours

38

Figure-ground segregationFigure-ground segregationnn Factors that influence itFactors that influence itnn SymmetrySymmetry

nn DevelopmentDevelopmentnn BeautyBeauty

nn MeaningMeaningnn See arrows?See arrows?

nn EmotionEmotionnn Perceptual autismPerceptual autismnn Pair faces w/ shockPair faces w/ shocknn Bias perception towardBias perception toward

vasevase

Black White

39

Chicken or the egg?Chicken or the egg?nn Meaning processed before or after figure-groundMeaning processed before or after figure-ground

segregation?segregation?

nn Top-down perceptual organization over-ridesTop-down perceptual organization over-ridesinitial processinginitial processingnn Only aware of the final products of this processOnly aware of the final products of this process

40

Substantial feedback ontoSubstantial feedback ontoearly visual processingearly visual processing

nn Higher-order cortexHigher-order cortexinfluences lower-orderinfluences lower-ordercortexcortex

nn More feedback thanMore feedback thanfeedforwardfeedforwardconnectionsconnections

nn Has delayed influenceHas delayed influencenn Neural processing isNeural processing is

dynamicdynamicnn Need to revise classicNeed to revise classic

definition of receptivedefinition of receptivefieldfield

Greater # offeedbackconnectionsapplies to earlycortex as well

41

Meaning and perceptualMeaning and perceptualprimitivesprimitives

nn Common fateCommon fatenn Can perceive objectsCan perceive objects

based on motionbased on motion

nn Higher-order meaningHigher-order meaninginfluences lower-orderinfluences lower-orderperceptual primitivesperceptual primitivesnn E.g. motionE.g. motion

42

Meaning influences perceptualMeaning influences perceptualprimitivesprimitives

nn Structure from motionStructure from motionnn Can perceive objectsCan perceive objects

based on motionbased on motion

nn Higher-order meaningHigher-order meaninginfluences lower-orderinfluences lower-orderperceptual primitivesperceptual primitivesnn E.g. motionE.g. motion

43

The EndThe End