Ch 5 1

Sensation & Perception

Ch. 5: Perceiving Objects and Scenes

© Takashi Yamauchi (Dept. of Psychology, Texas A&M University)

Main topicsThe challenge of object perception

Inverse projection problem

Gestalt psychology

Perceptual organization

Figure-ground segregation

Biederman’s Recognition by Components theory

Ch 5 2

Minsky’s estimation

• Why is it so difficult for the computer to recognize objects as we do?

• http://web.media.mit.edu/~minsky/

Ch 5 3

What AI can do

• 50th anniversary in 2006 (from Rodney Brooks)

– The most advanced AI can barely beat a 2-year-old child in object recognition.

Ch 5 4

Why AI betrayed?

• Recognizing objects– is much more than just detecting lines, colors,

or shapes.

– requires a large amount of background knowledge.

both implicit and explicit knowledge

Ch 5 5

The challenge of object perception

• The stimulus on the receptors is ambiguous

– Inverse projection problem: an image on the retina can be caused by an infinite number of objects

• Objects can be hidden or blurred

– Occlusions are common in the environment

Ch 5 6

Figure 5.3 The principle behind the inverse projection problem. The small square stimulus creates a square image on the retina. However, this image could also have been created by the other two shapes and many other stimuli. This is why we say that the image on the retina is ambiguous.

Ch 5 7

Fig. 5-4, p. 96

Ch 5 8

The Challenge of Object Perception - continued

• Objects look different from different viewpoints– Viewpoint invariance: the ability to recognize

an object regardless of the viewpoint

• The reasons for changes in lightness and darkness in the environment can be unclear

Ch 5 9

Fig. 5-6a, p. 96

Ch 5 10

Fig. 5-8, p. 97

Ch 5 11

Part + Part = Whole?

• Gestalt psychology

Ch 5 12

Group discussion

• Given the pictures shown in the slides, write down everything you see

Ch 5 13

Fig. 1.

Ch 5 14

Fig. 2

Ch 5 15

Fig. 3

Ch 5 16

Fig 4

Ch 5 17

Fig. 5

Ch 5 18

Fig. 7aFig. 7b

Fig. 7c

Ch 5 19

Three Musicians. P. Picasso

Ch 5 20

Woman with a book: P. Picasso

Ch 5 21

La lectrice: P. Picasso

Ch 5 22

Music. H. Matisse

Ch 5 23

Dance. H. Matisse

Ch 5 24

View of Toledo: El Greco

Ch 5 25

Baptism of Christ: El Greco

Ch 5 26

The Burial of Count Orgaz: El Greco

Ch 5 27

Metamorphosis of narcissus: S. Dali

Ch 5 28

War: S. Dali

Ch 5 29

Mont Sainte-Victoire (Cezanne)

Ch 5 30

Hokusai

Hiroshige

Ch 5 31

What do these pictures tell us?

• Perceiving things is a lot more than just detecting lines, colors, motion, so on.

• We see– Face, body, hands, apples, mountains, legs,

tables, houses, streets,……

– motion, movement, action, rhythm, expansion, contraction, upheaval,

Ch 5 32

How come?

• Your eyes receive 2-D flat information.

• All you got are activation of neurons.

• How come you perceive faces, trees, apples…?

Ch 5 33

Why & how do we see those?

• Perception involves– Organization

• We see a woman sitting on a sofa because we organize visual information in a certain way.

Ch 5 34

Ch 5 35

How do we organize visual information?

• Are there any principles behind it?

• What principles do we follow?

• Gestalt laws of organization

Ch 5 36

Tell me what you see.

Ch 5 37

Tell me what you see.

Ch 5 38

Perception is not just detection

• Perception is not just about detecting color or shape.

• Perception is about organizing visual information.

• How do we organize visual information?

Ch 5 39

Or when do we fail to organize visual information?

Ch 5 40

Gestalt psychology

• “Gestalt” means “whole.”

• Organizational principles:– Similarity– Proximity– Continuity

• And more (see the textbook)

Ch 5 41

Law of similarity

Similar things are put together

Ch 5 42

Ch 5 43

Items with similar colors are put together

Ch 5 44

Do you see 3 musicians? Why? Why not?

Do you see a table?

Do you see a sofa?

Do you see a blouse and a blue skirt?

Ch 5 45

Ch 5 46

Law of proximity

• Things that are close to each other are put together.

Ch 5 47

There are two separate worlds --- this world and that world?

Ch 5 48

Ch 5 49

Law of good continuity

We tend to put things together when they show nice continuity.

Ch 5 50

Do you see a spinal motion? This picture is called “Dance.” Do you see why?

Ch 5 51

Do you feel an upward motion or a downward motion?

Ch 5 52

Ch 5 53

• There are many many more ways to organize visual information (see the textbook).

Ch 5 54

Perceptual Segregation

• Figure-ground segregation– determining what part of environment is the figure

so that it “stands out” from the background

Ch 5 55

Figure-Ground segregation

• Distinguishing a figure from the ground.• We know the difference between the figure and the ground

Ch 5 56

How does the perceptual system distinguish the figure from the

ground?

Ch 5 57

Which one is the figure and which one is the ground?

Depending on the locus of attention, the figure and the ground switch rapidly.

Attention plays some role in determining the figure and the ground.

Ch 5 58

The figure represents “some thing.”

The contours belong to the figure rather than to the ground.

Ch 5 59

Which one if the figure and which is the ground?

This is easy.

The figure tends to have solid and continuous surface.

Ch 5 60

Which one is the figure and which one is the ground?

Symmetric items tend to be seen as a figure.

Ch 5 61

What did you see? Some arrows and what?

Do you see something else?

Meaningful items are seen as figure

Ch 5 62

Ch 5 63

Ch 5 64

What does this tell?

• Figure-ground segregation is influenced by our knowledge about the world in general.

– What things are, how they look like, – Two things can’t occupy the same space

simultaneously.

Ch 5 65

What does this tell?

• Gestalt laws of organization– Similarity, continuity, proximity,..

• Figure-ground segregation

• Perceptual organization is based on our world knowledge (what we know about the world).

Ch 5 66

Ch 5 67

Ch 5 68

Ch 5 69

Ch 5 70

Ch 5 71

Ch 5 72

Ch 5 73

Ch 5 74

Input & output to and from LGN

Ch 5 75

Any idea?

• Gestalt laws of organization

– Similarity, continuity, proximity,..

• Figure-ground segregation

• What do they tell you?

Some top-down processes are going on.

Ch 5 76

Ch 5 77

Modern research on object perception

• The perceptual system is tuned to capture the regularities in the environment.

• Biederman’s Recognition by Components model

Ch 5 78

How do we store objects in the brain?

• See an object.• Put it into your brain• Take out the one you stored

earlier.• compare it to what you

saw.

How do you store what you saw before?

Ch 5 79

You see billions of objects every day

• Does your brain have enough space to store them?

• Storing everything is ineffective (very expensive).

• So, you got to do something different.

• Better way to store things?

Ch 5 80

How do you do that?

Ch 5 81

Biederman’s Recognition by Components

• You just need about 36 components to represent millions of different objects.

Ch 5 82

Biederman’s Recognition by Components (RBC)

• Objects are described and stored by simple geometric components (geons).

• There are about 36 geons.

• To represent objects, we use geons and their arrangements.

Ch 5 83

Geons

Ch 5 84

Ch 5 85

Combinations of geons

Ch 5 86

Combinations of geons

Combining 4 geons can yield more than 1 million objects. (36x36x36x36)

Ch 5 87

Ch 5 88

Ch 5 89

Ch 5 90

Ch 5 91

Biederman & Ju (1988) Cognitive Psychology, 20, 38-64• Do people need information more than

geons provide?

• Contrast subjects’ performance for object recognition when two types of pictures (actual pictures and schematic pictures) are shown one by one.

Ch 5 92

Schematic pictures depicted by geons

Ch 5 93

Actual pictures

Ch 5 94

Experiment:

• The subject indicated the name of the object shown on the screen.

• In one case, an actual picture of the object was shown.

• In the other case, a schematic illustration of the object (depicted by geons) was shown.

Ch 5 95

Questions/design:

• Do subjects name the objects more quickly and accurately when actual pictures were shown or when schematic pictures (depicted by geons) were shown?

• Each picture (either actual/schematic) flashed on the computer screen only for 50ms, 60ms, or 400ms.

Ch 5 96

Why 50ms? Or 400ms?

• The task needs to be not too easy but not too difficult.

– Test college students’ math ability.

– test adding and subtracting?

– Test high school students’ math ability.

– Give quantum mechanics questions?

Ch 5 97

Results:

Error ratesPresentation 50(ms) 65(ms) 400 (ms)Line drawings 20 13 5(%)Photographs 35 9 3(%)

Response times (correct responses only)

Presentation 50(ms) 65(ms) 400 (ms)Line drawings 990 870 860Photographs 1010 875 850(ms)

Ch 5 98

Ch 5 99

Ch 5 100

The intelligence of human object perception

• Why are humans much better than computers at object recognition?

Ch 5 101

• Theory of unconscious inference

– Human object perception is like problem solving.

• We make an unconscious inference.

– Likelihood principle

• objects are perceived based on what is most likely to have caused the pattern.

– Humans have a vast array of knowledge (intelligence) that can disambiguate ambiguous stimuli.

Ch 5 102

Ch 5 103

Ch 5 104

Ch 5 105

Ch 5 106

Ch 5 107

Ch 5 108

Ch 5 109

Ch 5 110

Ch 5 111

Ch 5 112

Ch 5 113

Ch 5 114

Ch 5 115

Ch 5 116

Ch 5 117

Ch 5 118

Ch 5 119

Ch 5 120

Ch 5 121

Ch 5 122

Ch 5 123

Ch 5 124

Ch 5 125

Ch 5 126