Introduction to Mental Imagery Psychology 355: Cognitive Psychology Instructor: John Miyamoto 05/21 /2015: Lecture 08-4 This Powerpoint presentation may

Introduction to Mental Imagery

Psychology 355: Cognitive Psychology

Instructor: John Miyamoto

05/21/2015: Lecture 08-4

This Powerpoint presentation may contain macros that were used to create the slides. The macros aren’t needed to view the slides. If necessary, you can disable the macros without any change to the presentation.

Outline

• The Imagery Debate - Is Mental Imagery an Epiphenomenon?

• Evidence for the functional role of mental imagery

• Neuropsychological evidence for the similarity between perception and imagery

Psych 355, Miyamoto, Spr '15 2Mental Imagery - Definition

3Psych 355, Miyamoto, Spr '15

Visual Images – What Are They?

• Mental Imagery: Experiencing a sensory impression in the absence

of sensory inputo Visual imagery: “seeing” in the absence of a visual stimulus

Table Showing the Pro and Con View of the Functional Importance of Imagery


The Debate Over Mental Imagery

Same Slide with Emphasis Rectangles

Pro-Mental Imagery Anti-Mental Imagery

Mental images play a functional role in human cognition – they play an important role in memory, problem solving and reasoning.

Advocated by Stephen Kosslyn

Mental images are epiphenomenal;

Mental images accompany real mechanism but plays no functional role.

Cognitive theory does not need a theory of mental images.

Advocated by Zenon Pylyshyn

Human cognition makes use of mental images and propositional representations.

All cognitive representations are propositional representation.


The Debate Over Mental Imagery

Review: Basic Pattern of Argument in the Imagery Debate

Pro-Mental Imagery Anti-Mental Imagery

Mental images play a functional role in human cognition – they play an important role in memory, problem solving and reasoning.

Advocated by Stephen Kosslyn

Mental images are epiphenomenal;

Mental images accompany real mechanism but plays no functional role.

Cognitive theory does not need a theory of mental images.

Advocated by Zenon Pylyshyn

Human cognition makes use of mental images and propositional representations.

All cognitive representations are propositional representation.

6

Basic Approach to Arguing for the Functional Role of Mental Imagery

• Find behavioral or neuropsychological evidence that is ...♦ easy to explain if we assume that humans possess and use perception-based

representations, but ...

♦ hard to explain if we assume that we have only propositional representations.

• In general, these arguments are not air tight, but they can be strong and convincing.

♦ Example of Ptolmaic circles in astronomy

• Related Hypothesis: Humans use analog representations. I.e., Humans use representations that are analogous toworking with real physical objects.

Psych 355, Miyamoto, Spr '15 Review: Shepard's Mental Rotation Study


Shepard's Mental Rotation Experiments

• TASK: As quickly as possible, decide whether the two figures

have the same shape or different shapes.

• Mental rotation experiments: Influential argument for the

importance of mental imagery in cognitive processes.

Review: Mental Rotation Experiment - Results


Mental Rotation - Results

• Response time for "identical" figures is a linear (straight-line)

function of the angle of rotation between the figures.o Result is easy to explain if subjects are rotating a mental image. o Result is hard to explain if mental representation is exclusively propositional.

Image Scanning – Kosslyn's Experiment with Island Map

2-D Rotation 3-D Rotation

Angle of Rotation (Degrees)

Mea

n R

eact

ion

Tim

e (s

ec.)

Mea

n R

eact

ion

Tim

e (s

ec.)


Distance (cm)

Res

pon

se T

ime

(sec

)

Mental Scanning (Image Scanning)

• Figure 10.4: Image scanning is discussed in Goldstein, pp. 273 - 274.

• Subjects study the map of the island. Then take it away.

• Using only the mental image of the island, imagine a dot moving

from one point to another, e.g., from the mountain to the tree. Push

a button when you are done.

Zooming In or Zooming Out on a Mental Image


Zooming In and Zooming Out on a Mental Image

Top: Imagine a rabbit next to an elephant.

Yes or No: Does the rabbit have whiskers?

Bottom: Imagine a rabbit next to a fly.


Same Slide with Results Added


Zooming In and Zooming Out on a Mental Image

Top: Imagine a rabbit next to an elephant.


Bottom: Imagine a rabbit next to a fly.


RT result is hard to explain if you assume that

information is retrieved from a semantic

network without imagery.

Can Imagery Prime Perception

RT = 2,020 ms slower

RT = 1,870 ms faster


Can Imagery Prime Perception?

• Priming – Stimulus A can prime Stimulus B if A is similar to B.

Why? Theoretical reason is that A and B activate similar brain areas

or processes, so the recent activation by A facilitates the processing

of Stimulus B.

• Question: Can imagining a visual representation prime the

perception of a real visual stimulus?

Farah, M. J. (1985). Psychophysical evidence for a shared representational

medium for mental images and percepts. Journal of Experimental Psychology:

General, 114, 91-103.

Farah's Demonstration that Imagery Can Prime Perception


Images Can Act as Primes for Real Visual Displays

Step (a): Create mental image of H

or T while staring at a blank

screen.

Step (b): After forming a good

image, subject presses a button

that causes 2 screens to be

displayed one after the other. One

screen has an H or a T, the other

screen is blank.

Task: Say whether the letter was on

the 1st or the 2nd screen.

H

H

"2""1"

"2""1"

Will Priming Occur?


Images Can Act as Primes for Real Visual Displays

Will priming occur?

Will these responses be slower?

Will these responses be faster?H

HResults of Farah's Experiment


% C

orre

ct D

etec

tion

Results: Images Act as Primes for Real Visual Displays

Results:

When mental image is same

as target, percent correct

is higher.

Interpretation:

Forming the mental image

requires similar brain activity

to actual perception. Therefore

forming the mental image

primes the perception of the

target.

Conclusion: Imagery Plays a Functional Role in Cognition


Yes – Mental Imagery Plays a Functional Role in Cognition

• Lots of evidence –o Image scanning (RT pattern consistent with mental image)o Zooming in or out with image (RT pattern consistent with mental image)o Image can prime detection task

--------------------------------------------------------------------------------------------o Evidence is easy to explain if we postulate that people use mental imagery; o Evidence is hard to explain if we claim that people only use propositional

representations.

• Next Question:

How similar are perception and imagery?

END

Psych 355, Miyamoto, Spr '14 17

Neuropsychological Evidence for Similarity Between Perception & Mental Imagery

• Neural response is similar when perceiving an object or

imagining the object. o Single cell studies - neurons that respond to perceiving or imagining an object.

o fMRI studies - similar brain activity when perceiving an object or imagining

the object. o Transcranial magnetic stimulation (TMS) has similar effect (slower response)

on tasks based on perception and on imagery.

Single-Cell Evidence for Similarity Between Perception & Imagery


Single-Cell Studies of Perception & Imagery

• Kreiman, Koch & Fried (2000)

Single-cell recording in

medial temporal lobe

as precursor to surgery

for epilepsy.

• Cell is sensitive to

perception and

mental image

of baseballs, but

not to faces.

Same Slide with Emphasis on Face Stimuli

Perceivethe

Baseball

Imaginethe

Baseball







for epilepsy.


perception and

mental image

of baseballs, but

not to faces.

Same Slide with Final Comment re Selectivity of Other Individual Cells

Perceivethe FaceImaginethe Face







for epilepsy.


perception and

mental image

of baseballs, but

not to faces.

• Also found other cells that were selective for perception & imagery

of animals, famous people, and food.

Similarity in fMRI Activation During Perception & Imagery

fMRI Studies of Perception and Imagery

• Perception of objects and imagining an object produces similar activation in visual cortex.


Goldstein Figure 10.13

Results are from: Le Bihan, D., Turner, R., Zeffiro, T. A., Cuenod, A., Jezzard, P., & Bonnerdot, V. (1993). Activation of human primary visual cortex during visual recall: A magnetic resonance imaging study. Proceedings of the National Academy of Sciences, USA, 90, 11802–11805.

O'Craven & Kanwisher Study - Better Evidence of the Same Type

P 355, Miyamoto, Winter '09 22

Location of the fusiform face area and parahippocampal gyrus

• Graphic from the article: Haynes, J-D., & Rees, G. (2006).

Decoding mental states from brain activity in humans. Nature Reviews Neuroscience, 7, 523-534.

Parahippocampal place area (PPA)

and

Fusiform face area (FFA)

Diagram of BrainFrom the Side Facing Left

Next: fMRI Results for Face & Place Recognition

http://www.nature.com/nrn/journal/v7/n7/full/nrn1931.html


fMRI Study of Face and Place Perception/Imagery

FFA: Fusiform face area.

Specialized for faces.

PPA: Parahippocampal

place area. Specialized

for representing location

info. Subjects either view or imagine a face or place .

Perception Imagine

% S

igna

l Cha

nge

% S

igna

l Cha

nge

Next: Look at Just the Upper Left Quadrant of this Graph








info. Subjects view a face or place . Face stimulus activates FFA; place stimulus does not.

Perception Imagine

% S

igna

l Cha

nge

% S

igna

l Cha

nge

Next: Look at Just the Lower Left Quadrant of this Graph





PPA: Parahippocampal place

area. Specialized for

representing location info. Perception Imagine

% S

igna

l Cha

nge

% S

igna

l Cha

nge

Subjects view a face or place . Place stimulus activates PPA; face stimulus does not.

Next: Discuss Results for the Left Half of this Graph








info. Subjects view a face or place . Face stimulus activates FFA; place stimulus does not.

Perception Imagine

% S

igna

l Cha

nge

% S

igna

l Cha

nge

Next: Look at Just the Lower Left Quadrant of this Graph








% S

igna

l Cha

nge

% S

igna

l Cha

nge

Subjects view a face or place . Place stimulus activates PPA; face stimulus does not.

Next: Discuss Results for the Left Half of this Graph








% S

igna

l Cha

nge

% S

igna

l Cha

nge

Face and place stimuli have opposite effects on FFA and PPA.

Double dissociation when perceiving faces or places.

Next: Discuss the Right Half of the Graph








% S

igna

l Cha

nge

% S

igna

l Cha

nge

Face and place stimuli have opposite effects on FFA and PPA.

Double dissociation when imagining faces or places.

Do Neurological Impairments Have Equivalent Effects on Perception & Imagination?


Do Neurological Impairments Have Similar Effects on Perception and Imagery?

• Kosslyn's TMS study supports this hypothesis.(See Goldstein, p. 287 and Figure 10.17 on p. 288)

• What about permanent neurological impairments due

to lesions or strokes?

Reminder re Hemispatial Neglect – Transition to HemiSpatial Neglect in Imagery


Hemispatial Neglect (Goldstein calls this "unilateral neglect")T

his

slid

e is

bas

ed o

n in

stru

ctio

nal m

ater

ial t

hat w

as d

ownl

oade

d fr

om th

e P

ears

on P

ubli

sher

sw

ebsi

te (

http

://v

ig.p

renh

all.c

om)

for

Sm

ith

& K

ossl

yn (

2006

; IS

BN

978

0131

8250

86).

The patient’s copy in the right column neglects the left side of the visual field (opposite to the side of brain damage). Unilateral Neglect in Perception & Images

Hemispatial Neglect(Unilateral Neglect): A deficit of attention in which one entire half of a visual scene is simply ignored.

The cause of unilateral neglect is often a stroke that has interrupted the flow of blood to the right parietal lobe.

Figure to the right: Patient’s copy of an image (model) shows systematic deficits.

Psych 355, Miyamoto, Spr '14 32Transition to Question: Are Perception & Imagery Always Similar?

Left Unilateral Neglect in Perception & Images

Bisiach, E., & Luzzatti, C. (1978). Unilateral neglect of representational

space. Cortex, 14, 129-133.

• Patient with left unilateral

neglect was asked to

imagine himself standing at

one end of the Piazza del

Duomo in Milan.

• Patient neglected left side

of the visual image (in his

description) just as he

neglected the left side in

actual perception.


Are Perception & Imagery Always Similar?

Due to brain injury, Patient CK has

visual agnosia (inability to recognize

objects)

Figure (a) – incorrect identificationso Dart labeled “feather duster”o Tennis racquet labeled “fencer’s

mask”o Asparagus labeled “rose twig with

thorns”

Figure (b) – drawings from memoryo Outline of Englando Guitar

Figure (b) – If you show CK his

drawings at a later time, he cannot

recognize (label) what they are.Dissociations Between Perception & Imagery

Figure 10.19 (p. 284). (a) Pictures incorrectly labeled by CK who had visual agnosia. (b) Drawings from memory by CK. From study by Behrmann, Moscovitch, & Winocur (1994).

(a)

(b)


Dissociations Between Imagery & Perception

Same Slide without Emphasis Rectangles

Case Perception Imagery

Guariglia (1993) OK Unilateral neglect

Farah et al. (1993):Patient RM

OK: Recognizes objects & can draw pictures of objects

POOR: Can't draw objects from memory or answer questions that require mental imagery

Behrmann et al. (1994): Patient CK

POOR: Visual agnosia (can't recognize objects)

OK: Can draw objects from memory


Dissociations Between Imagery & Perception

Case Perception Imagery

Guariglia (1993) OK Unilateral neglect

Farah et al. (1993):Patient RM

OK: Recognizes objects & can draw pictures of objects

POOR: Can't draw objects from memory or answer questions that require mental imagery

Behrmann et al. (1994): Patient CK

POOR: Visual agnosia (can't recognize objects)

OK: Can draw objects from memory

Diagram Showing Bottom Up & Top Down Processing of Images


When Are Perception & Imagery Similar? When Are They Different?

• Behrmann et al. (1994) point out that perception is more bottom up;

Imagery is more top down.

Same Slide with Explanation of Behrmann’s Hypothesis


When Are Perception & Imagery Similar? When Are They Different?

• Hypothesis: o CK's injury blocks the bottom up input for object perception.o RM injury blocks the top down construction of a mental image.

Conclusion - END

Conclusion

• Mental manipulation of images is similar to perception of scenes as they undergo the analogous physical alterations.

• Perception and imagery engage similar cognitive processes, but they are not perfectly equivalent.

Psych 355, Miyamoto, Spr '14 38END

Documents

Introduction to Mental Imagery Psychology 355: Cognitive Psychology Instructor: John Miyamoto 05/21 /2015: Lecture 08-4 This Powerpoint presentation may