Visually-induced Visually-induced auditory spatial adaptation auditory spatial adaptation

in monkeys and humansin monkeys and humans

Norbert Kopčo, I-Fan Lin, Barbara Shinn-Cunningham, Jennifer Groh

Center for Cognitive Neuroscience, Duke UniversityHearing Research Center, Boston University

Technical University, Košice, Slovakia

2Nov 6, 2007 SFN 07 San Diego

Introduction

Visual stimuli can affect the perception of sound location

e.g. the Ventriloquism Effect

Way to go

Red Sox! Way to go

Red Sox!

But does effect persist?

3Nov 6, 2007 SFN 07 San Diego

Introduction

Visual stimuli can affect the perception of sound location

e.g. the Ventriloquist Effect

Way to go

Red Sox!

But does effect persist?- barn owls: prism adaptation (Knudsen et al.)- monkeys: “ventriloquism aftereffect” (Woods and Recanzone, Curr. Biol. 2004)

4Nov 6, 2007 SFN 07 San Diego

GOALS

1. Ventriloquism “aftereffect” in saccade task, in monkeys and humans?1. - well-defined sensory-motor paradigm

2. - bridge to barn owl prism adaptation studies (on different time scale)

2. Reference frame of plasticity?1. - Visual, auditory, or oculomotor reference frame?

5Nov 6, 2007 SFN 07 San Diego

Methods

Basic idea:

1. Pre-adaptation baseline: Measure auditory saccade accuracy

2. Adaptation phase: Present combined visual-auditory stimuli, with visual location shifted

3. Compare auditory saccade accuracy pre- and post-adaptation

6Nov 6, 2007 SFN 07 San Diego

Methods

Initial experiment: Does it work?

Design:MonkeyPre-adaptation baseline – ~100 Auditory-only trialsAdaptation phase –

80% V-A stimuli, visual stimulus shifted 6 deg. Left or Right

20% Auditory-onlyCompare Auditory-only trials from adaptation phase to pre-

adaptation phase

Sounds: Loudspeakers

Visual stimuli: LEDs

7Nov 6, 2007 SFN 07 San Diego

RESULTS

8Nov 6, 2007 SFN 07 San Diego

RESULTS

9Nov 6, 2007 SFN 07 San Diego

RESULTS

10Nov 6, 2007 SFN 07 San Diego

RESULTS

11Nov 6, 2007 SFN 07 San Diego

Methods: reference frame

Eye-centered?

Head (ear) -centered?

Oculomotor?

?

?

12Nov 6, 2007 SFN 07 San Diego

Method

Audiovisual display Expected behavior

Stimulus Location (°)

Mag

nitu

de (

°)

Induce shift: - in only one region of space- from a single fixation point

Test to see if shift generalizes to the same sub-region in:- head-centered space- eye-centered space

Experiment divided into 1-hour blocks:(12 for humans, 16

monkeys)

Within a block, 3 types ofrandomly interleaved trials:- AV 50%, - A-only, trained FP- A-only, shifted FP

13Nov 6, 2007 SFN 07 San Diego

Results: HumansAudiovisual display

Expected Responses

FPLEDs

Speakers

Stimulus Location (°)

Mag

nitu

de o

f Ind

uced

Shi

ft (°

)

or

Trained FP A-onlyresponses:- Shift induced in trained sub-region- Generalization to untrained regions (asymmetrical)

Shifted FP A-onlyresponses:- Shift reduced in center region

Head-centeredrepresentation,modulated by eye position

14Nov 6, 2007 SFN 07 San Diego

Results: MonkeysAudiovisual display

Expected Responses

FPLEDs

Speakers

Stimulus Location (°)

Mag

nitu

de o

f Ind

uced

Shi

ft (°

)

or

Trained FP A-onlyresponses:- Shift in trained sub- region weaker- Generalization to untrained regions stronger (asymmetry oppo- site to humans)

Shifted FP A-onlyresponses:- Shift decreases on the right- Shift increases on the leftHumans:

Representation

more mixed

15Nov 6, 2007 SFN 07 San Diego

SummaryThe main results are consistent across species:

Locally induced ventriloquist effect results in short-term adaptation, causing shifts in responses to A-only stimuli from trained sub-region.

The induced shift generalizes outside the trained sub-region, with gradually decreasing strength (However, the pattern of generalization differs across the species)

The pattern of induced shift changes as the eyes move. But, overall, it appears to be in a representation frame that is more head-centered than eye-centered.

16Nov 6, 2007 SFN 07 San Diego

DiscussionPosterior ParietalCortex

Neural adaptation could have beeninduced at several stages along thePathway (IC, MGB, AC, PPX, MC,SC).

In humans, multiple effects observed at different temporalscales likely adaptation atmultiple stages

Future workExtend Examine temporal and

spatial factors influencing the eye-centered modulation.

Look at other trained sub-regions.

Midbrain

Pons

Cerebrum

Thalamus

Midbrain

Pons

Thalamus

17Nov 6, 2007 SFN 07 San Diego

Summary and discussionthe main results are consistent across species:

Locally induced ventriloquist effect results in short-term adaptation, causing shifts in responses to A-only stimuli from trained sub-region.The pattern of induced shift is modified as the eyes move.

Bad news – there is a lot of differences between species:

Humans MonkeysRepresentation head-centered, eye-modulated eye-centered

Generalizationto untrained sub-regions more on the side away from FP opposite

Difference betweenhyper- and hypometricshifts no yes

Representation whenshift induced on side (data not shown) head-centered, no eye modulation eye-centered

18Nov 6, 2007 SFN 07 San Diego

Results: HumansAudiovisual display

Expected Responses

FPLEDs

Speakers

Human Behavior

Mean+SE

Stimulus Location (°)

Mag

nitu

de o

f Ind

uced

Shi

ft (°

)

19Nov 6, 2007 SFN 07 San Diego

Results: Humans

Stimulus Location (°)

Audiovisual display Expected Responses

FPLEDs

SpeakersHuman Behavior Data collapsed across

direction of inducedshift

Trained FP A-onlyresponses:- Shift induced in trained sub-region- Generalization to untrained regions (asymmetrical)

Shifted FP A-onlyresponses:- Shift reduced in center regionHead-centered repre-sentation, modulated by eye position

Mean+SE

20Nov 6, 2007 SFN 07 San Diego

Results: Humans vs. MonkeysAudiovisual display

FPLEDs

SpeakersHuman Behavior Monkey data

(only hypometric)

AV responses:- as expected

Trained FP A-onlyresponses:- Shift in trained sub- region weaker- Generalization to untrained regions stronger (asymmetry opposite to humans)

Shifted FP A-onlyresponses:- Shifted with eyes

Representation moreeye-centered

Mean+SE

Monkey Behavior

Mean

6

6-24 0 24

-24 0 24

-24 0 24

Mean+individualsMean+SE

Mean+SE Mean+individuals