Visual-auditory interactions assessed with ABR

Dr. W. David Hairston, Visual & Auditory Processes Branch

Human Research and Engineering Directorate

Army Research Laboratory

Overview

• Background

- Who am I, ARL, etc

- Background story

• The Plan

- Paradigm

- Analysis

Who is this guy?

My background….Exp Psych…. BS App State, MA Wake Forest U

Multisensory target localizationNeuroscience…. PhD (Neurobiology) Wake Forest U

(Mark Wallace) Multisensory perceptual interactions in time and space

Brain imaging…. Post doc (Radiology) Wake Forest U(Laurienti, Burdette, Maldjian)Cross-modal interactions with attention

(music conductors) and timing (dyslexia)Now…. Research fellowship w/ Army Research Lab

(Letowski, McDowell) Cross-modal workload influence on auditory processing

No experience in audiology

ARL?

US Army

Research, Development, and engineering Command

Human Research and Development

Visual and Auditory Processes Branch

Auditory Research Team

Officially – Oak Ridge Associated Universities

Army Research Lab

Interests…

My Interests:Visual-Auditory integrationMultisensory interactions – how processing in one sense affects others…

not just in good ways…(e.g. dark ugly downside to cross-modal interactions)

Role of experience – can focused training not only hone sense(s), but also affect how they interact with each other?(e.g. can detrimental interactions be avoided?)

ARL’s interests…Application – how can we use multisensory research to enhance US

Soldier?(e.g., can specific types of training enhance sensory awareness? Or, can detrimental affects be overcome?)

Programatic – How can neuroscientific methods be easily integrated into other ARL studies of human perception (e.g., sound localization, human factors, etc)

Research Background

Multisensory interactions….Many studied effects from multisensory stimulationClassically, examined as V+A vs V or A…. “Is 2 better than 1?”

Lots of good stuff - Behaviorally: faster response times, higher accuracy, better memory

- Neurophysiologically: bigger responses, decreased latency, lower S/N, etc.

(even in this lab – enhanced FFR, cortical response slope)

But what about cases where you don’t WANT integration?Not everything we do is multisensory…

Research Background

Cross-modal inhibition….

Sometimes you really need to focus on just one sense…Reading a book in a noisy roomTalking on phone (not while driving!)

“Cross-modal attention” (hate that word)shift mental resources from one sense to another

2.78 7.0

-2.78 -7.0

Deactivation of auditory cortex

during visual task

Research Background

Own recent work… role of task difficulty

440 Hz 660 Hz

440 Hz 660 Hz

Easier

Difficult

60 ms

20 ms

Temporal Discrimination

Subjects’ thresholds for each task acquired prior to fMRI scanning

Non-musicians: Moderate

Deactivation of visual cortex

+/-3.95 +/-8.0

Vis Response

Non-musicians: Difficult

Robust deactivation of visual cortex

+/-3.95 +/-8.0

Vis Response

Non-musicians vs. Conductors

Difference seen when task is difficult

0

40

80

120

160

200

Easier Difficult

N S

ign

if V

oxe

ls

Non-musicians

Conductors

n.s.

**

0

20

40

60

80

High M oderate

Acc

ura

cy(%

)

Highly trained concert

conductors (>7 yrs

experience) do NOT show

this trend

Suggests the inhibition can be mediated by experience/training

Research Background

So…. What about the brainstem?

Begs the question… does cross-modal inhibition affect most basic, incoming auditory processing?

We know that the ABR is affected by multisensory processing – (enhancement) and this is tied to cortical enhancement(Thanks Auditory Neuroscience Lab!)

… but what about suppression?

Increased FFR amplitude w/ Auditory attention vs visual(Galbraith)-but unclear if this is aud enhancement or visual suppression

(no no-stim baseline)

Question at hand…

• Does cross-modal task difficulty directly affect ABR-level processing?

• What are the implications at the level of both the brainstem and cortical response?

The Plan

Use ABR-based recording techniques while subjects perform visual, auditory, and baseline (no) tasks at different levels of difficulty

The Plan

ABR Stim

Oddball

Stim: 220 Hz tone, 100 ms duration 400 ms ISI

Infrequent oddball (5%?) 235 Hz

ABR

Task

Jittered, 4 sec ave ISI between stim

The Plan

Visual

Auditory

X 2 = Easy + Hard

No task

Tasks?

Tasks…?Temporal discrimination

N-Back

Motion detection

Must be similar in both modalities, and manipulated to “easy” and “hard” levels

440 Hz 660 Hz

440 Hz 660 Hz

Easier

Difficult

60 ms

20 ms

Analysis

• Interested in FFR of ABR – shown to be most affected by VA-interactions, cognitive influence etc

• Also cortical (MMN) responses associated with oddballs, and correlation with FFR

• Ideally limit # of trials to minimum – save time, more practical

Analysis

Typical FFR…

Trial 1 Trial X Mean Signal

+ =

FFT of Mean

Advtantage:

Proven method, have full data, derive timing (onset), signal amplitude and FFT

amplitude

Disadvantage: requires many trials for waveform to average out, requires pefect

time-lock response; time consuming

Analysis

Fourier-Average FFR….

Trial 1 Trial X

+ =

Mean FFT

Advantage:

Many fewer trials needed; no need for perfect phase locking – saves time!

Disadvantage: Loss of temporal structure; computationally intensive

Option 1 – FFT every trial,

average over 1000 trials

Option 2 – averge clusters

(10?), average over clusters

(100) MMN/cortical –

standard

averaging

technique

Expectations…

1. (I learn how to do ABR)2. Larger FFR response in A- task than V3. Decreased V (suppression) relative to baseline4. Effect (3) most pronounce in difficult task

(e.g., minimal/no diff in easy task)5. Latency of MMN correlated with FFR modulation6. (eventually) – Effects (4,5) diminished with training, or

specific kind of expertise

Alternative: cross-modal effect may be in baseline noise shift (SNR) rather than amplitude change

Equally enticing!

Done!

Open to ideas!

Unresolved:

Ideal tasks

Ideal ABR stim (pure vs complex tone)

How many trials

Individual vs cluster FFT

Min effective sample rate (data reduction)