Language & Brain

Why bother? What could we learn?

• something about how language works

• something about how the brain works

• nothing (interdisciplinary cross-sterilization)

Chomsky 1959

Franz Josef Gall

1758-1828

Hand scanner

Phrenology

(bad idea)

Organology

(good idea)

Two enduring ideas deriving from Gall

• Faculty psychology

The mind has a ‘parts list.’

• Experience-dependent plasticity

Using the parts changes

their neuronal realization.

The wave of the past

“Intuitive psychological organology ”

Phineas Gage, 1848

Before: responsible, well-

mannered, well-liked,

efficient worker, pious

After: capricious, impulsive,

irreverent, hypersexual

Damage involved VMPFC

Paul Broca 1861, 1865

Leborgne’s (Tan’s) brain

Geschwind 1985

Broca’s Area

“production”

“syntax”

Wernicke’s Are

“reception”

“semantics”

Visual system

(van Essen)

Auditory system

(Hackett)

The visual and auditory systems are highly articulated. Is there any a priori reason to believe

that language will be an order of magnitude simpler, captured by two brain areas?

Functional anatomy of speech perception.

Hickok & Poeppel 2007

Non-invasive

recording from

human brain

(Functional

brain imaging)

Positron emission

tomography

(PET)

Functional magnetic

resonance imaging

(fMRI)

Electro-

encephalography

(EEG)

Excellent spatial

resolution (<1 mm)

Limited temporal

resolution (~1sec)

Limited spatial

resolution (<1 cm)

Excellent temporal

resolution (<1msec)

Hemodynamic

techniques

Electro-magnetic

techniques

Magneto-

encephalography

(MEG)

D. Poeppel , A. Braun et al.

Language is not monolithic

Phonetics/phonology

sound structure

Morphology

word structure

Lexical semantics

word meaning

Syntax

sentence structure

Prosody

sentence melody

Compositional semantics

sentence meaning

Discourse

larger meaning scale

language-o-topy

The wave of the past

“Intuitive psychological organology ”

The wave of the present

“Cognitive psychological organology”

syntax

phonology

semantics

Linguistics Neuroscience

Fundamental elements of representation

distinctive feature dendrites, spines

syllable neuron

morpheme cell-assembly/ensemble

noun phrase population

clause cortical column

Fundamental operations on primitives

concatenation long-term potentiation

linearization receptive field

phrase-structure generation oscillation

semantic composition synchronization

?

?

Is there a future? Problems for interdisciplinarity and unification I

There is an absence of ‘linking hypotheses’ by which we explore how brain mechanisms

form the basis for linguistic computation.

Aligning the alphabets or primitives or atoms is a formidable challenge.

Is there a future? Problems for interdisciplinarity and unification II

Ontological Incommensurability Problem (neurolinguistics in principle):

The units of linguistic computation and the units of neurobio-

logical computation are incommensurable.

Therefore, an attempt at reduction makes no sense.

Why are there no linking hypotheses?

Granularity Mismatch Problem (neurolinguistics in practice):

Linguistic and neuroimaging studies of language operate

with objects of different granularity.

linguistics --- fine-grained distinctions

neuroscience --- broader conceptual distinctions

Neuroscience cannot succeed in seeking “syntax” (or “phonology”)

because syntax etc. are not monolithic but have many parts.

Poeppel & Embick, 2005

Is there a future? Problems for interdisciplinarity and unification III

Linguistics Neuroscience

distinctive feature dendrites, spines

morpheme cell-assembly/ensemble

noun phrase population

clause cortical column

concatenation long-term potentiation

linearization receptive field

phrase-structure generation oscillation

semantic composition synchronization

fractionate into

generic formal operations

segmentation

concatenation

comparison

recursion

identify basis for

generic formal operations

segmentation

concatenation

comparison

recursion

?

Desiderata for a model bridging neuronal mechanisms

and linguistic representation

x---y

yx

z

concatenation constituency recursion

Neurobiological mechanisms that can form the basis of elemental

steps involved in most linguistic computation:

Is there a future? Problems for interdisciplinarity and unification IV

This is the granularity - and level of abstractness - of operations

that can profitably be studied in animal research as well, doing

away with questions such as “are humans different or better or

higher, or not” and turning to the typical questions such as:

“How does this work?”

Putative primitives - the view on irreducible representations and operations

from semantics (Pietroski) and syntax (Hornstein)

• Variables, a way to link variables

• One-place predicates, thematic roles

• Operation with the power of conjunction and existential closure

• Concatenation (a-directional)

• Labeling: concatenate turns into one of its constituents

• Some mechanism (copy) to deal with positional specificity of variables

‘Unification Problem’

Marr’s computational approach

permits development of linking hypotheses

computational algorithmic implementational

How much can be gained by focusing only on localization by way of

imaging?

Not much, at this point - it is the ‘homework problem’, that is, an

important but ultimately uninteresting step from the point of view of

explanation.

Can we achieve unification by working on localization?

No! We need explicit linking hypotheses between well

characterized brain mechanisms and linguistic computation.

WRONG QUESTION: where are syntax/phonology/

semantics mediated?

RIGHT QUESTION: what kind of computations in the

brain form the basis of linguistic

representations and operations?

Is there a future? Problems for interdisciplinarity and unification V

xx yy

xxx yy

zzzz

zzz

ppppp

pppp

qqqq

qqq

oojjjoo

oooojj

ooooojjjj

There is localisation, but what is localized is tissue that executes specific

computations, such as, say, addition (xxx) or subtraction (zzz) or division (qq),

over representations (data structures) of certain types.

sss

sss

The cognitive faculties (the “parts”

of the human cognome) are not

monolithic but composed of multiple computational subroutines.

The wave of the past

“Intuitive psychological organology ”

The wave of the present

“Cognitive psychological organology”

syntax

phonology

semantics

The wave of the future

“Computational organology”

recursion

constituency

sequencing

linearization

Localization of generic

computational subroutines

÷÷÷

÷÷÷

! ! !"""

"""

[+ cons, -son] [-cons, +son] [+ cons, -son]

x xx

c ta

LAR/PHAR LAR/PHAR LAR/PHAR

[-cont] [-cont]

PLACE PLACEPLACE

GLOT

[-voice]

DORSAL [-ATR] DORSAL CORONAL

[-back, -high, +low]]

GLOT

[-voice] [+ant]

?

(a) (b)

(c)

(d)

phonological

primal sketch

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Functional anatomy of speech sound processing

Hickok & Poeppel, 2007, Nat Rev Neurosci

Lau et al. 2008, Nat Rev Neurosci

Newton, Principia

Our brain

Our brain, really

Our intuition

Axial ViewSagittal View

Neuromagnetic activity is recorded from the whole head (160 channels)

MEG: From signals to magnetic field maps to sources

Phase Patterns of Neuronal Responses Reliably

Discriminate Speech in Human Auditory Cortex

Huan Luo & David Poeppel

Single-unit responses robustly encode conspecific vocalizations

Machens et al., Nat. Neurosci. 2003

Grasshopper (peripheral auditory

neurons)

Narayan et al., J. Neurophys. 2006

Zebra Finch (Field L)

“Many natural sounds including vocal communication

sounds display striking time-varying structure over

multiple time scales”

“We demonstrate the existence of distinct time scales for

temporal resolution and temporal integration and explain

how they arise from cortical neural responses to complex

dynamic sounds.” [~10 ms and ~ 500 ms]

Design: evaluate coherence across single trials elicited by sentences

Luo & Poeppel, Neuron, 2007

Luo & Poeppel, Neuron, 2007

Theta phase

Materials:

Smith, Delgutte, and

Oxenham, Nature, 2002

Luo & Poeppel, Neuron, 2007

Theta phase has the sensitivity to discriminate based on single trials

Theta phase tracking displays the specificity to discriminate sentences

Luo & Poeppel, Neuron, 2007

Classification analysis

Luo & Poeppel, Neuron, 2007

A ~ 200 ms window analyzes the input signal -- The syllable as primitive

Endogenous cortical rhythms determine cerebral

specialisation for speech perception and production

Giraud et al. 2007, Neuron

Distribution of intrinsic cortical rhythms

Combined EEG/fMRI recordings

N=12 + 8 subjects at rest (twice 20 min.)

EEG, 32 channels, continuous acquisition.

fMRI, 1.5 T and 3 T Siemens, sparse acquisition

No auditory input beyond the MRI scanner noise.

Analyses from central electrodes to observe

temporal asymmetries with minimal lateralization

bias.

Giraud et al. 2007, Neuron

3–6 Hz

0 50 100 150 200 250 300

0

5

0 50 100 150 200 250 3000

5

time [scan number]

Continuous EEG recording (20 min.)

Sparse fMRI acquisition (20 min.)

Re

gre

sso

rs

FF

T

po

wer

[10

10

uV

2]

3sgap1s

fMRI3s

gap1s

fMRI3s

gap1s

Segments of

EEG trace

Sequence of

fMRI acquisition

Raw dataEstimated data (raw data convolved with hemodynamic function)

Combined EEG/fMRI recordings: Approach

28 – 40 Hz

Giraud et al. 2007, Neuron

Mouth

Tongue

3-6 Hz EEG band

28-40 Hz EEG band

Heschl

Experiment 1 (1.5T) Experiment 2 (3T)

Group results: topography of theta and gamma

in premotor cortex

Giraud et al. 2007, Neuron

Motor constraints on speech (Frame/Content theory, MacNeilage and Davis, Current Opinion Neurobiol. 2001)

Mechanical properties of the speech apparatus (e.g.,spontaneous

oscillation frequency of the jaw) determine rhythmic properties of spoken

language (e.g. syllabic rate - theta rhythm)

Giraud et al. 2007, Neuron

Speech

analysis

Functional interactions between perceptualand motor speech systems: internal forward

model at time scales ‘of interest’

Motor output

Sensory feedbackEfference copies

Fine articulatory tuning

Three messages

1. Language is not monolithic.

(Even subroutines of language

comprehension, such as

speech perception, are highly

complex.) The constituent

elementary computations are

likely mediated by an array of

cortical areas.

2. MEG is a useful -- and

underutilized - tool to

investigate a range of issues

in cognitive neuroscience.

The data can provide an

interesting bridge to

questions of neural coding.

3. The phase of low frequency

responses (e.g. theta)

provides a sensitive (trial by

trial) neurophysiological

index of online processing

and can be used to assess

the ‘temporal granularity’ of

perceptual analysis (sliding

temporal window).

UQAM -- Origins of Language, 6/24/10

David Poeppel

NYU Psychology and Neural Science

david.poeppel@nyu.edu

http://psych.nyu.edu/clash/poeppellab.html

http://talkingbrains.blogspot.com/