Tracking the timecourse of multiple context effects in assimilated speech

Tracking the timecourse of Tracking the timecourse of multiple context effects in multiple context effects in

assimilated speechassimilated speech David Gow

Massachusetts General HospitalBob McMurray

Dept. of Brain and Cognitive SciencesUniversity of Rochester

With thanks to Dana Subik, Joe Toscano & John Costalis

Overview

2) Coping with Coronal-Place Assimilation during online recognition.

Laboratory Phonology Spoken Word Recognition

1) Bridging fields yields:New solutions to old problems.New questions.

3) Implications for language processing & phonology.

Laboratory Phonology: How perceptual and articulatory constraints drive sound change and shape phonological systems

Bridging Fields: Laboratory Phonology

Rich information source in the signal: Constraints inferred through acoustic and articulatory measures.

Do phonological constraints inform word recognition?Can details of word recognition inform phonological constraints?

Perceptual models tend to come in two varieties:

Bridging Fields: Spoken Word Recognition

Spoken word recognition models that assume phonemic inputs as input to the lexicon and meaning.

Ignore systematic acoustic variation.

Phoneme perception models that relate acoustic properties to categorical perception

VOT0

100

PB

% /p/

ID (%/ pa/ ) 0

100Discrimination

Discrimination

VOT0

100

PB

% /p/

ID (%/ pa/ ) 0

100Discrimination

VOT0

100

PB

% /p/

ID (%/ pa/ ) 0

100

VOT0

100

PB

% /p/

ID (%/ pa/ ) 0

100

0

100Discrimination

Discrimination

Limits of categorical perception

Categorical perception (CP) is task-dependent, and doesnot appear to take place in tasks that involve spontaneous, naturalistic speech understanding.

McMurray, Aslin, Tanenhaus, Spivey & Subik (in prep)

Within category variation that should be lost in CP affectslexical processes

Andruski, Blumstein & Burton (1994), Gow & Gordon, 1995; Utman, Blumstein & Burton (2000), Dahan, Magnuson, Tanenhaus & Hogan (2001), Gow (2001; 2002; 2003)McMurray, Tanenhaus & Aslin (2003)

Speech perception and phonology relate signal properties to perception.

Systematic acoustic variation and SWR

Perc

ept

ion

Phonology

Meaning

Properties of the signal must be related to meaning—lexical activation.

Perception Ph

onol

ogy

Case study:

English Place Assimilation

English coronal place assimilation

/coronal # labial/ [labial # labial]

/coronal #velar/ [velar # velar]Prior work has treated this change as

• discrete• phonemically neutralizing

Assimilation

[ ]# berries nonword?

cat box?

cap box?[ ]# box

How are words recognized despite neutralization?

Phonological inference (Gaskell & Marslen-Wilson, 1996; 1998; 2001)

If [labial # labial] infer /coronal # labial/

greem beans green (Gaskell & Marslen-Wilson, 1996; Gow, 2001)

cap box cat (Gaskell & Marslen-Wilson, 2001; Gow, 2002)cap

Assimilatory modification is acoustically continuous.

Assimilation as Continuous Detail

F2 Transitions in /æC/ Contexts

Pitch Period155016001650

1700175018001850

Freq

uenc

y (H

z)

F3 Transitions in /æC/ Contexts

Pitch Period2550

2600

2650

2700

2750

2800

Freq

uenc

y (H

z)

coronalassimilatedlabial

Assimilation blends cues to two places of articulation

An Alternative View

Assimilation redistributes and blends information

cat box [# ]

Labiality of assimilatingitem

Coronality of assimilateditem

In theory: assimilation creates correlated cues…

[ # ]

Assimilating context might disambiguate blend

Blend might facilitate recognition of context

How can we determine if listeners use this information during recognition?

These questions require a method that:

• Measures lexical activation.• Sensitive to continuous acoustic detail.• Sensitive to temporal uptake of information.• Measures consideration of multiple items in parallel.

Visual World Paradigm

Visual World Paradigm

•Subjects hear spoken language and manipulate objects in a visual world.

•Visual world includes set of objects whose names represent competing hypotheses for the input.

•Eye-movements to each object are monitored throughout the task.

Tanenhaus, Spivey-Knowlton, Eberhart & Sedivy (1995)Allopenna, Magnuson & Tanenhaus (1998)

•Meaning based, natural task: Subjects

must interpret speech to perform task.

•Eye-movements fast and time-locked to speech—temporal dynamics.

•Fixation probability ~ lexical activation.

•Sensitive to within-category acoustic variability (McMurray, Tanenhaus & Aslin, 2003; Dahan, Magnuson, Tanenhaus & Hogan, 2001)

•Multiple competitors in same trial.

Present subjects with assimilated or non assimilated speech.

Measure activation for items that follow assimilation.

Experiment 1

Assimilation facilitates recognition.

Methods

Subject hears “select the maroon goose”“select the maroong goose”

Prediction: More fixations to goose after assimilated consonants.

34 Subjects. 16 sets of items. Subjects exposed to pictures/names before each block.

Stimuli cross-spliced from natural tokens—assimilation is not complete… continuous acoustic information.

“select the maroon duck”“select the maroon goose”“select the maroong duck” ***“select the maroong goose”

Spliced from “maroon duck”

Spliced from “maroon goose”

Eye-movements temporally aligned to onset of second word (goose or duck).

Target = Maroon GooseCompetitor = Maroon DuckUnrelated = Patriotic Duck and Goose

Time

200 ms Trials

1

2

3

4

5

… many more

trials

00.10.20.30.40.50.60.70.80.9

0 200 400 600 800 1000Time (ms)

Fixa

tion

Pro

port

ion Assimilated

Non Assimilated

0

0.1

0.2

0.3

0.4

0.5

0.6

0 100 200 300 400 500Time (ms)

Fixa

tion

Pro

port

ion

AssimilatedNon Assimilated

Results

Looks to the target (goose)

p = .03*

Looks to the competitor (duck)

0

0.05

0.1

0.15

0.2

0.25

0 200 400 600 800 1000

Time (ms)

Fixa

tion

Prop

ortio

n

Assimilated

Non Assimilated

Experiment 1: Summary

Continuous variation due to assimilation• not variability to be conquered… • signal to be used.

Assimilated coronals allow progressive operations.

• facilitate consistent targets• exclude inconsistent competitors earlier

Consistent with prior work using priming (Gow, 2001; 2003; Gow & Im, in press)

Lexical Ambiguity?

Even incomplete modification can create lexical ambiguity.

cat box catp box ?cat

cap

Does subsequent context regressively modify the interpretation of assimilated segments?

catp

box cat

p drawing

Experiment 2

Subject hears “select the cat

p box”“select the cat

p drawing”

Prediction:Fixations to cat or cap is a function of context.

0

0.1

0.2

0.3

0.4

0.5

0.6

0 400 800 1200 1600Time (ms)

Fixa

tion

Pro

port

ion

Coronal (cat)Non-Coronal (cap)

catp box Assim Non-Coronal

catp drawing Assim

Coronal

00.10.20.30.40.50.6

0 400 800 1200 1600Time (ms)

Fixa

tion

Pro

port

ion

Coronal (cat)Non-Coronal (cap)

Regressive effect is more biasing for non felicitous assimilation… perceptual locus?

?Progressive effects

too?

Regressive effects:Context biases interpretation of ambiguous token.

Will we see a progressive effect at the same time?

00.10.20.30.40.50.60.70.80.9

0 200 400 600 800 1000Time (ms)

Fixa

tion

Prop

ortio

n

Non-assimilatedAssimilated

0

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0 200 400 600 800 1000Time (ms)

Fixa

tion

Prop

ortio

n

Non-assimilatedAssimilated

Target Competitor

Weaker effectsPossibly due to item

variability

Experiment 2: Summary

Assimilated coronals allow regressive operations:

• Context useful in resolving ambiguity.

Similar Progressive operations to experiment 1.

What kind of computation is responsible?

… relationship to continuous detail in signal important

Progressive & Regressive effects vary continuously across items.

Continuous Signal Continuous Response

Experiment 1: Progressive effect on target

-0.100

-0.050

0.000

0.050

0.100

0.150

0 2 4 6 8 10 12 14 16

Item #

Prog

ress

ive

Effe

ct

-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5

0 2 4 6 8 10 12 14 16

Item #

Reg

ress

ive

Effe

ct

Experiment 2: Regressive effect

What can the acoustics properties of these items tell us about perceptual variability?

Measured F1, F2, F3, Closure Duration of original items.

Regression:

F1, F2, F3, Closure

Interaction with Labiality.

Not enough power (items) to reach significance but model accounted for:

Experiment 1• Progressive effect: 75% of variance

Continuous acoustic variation is related to perceptual processes… how?

Experiment 2• Progressive effect: 57% of variance.• Regressive effect: 37% of variance

Feature cue parsing (Gow, 2003)

Time (s)0 0.760454

0

3000

[

A Perceptual Account

Feature cue parsing (Gow, 2003)

Time (s)0 0.760454

0

3000

Any feature is encoded by multiple cues that are integrated

Feature cue parsing (Gow, 2003)

Time (s)0 0.760454

0

3000

Feature cue parsing (Gow, 2003)

Time (s)0 0.760454

0

3000

Assimilation creates cues consistent with multiple places

Feature cue parsing (Gow, 2003)

Extract feature cues

Feature cue parsing (Gow, 2003)

Group feature cues by similarity and resolve ambiguity

Integration is by the same process within a segment.

Standard component of information integration in perception.

Feature cue parsing (Gow, 2003)

catp # box cat

p # drawing catp

# | | | |

[cor] [cor] [COR] [cor] [lab] [LAB] [lab] [lab]

example: cat….

catp # Box cat

p # Drawing catp #

| | [cor] [cor] [COR] [cor] [lab] [LAB] [lab] [lab]

Feature cue parsing (Gow, 2003)

Progressive and regressive effects fall out of grouping

example: cat….

Feature cue parsing based on basic perceptual grouping principles:

• Not specific to assimilation.

Parsing errors may lead to sound change:

• Pressure on languages to avoid errors

• Maximize contrast between adjacent segments.• Minimize juxtaposition of similar segments

Implications for Phonology

Feature parsing errors may lead to sound change

e.g. Shona Dissimilation (Ohala, 1981)

Pre-Shona

-b w a [LAB] [labio-velar glide]

Shona

-b a [LAB] (labio) [velar glide]

Gow & Zoll (2002)

Conclusions

English coronal place assimilation neutralizes phonemic distinctiveness.

Perceptual recovery cannot be based on symbolic processes.

Continuous perceptual mechanisms sensitive to systematic acoustic variation yield

•Progressive activation of upcoming material

•Regressive ambiguity resolution.Such mechanisms may play a role in sound change.

Bridging spoken word recognition and laboratory phonology helps both fields.

Conclusions

SystematicPhonological Variation

Perceptual Processes

Sound Change

Perceptual Processes

…

•In SWR: assimilation is not noise to be conquered, but signal to be used.

•In phonology: perceptual mechanisms for handling variation may constrain languages’ sound structures.