Word Recognition (Sereno, 4/04) How long does it take to recognise a visual word? –What is meant by “recognition” or “lexical access”? –Can lexical access

Word Recognition (Sereno, 4/04)

How long does it take to recognise a visual word?

– What is meant by “recognition” or “lexical access”?

– Can lexical access be accurately measured?

– What factors affect lexical access and when?

The “magic moment” (Balota, 1990) of lexical access:“At this moment, presumably there is recognition that

the stimulus is a word, and access of other information (such as the meaning of the word, its syntactic class, its sound, and its spelling) would be rapid if not immediate.” (Pollatsek & Rayner, 1990)

Background: Basic Units of Language

A. ~5,000 languagesphonemes morphemes sentences conversations (sounds) & words

B. Phonemes = elementary sounds of speech• phonemes are not letters...

to, too, two, through, threw, shoe, clue, view

• vowel & consonant phonemes

• 11-144 phonemes in any given languageEnglish has ~ 40; Hawaiian has ~16

• combining phonemes is rule-governed

Wordness: For each row of 3 possible new words, which one will probably never make it : (

blick splunge rlight

sbarm wumple turl

mancer nserht crelurious

inther iwhucr neen

shace fring ngout

Basic Units of Language

C. Morphemes = smallest meaningful unit of lang.

• can be a word, word stem, or affix (prefix, suffix)word: help, loveword stem: spir, ceive, duceprefix/suffix: re-, dis-, un- / -less, -ful, -er

• derivational & inflectional morphemesderivational – change the grammatical class

V + -able = Adj (adorable, believable)V + -er = N (singer, runner)

inflectional – grammatical markersV + -ed = past tense (walked)N + -s = plural (cows)

“free” {

“bound”{


C. Words• Content vs. function (open- vs. closed-class) words

Content words = carry the main meaningnouns, verbs, adjectives, adverbs

Function words = grammatical wordsarticles (a, the, this), conjunctions (and,but), prepositions (in, above)

Psychological reality of the content-functionword distinction in aphasia selective impairment of content (Wernicke’s) orfunction words (Broca’s aphasia)

• Cattell (1886) & Stroop (1925)

Word superiority effect (Cattell, 1886)

– Reicher (1969); Wheeler (1970)

– tachistoscopic presentation

– more accurate identification of the letter when stimulus is a word

– pseudoword superiorty effect

---dk word

dk d

REDBLUE

BLACK

GREEN

RED

RED

GREENBLACK

BLUE

BLUE

RED

BLACK

BLUE

BLACK

BLUE

GREEN

BLUEGREEN

RED

GREEN

NAME THE COLOUR OF THE INK


C. Words• Ambiguity

1 word form, but 2 (or more) word meaningsEx: bank (N-N, “money” vs. “river”) watch (N-V, “clock” vs. “look”) bass (N-N, “guitar” vs. “fish”)

2 word forms, but 1 pronunciationEx: sail/sale, right/write

Generally unaware of ambiguity...even though it is quite pervasiveeven though it affects behaviour (RT, etc)

homographs

homophones


D. Sentences• Syntax = the rule-governed system for groupingwords together into phrases and sentences

• Sentences introduce a concept that they are about,the subject (or noun phrase), and then proposesomething about that concept, the predicate(or verb phrase).Ex: “The boy hit the ball.”

doer act done-to (thematic roles)

subject predicate


D. Sentences• Same deep structure, different surface structure

“The boy hit the ball.” (active)“The ball was hit by the ball.” (passive)

• Same surface structure, different deep structure [The French bottle]NP [smells.]VP

[The French]NP [bottle smells.]VP

THEY are boring. VISITING THEM is boring.

cf. ambig. figures in perception: 1 form, 2 interpretations

“The French bottle smells.”

“Visiting relatives can be boring.”

Necker cube

New obesity study looks for larger test groupReagan wins on budget, but more lies aheadMan struck by lightening faces battery chargeEnraged Cow Injures Farmer with AxeMilk Drinkers Are Turning to PowderLocal High School Dropouts Cut in HalfBritish Left Waffles on FalklandsDealers Will Hear Car Talk at NoonMiners Refuse to Work after DeathBeating Witness Provides NamesSquad Helps Dog Bite VictimKids Make Nutritious Snacks

Headlines

Stolen Painting Found by TreeProstitutes Appeal to PopeRed Tape Holds up BridgeDeer Kill 17,000Teenage Prostitution Problem is MountingChild Stool Great for Use in GardenShouting Match Ends Teacher’s HearingMan Robs then Kills HimselfLung Cancer in Women MushroomsMondale’s Offensive Looks Hard to BeatTuna Biting off Washington CoastChinese Apeman Dated

Headlines


D. Sentences• Syntactic ambiguities

“She hit the boy with the big stick.”

“She hit the boy with the runny nose.”

Interpretation depends on structural preferences(certain constructions used more often,

favoured),as well as the prior discourse context.


• Measures

• Components

• Models

• Eye movements (EMs)

• Event-related potentials (ERPs)

• Word frequency & lexical ambiguity

Measures• Standard behavioural techniques

– lexical decision, naming, categorisation; also RSVP, self-paced reading

– priming, masking, lateralised presentation

– Donders (1868): subtractive method• assumes strictly serial stages of processing• additive vs. interactive effects

– automatic vs. strategic (Posner & Snyder, 1975)

unconsciousexogenousbottom-upbenefit

controlledendogenoustop-downcost & benefit

Measures• Eye movements

• Neuroimaging– “Electrical”: EEG, MEG

– “Blood flow”: PET, fMRI

MEASURE

Normal reading

TASK

fixation duration (as well aslocation and sequence of EMs)

TIME RES.

GOOD

POOR“blood flow” imaging: fMRI, PET

“electrical” imaging: EEG, MEG

various word tasks

ms-by-ms

seconds

various word tasks

naming

categorisationlexical decision

Standard word recognition paradigms (± priming, ± masking):

RT~500 ms~600 ms~800 ms

~250 ms

Components

• Orthography of language– English vs. Hebrew or Japanese

• Language skill– beginning (novice) vs. skilled (expert) reader

– easy vs. difficult text

Components

• Intraword variables– word-initial bi/tri-grams clown vs. dwarf

– spelling-to-sound regularity hint vs. pint

– neighborhood consistency made vs. gave

– morphemes• prefix vs. pseudoprefix remind vs. relish

• compound vs. pseudocompound cowboy vs. carpet

Components

• Word variables– word length duke vs. fisherman

– word frequency student vs. steward

– AoA dinosaur vs. university

– ambiguity bank vs. edge, brim

– syntactic class open vs. closed; A,N,V

– concreteness tree vs. idea

– affective tone love vs. farm vs. fire

– etc.

Components• Extraword variables

– contextual predictabilityThe person saw the... moustache.The barber trimmed the...

– syntactic complexity Mary took the book. *Mary took the book was good. Mary knew the book. Mary knew the book was good.*Mary hoped the book. Mary hoped the book was good.

– discourse factors (anaphora, elaborative inferences)He assaulted her with his weapon.... knife... stabbed

Models• Dual-route account (Coltheart, 1978)

Direct route(addressed)

phonology

semantics

orthography

Indirect route(assembled)

Models• Interactive (Morton, 1969; Seidenberg & McClelland, 1989)

/m A k/

phonology

meaning

orthography

M A K E

context

Models• Modular (Forster, 1979; Fodor, 1983)

decision output

Lexicalprocessor

Syntacticprocessor

Messageprocessor

GeneralProblemSolver

input features

Models• Hybrid

– 2-stage: generate candidate set selection

– (Becker & Killion; Norris; Potter)


• Measures

• Components

• Models

• Eye movements (EMs)

• Event-related potentials (ERPs)

• Word frequency & lexical ambiguity

MEASURE

Normal reading

TASK

fixation duration (as well aslocation and sequence of EMs)

TIME RES.

GOOD

POOR“blood flow” imaging: fMRI, PET

“electrical” imaging: EEG, MEG

various word tasks

ms-by-ms

seconds

various word tasks

naming

categorisationlexical decision

Standard word recognition paradigms (± priming, ± masking):

RT~500 ms~600 ms~800 ms

~250 ms

Tools of choice:• Recording eye movements in reading

• Recording ERPs in language tasks

Eye Movements (EMs)

Best on-line measure of visual word recognition in the context of normal reading:

• Fast (avg fixation time ≈ 250 ms)

• Ecologically valid task

• Eye-mind span is tight

fixationonset

visualcortex

0 50 100 150 200 250 300 350 400

LEXICAL ACCESS

fixationonset

initiatesaccade

modify EMprogram

shift attention,initiate EM

motor program

signalto eye

muscles

EYE MOVEMENTS

P1

N1

P300

N400

Numberof trials

1

2

4

8

16

EEG

ERP

ERPs

Best real-time measure of brain activity associated with the perceptual and cognitive processing of words:

• Continuous ms-by-ms record of events

• Early, exogenous components (before 200 ms) should reflect lexical processing

(Sereno & Rayner, Trends in Cognitive Sciences, 2003)

DIVERSION

High-density ERP Analysis:A case of “too many notes”?

High-density ERP Analysis:Typical approaches for space &

time• Pick ‘n choose favourite electrode and ERP

component



component

• Hunt down where/when the effect is strongest and gather data from those electrodes/time window



component


• Procrustean regions analysis (turtle shell) or series of pre-set time windows (eg, 50, 100, 200 ms)



component


• Procrustean regions analysis (turtle shell) or series of pre-set time windows (eg, 50, 100, 200 ms)

• Spatial and/or temporal principal component analysis (PCA)

Scalp topography of the N1 @ 132-192 ms

SF1 loadings Voltages

(Sereno, Brewer, & O’Donnell, Psychological Science, 2003)

Word frequency

• Word frequency effect represents the differential response to commonly used high-frequency (HF) words versus low-frequency (LF) words that occur much less often.

• Presence of word frequency effects is used as a marker of successful word recognition or lexical access.

553 ms490 ms

259 ms275 ms

280 ms293 ms


Comparative EM & ERP studies

• Word frequency

• Word frequency X Context

The sore on Tam-Tam’s was swollen.(LF) rump(HF) back

(Neutral context) To our surprise we saw a(Biasing context) Flying to its nest was a hawk. (LF)

(Neutral context) She looked over the(Biasing context) She read the new book. (HF)

(Sereno, Rayner, & Posner, NeuroReport, 1998)(Sereno & Rayner, Perception & Psychophysics, 2000)(Sereno, Brewer, & O’Donnell, Psychological Science, 2003)

More comparative EM & ERP studies

• Word frequency X Orthographic regularity

• Lexical ambiguity X Context(Neutral context) James peered over at the(Biasing context) The mud was deep along the

bank.

Mike wasted the whole and then regretted it.

(LF-Reg) cask(LF-Exc) pint(HF-Reg) week(HF-Exc) hour

(Sereno, Pacht, & Rayner, Psychological Science, 1992)(Sereno, JEP: Learning, Memory, and Cognition, 1995)(Sereno, Rayner, & Posner, NeuroReport, 1998)(Sereno & Rayner, Perception & Psychophysics, 2000)(Sereno, Brewer, & O’Donnell, Psychological Science, 2003)


Lexical Ambiguity Resolution

• Interactive position– Access is selective: Context guides access towards the

appropriate sense of an ambiguous word; while both senses may be initially activated, only the contextually appropriate sense is fully accessed.

• Modular position– Access is exhaustive: All meanings of ambiguous words

are automatically accessed; context cannot directly affect lexical processing, but instead operates on the output of the lexical processor to select the appropriate sense.

Ambiguity: Cross-modal priming

• Paradigm: Aud amb VisualAuditory context prime target“The building was infested with BUGS” ANT “and it…” SPY SEW

• Results:– In general, support the modularity of lexical processing.

(Swinney, 1979)

Ambiguity: ERP unimodal priming

• Paradigm:– The only ERP ambiguity study employed a

unimodal (visual) version of the cross-modal paradigm (Van Petten & Kutas, 1987).

– Measured ERPs to targets that followed presentation of the ambiguous word prime.

• Results:– Support an interactive account of lexical

processing.

Ambiguity: EM “fast priming”

• Paradigm:

– Measured fixation duration on targets that followed presentation of the “fast” ambiguous word prime across various context conditions ...

*-------------*-----

A big black rhn captured her attention. PARAFOVEAL PREVIEW

----*A big black bug captured her attention. “FAST PRIME” ≈ 35 ms

*------------*----------------*A big black ant captured her attention. TARGET


Appropriately Related prime Unrelated prime

Dom The little girl liked to observe insects. The little girl ...A big black (bug)ant captured her attention. A big black (log)ant ...

Sub Mary’s apartment is under surveillance. Mary’s apartment ...A foreign (bug)spy may be hiding out with her. A foreign (log)spy ...

= = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = = =Inappropriately Related prime Unrelated prime

Dom Mary’s apartment is under surveillance. Mary’s apartment ...A big black (bug)ant captured her attention. A big black (log)ant ...

Sub The little girl liked to observe insects. The little girl ...A foreign (bug)spy may be hiding out with her. A foreign (log)spy ...

PrimingEffect

~30 ms

n.s.

n.s.

n.s.

(Sereno, JEP: LMC, 1995)


• Results: Support a modified interactive account of lexical

processing - “reordered access” - in which both (1) meaning frequency (Dom vs. Sub)

(2) prior context

affect access speed. Specifically,(1) Alternative meanings become activated in order of their

meaning frequency

(2) Context can “boost” the activation of one of the meanings, possibly reordering access procedures

Ambiguous The mud was deep along the bank ...(subordinate)

LF The mud was deep along the brim ...

HF The mud was deep along the edge ...

(Sereno, Pacht, & Rayner, Psych Sci, 1992)

Fixation time: Amb = LF > HF“Spillover” time: Amb > LF > HF

Support modified interactive account of access: “reordered access”.

• Paradigm:

• Results:

Ambiguity: EM normal reading

Critique of methods:Cross-modal priming

• RT (lexical decision, naming)– Slow (500-900 ms) compared to speed of lexical

access (~100-200 ms); susceptible to response bias.

• Secondary (indirect) measure– Effects of context on ambiguous word gauged by

priming effects on target downstream.

Critique of methods:ERP unimodal priming

• Time-locked averages of the EEG– Ms-by-ms voltage fluctuations reflect processing in

real time.

• Secondary (indirect) measure– Effects of context on ambiguous word gauged by

priming effects on target downstream.

Critique of methods:EM “fast priming”

• Fixation time– Relatively fast (~375 ms) and on-line, but can

reflect lexical and post-lexical integration effects.

• Secondary (indirect) measure– Although much quicker time course than cross-

modal or ERP unimodal, effects of context on ambiguous word still gauged by priming effects on target downstream

Critique of methods:EM normal reading

• Fixation time– Fast (~250 ms) and on-line, but can reflect lexical

and post-lexical integration effects.

• Primary (direct) measure– Effects of context on ambiguous word gauged by

comparing its fixation time to control word.

ERP Ambiguity Experiment

• Biased ambiguous words were presented in neutral and biasing contexts in a word-by-word sentence reading paradigm. Biasing contexts always instantiated the subordinate sense.

• ERPs on the ambiguous words, themselves, were measured.

• ERPs to ambiguous words were then directly compared to ERPs to unambiguous control words.

• Control words - matched either to the dominant (HF) sense of the ambiguous word or to the contextually instantiated subordinate (LF) sense of the ambiguous word - were presented in neutral and biasing contexts.

• Comparisons across conditions were made at an early, lexical stage of processing (N1, 132-192 ms).

Example Stimuli

Group 1 Group 2

Neutral Biasing

LF

HF

Amb

To our surprise we saw a hawk.

She looked over the book.

James peered over at the bank.

Flying to its nest was a hawk.

She read the new book.

The mud was deep along the bank.

Biasing Neutral

LF

HF

Amb

Pirates headed out to the cove.

The pharmacist distributed the drug.

They measured in terms of feet.

They navigated through the cove.

Sally knew about the drug.

They counted the number of feet.





± 0.7 factor loading contours

N1 SF1 Scores

-0.2

-0.1

0

0.1

0.2

Neutral Biasing

Context

Factor Score

LFHFAmb

N1 Voltages

1.2

1.4

1.6

1.8

Neutral Biasing

Context

µVLFHFAmb

Factor scores for SF1 Voltages

(electrodes with SF1 loading > +0.7

or SF1 loading < -0.7)

N1 @ 132-192 ms


Summary• From the EM record, we can infer when lexical

processing should occur (~100-200 ms).

• From the ERP record, we can see when certain differences first appear in real time.

• Effects of word frequency as well as context initially appear very early in the ERP record (N1 @ 132 ms post-stimulus).

• We can begin to establish a realistic time-line of word recognition in reading.

ERP Word Recognition

Words

Reg ExcLF cask pintHF time hour

PseudoWords

welf

Consonant Strings

fhvr

Lexicality W vs PW vs CS P1 @ 100-132 msFrequency LF vs HF N1 @ 132-164 msRegularity LF Reg vs LF Exc P2 @ 164-196 ms

Stimuli

Results

(Sereno, Rayner, & Posner, NeuroReport, 1998)

.001

.01.05

.001.01.05

CS-W PW-W CS-PW p <

Lexicality Effects:P1 (100-132 ms)


Frequency Effects:N1 (132-164 ms)

100 200 300 400 500ms

-1

1

LF words

HF words

µV

LF-HF


Regularity Effects:P2 (164-196 ms)

LF Exc -LF RegSs with RT effect Ss with no RT effect


ERPs

fixationonset

visualcortex

0 50 100 150 200 250 300 350 400

LEXICAL ACCESS

fixationonset

initiatesaccade

modify EMprogram

shift attention,initiate EM

motor program

signalto eye

muscles

EYE MOVEMENTS

N1

P1

P300

μV123

−1−2−3−4

400N

2N2P

LEXICALITY

REGULARITY

FREQUENCY



± 0.7 factor loading contours(Sereno, Brewer, & O’Donnell, Psychologcial Science, 2003)

Emotion words

Valence

Arousal

+ ve

– ve

Lo Hi

peace love

bored fire

Neutral controls: hotel, farm

Questions of interest:

• How fast are words recognised?

• What factors affect lexical access?

• How early do these factors operate?

Models• Interactive vs. Modular

– Logogen (Morton, 1969)

– PDP (Seidenberg & McClelland, 1989)

• Read-out– Search (Forster & Bendall, 19xx; )

• Hybrid – 2-stage: generate candidate set selection

– Becker & Killion, 19xx; Norris, 1984; Potter

Q: Why are camels called “ships of the desert?”A: Because they’re always filled with Arab sea-men.

Q: Would you prefer roses on your piano ortulips on your organ?

Q: What’s the difference between a rolling stoneand a Scotsman?

A: A Rolling Stone says “Hey you get off of my cloud!”and a Scotsman says “Hey McLeod get off my ewe!”

Q: Why is men’s ‘sea-men’ white and their urine yellow?A: So they can tell if they’re coming or going.

Jokes

Current Directions

• Emotion word processing

• Contextual constraint

ERP Ambiguity Experiment

• Design/Stimuli6 experimental conditions:

Word Type X ContextLF NeutralHF BiasingAmb

Word specifications:LF = 6 per millionHF = 60 per millionAmb = 63 per million (Dominant sense = 89

% Subordinate sense = 9%)

Spatial Principal Components Analysis

• Sample-by-sample voltage data at all 129 electrodes for all 14 Ss in all 6 conditions were submitted to a spatial PCA with a Quartimax rotation (cf. Spencer, Dien, & Donchin, 1998).

• The first Spatial Factor (SF1) accounted for a high degree of the variance (44%).

• 3(Word Type) x 2(Context) ANOVA was performed on the factor scores.

Documents

Word Recognition (Sereno, 4/04) How long does it take to recognise a visual word? –What is meant by “recognition” or “lexical access”? –Can lexical access