View
269
Download
2
Tags:
Embed Size (px)
Citation preview
The Human Eye At its center is the fovea, a
pit that is most sensitive to light and is responsible for our sharp central vision.
The central retina is cone-dominated and the peripheral retina is rod-dominated.
Eye Movements Within the visual field, eye movements serve
two major functions Saccades to Fixations – Position target objects of
interest on the fovea Tracking – Keep fixated objects on the fovea
despite movements of the object or head
Fixations The eye is (almost) still – perceptions are
gathered during fixations The most important of eye “movements”
90% of the time the eye is fixated duration: 150ms - 600ms
Saccades Saccades are used to move the fovea to the
next object/region of interest. Connect fixations Duration 10ms - 120ms
Very fast (up to 700 degrees/second) No visual perception during saccades
Vision is suppressed Evidence that some cognitive processing may also be
suppressed during eye-movements (Irwin, 1998)
Saccades Saccades are used to move the fovea to the
next object/region of interest. Connect fixations Duration 10ms - 120ms
Very fast (up to 700 degrees/second) No visual perception during saccades
Vision is suppressed Ballistic movements (pre-programmed) About 150,000 saccades per day
Smooth Pursuit
Smooth movement of the eyes for visually tracking a moving object
Cannot be performed in static scenes (fixation/saccade behavior instead)
Smooth Pursuit versus Saccades
Saccades Jerky No correction Up to 700
degrees/sec Background is not
blurred (saccadic suppression)
Smooth pursuit Smooth and
continuous Constantly corrected
by visual feedback Up to 100 degrees/sec Background is blurred
Eye-movements in reading
Clothes make the man. Naked people have little or no influence on society.
Eye-movements in reading are saccadic rather than smooth
Eye-movements in reading
Clothes make the man. Naked people have little or no influence on society.
Eye-movements in reading are saccadic rather than smooth
Eye-movements in reading
Clothes make the man. Naked people have little or no influence on society.
Eye-movements in reading are saccadic rather than smooth
Eye-movements in reading
Clothes make the man. Naked people have little or no influence on society.
Eye-movements in reading are saccadic rather than smooth
Eye-movements in reading
Clothes make the man. Naked people have little or no influence on society.
Eye-movements in reading are saccadic rather than smooth
Eye-movements in reading
Clothes make the man. Naked people have little or no influence on society.
Eye-movements in reading are saccadic rather than smooth
Eye-movements in reading
Clothes make the man. Naked people have little or no influence on society.
Eye-movements in reading are saccadic rather than smooth
Eye-movements in reading
Clothes make the man. Naked people have little or no influence on society.
Eye-movements in reading are saccadic rather than smooth
Eye-movements in reading Limitations of the visual field
130 degrees vertically, 180 degrees horizontally (including peripheral vision
Perceptual span for reading: 7-12 spaces
Clothes make the man. Naked people have little or no influence on society.
Purkinje Eye Tracker
Laser is aimed at the eye. Laser light is reflected by cornea
and lens Pattern of reflected light is
received by an array of light-sensitive elements.
Very precise Also measures pupil
accomodation No head movements
Measuring Eye Movements
Measuring Eye Movements
Video-Based Systems Infrared camera directed at eye Image processing hardware determines pupil position and size (and possibly
corneal reflection) Good spatial precision (0.5 degrees) for head-mounted systems Good temporal resolution (up to 500 Hz) possible
S
NP VPV NP
NP
Ndet Ndet
Themanhit dogwiththeleash.the
The manhit dog
PP
withtheleash
the
Instrument
Parsing
The syntactic analyser or “parser” Main task: To construct a syntactic
structure from the words of the sentence as they arrive
Different approaches Serial Analysis (Modular): Build just one based on
syntactic information and continue to try to add to it as long as this is still possible
Interactive Analysis: Use multiple levels (both syntax and semantics) of information to build the “best” structure
Parallel Analysis: Build both alternative structures at the same time
Minimal Commitment: Stop building - and wait until later material clarifies which analysis is the correct one.
Sentence Comprehension
ModularWord Recognition
SyntacticModule
Semantic Module
Word Recognition
SyntacticModule
Semantic Module
Interactive models
Sentence Comprehension Garden path sentences
A garden path sentence invites the listener to consider one possible parse, and then at the end forces him to abandon this parse in favor of another.
Real Headlines Juvenile Court to Try Shooting Defendant Red tape holds up new bridge Miners Refuse to Work after Death Retired priest may marry Springsteen Local High School Dropouts Cut in Half Panda Mating Fails; Veterinarian Takes Over Kids Make Nutritious Snacks Squad Helps Dog Bite Victim Hospitals are Sued by 7 Foot Doctors
Sentence Comprehension Garden path sentences
The horse raced past the barn fell.
VP
V
raced
S
NP
The horse
Sentence Comprehension Garden path sentences
The horse raced past the barn fell.
VP
V PP
P NP
racedpast
S
NP
The horse
Sentence Comprehension Garden path sentences
The horse raced past the barn fell.
VP
V PP
P NP
racedpastthe barn
S
NP
The horse
Sentence Comprehension Garden path sentences
The horse raced past the barn fell.
VP
V PP
P NP
racedpastthe barn
S
NP
The horse fell
Sentence Comprehension Garden path sentences
The horse raced past the barn fell.
VP
V PP
P NP
racedpastthe barn
S
NP
The horse
raced is initially treated as a past tense verb
Sentence Comprehension Garden path sentences
The horse raced past the barn fell.
VP
V PP
P NP
racedpastthe barn
S
NP
The horse fell
raced is initially treated as a past tense verb
This analysis fails when the verb fell is encountered
Sentence Comprehension Garden path sentences
The horse raced past the barn fell.
VP
V PP
P NP
racedpastthe barn
S
NP
The horse fell
raced is initially treated as a past tense verb
This analysis fails when the verb fell is encountered
raced can be re-analyzed as a past participle.
VP
V
raced
PP
P NP
pastthe barn
S
NP
The horse fell
NP RR
V
A serial model Formulated by Lyn Frazier (1978, 1987)
Build trees using syntactic cues: phrase structure rules plus two parsing principles
Minimal Attachment Late Closure
A serial model Minimal Attachment
Prefer the interpretation that is accompanied by the simplest structure.
simplest = fewest branchings (tree metaphor!) Count the number of nodes = branching points
The girl hit the man with the umbrella.
S
NP
the girl
VP
V
hit
NP
the man
PP
P
with
NP
the umbrella
S
NP
the girl
VP
V
hit
NP
NP
the man
PP
P
with
NP
the umbrella
The girl hit the man with the umbrella.
8 Nodes
9 nodes
Minimal attachment
Preferred
A serial model Late Closure
Incorporate incoming material into the phrase or clause currently being processed.
OR Associate incoming material with the most recent material possible.
She said he tickled her yesterday
Parsing Preferences .. late closure
She said he tickled her yesterday
S
np
she
vp
v
said
S'
np
he
vp
v
tickled
np
her
adv
yesterday
S
np
she
vp
v
said
S'
np
he
vp
v
tickled
np
her
adv
yesterday
Preferred
(Both have 10nodes, so use LCnot MA)
Minimal attachment Garden path sentences
The spy saw the cop with a telescope.
minimal attach
non-minimal attach
Modular predictionBuild this structure first
Interactive predictionBuild this structure first
(Rayner & Frazier, ‘83)
Minimal attachment Garden path sentences
The spy saw the cop with a revolver.
minimal attach
non-minimal attach
Modular predictionBuild this structure first
Interactive predictionBuild this structure first
Lexical information rules this one out
(Rayner & Frazier, ‘83)
MA Non-MAS
NP
the spyVP
V
sawNP
the cop
PP
P
with
NP
the revolver
S’
but the cop didn’t see him
S
NP
the spyVP
V
saw
NP
NP
the cop
PP
P
with
NP
the revolver
S’
but the cop didn’t see him
The spy saw the cop with the binoculars.. The spy saw the cop with the revolver … (Rayner & Frazier, ‘83)
<- takes longer to read
Interactive Models
The evidence questioned in the trial … The person questioned in the trial …
evidence typically gets questioned, but can’t do the questioning
Other factors (e.g., semantic context, co-occurrence of usage & expectation) may provide cues about the likely interpretation of a sentence
Interactive Models Other factors (e.g., semantic context, co-occurrence of
usage & expectation) may provide cues about the likely interpretation of a sentence
The evidence questioned in the trial … The person questioned in the trial …
A lawyer often asks questions (more often than answering them)
Semantic expectations
Taraban & McCelland (1988) Expectation
The couple admired the house with a friend but knew that it was over-priced.
The couple admired the house with a garden but knew that it was over-priced.
Other factors (e.g., semantic context, co-occurrence of usage & expectation) may provide cues about the likely interpretation of a sentence
The Non-MA structure may be favoured
Semantic expectations Taraban & McCelland, 1988
The couple admired the house with a friend but knew that it was over-priced.
The couple admired the house with a garden but knew that it was over-
priced.
Intonation as a cue
A: I’d like to fly to Davenport, Iowa on TWA.
B: TWA doesn’t fly there ...
B1: They fly to Des Moines.
B2: They fly to Des Moines.
Chunking, or “phrasing”
A1: I met Mary and Elena’s mother at the mall yesterday.
A2: I met Mary and Elena’s mother at the mall yesterday.
50
100
150
200
250
300
350
400
Phrasing can disambiguate
I met Mary and Elena’s mother at the mall yesterday
Mary & Elena’s mothermall
One intonation phrase with relatively flat overall pitch range.
50
100
150
200
250
300
350
400
Phrasing can disambiguate
I met Mary and Elena’s mother at the mall yesterday
Marymall
Elena’s mother
Separate phrases, with expanded pitch movements.