Upload
angelo-faires
View
219
Download
3
Tags:
Embed Size (px)
Citation preview
Dynamic attention and predictive tracking
Todd S. HorowitzVisual Attention Laboratory
Brigham & Women’s Hospital
Harvard Medical School
Lomonosov Moscow State University Cognitive Seminar, 6/10/2004
lab photo
Jeremy Wolfe
David FencsikGeorge Alvarez
Sarah Klieger
Randy Birnkrant
Jennifer DiMase
Helga Arsenio Linda Tran (not pictured)
Multi-element visual tracking task (MVT)
• Devised by Pylyshyn & Storm (1988)
• Method for studying attention to dynamic objects
Multi-element visual tracking task (MVT)
• Present several (8-10) identical objects
• Cue a subset (4-5) as targets
• All objects move independently for several seconds
• Observers asked to indicate which objects were cued
Demo
mvt4
demo
Interesting facts about MVT
• Can track 4-5 objects (Pylyshyn & Storm, 1988)
• Tracking survives occlusion (Scholl & Pylyshyn, 1999)
• Involves parietal cortex (Culham, et al, 1998)
• “Clues to objecthood” - Scholl
Accounts of MVT performance
• FINSTs (Pylyshyn, 1989)
• Virtual polygons (Yantis, 1992)
• Object files (Kahneman & Treisman, 1984)
• “Object-based attention”
These are all (partially) wrong
• FINSTs (Pylyshyn, 1989)
• Virtual polygons (Yantis, 1992)
• Object files (Kahneman & Treisman, 1984)
• “Object-based attention”
Common assumptions
• Low level (1st order) motion system updates higher-level representation– FINST– Object file– Virtual polygon
• Continuous computation in the present
Overview
• MVT and attention
• Tracking across the gap
• Tracking trajectories
MVT and attention
• Clearly a limited-capacity resource
• Attentional priority to tracked items (Sears & Pylyshyn)
• Hypothesis: MVT is mutually exclusive with other attentional tasks
George Alvarez, Helga Arsenio, Jennifer DiMase, Jeremy Wolfe
MVT and attention
• Clearly a limited-capacity resource
• Attentional priority to tracked items (Sears & Pylyshyn)
• Hypothesis: MVT is mutually exclusive with visual search
MVT and attention
• Clearly a limited-capacity resource
• Attentional priority to tracked items (Sears & Pylyshyn)
• Hypothesis: MVT is mutually exclusive with visual search
• Method: Attentional Operating Characteristic (AOC)
AOC Theory.
% Correct (Search)
% C o r r e c t ( T r a c k i n g )
chanceperformancefor each task
tracking aloneperformance
s e a r c h a l o n ep e r f o r m a n c e
completeindependence
mutuallyexclusive data
fall on this line
General methods - normalization
• Single task = 100
• Chance = 0
• Dual task performance scaled to distance between single task performance and chance
General methods - staircases
• Up step (following error) = 2 x down step
• Asymptote = 66.7% accuracy
• Staircase runs until 20 reversals
• Asymptote computed on last 10 reversals
General methods - tracking
• 10 disks
• 5 disks cued
• Speed = 9°/s
AOC Theory.
% Correct (Search)
% C o r r e c t ( T r a c k i n g )
chanceperformancefor each task
tracking aloneperformance
s e a r c h a l o n ep e r f o r m a n c e
completeindependence
mutuallyexclusive data
fall on this line
AOC reality
• Tasks can interfere at multiple levels
• Interference can occur even when resource of interest (here visual attention) is not shared
• How “independent” are two attention-demanding tasks which do not share visual attention resources?
Gold standard: tracking vs. tone detection
Gold standard method
• Tracking– Duration = 6 s
• Tone duration– 10 600 Hz tones– Onset t = 1 s– ITI = 400 ms– Distractor duration = 200 ms– Task: target tone longer or shorter?– Target duration staircased (31 ms)– Dual task priority varied
N = 10
0 25 50 75 100 1250
25
50
75
100
125
tone accuracy
Gold standard AOC
Tracking + search method
• Tracking– Duration = 5 s
• Search– 2AFC “E” vs. “N”– Distractors = rest of alphabet– Set size = 5– Duration staircased (mean = 156 ms)– Onset = 2 s
N = 9
Tracking + search method
.
E
T
R
B
H
Tracking + search AOC
0 25 50 75 100 1250
25
50
75
100
125
search accuracy
Tracking + search AOC
0 25 50 75 100 1250
25
50
75
100
125visual search tone
tone|search accuracy
Does tracked status matter?
T
L
L
L
T
L
method
• Tracking– Duration = 3 s
• Search– 2AFC left- or right-pointing T– Distractors = rotated Ls– Set size = 5– Duration staircased (mean = 218 ms)– Onset = 1 s
N = 9
search inside tracked set
T
LT
LL
L
L
search outside tracked set
T
LTL
L
L
L
mixed blocked
search inside tracked set
search outside tracked set
inside vs. outside AOC
0 25 50 75 100 1250
25
50
75
100
125 mixed - out
blocked - out
mixed - in
search accuracy
Does spatial separation matter?
E
F
V
H
P
method
• Tracking– Duration = 5 s
• Search– 2AFC “E” vs. “N”– Distractors = rest of alphabet– Set size = 5– Duration = 200 ms– Onset = 2 s
N = 9
spatial separation AOC
0 25 50 75 100 1250
25
50
75
100
125
search accuracy
search v track summary
0 25 50 75 100 1250
25
50
75
100
125
tone|search accuracy
MVT and search
• Clearly not mutually exclusive
• Not pure independence
• Close to gold standard
• MVT and search use independent resources?
Two explanations
• Separate attention mechanisms
• Time sharing
Predictions of time sharing hypothesis
• Should be able to leave tracking task for significant periods with no loss of performance
• Should be able to do something in that interval
Track across the gap method
Track across the gap method
• Track 4 of 8 disks
• Speed = 6°/s
• Blank interval onset = 1, 2, or 3 s
• Trajectory variability: 0°, 15°, 30°, or 45° every 20 ms
• Blank interval duration staircased (dv)
• N = 11
0 10 20 30 40 50
350
400
450
500
550
variability (°)
track across the gap asymptotes
Predictions of time sharing hypothesis
• Should be able to leave tracking task for significant periods with no loss of performance (see also Yin & Thornton, 1999) - confirmed
• Should be able to do something (e.g. search) in that interval
search during gap method
• AOC method• Tracking task same as before• Search task in blank interval
– Target = rotated T– Distractors = rotated Ls– Set size = 8– 4AFC: Report orientation of T
• Duration of search task staircased (326 ms)
0 25 50 75 100 1250
25
50
75
100
125
search accuracy
search during gap AOC
0 25 50 75 100 1250
25
50
75
100
125
tone|search accuracy
Predictions of time sharing hypothesis
• Should be able to leave tracking task for significant periods of time with no loss of performance (see also Yin & Thornton, 1999) - confirmed
• Should be able to do something (e.g. search) in that interval - confirmed
Summary
• MVT and visual search can be performed independently in the same trial
• May support independent “visual attention” mechanisms
• May support time-sharing
Summary
• Tracking across the gap data support time sharing
• Tracking across the gap data raise new questions
What is the mechanism?
• Not a continuous computation in the present
• Not first order motion mechanisms
• Not apparent motion
Randall Birnkrant, Jennifer DiMase, Sarah Klieger, Linda Tran, Jeremy Wolfe
None of these theories fit
• FINSTs (Pylyshyn, 1989)
• Virtual polygons (Yantis, 1992)
• Object files (Kahneman & Treisman, 1984)
What is the mechanism?
• Some sort of amodal perception? (e.g. tracking behind occluders, Scholl & Pylyshyn, 1999)
• … but there are no occlusion cues!
Scholl & Pylyshyn, 1999
Maybe the gap is just an impoverished occlusion stimulus
• No occlusion/disocclusion cues
• Synchronous disappearance
Predictions of impoverished occlusion hypothesis
• Occlusion cues will improve performance
• Asynchronous disappearance will improve performance
Method
• Track for 5 s• Speed = 12°/s• Track 4 of 10 disks• Independent variables (blocked)
– Gap duration:107 ms, 307 ms, 507 ms– Occlusion cues absent, present– Disappearances synchronous, asynchronous
• N = 15
synchronous disappearance
all items reappear simultaneously
items invisible but continue to move
synchronous disappearance + occlusion
occlusion begins
disocclusion begins
Occlusion/Disocclusion
asynchronous disappearance
item reappears
one item at a time disappears but continues to move
asynchronous disappearance + occlusion
... moves while invisible...
... then disoccludes
one item at a time begins to be occluded...
synchronous asynchronous0.75
0.80
0.85
0.90
0.95
1.00 disappearocclude
comparing cue types
Occlusion hypothesis fails
• Occlusion cues don’t help
• Asynchronous disappearance doesn’t help
Method
• Track for 5 s• Speed = 12°/s• Synchronous condition only• Independent variables (blocked)
– Gap duration:107 ms, 307 ms, 507 ms– Occlusion cues absent, present– Track 4, 5, or 6 of 10 disks
• N = 11
comparing cue types
4 5 60.75
0.80
0.85
0.90
0.95
1.00disappearocclude
number of targets
Occlusion hypothesis fails
• Occlusion cues don’t help
• Occlusion cues can actually harm performance
• Asynchronous disappearance doesn’t help
What is the mechanism?
• Not a continuous computation in the present
• Not first order motion mechanisms
• Not apparent motion
• Not amodal perception (occlusion)
How do we reacquire targets?
• remember last location (backward)
• store trajectory (forward)
David Fencsik, Sarah Klieger, Jeremy Wolfe
location-matching account
Memorizedpre-gap targetlocation.
Nearest tomemorizedlocation:identified as target.
First Post-Gap Frame
trajectory-matching account
Memorizedpre-gap targettrajectory.
On target trajectory: identified as target.
First Post-Gap Frame
Shifting post-gap location
d
d
0
Last visible pre-gap location
opposite of expected location
-1
Expected post-gap location
+1
= Stimulus trajectory
shifting post-gap location predictions
-1 0 10.60
0.65
0.70
0.75
0.80locationtrajectory
shift
Shifting post-gap location methods
• track for 5 s
• speed = 8°/s
• track 5 of 10 disks
• gap duration = 300 ms
• post-gap location condition blocked
• stimuli continue to move after gap
shifting post-gap location
-1 0 10.60
0.65
0.70
0.75
0.80
shift
Location vs. trajectory-matching
• support for location-matching– see also Keane & Pylyshyn 2003; 2004
• but advantage for -1 is suspicious
Location vs. trajectory-matching
time +1.0
+time +2.0
++
time +1.5
shift & stop methods
• track for 4-6 s
• speed = 9°/s
• track 2 or 5 of 10 disks
• gap duration = 300 ms
• post-gap location condition blocked
• stimuli stop after gap
moving vs. static after gap
-1 0 10.60
0.65
0.70
0.75
0.80
motion after gap
shift
moving vs. static after gap
-1 0 10.60
0.65
0.70
0.75
0.80
motion after gap
static after gap
shift
2 vs. 5 targets
-1 0 10.60
0.65
0.70
0.75
0.80
0.85
0.90
0.95
1.00 5 targets2 targets
shift
Location vs. trajectory-matching
• support for location-matching
• However...– conditions are blocked– observers might see their task not as tracking
across the gap, but learning which condition they’re in
– might not tell us about normal target recovery
Location vs. trajectory-matching
• can subjects use trajectory information?
• always have items move during gap
• vary whether trajectory information is available or not
moving condition
invisible motion
static condition
invisible motion
manipulate pre-gap information methods
• track for 4 s
• speed = 9°/s
• track 1 to 4 of 10 disks
• gap duration = 300 ms
manipulate pre-gap information
0 1 2 3 4 50.75
0.80
0.85
0.90
0.95
1.00
movingstatic
targets
manipulate pre-gap information
0 1 2 3 4 50.75
0.80
0.85
0.90
0.95
1.00
movingstatic
targets
Location vs. trajectory-matching
• observers can use trajectory information
• unlimited (or at least > 4) capacity for locations
• smaller (1 or 2) capacity for trajectories
Conclusions
• Flexible attention system allows rapid switching between MVT and other attention-demanding tasks
• Some representation allows recovery of tracked targets after 300-400 ms gaps
• This representation includes location and trajectory information
Speculation
• MVT reveals two mechanisms, rather than just one
• Frequently (but perhaps not continuously) updated location store
• Attention to trajectories