Upload
branden-black
View
214
Download
1
Tags:
Embed Size (px)
Citation preview
The role of contours in form-The role of contours in form-based motion processingbased motion processing
P.U. Tse, G. P. Caplovitz, and P.-J. Hsieh
Form from motion orStructure from motion
Form from motion
Motion from form
Form from motion
Motion from form
1. Transformational Apparent Motion
Form from motion
Motion from form
1. Transformational Apparent Motion
2. Trackable features: Object shape can influence how fast it is perceived to rotate
Form from motion
Motion from form
1. Transformational Apparent Motion
2. Trackable features: Object shape can influence how fast it is perceived to rotate
3. Emergent motion signals
Overview of talk
• Theoretical background to contour discontinuities
• The role of contour discontinuities in form/motion processing• Transformational apparent motion • Motion from feature tracking• Emergent motion signals
Overview of talk
• Theoretical background to contour discontinuities
• The role of contour discontinuities in form/motion processing• Transformational apparent motion • Motion from feature tracking• Emergent motion signals
Contour discontinuities
• What are they?
• Why are they important?
Attneave, 1954
Attneave, 1954
Attneave, 1954
Attneave, 1954
Attneave, 1954
Attneave, 1954
Biederman, 1987
Biederman, 1987
Biederman, 1987
Curvature detectors
Curvature detectors
(
Wolfe, Yee & Friedman-Hill (1992).
Curvature detectors
( (
(
((
(
(
(
(
(
((
(
(
((
(( (
((
((
(
(
(
(
(
(
(
( (((
(
( (
Wolfe, Yee & Friedman-Hill (1992).
Curvature detectors
( (
(
((
(
(
(
(
(
((
(
(
((
(( (
((
((
(
(
(
(
(
(
(
( (((
(
( (
There exist filters tuned to curvature
Wolfe, Yee & Friedman-Hill (1992).
Curvature as a cue to 3D shape
Tse, 2002, Psych. Review
Theory: The visual system locatespoints of contour curvature discontinuityin order to generate shape-from-contours
Tse (2002, Psych. Review)
Caveat: much more is going on in shape formation
Caveat: much more is going on in shape formation
Caveat: much more is going on in shape formation
Caveat: much more is going on in shape formation
Curvature as a cue to 3D shape
Curvature as a cue to 3D shape
Curvature as a cue to 3D shape
Curvature as a cue to 3D shape
Curvature as a cue to 3D shape
a
b
c
ba
+
=
bump ellipse
ad
d
d = 1/2(b+c)
a)
c)
b)
a
c
b
d)bumps among ellipses ellipses among bumps
Curvature discontinuities pop out
Kristjansson & Tse, 2001, P&P
Curvature discontinuities pop out
Kristjansson & Tse, 2001, P&P
Curvature discontinuities pop out
Kristjansson & Tse, 2001, P&P
Curvature discontinuities pop out
Kristjansson & Tse, 2001, P&P
Curvature discontinuities pop out
Kristjansson & Tse, 2001, P&P
Curvature discontinuities pop out
Kristjansson & Tse, 2001, P&P
Curvature discontinuities pop out
Kristjansson & Tse, 2001, P&P
There exist filters tuned to curvature discontinuities
Main theoretical point
• Contour discontinuities include:
*corners
*junctions
*terminators
*deep concavities and convexities
• These are highly informative when constructing 3D shape and 3D motion
Overview of talk
• Theoretical background to contour discontinuities
• The role of contour discontinuities in form/motion processing
• Transformational apparent motion
• Motion from feature tracking
• Emergent motion signals
Overview of talk
• Theoretical background to contour discontinuities
• The role of contour discontinuities in form/motion processing
• Transformational apparent motion
• Motion from feature tracking
• Emergent motion signals
Overview of talk
• Theoretical background to contour discontinuities
• The role of contour discontinuities in form/motion processing
• Transformational apparent motion
• Motion from feature tracking
• Emergent motion signals
Why did you brain match like this?
And not like this?
Does form influence matching?
Does form influence matching?
Does form influence matching?
Does form influence matching?
?
?
• Form did not seem to play much if any role in matching. People assumed that motion-energy and the nearest neighbor principle dominated matching.
• Form did not seem to play much if any role in matching. People assumed that motion-energy and the nearest neighbor principle dominated matching.
• Tse, Nakayama, and Cavanagh argued that this was an artifact of using translational apparent motion as a probe.
Frame 1 PerceptFrame 2
Transformational Apparent Motion
Translational Apparent Motion
Transformational apparent motion
Tse, Nakayama, and Cavanagh, 1996, 1998
Transformational apparent motion
Transformational apparent motion
Transformational apparent motion
Transformational apparent motion
Transformational apparent motion
Tse, Nakayama, and Cavanagh, 1996, 1998
Transformational apparent motion
Tse, Nakayama, and Cavanagh, 1996, 1998
Transformational apparent motion
Tse, Nakayama, and Cavanagh, 1996, 1998
Transformational apparent motion
Tse, Nakayama, and Cavanagh, 1996, 1998
Transformational apparent motion
Tse and Logothetis, 2002
Transformational apparent motion
Tse and Logothetis, 2002
High-level vs Low-level motion processing
Low-level: Motion energy dependent,
nearest neighbor matching
High-level vs Low-level motion processing
Low-level: Motion energy dependent,
nearest neighbor matching
High-level: Matching takes place on the basis 3D form relationships. High-level motion paths are determined following a stage of segmentation, grouping, and 3D analysis that is particularly sensitive to contour relationships and occlusion among attended figures.
Mri data specs• GE 1.5T signa
• 1-shot T2*-weighted gradient-recalled EPI
• block design, quasi-random order
• 20 second epochs, 8 vols
• TR=2.5s, 25 slices, FA=90degrees
• voxel size: 3.75x3.75x5mm
• EPI FOV 240x240x256mm
• Data analysis BV4.9, matlab
• Stimulus back-projected, FOV 22x16 degrees
Stimuli
T C C CT T
T C CT TC
Stimuli
T C C CT T
T C CT TC
Task: press button when fixation point transientlydisappears for 17ms.
• See TAM demo
Back Right
Left Bottom
right left
rearbottom
n=19
Back Right
Left Bottom
right left
rearbottom
hMT+
n=19
Back Right
Left Bottom
right left
rearbottom
hMT+ LOC
n=19
**
**
*
*
*
Average Bilateral BOLD Response by Area (n=10; MT/LOC n=7)
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
V1 V2d V2v V3d V3v V3A/B V4v MT-LOC
LOC-MT
% B
OL
D S
ign
al C
ha
ng
e
TAM Control
*
*
*
* **
*
Average Bilateral Beta Weight by Area
0
0.5
1
1.5
2
2.5
V1 V2d V2v V3d V3v V3a V4v MT-LOC
LOC-MT
Nor
mal
ized
Bet
a W
eigh
t
TAM Control
*
* **
*
*
Ventral pathway Dorsal pathway
V1
V2 thick stripe
V3A
MT
V2 thin stripe
V4v
MSTLOC
V3v V3d
Ventral pathway Dorsal pathway
V1
V2 thick stripe
V3A
MT
V2 thin stripe
V4v
MSTLOC
V3v V3d
The key idea here
• Form analysis occurs in retinotopic and ventral non-retinotopic areas BEFORE the perception of motion.
• Perceived motion direction follows a spatiotemporal analysis of GLOBAL FORM relationships among contours.
The key idea here
• Form analysis occurs in retinotopic and ventral non-retinotopic areas BEFORE the perception of motion.
• Perceived motion direction follows a spatiotemporal analysis of GLOBAL FORM relationships among contours.
The key idea here
• Form analysis occurs in retinotopic and ventral non-retinotopic areas BEFORE the perception of motion.
• Perceived motion direction follows a spatiotemporal analysis of GLOBAL FORM relationships among contours.
• Any model of motion processing that fails to explicitly incorporate a stage of global contour-based form analysis will be incomplete.
• Any model of motion processing that fails to explicitly incorporate a stage of global contour-based form analysis will be incomplete.
• For example, high-level motion cannot just be a motion-energy detector that receives input from a salience map. There must be an explicit stage of form analysis.
Overview of talk
• Theoretical background to contour discontinuities
• The role of contour discontinuities in form/motion processing
• Transformational apparent motion
• Motion from feature tracking
• Emergent motion signals
Overview of talk
• Theoretical background to contour discontinuities
• The role of contour discontinuities in form/motion processing
• Transformational apparent motion
• Motion from feature tracking
• Emergent motion signals
Velocity Space
The infinite number of motions that lead to the same
component vector all lie on a “constraint line” in vector
space
Percepts of Translational Motion
• The examination of Plaids – two oriented, overlapped, independently
moving sinewave gratings will appear to move as a plaid
– The direction and speed of motion can be predicted by various component-motion integration models
Integration
Example Predicted by
IOC or VS
Example Predicted by IOC
and not VS
Trackable Feature Solution to Aperture Problem
• Eyetracking and Neurophysiological Evidence suggests that certain form features (Endpoint Terminators) can be used by the visual system to disambiguate percepts of motion independent of component motion (Area MT: Pak and Born (2003)) (V1: Pak and Livingston (2003))
Components generated along a curved contour
IOC succeeds for Translational Motion
IOC succeeds for Translational Motion
IOC succeeds for Translational Motion
IOC succeeds for Translational Motion
IOC succeeds for Translational Motion
IOC Fails for Rotation
IOC Fails for Rotation
IOC Fails for Rotation
IOC Fails for Rotation
IOC Fails for Rotation
IOC Fails for Rotation
IOC for translating ellipse
IOC for rotating ellipse
• See ellipse demo
Rigidity = Preservation of Shape
Regions of High Curvature are Critical to the perception
of shape
Component Motion will always be non-rigid
Although Component Motion predicts variable speeds
Some form of form analysis must occur for stimulus to remain rigid, presumably
regions of high curvature are critical for rigidity
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Paradox!
Experiment 1
Experiment 2
Experiment 3
Experiment 4
vs.
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Determined equiluminant green for each subject
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 1
Experiment 2
Experiment 3
Experiment 4
• Now pit equiluminant ellipse against luminance-defined ellipse
vs.
• Equiluminant ellipses appear to rotate more slowly than luminance-defined ellipses
• Equiluminant ellipses appear to rotate more slowly than luminance-defined ellipses
• There is an interaction: as the ellipse gets thinner, the perceived angular velocity of an equiluminant ellipse becomes closer to that of a luminance-defined ellipse.
vs.
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Experiment 1
Experiment 2
Experiment 3
Experiment 4
Summary
• Global form-based processing of trackable features such as corners and regions of relatively high positive curvature serve as a motion cue in high-level motion processing.
• Perceived motion magnitude is a function of both low- and high-level inputs.
Question
• Which areas of the brain vary parametrically with contour curvature discontinuity points in motion?
Question
• Which areas of the brain vary parametrically with contour curvature discontinuity points in motion?
• All stimuli had the same area, but the sharpness of curvature discontinuities varied greatly across conditions.
• All stimuli had the same area, but the sharpness of curvature discontinuities varied greatly across conditions.
• All stimuli had the same area, but the sharpness of curvature discontinuities varied greatly across conditions.
• All stimuli had the same area, but the sharpness of curvature discontinuities varied greatly across conditions.
1 2 3 4 5
• Five types of stimuli were used, consisting of two half ellipses joined along their common major axis.
• Five types of stimuli were used, consisting of two half ellipses joined along their common major axis.
• Five types of stimuli were used, consisting of two half ellipses joined along their common major axis.
• Five types of stimuli were used, consisting of two half ellipses joined along their common major axis.
• Five types of stimuli were used, consisting of two half ellipses joined along their common major axis.
All stimuli rotated at the same speed.
• See Bumps stimuli
Stimuli
1 5 23 4
2 5 134
1 2 534
Stimuli
1 5 23 4
2 5 134
1 2 534
In order to guarantee that they were maintaining fixation, observers carried out a simple task at the fixation point that was not directly related to issues of curvature processing (press button when fixation point blinks).
• Hold area constant
• Hold max velocity constant
• Hold perceived angular
velocity constant
• Hold area constant
• Hold max velocity constant
• Hold perceived angular
velocity constant
• Hold area constant
• Hold max velocity constant
• Hold perceived angular
velocity constant
Summary
• Significant parametric variation between contour curvature discontinuities and BOLD signal occurs in V3A/B.
• V3A/B may process curvature discontinuities, junctions, and terminators that provide hMT+ with unambiguously moving features to disambiguate motion energy signals.
Summary
• Significant parametric variation between contour curvature discontinuities and BOLD signal occurs in V3A/B.
• V3A/B may process curvature discontinuities, junctions, and terminators that provide hMT+ with unambiguously moving features to disambiguate motion energy signals.
Summary
• Significant parametric variation between contour curvature discontinuities and BOLD signal occurs in V3A/B.
• V3A/B may process curvature discontinuities, junctions, and terminators that provide hMT+ with unambiguously moving features to disambiguate motion energy signals.
Ventral pathway Dorsal pathway
V1
V2 thick stripe
V3A
MT
V2 thin stripe
V4v
MSTLOC
V3v V3d
Ventral pathway Dorsal pathway
V1
V2 thick stripe
V3A
MT
V2 thin stripe
V4v
MSTLOC
V3v V3d
TAM results
Ventral pathway Dorsal pathway
V1
V2 thick stripe
V3A
MT
V2 thin stripe
V4v
MSTLOC
V3v V3d
Bumps results
Ventral pathway Dorsal pathway
V1
V2 thick stripe
V3A
MT
V2 thin stripe
V4v
MSTLOC
V3v V3d
Bumps results
Overview of talk
• Theoretical background to contour discontinuities
• The role of contour discontinuities in form/motion processing• Transformational apparent motion • Motion from feature tracking• Emergent motion signals
hMT
+
V2 V4
V3A
LOC
Ventral form system
: 3D surfaces
Dorsal form system
: Trackablefeatures
Key idea: Form determines motion
• There exist two interacting form analyses
• Ventral (V4v etc) 3D surfaces, volumes
• Dorsal (V3A etc)3D trajectories
• Both rely on contour discontinuities
Overview of talk
• Theoretical background to contour discontinuities
• The role of contour discontinuities in form/motion processing• Transformational apparent motion • Motion from feature tracking• Emergent motion signals
Conclusions
Conclusions
• Perceived motion direction follows global form analysis of contour relationships
Conclusions
• Perceived motion direction follows global form analysis of contour relationships
• This global form analysis occurs in V3, V4v,V3A/B, LOC, and hMT+.
Conclusions
• Perceived motion direction follows global form analysis of contour relationships
• This global form analysis occurs in V3, V4v,V3A/B, LOC, and hMT+.
• Perceived motion speed follows form analysis of trackable features (contour discontinuities)
Conclusions
• Perceived motion direction follows global form analysis of contour relationships
• This global form analysis occurs in V3, V4v,V3A/B, LOC, and hMT+.
• Perceived motion speed follows form analysis of trackable features (contour discontinuities)
• This form analysis occurs in V3A/B
Conclusions
• Perceived motion direction follows global form analysis of contour relationships
• This global form analysis occurs in V3, V4v,V3A/B, LOC, and hMT+.
• Perceived motion speed follows form analysis of trackable features (contour discontinuities)
• This form analysis occurs in V3A/B
• CDs play a key role in parsing and as TFs.
Conclusions
• Perceived motion direction follows global form analysis of contour relationships
• This global form analysis occurs in V3, V4v,V3A/B, LOC, and hMT+.
• Perceived motion speed follows form analysis of trackable features (contour discontinuities)
• This form analysis occurs in V3A/B
• CDs play a key role in parsing and as TFs.
• Motion perception based on emergent motion
Thank you