Why isn’t vision perfect? An exercise in psychoanatomy

Why isn’t vision perfect?An exercise in psychoanatomy

Why isn’t vision perfect?An exercise in psychoanatomy

Tracing the flow of information through the nervous system

using functional experiments

V1

LGN

Parietal (action)

temporal (perception)

If neural representation fails at any stage, perception will fail

The most basic aspect of vision: spatial resolution

Resolution has a limit:Coarse patterns are seen, fine detail is not

How and where does resolution fail?

We start at the beginning with the photons themselves

Cause of imperfect resolution:

A spreading of light, or of the effects of light

Campbell & Robson (1968)

Spatial frequency (cycles/degree)

Sensitivity

Contrast-Sensitivity Function (CSF)

Resolution limit: 50cpd

Factors that might limit visual resolution:

Optics of the image (including diffraction)Sampling by the retinal mosaic Light collection by the cone aperturesNeural convergence:

intra-retinal or retino-thalamicthalamo-corticalintra-cortical

Factors limiting visual resolution:Optics of the image (including diffraction)

One test for the role of optics:

Do perfect optics make vision perfect?

Two possible approaches:

• Test resolution using interference fringe targets, bypassing the optics; or…

• Use Adaptive Optics to compensate for individual optical aberrations

DaveWilliams

Laser interferometrybypassses optical losses:target stripes are generated directly on the retina

by intereference of two uniform laser beams

Bypassing optics improves vision, but only from 45 cpd to about 60 cpd.Vision is still not perfect; neural losses are at least as important as optics.

AdaptiveOptics

Adaptive Optics: supernormal, yet still imperfect vision

Factors limiting visual resolution:Optics of the image (including diffraction)…important, but not sufficient

Two ways the retinal mosaic might limit resolution:

• Filtering

• Sampling

… Effective size of cones

… Spacing of cones

Sampling limits on resolution?

• Foveal photoreceptor mosaic frequency: 110 cpd

• Nyquist sampling limit for 1 row of cones: 55cpd

30 cpd

60 cpd

120 cpd

120 cpd processed by a single row of cones: aliasing

Roorda and Williams

Sampling limits on resolution?

• Foveal photoreceptor mosaic frequency: 110 cpd

• Nyquist sampling limit for 1 row of cones: 55cpd

• Nyquist limit for 8 rows of cones: 440 cpd ??

Factors limiting visual resolution:Optics of the image (including diffraction)...important, but less important than neural losses Sampling by the photoreceptor mosaic…unimportantLight collection by the cone apertures ?

Unresolvable high-contrast patterns appear desaturated

Sherif Shady and Don MacLeod(Nature Neurosci 2002)

10 30 50 70 90 110

Factors limiting visual resolution:Optics of the image and photoreceptor sampling…Light collection by the cone apertures… not severely limiting (>100cpd)Neural Losses:

intra-retinal or retino-thalamicthalamo-corticalintra-cortical

LGN

Peter Lennie…Matt McMahon,

…Dave Williams, and Martin Lankheet

McMahon et al.J.Neurosci.1999

Factors limiting visual resolution:Optics of the image (diffraction, etc.)Light collection by the cone aperturesnot severely limiting (>100cpd)Neural convergence:

intra-retinal or retino-thalamic still not limiting (>80cpd)thalamo-corticalintra-cortical ?

LGNV1

Seeing spatial pattern

Physicalstimuli

Perceptualexperience

Localized neural activity

correlation correlation

What does V1 see?

Orientation-Selective Adaptation

Adapt(30 sec)

Test(200 msec)

Same OrientationHard to see!

Need high contrast

Orthogonal OrientationEasy to see!

Need low contrast

V1 is the first stage in the visual system where orientation information is extracted:

orientation-selective adaptation is only possible at or after V1.

If we find evidence for orientation-selective adaptation, then it implies that the orientation information has at least reached V1.

Adaptation: Psychophysicists’ microelectrode

a.

adapt

test

5000ms

fig. 1

250 ms

250 ms

250 ms

? ?

respond/adaptor

250 ms

unresolvable

0.8

0.6

0.4

0.2

0

1.0

1.2

SH DM

unresolvable

orientationdiscrimination

orientation-specificadaptation

orientationdiscrimination

orientation-specificadaptation

testtest

30 40 50 60 7035 45 55 65

(adapting) spatial frequency (cpd)(adapting) spatial frequency (cpd)

30 40 50 60 7035 45 55 65

b.

log sensitivityfor orientationdiscrimination

andlog threshold

elevation fromorientation-

specificadaptation

Invisibleverticalgrating

Adapt(30 sec)

Test(200 msec)

Same OrientationNeed higher contrast

Orthogonal OrientationNeed lower contrast

Result

a.

adapt

test

5000ms

fig. 1

250 ms

250 ms

250 ms

? ?

respond/adaptor

250 ms

unresolvable

0.8

0.6

0.4

0.2

0

1.0

1.2

SH DM

unresolvable

orientationdiscrimination

orientation-specificadaptation

orientationdiscrimination

orientation-specificadaptation

testtest

30 40 50 60 7035 45 55 65

(adapting) spatial frequency (cpd)(adapting) spatial frequency (cpd)

30 40 50 60 7035 45 55 65

b.

log sensitivityfor orientationdiscrimination

andlog threshold

elevation fromorientation-

specificadaptation

adapt test (250 ms)

250 ms

250 ms

perception (subjective horizontal) (5 sec)

a.

b.

36 48 66

-0.5

0.0

0.5

1.0

1.5

2.0

2.5

Adapting spatial frequency (cpd)

test

SH

60

SH DM

SH

Tilt Aftereffect in degrees

(mean s.e.m.)

adapt test (250 ms)

250 ms

250 ms

perception (subjective horizontal) (5 sec)

a.

b.

36 48 66

-0.5

0.0

0.5

1.0

1.5

2.0

2.5

Adapting spatial frequency (cpd)

test

SH

60

SH DM

SH

Tilt Aftereffect in degrees

(mean s.e.m.)

Orientation-selective adaptation from an invisible spatial pattern at 60 to 70 cpd:

invisible to us but visible at V1.

Factors that might limit visual resolution for interference targets:

Optics of the image (including diffraction): not applicableLight collection by the cone apertures: >100 cpdNeural convergence:

intra-retinal or retino-thalamic: 80 cpdthalamo-cortical…lumped with intracortical…intra-cortical: 70 cpd Perception: <= 60 cpd

Conclusions

• Neural losses slightly exceed optical losses at the limit.• Sampling and light collection in the photoreceptor mosaic are not limiting.• Neural losses are distributed through the system; some are intracortical, since cortex can respond to patterns too fine for conscious perception.

Optional Bonus Conclusion about Consciousness

• Primary visual cortex is not directly represented in conscious experience (Crick and Koch).

Paper topics

1) When a computer monitor that flickers too fast for the flicker to be perceived, can the unseen flicker nevertheless activate the visual cortex of your brain? Design an experiment to investigate this.

2) Is it possible to explain He and MacLeod’s results without accepting Crick and Koch’s conclusion that primary visual cortex has no immediate representation in conscious experience?

3) How might brain imaging experiments follow up on He and MacLeod’s observations?

Temporal Resolution: When Vision is Grossly

Imperfect

120 S 3 S 1 S

AdaptPre-Adapt Test

Time

3.5

3.0

2.5

2.0

1.5

1.0

806040200

4.0

3.5

3.0

2.5

2.0

1.5

1.0

806040200

Scaled adapting modulation

3.5

3.0

2.5

2.0

1.5

1.0

100806040200

60

50

4035

30

20 Hz10

4.0

3.5

3.0

2.5

2.0

1.5

1.0

100806040200

Adapting modulation (%)

45

30

20 Hz

10

40

HF

JL

Te

st m

od

ula

tio

n th

resh

old

a c

b d

Te

st m

od

ula

tio

n th

resh

old

Shady_Fig2

50403020100

Frequency (Hz)

-1.5

-1.0

-0.5

0.0

6050403020100

Frequency (Hz)

-2.0

-1.5

-1.0

-0.5

0.0

JL HF

Perception Adaptation

Shady_Fig3

Invisible Invisible

2.0

1.5

1.0

0.5

0.0

3020100

Frequency (Hz)

1.5

1.0

0.5

0.0

3020100

Frequency (Hz)

Perception Adaptation

invisible invisible

Shady_Fig4

Conclusions

• Oblique adapting and masking gratings are not less powerful than horizontal ones

• We conclude that the Oblique Effect arises after the site of pattern adaptation and masking

• Unexpectedly, oblique adapters are slightly more powerful than horizontal ones

• Additional experiments and modeling will allow us to quantitatively test the possible models we have presented

Data for subject TL

-3

-2.5

-2

-1.5

-1

0 1 2 3 4

Data for subject MW

-2.5

-2

-1.5

-1

-0.5

0 1 2 3 4

Modulation Frequency (cpd)

RF Width: Results

=

=

Th

resh

old

(lo

g C

)

Modulation Frequency (cpd)

Spatial integration is anisotropic

Data for subject TL

-3

-2.5

-2

-1.5

-1

0 1 2 3 4

Th

resh

old

(lo

g C

)

8´

Does the extent of spatial integration vary

with carrier contrast?

50% Contrast5% Contrast

-3

-2.5

-2

-1.5

-1

0 1 2 3 4Modulation Frequency (cpd)

low contrastlow contrast

high contrast

high contrast

Results: High Contrast

=

=

Th

resh

old

(lo

g C

)

-2.5

-2

-1.5

-1

-0.5

0 1 2 3 4

Data for subject TL Data for subject MW

High Contrast Properties

• Loss in sensitivity at lower modulation frequencies

• Spatial integration minimal both along and across contours

Modulation Frequency (cpd)

At high contrast:

-3

-2.5

-2

-1.5

-1

0 1 2 3 4

low contrast

high contrast

Th

resh

old

(lo

g C

)

Data for subject TL

LOW frequency: HIGH frequency:

Contrast Gain Control

Contrast gain locally compensates for low frequency modulation

Loss in sensitivity at lower modulation frequencies

Receptive Field ShrinksLOW contrast: HIGH contrast:

• Receptive field shrinks at high contrast.- Supported by recent physiological evidence

• Leads to similar behavior for integration along and across image contours.

Contrast

Constant

PerceivedBrightness

MeanLuminance

time

flicker

Orientation-selective adaptation from an invisible spatial pattern:

invisible to us but visible at V1.

There is intra-cortical loss of spatial information.

V1 does not directly support conscious vision (Crick and Koch)

With 8.5 cpd grating: Modulation threshold doubled at 2.5 cpdImplied RF height: 6 min, or 12 cone rows

With 40 cpd grating: Modulation threshold doubled near 4 cpdImplied RF height: 4 min, or 8 cone rows

Would integration over 8 rows prevent aliasing?

120 cpd