V1 Computational Theory

Lateral view

Retinogeniculate visual pathway

Ventral view

optic nerve

optic chiasm

optic tract

LGN

optic radiation

primary visual cortex

Retinogeniculate visual pathway

Stimulus drive & response selectivity

V1 orientation tuning

Neu

ral r

espo

nse

(spi

kes/

sec)

Stimulus orientation (deg)

Simple cell

Complex cell

Hubel & Wiesel movie

V1 physiology

Simple cells:• orientation selective• some are direction selective• some are disparity selective• monocular or binocular• separate ON and OFF subregions• length summation(best response to long bar)

Complex cells:• orientation selective• some are direction selective• some are disparity selective• nearly all are binocular• no separate ON and OFF subregions• length summation

Hypercomplex cells:end-stopping (best response to short bar)

No stimulus in receptive field: no response

Non-preferred stimulus: no response

Preferred stimulus: large response

Orientation selectivity model

Stimulus: vertical bar

Responses of each of severalorientation tuned neurons.

Peak (distribution mean) codesfor stimulus orientation.

Distributed representation of orientation

Broad tuning can code for small changes

Neural code depends on multiple factors

linearweighting function

rectification

firing ratestimulus

Complementaryreceptive fields

Rectification and squaring

Rectification and squaring

Rectification approximates relationship between membrane potential and spiking

Greg DeAngelis

Complex cells: theory

Complex cells & position invarianceOriented stimulus as seen by both subunits at two different locations:

Theory of spatial pattern analysis by the visual system

Input image(cornea)

“Neural image”(retinal ganglion cells)

Neural image: retinal ganglion cell responses

Array of center-surround receptive fields

Neural image: simple cell responses

Input image(cornea)

“Neural image”(V1 simple cells)

Array of orientation-selective receptive fields

Lots of neural images: V1 simple cells

A

B

C

Lots of neural images

V1 simple cells

V1 complex cells

Psychophysical/perceptual evidence for spatial-frequency and orientation selective channels

++

Orientation selective adaptation

Orientation selective adaptation

+

+

Orientation selective adaptation

+

Normalization in monkey V1

Response saturation and phase advance

25%

50%

0 1540

Time (ms)

Res

pons

e (s

p/s)

100%

12.5%

2

5

10

20

50

100

Ampl

itude

(sp/

s)

5 10 20 50 100-90

-45

0

Contrast (%)

Rel

ativ

e ph

ase

(deg

)

Time (ms)

Resp

onse

(sp/

s)

Contrast (%)

Rela

tive

pha

se (d

eg)

Am

plit

ude

(sp/

s)

Can no longer discriminate orientations near vertical

Failure of invariance with saturation?

10 20 50 100

0.5

1

2

5

10

20

50

Contrast of grating 1 (%)

Res

pons

e A

mpl

itude

(sp

/s)

10 20 50 100

Contrast of grating 2 (%)

6.25%

25%

50%

6.25% 25%

0%

50%

6130

76

Time (ms)

Res

pons

e (s

p/s)

0%

0

Masking

Normalization model

Σ( )normalizedresponse

(unnormalized response)2

unnormalized 2responses + σ

2

=

Response saturation and cross-orientation suppression

25%

50%

0 1540

Time (ms)

Res

pons

e (s

p/s)

100%

12.5%

2

5

10

20

50

100

Ampl

itude

(sp/

s)

5 10 20 50 100-90

-45

0

Contrast (%)

Rel

ativ

e ph

ase

(deg

) Contrast (%)

Am

plit

ude

(sp/

s)

r = α c2

c2 +σ 2

10 20 50 100

0.5

1

2

5

10

20

50

Contrast of grating 1 (%)

Res

pons

e A

mpl

itude

(sp

/s)

10 20 50 100

Contrast of grating 2 (%)

6.25%

25%

50%

6.25% 25%

0%

50%

6130

76

Time (ms)

Res

pons

e (s

p/s)

0%

0

r = α ct2

ct2 + cm

2 +σ 2

Resp

onse

am

plit

ude

(sp/

s)

Contrast of grating 1 (%)

Ratio of responses to pref and non-pref directions constant overfull range of contrasts.

Contrast invariance

Contrast

Resp

onse

(spi

kes/

sec)

Tolhurst & Dean (1980) Model

Surround suppression

CRF

SurroundSurround ck

sk

CRF

Surround

0 < β < 1 surround suppression

80

60

40

20

0

0 1 2 3 4 5 6452l021.p05

Grating patch diameter (deg)

Res

pons

e (im

p/se

c)

Suppression

Surrounddiameter

Optimaldiameter

Surround suppression

CRF

SurroundSurround ck

sk

CRF

Surround

Psychophysical/perceptual evidence for normalization

Masking

Target Surround masking

Overlay masking

Canonical computation hypothesis: normalization in other brain areas

Light adaptation in the retina (revisited)

R =α ItIt + Ib +σ

1 10 1000

50

100

150

200

1 10 100ORN response (spikes/sec)

PN r

espo

nse

(spi

kes/

sec)

Normalization in fruit fly olfaction

R = α Itn

Itn + Im

n +σ n

Mask odor Test odorORN: olfactory receptor neuronPN: projection neuron

Test only Mask + Test

Also...

•Visual cortical areas V4 (pattern), MT (motion perception), and IT (object recognition).

•Other sensory modalities: auditory cortex; multisensory integration (visual motion and vesibular system) in MST.

•Encoding of value in posterior parietal cortex.•Superior colliculus: saccade averaging.•Attention: modulation of activity in visual cortex.

Why normalize?

• Limited dynamic range (Heeger, Vis Neurosci, 1992).•Simplify read-out, thinking of population code as a probability

density (Simoncelli & Heeger, Vis Res, 1998; Simoncelli, in The Visual Neurosciences, 2003; Beck, Latham & Pouget, J Neurosci, 2011).

• Invariance w.r.t. one or more stimulus dimensions, e.g., contrast, odorant concentration (Heeger, Vis Neurosci, 1992; Heeger, Simoncelli & Movshon, PNAS, 1996; Simoncelli & Heeger, Vis Res, 1998; Ringach, Vis Res, 2009; Olsen, Bhandawat & Wilson, Neuron, 2010).

•Averaging vs. winner-take-all (Busse, Wade & Carandini, Neuron, 2009).•Decorrelation & statistical independence (Schwartz & Simoncelli, Nat

Neurosci, 2001; Lyu & Simoncelli, Neural Comput, 2009; Olsen, Bhandawat & Wilson, Neuron, 2010; Lyu, Axel & Abbott, PNAS, 2010).

Possible circuits & mechanisms

•Might be feedforward, feedback, or a combination of the two (Heeger, J Neurophysiol, 1993)

•Shunting inhibition (Carandini & Heeger, Science, 1994; Carandini, Heeger & Movshon, J Neurosci, 1997)

•Synaptic depression (Carandini, Heeger & Senn, J Neurosci, 2002)• Presynaptic inhibition (Olsen & Wilson, Nature 2008)•Balanced amplification (Murphy & Miller, Neuron, 2009)•Background synaptic activity (Chance, Abbott & Reyes, Neuron, 2002)•Saturation of the inputs combined with spike threshold and

spike-rate rectification (Priebe & Ferster, Neuron, 2008)

•Etc.

From circuits & mechanisms to behavior

Computation

Circuits & cellular/molecular mechanisms Behavior

Carandini, Nat Neurosci (2012)

Mechanism?

Canonical computation deficit hypothesis

Possible dysfunction of normalization underlying schizophrenia, epilepsy, and other developmental and neurological disorders.

Schizophrenia: a dysfunction of normalization?

Dakin, Carlin & Hemsley, Curr Biol, 2005

Possible mechanism for normalization deficit

MR spectroscopy

Yoon et al, J Neurosci, 2010