Modeling the Visual Pathway: Some History and a …ins.sjtu.edu.cn/files/common/20121224080939_V1_Model-SJTU-Short... · Modeling the Visual Pathway: Some History and a Neuronal Network

Modeling the Visual Pathway: Some History and a Neuronal

Network Model of V1 SJTU Computational Neuroscience Winter Short Course

12/21/2012

Lecturer: Louis Tao, PKU

taolt @ mail.cbi.pku.edu.cn or letaotao @ pku.edu.cn

© Copyright 2012 Center for Bioinformatics, Peking University

© Copyright 2012 Center for Bioinformatics, Peking University

Line Motion Illusion

Neuronal Networks Are Complex

~1011 neurons & 1015 connections

104 cells & 1 km wiring in 1 mm3 of cortex

Computational Neuroscience

• What “computations” are performed by neurons & neuronal

networks?

• How are these computations done?

• What?

– Feature detection (visual system, olfactory system, …)

– Coincidence / timing (auditory system)

– Memory (hippocampus)

– Sensory-motor (eye saccades, …)

– Neural Code: firing rate, spike timing

• How?

– Cell level: molecular and biophysical

– Network & systems level

• What & How to Study?

– Cellular: membrane potential, ion channels, synaptic mechanisms

– Extracellular: firing rates, spike times, statistics of spike trains, …

– Systems: fMRI, optical imaging, …

Primary Visual Cortex (V1)

Lateral Geniculate

Nucleus (LGN)

V1 & the Visual Pathway

V1 laminaeV1 laminae

4C primary input layer

• A neuron receives inputs via dendrites (thick) and sends

outputs via axons (thin)

• Neurons are ‘connected’ via synapses

Callaway Ann Rev Neurosci 1998

Pre-synaptic neuron

axon

Post-synaptic neuron

dendrite

Excitatory 4C

neurons

Yobuta & Callaway

1998

Excitatory

Pre-synaptic

Post-synaptic

V

Excitatory neuronal action

potentials induce

positive changes in

postsynaptic membrane

potential

V1 Inhibitory Neurons

Wiser & Callaway 1996 Inhibitory

Pre-synaptic

neuron

Post-synaptic

V

Marino et al, Nature Neurosci 2005

Local, inter-laminar

connectivity tends to

be isotropic

Exc/Inh lengthscales

are roughly the same

Fitzpatrick et al ’85, Lund

‘87, Callaway & Wiser ‘96,

Marino et al ‘05

Linear Systems Analysis: Early Visual System

• Concept of the Receptive Field as a Model

• Input / Output Analysis

• Experimental Results

Recording of retinal ganglion cells by Kuffler (1950s)

Hubel & Wiesel Movie!!!

Reverse-Correlation Methods

1

1, , , ,

n

i

i

C x y s x y tn

Spike-triggered average stimulus:

Increment counter

corresponding to this

stimulus by +1

Reverse-Correlation Methods

1

1, , , ,

n

i

i

C x y s x y tn

0

1, , , ,

T

rsQ x y r t s x y t dtT

, ,

, , rsQ x yC x y

r

/r n T

Spike-triggered average stimulus:

Input-output correlation (between stimulus and firing rate):

Where the firing rate is

The two can be related by

Linear Systems Analysis : Early Visual System

, , ,s tD x y D x y D

For most RGC and LGN cells,

we can model D as separable in space and in time:


2 2 2 2

2 2 2 2, , exp exp

2 2 2 2

cen sur

t t

cen cen sur sur

D Dx y x yD x y


For most RGC and LGN cells, we can model D as separable in space and in time:

DeAngelis and Freeman, ‘97

2

0( ) ( ) ( , )( )

t

kk B tR t R ds d x A xD x It s x s

Benardete and Kaplan, Visual Neurosci 1999


, 2 2

, , , ,exp expcen sur

t cen sur cen sur cen sur cen surD


Summary Linear Systems Analysis Applied to RGC and LGN cells

• Approximate linearity

• First kernel measured using reverse correlation

• Receptive fields

• Model of spatial RF (difference of Gaussians)

• Model of temporal RF (difference of exponentials)

• Nonlinearity (e.g., Y/M cells in retina, …)

• Success! -> further along the visual pathway…

Many neurons in

V1 are selective

for stimulus

orientation,

stimulus direction,

stimulus phase,

On/Off regions

j

q

Anderson et al, Science 2000

j

q

V1 neurons are orientation-selective

high contrast

medium

low

0( , ) 1 sin( )x t I t k xI j

Drifting Grating

cos , sin , k k k k kq q

grating contrast

Orientation selectivity is “independent” of stimulus contrast

(so-called “contrast invariance”)

Cortical Map of Orientation Preference

right eye

left eye

----

0.5mm

----

Bla

sdel (1

992)

Typical experimental setup for

optical imaging

(above figure taken from Tsodyks et al 1999)

Show Larry’s Movie!!!

1 mm

orientation

hypercolumn

Simple & Complex Classification

Hubel & Wiesel (1962):

- Simple: “linear” spatio-temporal filters

Contrast Reversal:

(1) temporal response at driving frequency

(2) sensitive dependence on spatial phase

(grating location);

- Complex: everything else

Contrast Reversal :

(1) frequency doubled

(2) phase insensitive

V1: 40% Simple

q

j

Simple Complex

j

Contrast Reversal at 8 Different Spatial Phases

De Valois et al. (Vis. Res. 1982)




Drifting Grating:

(1) follows grating location (spatial phase);



Drifting Grating:

(1) phase insensitive;

(2) time-independent response

V1: 40% Simple

q

j

Simple Complex

0estr t r F L t

0

, , , ,L t d dxdyD x y s x y t

2 2

, ,, ,, , rs

s s

r C x yQ x yD x y


Estimate the firing rate:

Where the linear part is given by

D is the first Wiener kernel / spatio-temporal receptive field, and, using “white noise” stimulus, can be estimated from spike-triggered avg. stimulus via

For some neurons, we can model D as separable in space and in time:

2

sVariance of the white noise

Modeling a V1 Simple Cell

2 21

, exp cos2 2 2

s

x y x y

x yD x y kx

Approximate the “receptive field” with a Gabor function

2 21

, exp cos2 2 2

s

x y x y

x yD x y kx

cos sin

sin cos

x x

y y

q q

q q

A more general form of the Gabor function:

Coordinate transformation (here, a rotation)

spatial length-scales (preferred) spatial frequency (preferred) orientation (preferred) spatial phase

x y

k

q

cos , sink k kq q

q

At this point, we have the “receptive field” (i.e., a “model”) of a Simple cell.

Now what?!?!

5 7

exp, 05! 7!

0, 0

tD

2 21

, exp cos2 2 2

s

x y x y

x yD x y kx

0

, , , ,L t d dxdyD x y s x y t

, , ,s tL x y L x y L

0

, cos cos sin

cos

S S

t t

L dxdyD x y A Kx Ky

L t d D t

Response of a Model Simple Cell to a Counterphase Grating

When D is separable, L is separable

0, , 2x y kq

,K k ,q ,K k q

2 2

2exp cos cos cos sin

2S

x yL A dxdy kx Kx Ky


0

,K k ,q

2 2 2

2exp cosh cos2

S

k KL A kK

2 2 2

2 2

exp2 2

cos exp cos cos exp cos

S

k KAL

kK kK

2 2

2exp cos cos cos sin

2S

x yL A dxdy kx Kx Ky


2 2 2

2 2

exp2 2

cos exp cos cos exp cos

S

k KAL

kK kK

0q

2 2 2

exp cos2 2

S

k KAL

assuming

2exp 0kK

,K k q


,K k ,q ,K k q

0q

2 2 2

exp cos2 2

S

k KAL

0

2 2 2

2exp cosh cos2

S

k KL A kK

0

cost tL t d D t


5 7

exp, 05! 7!

0, 0

tD

6 2 2

42 2

4costL t t

28arctan arctan


6 2 2

42 2

4costL t t

28arctan arctan

Summary: Early Visual System modeling

From RGC to LGN to Simple Cells

• Reverse-correlation methods

• Receptive fields

– Model of spatial receptive fields (Gabor)

– Model of temporal receptive fields

• Nonlinearities (How to model?)

• Again, these are descriptive and not mechanistic models!!!

• Think about extensions to other visual pathway neurons, other sensory neurons, …

Reverse-Time Correlation

B

Yeh et al, PNAS 2009




Contrast Reversal:


(2) sensitive dependence on spatial phase

(grating location);


Contrast Reversal :

(1) frequency doubled

(2) phase insensitive

V1: 40% Simple

q

j

Simple Complex

j

Contrast Reversal at 8 Different Spatial Phases

De Valois et al. (Vis. Res. 1982)




Drifting Grating:

(1) follows grating location (spatial phase);



Drifting Grating:

(1) phase insensitive;

(2) time-independent response

V1: 40% Simple

q

j

Simple Complex

Martinez & Alonso (2003)

Phenomenology of V1 Complex Cell

Hubel & Wiesel Movie!!!


Nonlinear response to counter-phase grating:

1) Invariance to spatial phase of grating

2) (temporal) frequency-doubled response

DeValois et al (1982)


Constant response to drifting grating


Complex Cell: Modulation Ratio


1

. . 1 exp 2A C F m t i t dtT

0

1. . 0

T

D C F m t dtT

1

0 spontaneous rate

F

F Modulation Ratio =


Skottun et al (1991)

1

0 spontaneous rate

F



Ibbotson et al (2005)

1

0 spontaneous rate

F


Physiology of Complex Cells

• Receptive fields do not have on-, off- subunits

• Complex cells show selectivity to orientation, spatial frequency, but not spatial phase

• Constant response to drifting gratings

• Frequency-doubled response to counter-phase gratings

• Model?

Modeling Complex Cells

• Let us suppose there are 2 Simple cells

Also frequency-doubled response!

0q

1 , cos cosL AB K t

2 , sin cosL AB K t

preferred phase

preferred phase

/ 2

2 2 2 2 2 2 2

1 2

1, cos , 1 cos 2

2L L A B K t A B K t

2 2

1 2F L G L L

0estr t r F L t

Independent of grating phase

Static Nonlinearities: Complex Cells

2 2 2 2 2 2 2

1 2

1, cos , 1 cos 2

2L L A B K t A B K t

Static Nonlinearities: Complex Cells

From Ringach ‘04

Hubel-Wiesel Model of V1 Receptive Fields (aka Hierarchical Model)

Hubel-Wiesel Model of V1 Receptive Fields (aka Hierarchical Model)

Summary Reverse-Correlation Methods and Receptive Fields

• Complex cells

• Hubel-Wiesel Model of V1 receptive fields

• Problems with the HW model …

• Miscellaneous:

– Higher order Wiener kernels???

– Beyond LN models, dynamics and function

Convergent LGN input confers orientation & spatial phase preference

V1 Simple

Cell

Reid & Alonso ‘95

(q,j) preference

Hubel & Wiesel ‘62

Hubel-Wiesel Model of V1 Receptive Fields Experimental support for Simple cells

The Hierarchical Model (Hubel & Wiesel, 1962)

Two Distinct Populations?!?!

The Classical View: Experiment

S S

LGN

S

C

}

LGN drives Simple cells, whose summed

outputs drive Complex cells

Ringach, Shapley & Hawken, 2002

Comparison to experiment (data from

Ferster and colleagues - replotted by F. Mechler)

But unimodal F1/F0 distribution measured in

experiments (cat)

Some Problems of Orientation Tuning

Let’s consider drifting gratings

• Individual RG/LGN cells are not orientation selective

• Therefore, average response of a single LGN cell does not change with orientation

• Therefore, the sum of the average responses of a collection of LGN cells does not change with orientation

• Therefore, the average synaptic input into a Simple cell does not change with orientation

• Where does its orientation selectivity come from?

Some Problems of Orientation Tuning

• Contrast invariance

• Noise?

Anderson et al Science 2000

j

q

Contrast Invariance Orientation Selectivity

Contrast Invariant Orientation Selectivity

high contrast medium low

0( , ) 1 sin( )x t I t k xI j

Drifting Grating

cos , sin , k k k k kq q

grating contrast

Features of a V1 Neuronal Network Model

~ 4,000 I & F neurons, 1 mm2, local patch of V1 4Ca

KijEXC Gaussian with EXC = 200 mm (spatial coupling isotropic)

tjk = kth spike-time of jth neuron

Fi(t) models LGN forcing and activity in other layers

[ gi

INH(t) similar, without LGN drive, INH = 100-200 mm EXC ]

( ) ( ) ( )j j

E I

jj j j

L R E Idv

g v V g v V g v Vdt

( ) ( ) ( )j

E

j E kEE j k l

k l

g t F t S K G t t

( ) ( ) ( )j j j

LGN noiseF t f t f t ( ) ( )j kELGN E l

k l

f t c G t s


• Convergent LGN input confers orientation & spatial phase preference

(Reid & Alonso, ‘95)

V1

Reid & Alonso ‘95

(q,j) preference

of cortical neuron

Hubel & Wiesel ‘62




Regular Map of Orientation in Pinwheels

(Optical Imaging: Bonhoeffer & Grinvald

1991; Blasdel 1992; Maldonado et al. 1997)

Random Map of Spatial Phase

(DeAngelis et al. 1999: preferred phase

of 2 nearby neurons uncorrelated)




• Variability in strength of LGN excitation (Tanaka, ’86); total excitation

(LGN + Cortex) roughly constant (Miller ’96, Royer & Pare, ’02)

I

E

I

E

LGN

Simple Complex

Inhibitory

Excitatory

V1






• Local (<500 mm) connections isotropic & non-specific (Fitzpatrick et al.,

’85; Lund, ’87; Callaway & Wiser, ’96; Marino et al, ’05)








• Cortical inhibition dominant (Borg-Graham et al, ’98; Hirsch et al, ’98,

Anderson et al, ’00)








• Cortical inhibition dominant (Borg-Graham et al, ’98; Hirsch et al, ’98,

Anderson et al, ’00)

• Dynamics of individual neurons dominated by fluctuations

Data from Anderson et al, Science 2000

Membrane potential: Average over trials (left, 3 contrasts)

vs. Individual trials (right, at medium contrast)

Response to CR: Comparison to Experiments

Simple Complex Contrast Reversal at 8 different spatial phases

DeValois et al. 1982

orthogonal

preferred

Response to DG: Comparison to Experiments

(2 / )

1 00 0

/ ( ) ( )i tF F dt m t e dt m t

F1/F0 Distributions: Comparison to Experiments

Intracellular f1/f0

Extracellular F1/F0

Model of Tao et al (2006)

Model of V1 Simple Cell

• On- / off- segregated input from LGN gives frequency doubled input at “orthogonal” phase

• At “orthogonal” phase, the LGN input looks like a complex cell output!

Wielaard et al (2001)



• Cortical “inputs” also appear to be frequency-doubled in a network of Simple cells!

• Why?



• Cortical “inputs” also appear to be frequency-doubled in a network of Simple cells!

• Why?

Model of V1 Complex Cell

Tao and Cai, Acta Physiol. Sinica (review of network model of V1 Simple and Complex cells); nonlinear network model of “HW mechanism”, explained in Tao et al PNAS (2004) ; interesting issues with network dynamics and stability, see Tao et al PNAS (2006)


Nonlinear LGN Input gLGN at Various Spatial Phases

Simple Cell at Preferred & Orthogonal Phase

(LGN & cortex)

"Simple" Response: Basically, an

interaction between LGN excitation, cortical inhibition, and thresholding

Frequency-doubled & phase-insensitive cortical inhibition removes frequency-doubled LGN excitation at orthogonal phase

Wielaard, Shelley, McLaughlin, Shapley JNS 2001

Two Distinct Populations???

Mechler & Ringach (2002)

Mechanism of Orientation Selectivity

• Recurrent I & F network

• Role of coupling length-scales

• Contrast invariance

• Mean vs. fluctuation driven

• Relation to bistable states

Anderson et al

Science 2000

Tao et al, PNAS 2006

i

i i i i i

L L exc E inh I

dVg V V g V V g V V

dt

Whenever , the neuron fires and i

ThresV t V i

resetV t V

T S

dVg t V V t

dt

S ThresV V

S ThresV V

“Mean-driven”

“Fluctuation-driven”

i i ij k

exc LGN exc exc EXC j

j k

g t F t S K G t t

2 1 2 1/ / / /

2 1 2 1

1 AMPA AMPA NMDA NMDAt t t tNMDA NMDAEXC AMPA AMPA NMDA NMDA

f fG t e e e e

1 2 1 2 1 211 , 5 , 2 , 80 , , 10A AAMPA AMPA NMDA NM GABA GABADAms ms ms mms sms

Individual neurons are orientation selective and contrast invariant

Firing Rate at 2 contrasts

Time Avg Membrane Potential

LGN Input

Cortical Excitation

Cortical Inhibition

Neuronal populations are orientation selective and contrast invariant

CV = circular variance

2

0 01 ( ) ( )iCV dt m t e dt m t

q CV ~ 0 Selective

CV ~ 1 Not selective

Sparsity vs. Synaptic Failure

• Synaptic failure: Self-averaging system

• Sparsity: consequences in spatially extended systems (spatial homogenization)

Network model results consistent with experiments of Sur’s lab at MIT

showing similar orientation selectivity across cortex (Marino et al. 2005)

(independent of location to pinwheel singularity)

Effective coupling orientation is a function of location

Co-variation of tuning

of E/I conductances

Fluctuation-Controlled Critical States

Intrinsic Fluctuations in Recurrent Excitation controlled by

• AMPA vs. NMDA (fast vs. slow synapses)

• finite-size networks, sparse coupling, and synapatic failure

Tao et al, PNAS 2006

Some Problems with Linear Systems Analysis Applied to V1

• Different stimulus gives different ‘receptive fields’ – Spots/dots, bars, gratings,… give different answers!

– Stimulus not ‘white’ noise

– Not nearly the case with retina, thalamus, …

• Fundamentally nonlinear! – Linear model inadequate to

explain many data

– Network Model – Kanizsa Triangle

– lm1 movie

Some References

Dayan, P and Abbott, L.F., Theoretical Neuroscience

Enroth-Cugell, C. and Robson, J.G., “The contrast sensitivity of retinal ganglion cells of the cat,” J. Physiol. 187, 517 (1966)

Skottun, B.C., et al, “Classifying Simple and Complex cells on the basis of response modulation,” Vision Res. 31, 1079 (1991)

Martinez, L.M. and Alonso, J.-M., “Complex receptive fields in primary visual cortex,” The Neuroscientist 9, 317 (2003)

Ringach, D.L., “Mapping receptive fields in primary visual cortex,” J. Physiol. 558, 717 (2004)

Wielaard D.J. et al, “How Simple cells are made in a nonlinear network model of the visual cortex,” J. Neurosci. 21, 5203 (2001)

Mechler, F. and Ringach, D.L., “On the classification of Simple and Complex cells,” Vision Res. 42, 1017, 2002

Rangan et al. PNAS, 2005

Tao, et al. PNAS 2004, 2006

Some of My Other Work

• Optical imaging of neural circuits

– Zebrafish

– C. Elegans: simultaneous tracking behavior and imaging activity of individual neurons

• Empirical, data-driven dimension reduction

• More visual modeling …

Documents

Modeling the Visual Pathway: Some History and a …ins.sjtu.edu.cn/files/common/20121224080939_V1_Model-SJTU-Short... · Modeling the Visual Pathway: Some History and a Neuronal Network