Winner take all - Cornell University · 2012. 9. 12. · Winner take all. In many sensory systems, the tuning curves of neurons are not identical to the receptive fields projected

Tuning tuning curves

So far: Receptive fieldsRepresentation of stimuliPopulation vectors

Today: Contrast enhancment, cortical processing

0 45 90 135 180 225 270 369

Firin

g fr

eque

ncy

N1 N2

N3N4 N5

smax(N1) = 40o

smax(N2) = 110o

smax(N3) = 135o

smax(N4) = 180o

smax(N5) = 230o

s1

10

90o

x

y

0o 180oN1

N2 N3

N4

N5

90o

x

y

0o 180o

N2 N3

90o

x

y

0o 180o

N2

N3N2+N3

0 45 90 135 180 225 270 369

Firin

g fr

eque

ncy

N1 N2

N3N4 N5

smax(N1) = 40o

smax(N2) = 110o

smax(N3) = 135o

smax(N4) = 180o

smax(N5) = 230o

s1

10

90o

x

y

0o 180oN1

N2 N3

N4

N5

90o

x

y

0o 180o

N2 N3

90o

x

y

0o 180o

N2

N3N2+N3

Winner take all

In many sensory systems, the tuning curves of neurons are not identical to the receptive fields projected to these neurons by sensory neurons. They may be more narrow, include inhibitory parts of the curve, they may be wider or more separated from each other. There are a number of processes that “tune” tuning curves, these include interactions between neurons such as inhibition, excitation and feedback interactions. As we noted before, sensory receptive fields are often broad and relatively non-specific, for example frequency tuning curves in the auditory nerve can span a large range of frequencies.

Frequency (100 Hz)1 2 3 4 5 6 7 8 9 10

Firin

g ra

te (H

z)100

80

60

40

20

0Stimulus

Frequency (100 Hz)1 2 3 4 5 6 7 8 9 10

Firi

ng r

ate

(Hz)

100

80

60

40

20

0 Stimulus

N1 = 20-50 = -30 (=0)N2 = 50 – 20 – 10 = 20N3 = 10-50 = -40 (=0)

Exercise: Assuming linear interactions and all synaptic weights being zero, construct the approximate resulting receptive fields for these neurons and this network:

frequency

100 200 300 400 500 600

100

80

60

40

20

0

Excitatory neurons N1, N2, N3

N1

N2 N3

frequency

100 200 300 400 500 600

100

80

60

40

20

0

Inhibitory neurons I1, I2, I3

N1

N2 N3

for all neurons: x = infor all synapses: w=-1

outp

ut fr

eque

ncy

outp

ut fr

eque

ncy

Firin

g ra

te

Stimulus 1 Stimulus 2 Stimulus 2

Exercise: Look at the recordings below. Think about what you could learn from these and what additional information you would need to get useful information from this experiment.

Because of broad receptive fields and tuning curves, neural circuits are thought to enhance “contrast” or increase the difference between sensory stimuli in order to make them more easily recognizable, their features more salient, more distinguishable from each other.

Exercise: (a) You have a number of chairs and a number of tables. List features these have in common. Now list features that differentiate them. Write a list of yes no questions that would allow you to decide (i) if an object does belong to either category and (ii) if it is a chair or a table. Now find some examples that would not easily be classified. Create a neural network with a layer of feature detectors (respond to a specific feature), a layer of inhibitory neurons and one or two more layers of neurons including a layer of output neurons. At the output, you want to know if the object you detect is a chair or a table. Think about which features you want to suppress (inhibit) and which you want to have compete against each other.

Exercise: (a) You have a number of chairs and a number of tables.

List features these have in common.

Now list features that differentiate them.

Write a list of yes no questions that would allow you to decide (i) if an object does belong to either category and (ii) if it is a chair or a table.

Now find some examples that would not easily be classified. Create a neural network with a layer of feature detectors (respond to a specific feature), a layer of inhibitory neurons and one or two more layers of neurons including a layer of output neurons. At the output, you want to know if the object you detect is a chair or a table. Think about which features you want to suppress (inhibit) and which you want to have compete against each other.

#of carbons

act ivity

U3

U4

(4)CHO

U3

U4

(5)CHO

U3

(6)CHO

U3

U4

U3

U4

U3

U4

(5)CHO-(4)CHO (6)CHO-(5)CHO (4)CHO-(6)CHO

Distance: 2)43( UUD −=

U3

U4

U3

U4

U3

U4

(4)CHO


Dot product AB = A * B * cos(α)

(5)-(4) < (4)-(5) < (5)-(6)

#of carbons

act ivity

U3

U4

(4)CHO

U3

U4

(5)CHO

U3

(6)CHO

U3

U4

U3

U4

U3

U4



U3

U4

#of carbons

act ivity

U3

U4

(4)CHO

U3

U4

(5)CHO

U3

(6)CHO

U3

U4

U3

U4

U3

U4



U3

U4

U3

U4

U3

U4

(4)CHO



(5)-(4) < (4)-(5) < (5)-(6)

How to compare vectors

#of carbons

act ivity

U3

U4

U3

U4

U3

(6)CHO

U3

U4

U3

U4

U3

U4



U3

U4

U3

U4

U3

U4

(4)CHO



(5)-(4) < (4)-(5) < (5)-(6)

Exercise: What happens to the distance measure and dot product measure if the vectors are “normalized” first (this means they all have length 1.0 and span the unit circle).

Photoreceptors in eye Auditory receptors in cochlea

Olfactory receptors in nose

Other retinal neurons Brain stem neurons Olfactory bulb

LGN in thalamus MGN in thalamus ?

Primary visual cortex Primary auditory cortex

Olfactory cortex

Secondary visual cortex

Secondary auditory cortex

Hippocampus

Association cortex

AcetylcholineNoradrenalineSerontonineDopaminePeptides ....

I. I. Molecular LayerMolecular Layer

II. II. External Granular LayerExternal Granular Layer

III.III. External Pyramidal LayerExternal Pyramidal Layer

Line of Line of KaesKaes--BechterewBechterew

IV.IV. Internal Granular LayerInternal Granular Layer

Outer band of Outer band of BaillargerBaillarger

-- Line of Line of GennariGennari in area 17 in area 17

V. V. Internal Pyramidal LayerInternal Pyramidal Layer

Giant pyramidal cell of BetzGiant pyramidal cell of Betz

Inner Band of Inner Band of BaillargerBaillarger

VI. VI. Polymorphic LayerPolymorphic Layer

Golgi Golgi NisslNissl WeigertWeigert

Inputs Outputs

Cell body

Pyramidal cell

Piriform cortex circuitry

Haberly, L.B. Chem. Senses, 10: 219 -38 (1985)

P P P

Ia

Ib

II

III

Afferent Input from OB mitral cells (LOT)

AssociationFibers

afferent input from olfactory bulb

association fibersfrom other pyramidalcells

cell body layer

deep interneuronsP

output



P P P

Ia

Ib

II

III


AssociationFibers



cell body layer

deep interneuronsP

output



P P P

Ia

Ib

II

III


AssociationFibers

FF

FB



cell body layer

deep interneuronsP

Feedforwardinterneurons

Feedback interneurons

output



P P P

Ia

Ib

II

III


AssociationFibers

FF

FB



cell body layer

deep interneuronsP

Feedforwardinterneurons

Feedback interneurons

output

Neuromodulatory inputsOther association fiber inputs

Layer II (cell bodies)

Layer Ib

Layer Ia

Stimulus : citronellal

activity patternacross mitral cells

activity patternacrosspyramidal cells



activity patternacross pyramidal cells

feedforward inhibition



activity patternacross pyramidal cells

feedforward inhibition

association fibers between pyramidal cells

xsj = cos(α-offsetj) when cos(α-offsetj) >= θandXsj = 0.0 when cos(α-offsetj) < θ

where offsetj = 0, -30, -60, -90 and -120.

Sensory neurons

Local interneurons

Pyramidal cells

α

Ipj = xsj; Iij = xsjxpj = Ipj and xij = Iij if I > θ and xpj = 0 and xij = 0 if I < θ.

Sensory neurons

Local interneurons

Pyramidal cells

α

Exercise. Lets work through an example[1] to get used to writing and reading equations. 1) We will start by constructing a network of neurons in motor cortex receiving inputs from presynapticneurons s with the following tuning curves as a function of the angle α of arm movement and α threshold θ:xsj = F(cos(α−offseti), θ) where offsetj = 0, -30, -60, -90 and -120.

Now, we will create 10 postsynaptic neurons (5 excitatory pyramidal cells and 5 inhibitory local interneurons), each receiving presynaptic neurons, assuming all synaptic weights w = 1:Ipj = xsj; Iinj = xsj with Ipj being the input to pyramidal cell j and Iij the input to interneuron j. These are linear threshold neurons with continuous output for now, so xpj = Ipj and xij = Iinj if I > θ and x = 0 and xij = 0 if I < θ. In vector notation:

[1] Notations: I: input to a neuron; x: output from a neuron; j: neuron index; θ: threshold; α: movement angle; s: sensory neuron; p: pyramidal neuron; in: interneuron.

Ipj = xsj – xi(j-1) - xi(j+1)

Sensory neurons

Local interneurons

Pyramidal cells

α

Sensory neurons

Local interneurons

Pyramidal cells

α

Ipj = xsj – xi(j-1) - xi(j+1) + 0.5*xpj

Ipj = xsj – xi(j-1) - xi(j+1) + 0.5*F(Ipj, Θ2)

Sensory neurons

Local interneurons

Pyramidal cells

α

Sensory neurons

Local interneurons

Pyramidal cells

α

Ipj = xsj – xi(j-1) - xi(j+1) + 0.5*F(Ipj, Θ2) + SUMk=j (wjk*xpk)

prob (xjp = 1.0) = F (Ijp, θ)

I

X

θ

X=F(I, θ) linear threshold functionX = 0 if I <= θX = I if I > θ


I

X

θ

X=F(I, θ) linear threshold functionX = 0 if I <= θX = I if I > θ

I

Prob (X=1)

θ

Prob (x=1) = 0 if I <= θProb (x=1) = I if I > θ


I

Prob (X=1)

θ

Prob (x=1) = 0 if I <= θProb (x=1) = I if I > θ

time

Θ = 0.0; I = 0.1 -> p = 0.1

On average, one spike every 10 time steps

time

Θ = 0.0; I = 0.9 -> p = 0.9

On average, nine spikes every 10 time steps

Sensory neurons

Local interneurons

Pyramidal cells

α

α = -100 α = 50

Documents

Winner take all - Cornell University · 2012. 9. 12. · Winner take all. In many sensory systems, the tuning curves of neurons are not identical to the receptive fields projected