Dante R. Chialvo

Learning, Memory and Criticality

“Most of our entire life is devoted to learn. Although its importance in medicine is tremendous, the field don’t quite have yet an understanding of what is the essence of brain learning. We have the intuition that brain learning must be a collective process (in the strong sense) for which there is not yet theory. Main stream efforts runs in a direction we argue will not leads to the solution. In this “motivational” talk we illustrate briefly the main point.”

1. (blah blah) Complex vs. Complicated .

2. (numerics) Toy model of learning -> is critical.

Why We Do What We Do?

1. Brains self-organize to survive predators escaping, moving.2. Immune systems self-organize to survive predators(when is

inside and escaping is useless).3. Societies self-organize to survive predators (when the

individual response is useless) .4. …. More.

All these systems are complex dynamical systems, with very large number of nonlinear degrees of freedom, curiously share a property: memory… would it be possible to learn something

relevant about memory studying societies, brains etc?..

Brains found useful to be the way they are

many linear pieces + a central supervisor + blueprint = “whole”

Example: a tv set

many nonlinear pieces + coupling + injected energy = “emergent properties”

Example: society

Complex system

Complicated system

Complicated or Complex?Complicated or Complex?

Is Learning & Memory Is Learning & Memory a Complex or a Complicated Problem?a Complex or a Complicated Problem?

•If learning & memory is just complicated, then somebody will eventually figure out the whole problem.

•But if happen to be But if happen to be complex complex … we can seat and wait … we can seat and wait forever… forever…

Note that: Current experiments explore isolated details (i.e. one neuron, few synapses… etc.)

What Is the Problem?

The current emphasis is …

• To understand how billions of neurons learn, remember and forget on a self-organized way.

I Don’t Know the Solution! The problem belong to biology but the solution

to physics.

• To find a relationship between hippocampal long-term potentiation, (“LTP”) of synapses and memory.

Steps of Long-term PotentiationSteps of Long-term Potentiation1. Rapid stimulation of neurons depolarizes them.2. Their NMDA receptors open, Ca2+ ions flows into

the cell and bind to calmodulin.3. This activates calcium-calmodulin-dependent

kinase II (CaMKII).4. CaMKII phosphorylates AMPA receptors making

them more permeable to the inflow of Na+ ions (i.e., increasing the neuron’ sensitivity to future stimulation.

5. The number of AMPA receptors at the synapse also increases.

6. Increased gene expression (i.e., protein synthesis - perhaps of AMPA receptors) and additional synapses form.

Biology is concerned with “Long-Term Potentiation”

If A and B succeed together to fire the neuron (often enough) synapse B will be reinforced

What Is Wrong With “LTP”?

First of all:There is no evidence* linking memory LTP

Furthermore:• It is a process purely local (lacking any global coupling).• It implies a positive feedback (“addictive”).• It needs multiple trials (“rehearsal”).

Finally: Network components are not constant, neurons are

replaced (even in adults).

*(non-circumstantial)

How difficult would be for a neuronal network to learn

The idea was not to invent another “learning algorithm” but to play with the simplest, still biologically realistic, one.

• Chialvo and Bak, Neuroscience (1999)

• Bak and Chialvo, Phys. Rev. E (2001).

• Wakeling J. Physica A, 2003)

• Wakeling and Bak, Phys.Rev. E (2001).

Self-organized Learning: Toy Model

1) Neuron “I*” fires

2) Neuron “j*” with largest W*(j*,I*) fires

and son onneuron with largest W*(k*,j*) fires…

3) If firing leads to success: Do nothingDo nothing

otherwiseotherwise decrease W* by

That is allThat is all

• Bak and Chialvo. Phys. Rev. E (2001).

• Chialvo and Bak, Neuroscience (1999)

• Wakeling J. Physica A, 2003)

How It Works on a Simple Task

Connect one (or more) input neurons with a given output neuron.

Chialvo and Bak, Neuroscience (1999)

A simple gizmo

a)left <->right

b)10% “blind”

c)10% “stroke”

d)40% “stroke”

Chialvo and Bak, Neuroscience (1999)

How It Scales With Brain Size

More neurons -> faster learning.

It makes sense!The only model where larger is better

Chialvo and Bak, Neuroscience (1999)

How It Scales With Problem Size (on the Parity Problem)

• A) Mean error vs Time for various problem’ sizes (i.e., N=2m bit strings)

• B) Rescaled Mean error (with k=1.4)

Chialvo and Bak, Neuroscience (1999)

Order-Disorder Transition

Learning time is optimized for > 1

Order-Disorder Transition

At = 1 the network is critical!

Synaptic landscape remains rough• Elimination of the least-

fit connections• Activity propagates

through the best-fit ones• At all times the synaptic

landscape is rough

Fast re-learning

Chialvo and Bak, Neuroscience (1999)

Summing up:

1.1. We discusses why we don’t share the main-stream idea that We discusses why we don’t share the main-stream idea that learning in the brain is based on LTP. Probably LTP is an epi-learning in the brain is based on LTP. Probably LTP is an epi-phenomena. phenomena.

2.2. Intuition tell us that learning in brains must be a collective Intuition tell us that learning in brains must be a collective process. Theory is needed here.process. Theory is needed here.

3.3. As an exerciseAs an exercise we showed an alternative toy model of self- we showed an alternative toy model of self-organized learning (not based on LTP) which is biologically organized learning (not based on LTP) which is biologically plausible.plausible.