Abstract

This talk summarizes our recent attempts to integrate action and perception within a single optimization framework. We start with a statistical formulation of Helmholtz's ideas about neural energy to furnish a model of perceptual inference and learning that can explain a remarkable range of neurobiological facts. Using constructs from statistical physics it can be shown that the problems of inferring the causes of our sensory inputs and learning regularities in the sensorium can be resolved using exactly the same principles. Furthermore, inference and learning can proceed in a biologically plausible fashion. The ensuing scheme rests on Empirical Bayes and hierarchical models of how sensory information is generated. The use of hierarchical models enables the brain to construct prior expectations in a dynamic and context-sensitive fashion. This scheme provides a principled way to understand many aspects of the brain's organization and responses. We will demonstrate the brain-like dynamics that this scheme entails by using models of bird songs that are based on chaotic attractors with autonomous dynamics. This provides a nice example of how nonlinear dynamics can be exploited by the brain to represent and predict dynamics in the sensorium..

Attractors in Song

Cognitive Neuroimaging Seminar on Monday 18th

Inference and learning under the free energy principleHierarchical Bayesian inference

Bird songs (inference)Perceptual categorisationPrediction and omission

Bird songs (learning)Repetition suppressionThe mismatch negativity

A simple experiment

Overview

agent - menvironment

( , )s g w

argmin ( , )s

F

argmin ( , )s

F( , )f z

Separated by a Markov blanket

External states

Internal states

Sensation

Action

Exchange with the environment

Perceptual inference Perceptual learning Perceptual uncertainty

Action to minimise a bound on surprise

The free-energy principle

Perception to optimise the bound

( | ) ( | ) ( | ) ( | )uq q u q q

argminu

F argmin dt

F argmin dt

F

The conditional density and separation of scales

{ , , }u

ln ( , | ) ln ( )q q

p s m q F

ln ( ( ) | , ) ( || ( ))

argmin

argmax ln ( | , )

KLq

q

p s m D q p

p s m

F

F

ln ( | ) ( ( | ) || ( | ))

argmin

( | ) ( | )

KLp s m D q p s

q p s

F

F

Mean-field partition

argminu

F argmin

F

argmin

FSynaptic gain

Synaptic activity Synaptic efficacy

Activity-dependent plasticity

Functional specialization

Attentional gain

Enables plasticity

( | ) ( | ) ( | ) ( | )uq q u q q

Perception and inference

Attention and uncertainty

Learning and memory

)1(~x )1(

s

)2((2)

(1)

)2(~x

)2(~v

)1(~v

x~

s

v~

( | , , , ) ( , ( ))sp s v x N g

( | , , , ) ( , ( ))xp x x v N f

( ) ( , )p v N

( , , )

( , , )

s g zs g x v z

x f x v w Dx f w

Hierarchical (deep) dynamic models

( 1) ( ) ( )( 1) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

( , , )

( , , )

i i ii i i i i

i i i i i i i i

v g zv g x v z

x f x v w Dx f w

{ ( ), ( ), , }x t v t

Empirical Bayes and DEMBottom-up Lateral

E-StepPerceptual learning

M-StepPerceptual uncertainty

D-StepPerceptual inference

Recurrent message passing among neuronal populations, with top-down predictions changing to suppress bottom-up prediction error

Friston K Kilner J Harrison L A free energy principle for the brain. J. Physiol. Paris. 2006

Associative plasticity, modulated by precision

Encoding of precision through classical neuromodulation or plasticity of lateral connections

Top-down

( ) ( ) ( ) ( ) ( 1)

( ) ( ) ( ) ( )

i v i v i T i i vv

i x i x i T ix

D

D

( ) ( 1) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( )

( ) ( )

( ) ( )

i v i v i i z i v

i x i x i i w i x

g

D f

ij

ij ij ij ij

Tij

A

A

12 ( ( ))

i i ii i

Ti i

A

A tr R

L4

SG

IG

( 1)i v

( )i v( )i x

( )i x

( )i v

( 1)i v ( 1)i x

( 1)i x

( 2)i v

( 1)i v ( 1)i x

( 1)i x

Message passing in neuronal hierarchies

Backward predictions

Forward prediction error

Inference and learning under the free energy principleHierarchical Bayesian inference

Bird songs (inference)Perceptual categorisationPrediction and omission

Bird songs (learning)Repetition suppressionThe mismatch negativity

A simple experiment

Overview

Synthetic song-birds

SyrinxVocal centre

1

2

vv

v

Time (sec)

Freq

uenc

y

Sonogram

0.5 1 1.5

2 1

1 1 3 1 2

1 2 2 3

18 18

( , ) 2

2

x x

x x v v x x x x

x x v x

f

( )i x

( )i x

( )i v( )s t

( )i v

10 20 30 40 50 60-5

0

5

10

15

20

prediction and error

time

10 20 30 40 50 60-5

0

5

10

15

20

hidden states

time

Backward predictions

Forward prediction error

10 20 30 40 50 60-10

-5

0

5

10

15

20

Causal states

time (bins)

2 1

1 1 3 1 2

1 2 2 3

18 18

( ) 2

2

x x

f x v x x x x

x x v x

Hierarchical recognition

stimulus

0.2 0.4 0.6 0.82000

2500

3000

3500

4000

4500

5000

time (seconds)

Perceptual categorization

Freq

uenc

y (H

z) Song A

0.2 0.4 0.6 0.82000

3000

4000

5000

time (seconds)

Song B

0.2 0.4 0.6 0.82000

3000

4000

5000

Song C

0.2 0.4 0.6 0.82000

3000

4000

5000

Inference and learning under the free energy principleHierarchical Bayesian inference

Bird songs (inference)Perceptual categorisationPrediction and omission

Bird songs (learning)Repetition suppressionThe mismatch negativity

A simple experiment

Overview

Sequences of sequences

SyrinxNeuronal hierarchy

Time (sec)

Freq

uenc

y (K

Hz)

sonogram

0.5 1 1.5

(2) (2)2 1

(2) (2) (2) (2) (2)1 3 1 2

(2) (2) (2)81 2 33

(2) (1)(2) 2 1

(2) (1)3 2

18 18

32 2

2

x x

f x x x x

x x x

x vg

x v

(1)1(1)2

v

v

1

2

y

y

(1) (1)2 1

(1) (1) (1) (1) (1) (1)1 1 3 1 2

(1) (1) (1) (1)1 2 2 3

(1)1(1) 2

(1)23

18 18

2

2

x x

f v x x x x

x x v x

yxg

yx

( )i x

( )i x

( )i v

( 1)i v

( )s t

( )i v( 1)i x

( 1)i x

( 1)i v

Hierarchical perception

20 40 60 80 100 120-20

-10

0

10

20

30

40

50prediction and error

time

20 40 60 80 100 120-10

0

10

20

30

40

50

causes

time20 40 60 80 100 120

-20

-10

0

10

20

30

40

50

hidden states

time

20 40 60 80 100 120-20

-10

0

10

20

30

40

50

hidden states

time

time (seconds)0.5 1 1.5

2000

2500

3000

3500

4000

4500

5000

stimulus

percept

Prediction error encoded by superficial pyramidal cells that generate ERPs

… omitting the last chirps

20 40 60 80 100 120-20

-10

0

10

20

30

40

50prediction and error

time20 40 60 80 100 120

-20

-10

0

10

20

30

40

50hidden states

time

20 40 60 80 100 120-10

0

10

20

30

40

50causes - level 2

time20 40 60 80 100 120

-20

-10

0

10

20

30

40

50hidden states

time

omission and violation of predictions

Stimulus but no percept

Percept but no stimulus

Freq

uenc

y (H

z)

stimulus (sonogram)2500

3000

3500

4000

4500

Time (sec)

Freq

uenc

y (H

z)

percept

0.5 1 1.5

2500

3000

3500

4000

4500

5000

500 1000 1500 2000-100

-50

0

50

100

peristimulus time (ms)

LFP

(micr

o-vo

lts)

ERP (error)

without last syllable

Time (sec)

percept

0.5 1 1.5

500 1000 1500 2000-100

-50

0

50

100

peristimulus time (ms)

LFP

(micr

o-vo

lts)

with omission

Inference and learning under the free energy principleHierarchical Bayesian inference

Bird songs (inference)Perceptual categorisationPrediction and omission

Bird songs (learning)Repetition suppressionThe mismatch negativity

A simple experiment

Overview

Suppression of inferotemporal responses to repeated faces

Main effect of faces

Henson et al 2000

Repetition suppression and the MMN

The MMN is an enhanced negativity seen in response to any change (deviant) compared to the standard response.

( )i x

( )i x

( )i v( )s t

( )i v

Hierarchical learning

10 20 30 40 50 60-2

-1

0

1

2

hidden states

time

10 20 30 40 50 60-0.2

-0.1

0

0.1

0.2

0.3

0.4

0.5causes

time

10 20 30 40 50 60-5

0

5

10

15

20

25

30

35

prediction and error

time

1

2

0 2 0( )

2 16

xg x

x

Time (sec)

Freq

uenc

y (H

z)

0.1 0.2 0.3 0.42000

2500

3000

3500

4000

4500

5000

Synaptic adaptation Synaptic efficacy

ij

ij ij

Tij

A

A

12 ( ( ( )))

i i

Ti i

A

A tr R

1 41 116 16

2 1216 16

( )0

x vf x

x

argmin

argmin

dt

dt

F

F

Perceptual inference: suppressing error over peristimulus time

Perceptual learning: suppression over repetitions

Simulating ERPs to repeated chirps

argminu

F

100 200 300

-10

0

10

LFP

(micr

o-vo

lts)

prediction error

0 0.2 0.4-5

0

5hidden states

Freq

uenc

y (H

z)

percept

0.1 0.2 0.32000

3000

4000

5000

100 200 300

-10

0

10

LFP

(micr

o-vo

lts)

0 0.2 0.4-5

0

5

Freq

uenc

y (H

z)

0.1 0.2 0.32000

4000

100 200 300

-10

0

10

LFP

(micr

o-vo

lts)

0 0.2 0.4-5

0

5

Freq

uenc

y (H

z)

0.1 0.2 0.32000

4000

100 200 300

-10

0

10

LFP

(micr

o-vo

lts)

0 0.2 0.4-5

0

5

Freq

uenc

y (H

z)

0.1 0.2 0.32000

4000

100 200 300

-10

0

10

LFP

(micr

o-vo

lts)

0 0.2 0.4-5

0

5Fr

eque

ncy

(Hz)

0.1 0.2 0.32000

4000

100 200 300

-10

0

10

peristimulus time (ms)

LFP

(micr

o-vo

lts)

0 0.2 0.4-5

0

5

Time (sec)

Freq

uenc

y (H

z)

0.1 0.2 0.32000

4000

Time (sec)

Synthetic MMN

Last presentation(after learning)

First presentation(before learning)

1 2 3 4 50

0.5

1

1.5

2

2.5

3Synaptic efficacy

presentation

chan

ges

in p

aram

eter

s

1 2 3 4 50

1

2

3

4

5

6Synaptic gain

presentation

hype

rpar

amet

ers

100 200 300

-10

0

10

100 200 300

-0.4-0.2

0

0.2

100 200 300

-10

0

10

100 200 300

-0.4-0.2

00.2

100 200 300

-10

0

10

100 200 300

-0.4-0.2

00.2

100 200 300

-10

0

10

100 200 300

-0.4-0.2

00.2

100 200 300

-10

0

10

100 200 300

-0.4-0.2

00.2

0 100 200 300 400-15

-10

-5

0

5

10

15

20primary level (N1/P1)

peristimulus time (ms)

Diffe

renc

e wa

vefo

rm

0 100 200 300 400-0.6

-0.4

-0.2

0

0.2

0.4secondary level (MMN)

peristimulus time (ms)

Diffe

renc

e wa

vefo

rm

primary level prediction error

secondary level

Synthetic and real ERPs

1 2 3 4 50

0.5

1

1.5

2

2.5

3

presentation

chan

ges

in p

aram

eter

s

1 2 3 4 50

1

2

3

4

5

6

presentationhy

perp

aram

eter

s

A1 A1

STG

subcortical input

STG

repetition effects

Intrinsic connections

1 2 3 4 50

20

40

60

80

100

120

140

160

180

200

1 2 3 4 50

Extrinsic connections

20

40

60

80

100

120

140

160

180

200

presentation presentation

Synaptic efficacy Synaptic gain

( )i x

( )i x

( )i v( )y t

( )i v

Inference and learning under the free energy principleHierarchical Bayesian inference

Bird songs (inference)Perceptual categorisationPrediction and omission

Bird songs (learning)Repetition suppressionThe mismatch negativity

A simple experiment

Overview

01

A brain imaging experiment with sparse visual stimuli

V2

V1

Angelucci et al

Coherent and predicable

Random and unpredictable

top-down suppression of prediction error when coherent?

CRF V1 ~1o

Horizontal V1 ~2o

Feedback V2 ~5o

Feedback V3 ~10o

Classical receptive field V1

Extra-classical receptive field

Classical receptive field V2

?

V2

V1

V5

pCGpCG

V5

RandomStationaryCoherent

V1V5V2

Suppression of prediction error with coherent stimuli

regional responses (90% confidence intervals)

decreases increases

Harrison et al NeuroImage 2006

This model of brain function explains a wide range of anatomical and physiological facts; for example, the hierarchical deployment of cortical areas, recurrent architectures using forward and backward connections and their functional asymmetries (Angelucci et al, 2002).

In terms of synaptic physiology, it predicts Hebbian or associative plasticity and, for dynamic models, spike–timing-dependent plasticity.

In terms of electrophysiology it accounts for classical and extra-classical receptive field effects and long-latency or endogenous components of evoked cortical responses (Rao and Ballard, 1998).

It predicts the attenuation of responses encoding prediction error, with perceptual learning, and explains many phenomena like repetition suppression, mismatch negativity and the P300 in electroencephalography.

In psychophysical terms, it accounts for the behavioural correlates of these physiological phenomena, e.g., priming, and global precedence.

Predictions about the brain

Summary

• A free energy principle can account for several aspects of action and perception

• The architecture of cortical systems speak to hierarchical generative models

• Estimation of hierarchical dynamic models corresponds to a generalised deconvolution of inputs to disclose their causes

• This deconvolution can be implemented in a neuronally plausible fashion by constructing a dynamic system that self-organises when exposed to inputs to suppress its free-energy

Thank you

And thanks to collaborators:

Jean DaunizeauLee HarrisonStefan KiebelJames Kilner

Klaas Stephan

And colleagues:

Peter DayanJörn DiedrichsenPaul Verschure

Florentin Wörgötter