Dynamic causal modelling of visual imagery and perception · Dynamic causal modelling of visual imagery and perception Nadine Dijkstra DondersInstitute for Brain, Cognition and Bejaviour

Dynamiccausalmodellingofvisualimageryandperception

NadineDijkstra

Donders InstituteforBrain,Cognition andBejaviourDepartmentofArtificialIntelligenceRadboud university, Nijmegen, theNetherlands

SPMcourseMay24th 2019UCL,London,UnitedKingdom

Stimulus from Buchel andFriston, 1997Figure 3 from Friston et al.,Neuroimage,2003Brain by Dierk Schaefer,Flickr, CC 2.0

Experimental Stimulus (u) Observations (y)

z = f(z,u,θn).

How brain activity z

changes over time

y = g(z, θh)

What we would see in the scanner, y, given the

neural model?

Neural Model Observation Model

DCMFramework

TheNeuralmodel

�̇� =𝑧$̇⋮𝑧&̇

= 𝑓(𝑧,𝑢, 𝜃)

Deterministic DCM for fMRI

(Taylor approximation)

DCM for CSD

�̇� = 𝐴𝑧 + 𝑣

Task Resting StateCanonical Microcircuit

Coming soon

Friston et al., Neuroimage, 2014 Friston et al., Neuroimage, 2003 Friston et al., Neuroimage, 2017

�̇� = (𝐴 +0𝑢1

2

13$𝐵1)𝑧 + 𝐶𝑢

“Howdoesbrainactivity,z,changeovertime?”

TheNeuralmodel

�̇� = (𝐴 +0𝑢1

2

13$

𝐵1)𝑧 + 𝐶𝑢

Friston et al. 2003

brain activity

baseline/averaged connectivity

modulation ofconnectivity

driving input

TheNeuralModel“Howdoesbrainactivity,z,changeovertime?”

• Subjects viewed moving dots during fMRI• On some trials, subjects were instructed to pay attention to the

speed of the dots’ motion• Question: How does attention to motion change the strength of

the connections between V1, V5 and Superior Parietal Cortex?

V1V5SPC


V1z1a

c

u1

z1

z2

�̇�$ = 𝑎𝑧 + 𝑐𝑢$ Inhibitory self-connection (Hz).Rate constant: controls rate of decay in region 1. More negative = faster decay.

Driving input u1


V5

V1

Driving input u1

z1

z2

a11

a22

a21

c11

�̇�$ = 𝑎$$𝑧$ + 𝑐$$𝑢$

Change of activity in V1:

�̇�8 = 𝑎88𝑧8 + 𝑎8$𝑧$


Self decay V1 input


V5

V1z1

z2

a11

a22

c11

u1

z1

z2

Driving input u1

a21


V5

V1z1

z2

a11

a22

c11�̇� = 𝐴𝑧 + 𝐶𝑢$

𝑧$̇𝑧8̇

= 𝑎$$ 0𝑎8$ 𝑎88

𝑧$𝑧8 + 𝑐$$

0 𝑢$

Columns are outgoing connectionsRows are incoming connections

Driving input u1

a21


V5

V1z1

z2

a11

a22

c11

Driving input u1

a21u2

u1

z1

z2

b21

Attention u2


V5

V1z1

z2

a11

a22

c11

Driving input u1

a21

𝑧$̇ = 𝑎$$𝑧$ + 𝑐$$𝑢$


b21

Attention u2

𝑧8̇ = 𝑎88𝑧8 + 𝑎8$𝑧$ + (𝑏8$𝑢8)𝑧$


Self decay V1 input Modulatory input


V5

V1z1

z2

a11

a22

c11

Driving input u1

a21

b21

Attention u2

�̇� = (𝐴 +0𝑢1

2

13$𝐵1)𝑧 + 𝐶𝑢

For m experimental inputs:

�̇�$�̇�8

= 𝑎$$ 0𝑎8$ 𝑎88

+ 𝑢80 0𝑏8$ 0

𝑧$𝑧8 + 𝑐$$ 0

0 0𝑢$𝑢8

A: Structure B: ModulatoryInput

C: DrivingInput

Change in activity per

region

External input 2(attention)

Currentactivity

per region

All external input

Columns: outgoing connectionsRows: incoming connections

TheHaemodynamic Model

Stimulus from Buchel andFriston, 1997Figure 3 from Friston et al.,Neuroimage,2003Brain by Dierk Schaefer,Flickr, CC 2.0

Experimental Stimulus (u) Observations (y)

z = f(z,u,θn).

How brain activity z

changes over time

y = g(z, θh)

What we would see in the scanner, y, given the

neural model?

Neural Model Observation Model

DCMFramework

DistinctTop-DownandBottom-UpBrainConnectivityduringVisualPerceptionandImagery

NadineDijkstra,PeterZeidman,SashaOndobaka,MarcelvanGerven &KarlFriston (2017)ScientificReports

UsingDCMtoinvestigatedirectionalconnectivityduringvisualperceptionandimagery

Background:overlap

Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports

Dijkstra,Bosch&vanGerven (2017)Journal ofNeuroscience

Leeetal.(2012)Neuroimage

Large overlap in neural representations of perceived and imagined stimuli Dijkstra, N. et al., (2019) Trends in Cognitive Sciences

Background:twomechanisms


MemoryEye

Perception ImageryVisual buffer

Kosslyn, 1980

Dentico et al. (2014)Mechelli et al. (2004)

Experimentaldesign

Dijkstra, N., Bosch, S., & van Gerven, M. (2017) Journal of NeuroscienceDijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports

Perception

Imagery

Selectionofregions


Timeseriesextraction


• 1 time series per region

• Mask (e.g. anatomical mask)

• Find peak relevant contrast

• Sphere (x mm)

• Box (x mm by y mm by z mm)

• Cluster (all voxels exceeding threshold)



• Regress out unrelated variance (e.g. head movement)

• Adjust based on effect of interest

• F-contrast, e.g.

• Take 1st principal component, i.e. eigenvariate

1 000 10001















• 8 mm sphere per subject within 16 mm sphere of group effect

check: https://en.wikibooks.org/wiki/SPM/Timeseries_extraction

Definitionofmodel


• Fully connected

• Bayesian Model Reduction

Parametric Empirical Bayes

PP

P

P

PPI

I I

II

I

Definitionofmodel


A =1 11 1

1 11 1

1 11 1

1 11 1

B(:,:,1) =0 11 0

1 11 1

1 11 1

0 11 0

B(:,:,2) =0 11 0

1 11 1

1 11 1

0 11 0

B(:,:,3) =1 11 0

1 11 1

1 11 1

0 11 0

Perception Imagery

Vividness

Definitionofmodel:drivinginput


P

I

II

I

Definitionofmodel:drivinginput


C(:,:,1) =1 00 0

0 00 0

0 00 0

0 00 0

C(:,:,2) =1 00 0

0 00 0

0 00 0

0 00 1

Perception Imagery

�̇� = (𝐴 +0𝑢1

2

13$

𝐵1)𝑧 + 𝐶𝑢

Results


Results


Conclusions


• Perception and imagery are both associated with increases in top-down

coupling

• Connectivity from IFG to OCC is present in both

• (2 x stronger in imagery)

• increase of visual activity relevant to current task

• Bottom-up coupling is only increased during perception

• Vividness is specifically associated with increases in top-down coupling

to OCC

Thankyou

Marcel van Gerven

Peter Zeidman Karl FristonSasha Ondobaka

Documents

Dynamic causal modelling of visual imagery and perception · Dynamic causal modelling of visual imagery and perception Nadine Dijkstra DondersInstitute for Brain, Cognition and Bejaviour