Dynamiccausalmodellingofvisualimageryandperception
NadineDijkstra
Donders InstituteforBrain,Cognition andBejaviourDepartmentofArtificialIntelligenceRadboud university, Nijmegen, theNetherlands
SPMcourseMay24th 2019UCL,London,UnitedKingdom
Stimulus from Buchel andFriston, 1997Figure 3 from Friston et al.,Neuroimage,2003Brain by Dierk Schaefer,Flickr, CC 2.0
Experimental Stimulus (u) Observations (y)
z = f(z,u,θn).
How brain activity z
changes over time
y = g(z, θh)
What we would see in the scanner, y, given the
neural model?
Neural Model Observation Model
DCMFramework
TheNeuralmodel
�̇� =𝑧$̇⋮𝑧&̇
= 𝑓(𝑧,𝑢, 𝜃)
Deterministic DCM for fMRI
(Taylor approximation)
DCM for CSD
�̇� = 𝐴𝑧 + 𝑣
Task Resting StateCanonical Microcircuit
Coming soon
Friston et al., Neuroimage, 2014 Friston et al., Neuroimage, 2003 Friston et al., Neuroimage, 2017
�̇� = (𝐴 +0𝑢1
2
13$𝐵1)𝑧 + 𝐶𝑢
“Howdoesbrainactivity,z,changeovertime?”
TheNeuralmodel
�̇� = (𝐴 +0𝑢1
2
13$
𝐵1)𝑧 + 𝐶𝑢
Friston et al. 2003
brain activity
baseline/averaged connectivity
modulation ofconnectivity
driving input
TheNeuralModel“Howdoesbrainactivity,z,changeovertime?”
• Subjects viewed moving dots during fMRI• On some trials, subjects were instructed to pay attention to the
speed of the dots’ motion• Question: How does attention to motion change the strength of
the connections between V1, V5 and Superior Parietal Cortex?
V1V5SPC
TheNeuralModel“Howdoesbrainactivity,z,changeovertime?”
V1z1a
c
u1
z1
z2
�̇�$ = 𝑎𝑧 + 𝑐𝑢$ Inhibitory self-connection (Hz).Rate constant: controls rate of decay in region 1. More negative = faster decay.
Driving input u1
TheNeuralModel“Howdoesbrainactivity,z,changeovertime?”
V5
V1
Driving input u1
z1
z2
a11
a22
a21
c11
�̇�$ = 𝑎$$𝑧$ + 𝑐$$𝑢$
Change of activity in V1:
�̇�8 = 𝑎88𝑧8 + 𝑎8$𝑧$
Change of activity in V5:
Self decay V1 input
TheNeuralModel“Howdoesbrainactivity,z,changeovertime?”
V5
V1z1
z2
a11
a22
c11
u1
z1
z2
Driving input u1
a21
TheNeuralModel“Howdoesbrainactivity,z,changeovertime?”
V5
V1z1
z2
a11
a22
c11�̇� = 𝐴𝑧 + 𝐶𝑢$
𝑧$̇𝑧8̇
= 𝑎$$ 0𝑎8$ 𝑎88
𝑧$𝑧8 + 𝑐$$
0 𝑢$
Columns are outgoing connectionsRows are incoming connections
Driving input u1
a21
TheNeuralModel“Howdoesbrainactivity,z,changeovertime?”
V5
V1z1
z2
a11
a22
c11
Driving input u1
a21u2
u1
z1
z2
b21
Attention u2
TheNeuralModel“Howdoesbrainactivity,z,changeovertime?”
V5
V1z1
z2
a11
a22
c11
Driving input u1
a21
𝑧$̇ = 𝑎$$𝑧$ + 𝑐$$𝑢$
Change of activity in V1:
b21
Attention u2
𝑧8̇ = 𝑎88𝑧8 + 𝑎8$𝑧$ + (𝑏8$𝑢8)𝑧$
Change of activity in V5:
Self decay V1 input Modulatory input
TheNeuralModel“Howdoesbrainactivity,z,changeovertime?”
V5
V1z1
z2
a11
a22
c11
Driving input u1
a21
b21
Attention u2
�̇� = (𝐴 +0𝑢1
2
13$𝐵1)𝑧 + 𝐶𝑢
For m experimental inputs:
�̇�$�̇�8
= 𝑎$$ 0𝑎8$ 𝑎88
+ 𝑢80 0𝑏8$ 0
𝑧$𝑧8 + 𝑐$$ 0
0 0𝑢$𝑢8
A: Structure B: ModulatoryInput
C: DrivingInput
Change in activity per
region
External input 2(attention)
Currentactivity
per region
All external input
Columns: outgoing connectionsRows: incoming connections
TheHaemodynamic Model
Stimulus from Buchel andFriston, 1997Figure 3 from Friston et al.,Neuroimage,2003Brain by Dierk Schaefer,Flickr, CC 2.0
Experimental Stimulus (u) Observations (y)
z = f(z,u,θn).
How brain activity z
changes over time
y = g(z, θh)
What we would see in the scanner, y, given the
neural model?
Neural Model Observation Model
DCMFramework
DistinctTop-DownandBottom-UpBrainConnectivityduringVisualPerceptionandImagery
NadineDijkstra,PeterZeidman,SashaOndobaka,MarcelvanGerven &KarlFriston (2017)ScientificReports
UsingDCMtoinvestigatedirectionalconnectivityduringvisualperceptionandimagery
Background:overlap
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
Dijkstra,Bosch&vanGerven (2017)Journal ofNeuroscience
Leeetal.(2012)Neuroimage
Large overlap in neural representations of perceived and imagined stimuli Dijkstra, N. et al., (2019) Trends in Cognitive Sciences
Background:twomechanisms
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
MemoryEye
Perception ImageryVisual buffer
Kosslyn, 1980
Dentico et al. (2014)Mechelli et al. (2004)
Experimentaldesign
Dijkstra, N., Bosch, S., & van Gerven, M. (2017) Journal of NeuroscienceDijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
Perception
Imagery
Selectionofregions
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
Timeseriesextraction
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
• 1 time series per region
• Mask (e.g. anatomical mask)
• Find peak relevant contrast
• Sphere (x mm)
• Box (x mm by y mm by z mm)
• Cluster (all voxels exceeding threshold)
Timeseriesextraction
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
• Regress out unrelated variance (e.g. head movement)
• Adjust based on effect of interest
• F-contrast, e.g.
• Take 1st principal component, i.e. eigenvariate
1 000 10001
Timeseriesextraction
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
Timeseriesextraction
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
Timeseriesextraction
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
Timeseriesextraction
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
Timeseriesextraction
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
Timeseriesextraction
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
Timeseriesextraction
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
• 8 mm sphere per subject within 16 mm sphere of group effect
check: https://en.wikibooks.org/wiki/SPM/Timeseries_extraction
Definitionofmodel
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
• Fully connected
• Bayesian Model Reduction
Parametric Empirical Bayes
PP
P
P
PPI
I I
II
I
Definitionofmodel
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
A =1 11 1
1 11 1
1 11 1
1 11 1
B(:,:,1) =0 11 0
1 11 1
1 11 1
0 11 0
B(:,:,2) =0 11 0
1 11 1
1 11 1
0 11 0
B(:,:,3) =1 11 0
1 11 1
1 11 1
0 11 0
Perception Imagery
Vividness
Definitionofmodel:drivinginput
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
P
I
II
I
Definitionofmodel:drivinginput
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
C(:,:,1) =1 00 0
0 00 0
0 00 0
0 00 0
C(:,:,2) =1 00 0
0 00 0
0 00 0
0 00 1
Perception Imagery
�̇� = (𝐴 +0𝑢1
2
13$
𝐵1)𝑧 + 𝐶𝑢
Results
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
Results
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
Conclusions
Dijkstra, N., Zeidman, P., Ondobaka,S., van Gerven, M., & Friston, K. (2017) Scientific Reports
• Perception and imagery are both associated with increases in top-down
coupling
• Connectivity from IFG to OCC is present in both
• (2 x stronger in imagery)
• increase of visual activity relevant to current task
• Bottom-up coupling is only increased during perception
• Vividness is specifically associated with increases in top-down coupling
to OCC
Thankyou
Marcel van Gerven
Peter Zeidman Karl FristonSasha Ondobaka