Upload
others
View
5
Download
0
Embed Size (px)
Citation preview
Laboratory for Computational Cognitive NeuroscienceUniversity of California, Santa Barbara
1) category learningA) ExperimentsB) TheoryB) Theory
2) categorization automaticityA) C t ti l i d lA) Computational neuroscience modelB) Tests of the model
A B
Except with many exemplars per categoryn
A B
ntat
ion
Ori
enO
Bar Width
A B
Categorization rule is
ntat
ion
easy to describe
Ori
en
ABar WidthA
Effective learning requires:
• no distractions
• active and effortful processing of feedback
h d i i f f db k i i i lBut the nature and timing of feedback is not critical
(cluster learning is possible)
A
B
Does this mammogram show a tumor?
i e is it in the categoryi.e., is it in the category “tumor” or the category
“nontumor”?nontumor ?
Tumor!
A
Categorization rule is enta
tion
gdifficult to describeO
rie
BBar Width
Effective learning requires:
• consistent feedback immediately after responsey p
• consistent mapping from category to response location
i f db k i• no active feedback processing
(no evidence that cluster learning is possible)
Rule Based Information IntegrationRule-Based Information-Integration
Bar Width Bar WidthBar WidthBar Width
Is the information-integration task inherently more difficult?
(Ashby, Alfonso-Reese, Turken, & Waldron, Psychological Review, 1998)
• explicit, logical-reasoning systemp g g y-- quickly learns explicit rules
• procedural- or habit-learning system-- slowly learns similarity-based rules
• simultaneously active in all tasks (at least initially)
Romo, Merchant et al.
Low Speed Cell High Speed Cell
Merchant et al. (1997, J. of Neurophysiology)
• logical reasoning system ki d i i• uses working memory and executive attention
• prefrontal cortex, anterior cingulate, head of the d t l th l ti l l di l caudate nucleus, thalamo-cortical loops, medial
temporal lobe structures
• Working memory & attentional switching component FROST (Ashby Ell Valentin & Casale component – FROST (Ashby, Ell, Valentin, & Casale, 2005, J. of Cognitive Neuroscience)
ACC = Anterior CingulatePFC = Lateral Prefrontal CortexMDN = Medial Dorsal Nucleus of the ThalamusGP = Globus PallidusCD = Head of the Caudate NucleusCD = Head of the Caudate NucleusVTA = Ventral Tegmental AreaSN = Substantia Nigra pars compactaHC = Hippocampus
Excitatory projection
19
Excitatory projectionInhibitory projectionDopamine projection
The Striatal Pattern Classifier (Ashby & Waldron, 1999)
PremotorA
VisualAreas
suaAreas
• Information-integration category learning should• Information-integration category learning shouldbe sensitive to feedback delay
• Rule-based category learning should not beiti t f db k d lsensitive to feedback delay
Maddox, Ashby, & Bohil (2003, JEP:LM&C)
Maddox, Ashby, & Bohil (2003, JEP:LM&C)
• Results identical with 2.5 and 10 sec delaysy
• RB results replicated at 4 increased levels of RB results replicated at 4 increased levels of difficulty
• Replication with a rule-based task that uses a conjunction rule?a conjunction rule?
(Note: Rule-based discriminability higher)
Rule-Based Information-IntegrationDimensional-4 (Dim4) Non-Dimensional-4 (Non-Dim4)
on on
Ori
enta
tio
Ori
enta
tio
Frequency Frequency
Maddox & Ing (2005, JEP:LM&C)
1.00
0.80
orre
ct *
0.60
ortio
n C
o
DelayImm
0.40Prop
o
0.20Dim4 Non-Dim4
Rule-Based Information-Integration
Maddox & Ing (2005, JEP:LM&C)
Feedback delay interferes with Feedback delay interferes with information-integration category learning, but not with rule-based category learningbut not with rule-based category learning.
Asaad, Rainer, & Miller, 2000; Hoshi, Shima, & Tanji, 1998; Merchant, Z i H d S li & R 1997 R M h t R i C
Single-cell recording studies
Zainos, Hernadez, Salinas, & Romo, 1997; Romo, Merchant, Ruiz, Crespo, & Zainos, 1995; White & Wise, 1999
Animal lesion experimentsAnimal lesion experiments Eacott & Gaffan, 1991; Gaffan & Eacott, 1995; Gaffan & Harrison, 1987; McDonald & White, 1993, 1994; Packard, Hirsch, & White, 1989; Packard & McGaugh 1992; Roberts & Wallis 2000McGaugh, 1992; Roberts & Wallis, 2000
Neuropsychological patient studiesA hb N bl Fil t W ld & Ell 2003 B & M d 1988 C lAshby, Noble, Filoteo, Waldron, & Ell, 2003; Brown & Marsden, 1988; Cools et al., 1984; Downes et al., 1989; Filoteo, Maddox, & Davis, 2001a, 2001b; Filoteo, Maddox, Ing, Zizak, & Song, in press; Filoteo, Maddox, Salmon, & Song 2005; Janowsky Shimamura Kritchevsky & Squire 1989; KnowltonSong, 2005; Janowsky, Shimamura, Kritchevsky, & Squire, 1989; Knowlton, Mangels, & Squire, 1996; Leng & Parkin, 1988; Snowden et al., 2001
Konishi et al., 1999; Lombardi et al., 1999; Nomura et al., in press; P ld k t l 2001 R t l 1997 R A d G b B k
Neuroimaging experiments
Poldrack, et al., 2001; Rao et al., 1997; Rogers, Andrews, Grasby, Brooks, & Robbins, 2000; Seger & Cincotta, 2002; Volz et al., 1997
Traditional cognitive behavioral experimentsTraditional cognitive behavioral experiments Ashby & Ell, 2002; Ashby, Ell, & Waldron, 2003; Ashby, Maddox, & Bohil, 2002; Ashby, Queller, & Berretty, 1999; Ashby, Waldron, Lee, & Berkman, 2001; Maddox Ashby & Bohil 2003; Maddox Ashby Ing & Pickering2001; Maddox, Ashby, & Bohil, 2003; Maddox, Ashby, Ing, & Pickering, 2004; Maddox, Bohil, & Ing, in press; Waldron & Ashby, 2001; Zeithamova& Maddox, in press
"As I write, my mind is not preoccupied with how my fingers form the letters; my attention is fixed simply on the thought the words express. But there was a time when the formation of the letters, as each one was written would have occupied my whole one was written, would have occupied my whole attention.”
Sir Charles Sherrington (1906)
“It has been widely held that although memory traces are at first formed in the cerebral cortex they are finally are at first formed in the cerebral cortex, they are finally reduced or transferred by long practice to subcortical levels” (p. 466)
Karl Lashley (1950) In search of the engram.
“Routine, automatic, or overlearned behavioral sequences however complex do not engage the PFC sequences, however complex, do not engage the PFC and may be entirely organized in subcortical structures” (p. 323) (p )
Joaquin Fuster (2001). The prefrontal cortex – an update.
Category CategorizationLearning Expertise
Patients with Basal Ganglia Dysfunction
(Parkinson’s disease, Impaired Unimpaired
Huntington’s disease)
P ti t ith t i Unimpaired if Patients with certain visual cortex lesions
(category specific agnosia)
pstimuli are perceived Impaired
(category-specific agnosia) normally?
n
A
tatio
nO
rien
B Wid h
O
BBar Width
A
AB
A
Ashby Ennis & Spiering (2007 Psych Review)Ashby, Ennis, & Spiering (2007, Psych Review)
Excitatory projection (glutamate)
Inhibitory projection (GABA)
Dopamine projectionDopamine projection
Ashby, Ennis, & Spiering (2007, Psych Review)
Izhikevich (2003, IEEE Trans. on Neural Networks)
( )[ ]bIuvvvvkvC trest +−−−=
&
& ))((( )[ ]uvvbau rest −−=&
If v ≥ vpeak then reset v to v = cpeakand reset u to u + d
C = 50, vrest = -80, vt = -25, a = .01,b = -20, vpeak = 40, c = -55, d = 150
)(dS ⎤⎡
Activation in striatal unit J at time t, denoted SJ(t) equals
[ ] [ ] ( )[ ],)(1)()()()(1)()()(, tStStStStStStInw
dttdS
JJSbaseJSMSJK
KJKJ −+−−−−⎥
⎦
⎤⎢⎣
⎡= ∑ εσγβ
where IK(t) is the input from visual cortical unit K at time t, and wK,iJ(n) is the strength of the synapse between cortical
i K d i di i ll J d ( ) i hiunit K and spine i on medium spiny cell J, and ε(t) is white noise.
Globus Pallidus: [ ]baseJGJJGJ GtGtGtS
dtdG
−−−= )()()()( βα [ ]baseJGJJGdt)()()( β
Thalamus: ),()()()( tTtTtGdt
tdTJTJJT
J βα −−=
P t A
[ ] [ ] [ ],)(1)()()()()(1)()()()(, tEtEtEtEtEtEtInvtT
dttdE
JJEbaseJEKEJKK
JKJEJ −+−−−−⎥
⎦
⎤⎢⎣
⎡+= ∑ εσγβα
Premotor Area:
dt K ⎦⎣
Excitatory projection (glutamate)
Inhibitory projection (GABA)
Dopamine projectionDopamine projection
Hebbian learning
reinforcement
learning
reinforcement learning
vK,J(n) = strength of synapse between visual cortical cell K and premotor cell J on trial n
presynaptic postsynaptic activation
Global Feedback Algorithm
LTPpresynaptic activation
postsynaptic activation (above NMDA threshold)
[ ] [ ])(1)()()()1( ,,, nvtEtInvnv JKNMDAJkvJKJK −−+=+ +θα
[ ] )()()( , tvtEtI JKJNMDAkv+−− θβ
LTD postsynaptic activation p y p(below NMDA threshold)
CorticalCortical--StriatalStriatal Learning Learning (3 factor)(3 factor)(reinforcement learning (reinforcement learning –– also a global learning algorithm)also a global learning algorithm)
LTP activation above NMDA threshold
dopamine above baseline (Correct Response)
(reinforcement learning (reinforcement learning also a global learning algorithm)also a global learning algorithm)
)()1( nwnw iJKiJK =+
NMDA threshold (Correct Response)
[ ] [ ] [ ][ ] [ ]
)(1)()()( ,, nwDnDtrtS iJKbaseNMDAiJKJw++ −−−+ θα
)()1( ,, nwnw iJKiJK +
[ ] [ ][ ] ),()(
)()()()(
,,
,,
nwtr
nwnDDtrtS
iJKiJKNMDAw
iJKbaseNMDAiJKJw
+
++
−−
−−−
θγ
θβ
activation below
dopamine below baseline (error)
LTD
activation below NMDA threshold activation above
NMDA threshold
Obtained Reward Predicted RewardIncreases with:
Obtained Reward – Predicted Rewardwhere obtained reward on trial n equals
⎪⎨
⎧+= receivedisfeedbacknoif
received isfeedback correct if0 1
R⎪⎩
⎨− received isfeedback error if
receivedisfeedback noif 1 0nR
∑−
=
−=1
1
)(n on trial Reward Predicted andn
ii
in ReC θ
=1i
Bayer & Glimcher (2005 Neuron) Dopamine Release in SPEEDBayer & Glimcher (2005, Neuron) Dopamine Release in SPEED
Obtained Reward – Predicted RewardObtained Reward Predicted Reward
1
0.8
0 4
0.6
opam
ine
0.2
0.4Do
01 101 201 301 401 501 601
Trial
n
A
tatio
nO
rien
B Wid h
O
BBar Width
1
0.9
ect With Striatal
0.7
0.8
tion
Cor
re
With StriatalNoise
Without
0.6
Prop
ort Without
StriatalNoise
0.4
0.5
1 2 3 4 5 6 7 8 9Block
Romo, Merchant et al.
1
nses Monkeys
SPEED
Res
pon SPEED
0.5
of L
ow Low High
port
ion
012 14 16 18 20 22 24 26 28 30
Prop
Cycles per SecondSpeed (mm/s)
Low Speed Cell High Speed Cell
Merchant et al. (1997)
Low Speed Cells High Speed Cells
Romo et al., 1997
(1997 J of Neuroscience)(1997 J of Neuroscience)(1997, J. of Neuroscience)(1997, J. of Neuroscience)
Lever press to tone
70 trials/day
18 days18 days
Striatal Response
Carelli et al. (1997, Journal of Neuroscience)
Carelli et al.
(i.e., RT > 10 s)
F d ll t d d
(2005, J. of Neuroscience)(2005, J. of Neuroscience)
Food pellet dropped into compartment
Minimum 30 s between trials
28 trials/day
17 days17 days
Injected with dopamine (D1)
antagonist
Munsell Color Patches – 3 Subjects – 1800 Trials
ABs A A
A ABghtn
ess
A
A
AB
B
Brig
B B B
Saturation
Model Performance
90
100ec
t
70
80
ent C
orre
50
60Per
ce
5 20 35 50 65 80 95 110
125
140
Block (N)( )
SPEEDNosofsky & Palmeri (1997)
Participant 1
2000
SPEEDy ( )
1400
1600
1800
2000
pons
e Ti
me
800
1000
1200
5 0 5 0 5 0 5 0 5 0
Mea
n R
es
5 20 35 50 65 80 95110 125 140
Block (N)
•fMRI
• Model automaticity development in:
-- neuropsychological populations
bj t d i fl f d-- subjects under influence of drugs
•Automaticity in rule-based tasks•Automaticity in rule-based tasks
• Two category learning systems
• Explicit, logical reasoning system-- Uses working memory & executive attention-- Frontal cortex
• Procedural learning system• Procedural learning system-- Striatum
• Learning systems train long-term cortical representations
Collaborators:Learning
Leola Alfonso Reese Michael Beran Robert Cook Leola Alfonso-Reese, Michael Beran, Robert Cook, Shawn Ell, Vince Filoteo, Todd Maddox, Alan
Pickering, David Smith, And Turken, Elliott Waldron, gmany others
AutomaticityAutomaticityJohn Ennis, Brian Spiering
Funding:Public Health Services Grant MH3760 2Public Health Services Grant MH3760-2