F. Gregory Ashby

1

F. Gregory AshbyLaboratory for Computational Cognitive Neuroscience

University of California, Santa Barbara

Ori

enta

tion

A B

Except with many exemplars per category

Bar Width

Ori

enta

tion

Categorization rule is easy to describe

Bar Width

Ori

enta

tion

A B

A

Effective learning requires:

• no distractions

• active and effortful processing of feedback

But the nature and timing of feedback is not critical

Bar Width

A

B

Does this mammogram show a tumor?

i.e., is it in the category “tumor” or the category

“nontumor”?

Tumor!

Categorization rule is difficult to describe

Bar Width

Ori

enta

tion

A

B

Effective learning requires:

• consistent feedback immediately after response

• consistent mapping from category to response location

• no active feedback processing

Rule-Based Information-Integration

Rule Based Information Integration

Bar Width Bar Width

Is the information-integration task inherently more difficult?

Bar WidthBar Width

• explicit, logical-reasoning system

-- quickly learns explicit rules

(Ashby, Alfonso-Reese, Turken, & Waldron, Psychological Review, 1998)

• procedural- or habit-learning system

-- slowly learns similarity-based rules

• simultaneously active in all tasks (at least initially)

Romo, Merchant et al.

Low Speed Cell High Speed Cell

Merchant et al. (1997, J. of Neurophysiology)

• logical reasoning system

• uses working memory and executive attention

• prefrontal cortex, anterior cingulate, head of the caudate nucleus, thalamo-cortical loops, medial temporal lobe structures temporal lobe structures

• Working memory & attentional switching component – FROST (Ashby, Ell, Valentin, & Casale, 2005, J. of Cognitive Neuroscience)

15

ACC = Anterior CingulatePFC = Lateral Prefrontal CortexMDN = Medial Dorsal Nucleus of the ThalamusGP = Globus PallidusCD = Head of the Caudate NucleusVTA = Ventral Tegmental AreaSN = Substantia Nigra pars compactaHC = Hippocampus

Excitatory projection

Inhibitory projection

Dopamine projection

The Striatal Pattern Classifier (Ashby & Waldron, 1999)

PremotorPremotorPremotorPremotor

AreasAreasAreasAreasVisualVisualVisualVisual

AreasAreasAreasAreas

• Information-integration category learning should

be sensitive to feedback delay

• Rule-based category learning should not be• Rule-based category learning should not be

sensitive to feedback delay

Maddox, Ashby, & Bohil (2003, JEP:LM&C)

Maddox, Ashby, & Bohil (2003, JEP:LM&C)

• Results identical with 2.5 and 10 sec delays

• RB results replicated at 4 increased levels of difficulty difficulty

• Replication with a rule-based task that uses a conjunction rule?

Rule-Based Information-IntegrationDimensional-4 (Dim4) Non-Dimensional-4 (Non-Dim4)

(Note: Rule-based discriminability higher)

Frequency

Ori

enta

tion

Frequency

Ori

enta

tion

Maddox & Ing (2005, JEP:LM&C)

0.60

0.80

1.00

Pro

port

ion

Cor

rect

Delay

*

0.20

0.40

0.60

Dim4 Non-Dim4

Pro

port

ion

Cor

rect

Imm

Rule-Based Information-Integration

Maddox & Ing (2005, JEP:LM&C)

Feedback delay interferes with

information-integration category learning, information-integration category learning,

but not with rule-based category learning.

• Rule-based category learning requires active

processing of feedback signal

• Feedback processing is automatic in• Feedback processing is automatic in

information-integration category learning

ITI 2000 msec

7 (Y-N resp) Response

Terminated

Delay

500 msec

1000 msec

2500 msec

ITI 2000 msec

Delay

7 (Y-N resp) Response

Terminated

Delay

2 4 9 1

1000 msec

Time

2 4 9 1

2 4 9 1

Correct or

Error

Response Terminated

500 msec

500 msec 2500 msec Delay

Correct or

Error

Response Terminated

500 msec

Cat – Mem Scan – Delay (Interference)

Cat – Delay – Mem Scan (Control)

Maddox, Ashby, Ing, & Pickering (2004, Memory & Cognition)

Rule-Based Information-Integration

Ori

enta

tion

Ori

enta

tion

FrequencyO

rien

tati

onFrequency

Ori

enta

tion

Maddox, Ashby, Ing, & Pickering (2004, Memory & Cognition)

0.70

0.80

0.90

1.00

Short

0.40

0.50

0.60

0.70

Dim Non-Dim

Category Structure

Long

Control*

Rule-Based Information-Integration

Maddox, Ashby, Ing, & Pickering (2004, Memory & Cognition)

Asaad, Rainer, & Miller, 2000; Hoshi, Shima, & Tanji, 1998; Merchant, Zainos, Hernadez, Salinas, & Romo, 1997; Romo, Merchant, Ruiz, Crespo, & Zainos, 1995; White & Wise, 1999

Single-cell recording studies

Animal lesion experiments Eacott& Gaffan, 1991; Gaffan& Eacott, 1995; Gaffan& Harrison, 1987; Eacott& Gaffan, 1991; Gaffan& Eacott, 1995; Gaffan& Harrison, 1987; McDonald & White, 1993, 1994; Packard, Hirsch, & White, 1989; Packard & McGaugh, 1992; Roberts & Wallis, 2000

Neuropsychological patient studiesAshby, Noble, Filoteo, Waldron, & Ell, 2003; Brown & Marsden, 1988; Cools et al., 1984; Downes et al., 1989; Filoteo, Maddox, & Davis, 2001a, 2001b; Filoteo, Maddox, Ing, Zizak, & Song, in press; Filoteo, Maddox, Salmon, & Song, 2005; Janowsky, Shimamura, Kritchevsky, & Squire, 1989; Knowlton, Mangels, & Squire, 1996; Leng & Parkin, 1988; Snowden et al., 2001

Konishi et al., 1999; Lombardi et al., 1999; Nomura et al., in press; Poldrack, et al., 2001; Rao et al., 1997; Rogers, Andrews, Grasby, Brooks, & Robbins, 2000; Seger & Cincotta, 2002; Volz et al., 1997

Neuroimaging experiments

Traditional cognitive behavioral experiments Ashby & Ell, 2002; Ashby, Ell, & Waldron, 2003; Ashby, Maddox, & Bohil, Ashby & Ell, 2002; Ashby, Ell, & Waldron, 2003; Ashby, Maddox, & Bohil, 2002; Ashby, Queller, & Berretty, 1999; Ashby, Waldron, Lee, & Berkman, 2001; Maddox, Ashby, & Bohil, 2003; Maddox, Ashby, Ing, & Pickering, 2004; Maddox, Bohil, & Ing, in press; Waldron & Ashby, 2001; Zeithamova& Maddox, in press

"As I write, my mind is not preoccupied with how my fingers form the letters; my attention is fixed simply on the thought the words express. But there was a time when the formation of the letters, as each was a time when the formation of the letters, as each one was written, would have occupied my whole attention.”

Sir Charles Sherrington (1906)

“It has been widely held that although memory traces are at first formed in the cerebral cortex, they are finally reduced or transferred by long practice to subcortical levels” (p. 466)

Karl Lashley (1950) In search of the engram.Karl Lashley (1950) In search of the engram.

“Routine, automatic, or overlearned behavioral sequences, however complex, do not engage the PFC and may be entirely organized in subcortical structures” (p. 323)

Joaquin Fuster (2001). The prefrontal cortex – an update.

Category

Learning

Categorization

Expertise

Patients with Basal Ganglia Dysfunction

Impaired Unimpaired(Parkinson’s disease,

Huntington’s disease)

Impaired Unimpaired

Patients with certain visual cortex lesions

(category-specific agnosia)

Unimpaired if stimuli are perceived normally?

Impaired

A

A

B

A

Excitatory projection (glutamate)

Inhibitory projection (GABA)

Dopamine projection

Ashby, Ennis, & Spiering (2007, Psych Review)

[ ] [ ] ( )[ ],)(1)()()()(1)()()(

, tStStStStStStInwdt

tdSJJSbaseJSMSJ

KKJK

J −+−−−−

= ∑ εσγβ

Activation in striatal unit J at time t, denoted SJ(t) equals

where IK(t) is the input from visual cortical unit K at time t, and wK,iJ(n) is the strength of the synapse between cortical unit K and spine i on medium spiny cell J, and ,(t) is white noise.

dt K

Globus Pallidus: [ ]baseJGJJGJ GtGtGtS

dt

tdG −−−= )()()()( βα

Thalamus: ),()()()(

tTtTtGtdTJ βα −−=Thalamus: ),()()()(

tTtTtGdt

tdTJTJJT

J βα −−=

[ ] [ ] [ ],)(1)()()()()(1)()()()(

, tEtEtEtEtEtEtInvtTdt

tdEJJEbaseJEKEJK

KJKJE

J −+−−−−

+= ∑ εσγβα

Premotor Area:

Excitatory projection (glutamate)

Inhibitory projection (GABA)

Dopamine projection

3-factor learning

Hebbian learning

learning

vK,J(n) = strength of synapse between visual cortical cell K and premotor cell J on trial n

[ ] [ ]−−+=+ +θα

LTP

presynaptic activation

postsynaptic activation (above NMDA threshold)

[ ] [ ])(1)()()()1( ,,, nvtEtInvnv JKNMDAJkvJKJK −−+=+ +θα

[ ] )()()( , tvtEtI JKJNMDAkv+−− θβ

LTD postsynaptic activation (below NMDA threshold)

CorticalCortical--StriatalStriatal Learning Learning (3 factor)(3 factor)

[ ] [ ] [ ])(1)()()( nwDnDtrtS ++ −−−+ θα

)()1( ,, nwnw iJKiJK =+

LTP activation above NMDA threshold

dopamine above baseline (Correct Response)

[ ] [ ] [ ][ ] [ ]

[ ] ),()(

)()()()(

)(1)()()(

,,

,,

,,

nwtr

nwnDDtrtS

nwDnDtrtS

iJKiJKNMDAw

iJKbaseNMDAiJKJw

iJKbaseNMDAiJKJw

+

++

++

−−

−−−

−−−+

θγ

θβ

θα

LTD

activation below NMDA threshold activation above

NMDA threshold

dopamine below baseline (error)

Obtained Reward – Predicted Reward

Increases with:

where obtained reward on trial n + 1 equals

1 if correct feedback is received

1

1 if correct feedback is received

0 if no feedback is received

1 if error feedback is received nR +

= −

and

Bayer & Glimcher (2005, Neuron) Dopamine Release in SPEED

Obtained Reward – Predicted Reward

0.6

0.8

1

Do

pam

ine

0

0.2

0.4

1 101 201 301 401 501 601

Trial

Do

pam

ine

Romo, Merchant et al.

0.5

1P

rop

ort

ion

of

Lo

w R

esp

on

ses

Low High

Monkeys

SPEED

0

0.5

12 14 16 18 20 22 24 26 28 30

Cycles per Second

Pro

po

rtio

n o

f L

ow

Res

po

nse

s

Low High

Speed (mm/s)

Low Speed Cell High Speed Cell

Merchant et al. (1997)

Low Speed Cells High Speed Cells

Romo et al., 1997

Lever press to tone

70 trials/day

(1997, J. of Neuroscience)(1997, J. of Neuroscience)

Striatal Response

70 trials/day

18 days

Carelli et al. (1997, Journal of Neuroscience)

Carelli et al.

A

A

A

BB

rig

htn

ess

Munsell Color Patches – 3 Subjects – 1800 Trials

A

A

AB

B

B

B B

Bri

gh

tnes

s

Saturation

Model Performance

80

90

100P

erce

nt

Co

rrec

t

50

60

70

5 20 35 50 65 80 95 110

125

140

Block (N)

Per

cen

t C

orr

ect

Participant 1

1600

1800

2000

Mea

n R

espo

nse

Tim

e

SPEEDNosofsky & Palmeri (1997)

800

1000

1200

1400

1600

5 20 35 50 65 80 95110 125 140

Block (N)

Mea

n R

espo

nse

Tim

e

•fMRI

• Model automaticity development in:

-- neuropsychological populations -- neuropsychological populations

-- subjects under influence of drugs

• Automaticity in rule-based tasks

• Two category learning systems

• Explicit, logical reasoning system

-- Uses working memory & executive attention

-- Frontal cortex

• Procedural learning system-- Striatum

• Learning systems train long-term cortical representations

Collaborators:Learning

Leola Alfonso-Reese, Michael Beran, Robert Cook, Shawn Ell, Vince Filoteo, Todd Maddox, Alan

Pickering, David Smith, And Turken, Elliott Waldron, many others

AutomaticityJohn Ennis, Brian Spiering

Funding:Public Health Services Grant MH3760-2