Amygdala contributions to stimulus–reward encoding in the

Accepted manuscripts are peer-reviewed but have not been through the copyediting, formatting, or proofreadingprocess.

Copyright © 2017 the authors

This Accepted Manuscript has not been copyedited and formatted. The final version may differ from this version.

Research Articles: Systems/Circuits

Amygdala contributions to stimulus–reward encoding in the macaque medial andorbital frontal cortex during learning

Peter H. Rudebeck1,2, Joshua A. Ripple2, Andrew R. Mitz2, Bruno B. Averbeck2 and Elisabeth A. Murray2

1Friedman Brain Institute, Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Place, New York, NY,10029, USA2Section on the Neurobiology of Learning and Memory, Laboratory of Neuropsychology, Building 49, Suite 1B80,49 Convent Drive, Bethesda, Maryland 20892, USA

DOI: 10.1523/JNEUROSCI.0933-16.2017

Received: 21 March 2016

Revised: 14 January 2017

Accepted: 18 January 2017

Published: 25 January 2017

Author contributions: P.H.R. and E.A.M. designed research; P.H.R., J.A.R., and A.M. performed research;P.H.R., J.A.R., A.M., B.A., and E.A.M. analyzed data; P.H.R., A.M., B.A., and E.A.M. wrote the paper.

Conflict of Interest: The authors declare no competing financial interests.

This work was supported by the Intramural Research Program of the National Institute of Mental Health(ZIAMH002886, EAM), an NIMH BRAINS award (R01MH110822, PHR), and NARSAD Young InvestigatorAward (PHR). We thank Kevin Blomstrom, Kevin Fomalont and Ravi Chacko for assistance with data collection,and James Fellows, Ping Yu Chen and David Yu for help with surgery and histology. We are indebted to StevenWise for discussion and comments.

Correspondence to: Dr. Peter H. Rudebeck, Friedman Brain Institute, Icahn School of Medicine at Mount Sinai,One Gustave L. Levy Place, New York, NY, 10029, Tel: 212 824-9307, E-mail: peter.rudebeck@mssm.edu

Cite as: J. Neurosci ; 10.1523/JNEUROSCI.0933-16.2017

Alerts: Sign up at www.jneurosci.org/cgi/alerts to receive customized email alerts when the fully formattedversion of this article is published.

Subjects

Macaca mulatta

Apparatus and materials

Behavioral procedures

°

± °

°

°

Surgical procedures

Neural recordings

Ω

Imaging and histological reconstruction of recording sites and lesions

Data analysis

v i

r t

vi(t – 1)

(r(t) – vi(t – 1))

di(t)

ck(t) = 1 k t ck(t)

dk(t)

stimulus period

reward period

Amygdala lesion effects on stimulus–reward learning

Stimulus–reward encoding during learning

Expected and received reward encoding during learning

expected-reward

received-reward

Contrasting novel and familiar stimulus-reward versions of the task

Amygdala lesion effects on stimulus–reward encoding during learning

Amygdala lesion effects on neural activity near reward

Amygdala lesion effects across novel and familiar tasks

Amygdala contributions to stimulus-reward learning

Reward encoding in OFC and MFC during learning

Amygdala contributions to reward encoding in prefrontal cortex during learning

Interpretational issues

Conclusion

reference period stimulus-1 period

stimulus-2 period

stimulus

stimulus

reward

reward

stimulus

stimulus

reward

reward

B

1st stimulus 2nd stimulus GO signal ChoiceFixation

Time

Trial Sequence

1000 ms 0-1500 ms500-1500 msEvent Reward Delivered

500 ms< 500 ms

EXAMPLE STIMULI

0.2 ml 0.4 ml0.0 ml

Analysis period Reward periodStimulus period

A

Figure 1

4F 4F 4F

4F

POST

-OP

T2 M

RI <

7 da

ysPR

E-O

P T1

MR

IN

issl

-sta

ined

se

ctio

ns

Monkey N

5mm

Monkey V Monkey H

A

B

AP level + 36

+36

+30

10 5 1

PREOPPOSTOP

Ventral view

Medial orbital sulcus

Lateral orbital sulcus

OFC MFC

PREOPPOSTOP

10 5 1

Fundus of the cingulate sulcus

Principal sulcus

Dorsal view

AP level + 28

Figure 2

A

PreopPostop

Figure 3

B

PreopPostop

0.5

0.6

0.7

0.8

0.9

1

Correct+1 Error+1

Prop

ortio

n co

rrect

0 10 20 30 400

0.2

0.4

0.6

0.8

1

Trial

Prop

ortio

n co

rrect

C

p<0.05 (FDR corrected)

1 2 3 40.4

0.6

0.8

1

Pre E+1Post E+1

Pre C+1Post C+1

Block

Prop

ortio

n co

rrect

*

* p<0.05

* p<0.05

*

* *

0.4 ml0.2 ml0.0 ml

Stimulus 1 Stimulus 2

0 1.0

0

70

2.0

Firin

g ra

te(s

p/s)

2.00

10

20

Time from S1 onset (s)

Mea

n %

exp

. var

ianc

e

0

S1 RewardS2 RewardMovement Dir.

S1 x S2 RewardS1 x Move. Dir.S2 x Move. Dir.

0

5

10

15

20

25

% n

euro

ns

OFC

0

5

10

15

20

25MFC

Ref. S1 S2Period Period

Ref. S1 S2 1.0

OFC - S1 RewardMFC - S1 RewardOFC - S2 RewardMFC - S2 Reward

Stimulus 2

Time from S1 onset (s)

0

4

0 5 10 150

20

40

60

Trial (relative)

TrialRL

stim

val

ue

Firin

g ra

te /

Hz

40

2

0 5 10 150

20

40

60

Trial (relative)

0.4 ml0.2 ml0.0 ml

Stimulus 1 - reward Stimulus 2 - reward

p<0.05

A B

C D

Figure 4

0

Trial

-0.5 0 0.5

0

75

Choice Reward

Firin

g ra

te(s

p/s)

0

5

20

25

30

0

5

20

25

30

Chosen Reward ValueMovement Dir.Reward x Move. Dir.

Expected Received

Perc

ent o

f neu

rons

OFC

MFCExpected Received

Perc

ent o

f neu

rons

A

B

C0.4 ml0.2 ml0.0 ml

Figure 5

0

2

4

0 20 300

30

60

90

Trial (relative)

Firin

g ra

te (s

p/s)

RL

stim

val

ue

0.4 ml0.2 ml0.0 ml

p<0.05

400

Time from reward delivery (s)

S1Reward

0

10

20

30

40

S2Reward

ExpectedReward

ReceivedReward

FamiliarNovel

*

Perc

ent o

f neu

rons

OFC MFC

Figure 6

S1Reward

S2Reward

ExpectedReward

ReceivedReward

*

* p<0.01*

A

0 0.5 1.0

0

120

sp/s

0.4 ml0.2 ml0.0 ml

0 0.5

0

60

1.0

sp/s

0.8 ml0.4 ml0.2 ml0.1 ml0.0 ml

0

25

-1 -0.5 0 0.5 1

Time from reward delivery (s)

0

25

-1 -0.5 0 0.5 1

Fam

iliar

Nov

el

Choice

0.4 ml0.2 ml

0.8 ml0.4 ml0.2 ml0.1 ml

Stimulus 2

Time from stimulus 2 onset (s)

Fam

iliar

Nov

elsp

/ssp

/s

B

C

S1Reward

0

10

20

30

40

S2Reward

ExpectedReward

ReceivedReward

OFC

S1Reward

0

10

20

30

40

S2Reward

ExpectedReward

ReceivedReward

MFC

FamiliarFamiliar & Novel

Figure 7

Perc

ent o

f neu

rons

in fa

milia

r

22 % 28 % 50 %61 %

21 % 50 % 10 %41 %

Figure 8

S1 S20

10

20

A

0

10

20

MFC

PreopPostop

1.0 2.00

10

20

0M

ean

% E

xp. v

aria

nce

Preop - S1 RewardPostop - S1 RewardPreop - S2 RewardPostop - S2 Reward

Stimulus 2

OFC

S1 S2

1.0 2.00

10

20

0

Time from S1 onset (s)

Mea

n %

Exp

. var

ianc

e

Period

OFC

MFC

BPe

rcen

t of n

euro

nsPe

rcen

t of n

euro

ns

*

*

0

10

20

30

Monkey N Monkey VMonkey H

Perc

ent o

f neu

rons

Pre-Post S2 Encoding

0

10

20

30

Mea

n %

Exp

. var

ianc

e

Pre-Post S1 Encoding

Monkey N Monkey VMonkey H

Preop OFCPostop OFCPreop MFCPostop MFC

*

#

C*

*

-30

0

30

Mk. H

% c

hang

e

Mk. N Mk. V

0

20

% c

hang

e

- 20

Mk. H Mk. N Mk. V

Chosen reward

Perc

ent o

f neu

rons

OFC

PREOPPOSTOP

Perc

ent o

f neu

rons

Response

Expected ReceivedExpected Received

MFC

Expected ReceivedExpected Received

PeriodPeriod

Figure 9

0

10

20

30

0

10

20

30

0

10

20

30

0

10

20

30

A B

C

Time from reward (s)

20

40

60

80

-0.5 0 0.5 1

20

40

60

80

10

20

30

40

50

60

000-1

Time from reward (s)

Preop Postop

MFC

neu

ron

#

PEV

-0.5 0 0.5 1-1

*

*

Mk. NMk. H0

18

% o

f neu

rons

Mk. NMk. H0

18

% o

f neu

rons

Individ. Mks

Individ. Mks

OFC MFC

30

20

10

0

30

20

10

0

A

Figure 10

30

20

10

0

40

30

20

10

0

40

S1reward

S2reward

Expectedreward

Received reward

Period

S1reward

S2reward

Expectedreward

Received reward

Period

S1reward

S2reward

Exp.reward

Rece. reward

S1reward

S2reward

Exp.reward

Rece. reward

25% 28% 42% 6% 7% 0%33%22% 28% 50%

61%

21% 50% 10%

41%

FamiliarNovel

Preop FamiliarPostop FamiliarPreop Familiar & NovelPostop Familiar & Novel

Perc

ent o

f neu

rons

Perc

ent o

f neu

rons

in fa

milia

r

**

**

B OFC MFC

* p<0.05

* p<0.05

***

**

** p<0.01

25%

*

*

0

40

80

% o

f neu

rons

Mk. N Mk. V

Individ. Mks