© 2011 Kathryn Cain Dickerson ALL RIGHTS RESERVED

© 2011

Kathryn Cain Dickerson

ALL RIGHTS RESERVED

Multiple Memory Systems Involved in Human Probabilistic Learning

By: Kathryn Cain Dickerson

A Dissertation submitted to the Graduate School – Newark

Rutgers, The State University of New Jersey

in partial fulfillment of the requirements for the degree of

Doctor of Philosophy

Graduate Program in Behavioral and Neural Science

Written under the direction of

Professor Mauricio R. Delgado

and approved by:

Dr. Lila Davachi

Dr. Mauricio R. Delgado

Dr. Bart Krekelberg

Dr. Catherine Myers

Dr. Elizabeth Tricomi

Newark, New Jersey

October 2011

ABSTRACT OF THE DISSERTATION

Multiple Memory Systems Involved in Human Probabilistic Learning

By: Kathryn Cain Dickerson

Dissertation Director: Professor Mauricio R. Delgado

This dissertation investigated the nature of interactions between multiple

memory systems (MMS) in the human brain during probabilistic learning. How

MMS interact in the brain is highly debated in the literature, with evidence

supporting competition, cooperation, and parallel engagement observed across

several species and various experimental paradigms (e.g., Poldrack et al., 2001;

Poldrack and Rodriguez, 2004; Voermans et al., 2004; White and McDonald,

2002). In this dissertation, three functional neuroimaging experiments

investigated the relationship between the medial temporal lobes (MTL),

implicated in declarative learning, and the basal ganglia (BG), involved in

nondeclarative learning, during probabilistic learning. Specifically, tasks which

varied with respect to learning type (feedback, akin to nondeclarative learning

and observation akin to declarative learning) and cue difficulty (e.g., easy, hard)

were employed. Based on the evolutionary theory of MMS, neuroanatomical

connections between these regions, as well as connectivity with midbrain

dopaminergic centers involved in reward, memory, and reinforcement learning, it

ii

was hypothesized that rather than one specific relationship (e.g., competitive or

cooperative), MMS operate in parallel during probabilistic learning – at times

exhibiting parallel engagement, and at times interacting directly via cooperation

or competition depending on the context of the learning situation. Interactions

between memory systems were measured by the relative engagement of each

system during learning (as measured by BOLD responses), simple correlation

analyses, as well as functional and effective connectivity measures. Lastly, to

examine putative dopaminergic influences during learning, reinforcement

learning models were employed. Across all experiments, it was observed that a)

patterns of activation in MMS during learning varied depending on the learning

context; b) functional and effective connectivity existed across regions (MTL,

BG); and c) a dopaminergic learning signal correlated with activity in both the BG

and MTL. To conclude, the results of this dissertation support the hypothesis

that multiple memory systems operate in parallel during probabilistic learning in

humans. Understanding how such systems communicate during learning may

inform treatment of several neuropsychiatric disorders which affect learning and

memory in humans (e.g., Parkinson’s disease, schizophrenia, and MTL

amnesia).

iii

Dedication

To my mother, Christine Josephine Dickerson, for her constant support and love

To my father, Kenneth Gerard Dickerson, for his quiet and steady guidance

I would not be here without the two of you – I am eternally grateful for all you

have given me

&

To my grandparents, Dorothea Marie and Raymond James Cain – thank you for

teaching me the power of knowledge, hard work, and determination

You are deeply loved and missed

iv

Acknowledgments

There are many people who I would like to thank for their support and assistance

on my journey towards achieving my graduate degree. First I would like to thank

my advisor, Dr. Mauricio Delgado. He has been an excellent mentor and I am so

happy to have had the opportunity and pleasure of working with him. He has

taught me a great deal and I am very appreciative of his kindness and patience

over the years. I would also like to thank all the members of my dissertation

committee, both past and present, Dr. Elizabeth Tricomi, Dr. Bart Krekelberg, Dr.

Mark Gluck, Dr. Catherine Myers, and Dr. Lila Davachi. They have all been

extremely helpful and thoughtful in guiding me through my dissertation

experience. I am also very grateful for the advice and mentoring I received at the

University of Rochester, especially from Dr. Mark Mapstone, which helped me

decide to attend graduate school. My time at Rutgers would not have been the

same without my dear friends and colleagues in the Delgado lab. I am especially

thankful to Dominic Fareri, Laura Martin, Mike Niznikiewicz, Tony Porcelli, and

Swati Battacharya-Sharma with whom I began my journey as well as Kamila Sip,

Meredith Johnson, Andrea Lewis, Lauren Leotti and Anastasia Rigney, with

whom I am ending it. It has been such a joy and a pleasure knowing you all, and

I value your friendships enormously. Mike, Dominic, Tony and Laura especially

taught me so much and were wonderfully patient teachers. It was also my

pleasure to work with Victoria Lee, Martina Henderson, and Stephanie Herrera,

who were undergraduate students in the lab. Vicki especially has been such a

wonderful support, helping me with many aspects of my projects over the years.

v

I am very grateful to the Neuroscience Department, especially Dr. Ian Creese

and Dr. Steve Levison, who were the directors of the program for the majority of

my time at Rutgers, as well as the all of the faculty who taught me a great deal

about neuroscience. I am so happy I was accepted here and have very much

enjoyed being a part of the department as well as being affiliated with the

Psychology department. I am very grateful to all of my fellow neuroscience

students, especially Amber Ziegler, Kate Seip, Arielle Schmidt, Anushree Karnik,

Jessica Wright, Alex Kohl, Josh Callahan, Till Hartmann, Bengi and Tim Unal,

Julia Basso, and Liad and Alfonso Renart, for listening and providing excellent

comments on practice talks and for their general support and encouragement. I

would also like to thank the staff of the Neuroscience Department, Ann Kutyla,

Wayne Brown, and Connie Sadaka as well as Sandra Smith of the Psychology

Department for their assistance throughout the years. The staff at the UMDNJ

Advanced Imaging Center: Pat Singh, Bobby Singh, Evelyn Oliviera, and Melissa

Ortiz were most helpful during my time of data collection and I am very

appreciative of their assistance. I am also grateful to all of the students who

volunteered their time participating in my studies. I am so lucky to have the

support of my close friends Molly Reed, Lidza Kalifa, Megan Sweeny, Avinash

Reddy, Usman Khan, and my entire extended family, who have supported me

every step of the way. I am also so lucky to have the best parents and siblings –

Mom, Dad, Nate, Lilly, and Kate I truly could not have done this without you, your

love has given me so much encouragement when I needed it most. Lastly I

vi

would like to thank my partner Alon Amir for his love and support. I am so happy

that we had this journey together; it would not have been the same without you.

vii

Table of Contents

Abstract of the Dissertation ii

Dedication iv

Acknowledgments v

Table of Contents viii

List of Figures xii

List of Tables xiv

List of Appendices xvi

Chapter 1: Introduction 1

1.1 General Introduction 1

1.2 The Neuroanatomy of Multiple Memory Systems 4

1.2.1 Medial temporal lobes and declarative memory 5

1.2.2 Basal ganglia and nondeclarative memory 8

1.2.3 Anatomical circuitry of the MTL and BG 10

1.2.4 Neuroanatomical connections between memory systems 12

1.3 Interactions between Multiple Memory Systems – Functional Significance 16

1.3.1 Evidence supporting competitive interactions 17

1.3.2 Evidence supporting cooperative interactions 19

1.3.3 Evidence supporting parallel engagement 21

1.4 Description of Dissertation Experiments 25

1.5 Significance 29

viii

Chapter 2: General Materials, Methods, & Data Analyses 30

2.1 General Materials & Methods 30

2.1.1 Experimental designs 30

2.1.2 Participants 31

2.1.3 fMRI acquisition 32

2.2 General Data Analyses 33

2.2.1 General linear model 33

2.2.2 Granger causality analyses 35

2.2.3 Prediction error analyses 36

2.2.4 Cluster lever statistical threshold estimator 40

2.2.5 Sequential Bonferroni correction 41

2.2.6 Note regarding test phase neuroimaging results 42

Chapter 3: Experiment 1 43

3.1 Introduction: Background & Rationale 43

3.2 Materials & Methods 47

3.2.1 Experimental design 47

3.2.2 Data analysis 51

3.3 Results 54

3.3.1 Behavioral results 54

3.3.2 Neuroimaging results 57

3.4. Discussion 62

ix






4.3 Results 86



4.3.3 Exploratory analyses 107

4.4 Discussion 116






5.3 Results 153



5.4 Discussion 166

x

Chapter 6: General Discussion 176

6.1 Purpose & Summary of Experiments 176

6.2 Limitations 184

6.3 Conclusions & Future Directions 187

References 189

Figures 208

Tables 223

Appendices 236

Vita 255

xi

List of Figures

Figure 3.1 Schematic of the Task used in Experiment 1 208

Figure 3.2 Experiment 1 Behavioral Results 209

Figure 3.3 Experiment 1 Neuroimaging Results: Main Effect of Cue Difficulty 210

in the Striatum and Hippocampus

Figure 3.4 Experiment 1 Neuroimaging Results: Granger Causality Analysis 211

Functional Connectivity in the Caudate Nucleus using the Hippocampus

as the Seed Region

Figure 3.5 Experiment 1 Neuroimaging Results: Prediction Error Analysis in 212

the Hippocampus and Striatum



Figure 4.3 Experiment 2 Neuroimaging Results: Learning Phase Main Effect 215

of Cue Difficulty in the Striatum and MTL – Feedback Group

Figure 4.4 Experiment 2 Neuroimaging Results: Learning Phase Main Effect 216

of Cue Difficulty in the Striatum and MTL – Observation Group

Figure 4.5 Experiment 2 Neuroimaging Results: Update Phase Main Effect 217

of Cue Difficulty in the Striatum and MTL – Feedback and Observation

Groups

Figure 4.6 Experiment 2 Neuroimaging Results: Granger Causality Analysis 218

Effective Connectivity using the Midbrain as the Seed Region

Figure 4.7 Experiment 2 Neuroimaging Results: Prediction Error Analysis 219


xii


Figure 5.3 Experiment 3 Neuroimaging Results: ROI Analysis during the 222

Stimulus Presentation Period

xiii

List of Tables

Table 1.1 Evidence Supporting Interactions between Multiple 25

Memory Systems

Table 3.1 Experiment 1 Learning Phase ANOVA: Brain Regions Showing 223

a Main Effect of Cue Difficulty


a Main Effect of Learning Type

Table 3.3 Experiment 1 Test Phase Contrast of Previously Studied 225

vs. Novel


a Main Effect of Cue Difficulty – Feedback Group


a Main Effect of Cue Difficulty – Observation Group

Table 4.3 Experiment 2 Test Phase 1: Contrast of Previously Studied vs. 228

Novel – Feedback Group


Novel – Observation Group

Table 4.5 Experiment 2 Update Phase ANOVA: Brain Regions Showing 230

a Main Effect of Cue Difficulty – Feedback & Observation Groups

Combined


a Main Effect of Learning Type – Feedback & Observation Groups

Combined

xiv


a Main Effect of Group


Novel – Feedback Group


Novel – Observation Group

Table 5.1 Experiment 3 Test Phase: Contrast of Previously Studied 235

vs. Novel

xv

List of Appendices

Appendix 1 – Neuroimaging Test Phase Results for Experiments 1-3 236

Appendix 2 – Experiment 3 Correlations Observed across the 238

Independent Region of Interest Analysis

Appendix 3 – Behavioral Questionnaires Administered in Experiments 1-3 241

xvi

1

Chapter One: Introduction

1.1 General Introduction

Accumulating evidence from both animal and human-based research

supports the existence of multiple memory systems that rely on distinct neural

substrates in the human brain (Sherry and Schacter, 1987; Squire, 1992b; Squire

and Zola, 1996). An important division is between a declarative memory system,

which relies on the integrity of the medial temporal lobes (MTL), and a

nondeclarative memory system, which engages many brain regions including the

basal ganglia system (BG; Sherry and Schacter, 1987; Squire, 1992b). How

these two regions interact during learning, however, remains unclear and is a

topic of debate in the literature. While some evidence points towards competitive

interactions, such that when one system is engaged it suppresses or blocks the

other (Poldrack et al., 2001; Poldrack and Packard, 2003; Foerde et al., 2006;

Lee et al., 2008), other research supports cooperative interactions (Dagher et al.,

2001; Voermans et al., 2004; Cincotta and Seger, 2007; Sadeh et al., 2011).

Moreover, a third theory posits that the MTL and BG operate in parallel and may

interact in either a competitive or a cooperative manner depending on the

learning scenario (White and McDonald, 2002; Albouy et al., 2008; Atallah et al.,

2008). Considering the conflicting evidence regarding the nature of the

interactions between these two major learning and memory systems,

understanding how they interact remains a question of great interest. The goal of

the experiments presented in this dissertation was to investigate the interactions

between multiple memory systems, specifically the basal ganglia and the medial

2

temporal lobe, during probabilistic learning using functional magnetic resonance

imaging (fMRI) of healthy human participants.

One way of approaching this debate is to consider the evolutionary theory

supporting multiple memory systems as well as neuroanatomical and functional

connectivity between the MTL and BG. It has been suggested that multiple

learning and memory systems evolved to play distinct roles in the brain (Sherry

and Schacter, 1987). Sherry and Schacter propose that natural selection may

have driven the development of multiple memory systems in the brain, when

functional incompatibility, or the inability of one memory system to solve a

problem in a particular learning environment, lead to the development of multiple

systems with distinct functions. Furthermore, it has been theorized that these

memory systems may operate in parallel, encoding unique information (White

and McDonald, 2002). Anatomical considerations state that these structures

communicate directly via a unidirectional anatomical projection from the

hippocampus, part of the MTL, to the nucleus accumbens (Kelley and Domesick,

1982) as well as the medial, ventral, rostral, and caudal aspects of the caudate

nucleus/putamen, all subcomponents of the BG (Krayniak et al., 1981; Sorensen

and Witter, 1983; Swanson and Kohler, 1986; Groenewegen et al., 1987). The

MTL and BG also communicate indirectly via interconnectivity with dopaminergic

centers in the midbrain (Swanson, 1982; Bolam et al., 2000; Poldrack and

Rodriguez, 2004). Dopamine is a neurotransmitter that has been implicated in

playing a fundamental role in learning and memory processes (Wise, 2004).

Dopaminergic neural activation is observed in the initial acquisition of reward

3

contingencies (Schultz, 1997, 2002) and is associated with changes in blood

oxygen level dependent (BOLD) signals in the human striatum during processes

such as receipt of rewards and punishments, monitoring goal outcomes, and

prediction error encoding (for review see O'Doherty et al., 2004; Balleine et al.,

2007; Delgado, 2007). Moreover, dopamine neuronal activity is linked with long-

term memory formation (Frey et al., 1990; Frey et al., 1991; Li et al., 2003),

known to recruit the hippocampus (for review see Bliss and Collingridge, 1993)

and recently midbrain reward-related activation has been linked to the

enhancement of long-term memory in the hippocampus (Wittmann et al., 2005).

Furthermore, functional coupling of the midbrain and hippocampus enhances

memory formation (Adcock et al., 2006; Shohamy et al., 2008). Based on

evidence in the literature, it is hypothesized that this functional neuroconnectivity

with dopamine plays an integral role in facilitating interactions between multiple

memory systems.

Therefore, drawing from the evolutionary theory of multiple memory

systems and functional neuroconnectivity with dopamine, it is theorized that the

MTL and BG operate in parallel during probabilistic learning with the ability to

complement each other, which may manifest itself as both cooperation and

competition. For the purpose of this dissertation, the following definitions of

parallel, cooperative, competitive interactions are employed, modified for

interpreting neuroimaging data based on White and McDonald’s theory (2002) as

well as interpretations of current neuroimaging studies in the field (Poldrack et

al., 2001; Voermans et al., 2004; Foerde et al., 2006; Doeller et al., 2008; Sadeh

4

et al., 2011). Parallel engagement is defined as the engagement of multiple

memory regions simultaneously during learning, while displaying unique patterns

of BOLD activation, suggesting that they may be encoding distinct information in

parallel. Cooperative interactions are defined as the exhibition of similar patterns

of BOLD signal in both the MTL and BG during learning as well as positive

correlations across distinct areas. Lastly, competitive interactions are defined as

the exhibition of opposing patterns of BOLD signal in both the MTL and BG

during learning and negative correlations across distinct regions.

1.2 The Neuroanatomy of Multiple Memory Systems

Over the past two decades the predominant theory regarding human

memory suggests that memory is not a unitary process, but rather consists of

multiple systems that rely on distinct neural substrates (Sherry and Schacter,

1987; Squire, 1992b; Squire and Zola, 1996). One system is referred to as

declarative memory, which encompasses episodic (memory for distinct episodes)

and semantic memory (facts and events), both of which consist of flexible

associations that may be rapidly acquired (Squire et al., 2004). Recalling one’s

college graduation day (episode) and knowing that George Washington was the

first President of the United States (fact) are examples of declarative memory. A

second system is referred to as nondeclarative memory, and is a broader

collection of many types of memory, which are more inflexible in nature and take

longer to acquire (Reber et al., 1996; Squire et al., 2004). Habit and procedural

memory, emotional memory, as well as priming and perceptual memory are all

5

types of nondeclarative memory (Squire et al., 2004). Remembering how to ride

a bicycle is an example of a nondeclarative memory.

1.2.1 Medial temporal lobes and declarative memory: Declarative and

nondeclarative memory are thought to differ not only in the types of memory they

encompass and their respective properties, but also in the neural substrates they

engage. Numerous studies, spanning rodent, primate, and human-based

research, document that declarative memory relies on the integrity of the medial

temporal lobes (see Squire, 1992a for review). A significant portion of rodent-

based research focuses on spatial maze learning tasks (Packard et al., 1989;

Packard and White, 1989; Packard and White, 1991; Packard and McGaugh,

1992, 1994). In these maze learning tasks, a rodent is trained to navigate a

maze to retrieve a food reward or escape water. One example is the radial maze

task, which has two versions termed the win-shift and the win-stay radial maze

(Packard et al., 1989). The win-shift radial maze task requires animals to visit

each arm of the maze only once to retrieve a food reward. It is theorized to

represent declarative learning because the animal must store rapidly acquired

information based on a single experience, e.g., it must remember which arms of

the maze it has and has not visited in order to obtain all of the food rewards while

not making any errors (re-entering an arm where the food has already been

obtained; White and McDonald, 2002). Lesions to the fimbria-fornix region, the

fiber bundle which arises from the subiculum and projects to the hypothalamus,

impair performance on the win-shift version of the radial maze task (Packard et

al., 1989) and a water maze paradigm (Packard and McGaugh, 1992). Because

6

lesions to the fimbria-fornix impair performance on the win-shift radial maze task,

it is thought that the MTL plays an important role in episodic-like learning in

rodents. Additionally, lesions to the CA1 and CA3 subfields of the hippocampus

produce spatial and associative memory deficits in rodents (Stubley-Weatherly et

al., 1996).

In primate-based research, evidence suggesting that the MTLs are

important for declarative learning comes from a host of discrimination related

tasks (such as pattern discrimination, delayed-nonmatching-to-sample, delayed

retention of object discrimination, and concurrent discrimination). These studies

have shown that lesions to the medial temporal lobes impair discrimination tasks

(Zola-Morgan and Squire, 1984, 1985; Zola-Morgan and Squire, 1986), but leave

motor and cognitive skill learning intact (Zola-Morgan and Squire, 1984).

Furthermore, studies using human MTL amnesic patients also support the theory

that the medial temporal lobes are necessary for declarative memory, as patients

who are missing part or all of their MTLs suffer profound declarative memory

deficits (Scoville and Milner, 1957; Penfield and Milner, 1958; Corkin et al., 1984;

Rempel-Clower et al., 1996). Lastly, fMRI and positron emission tomography

(PET) studies corroborate the theory that the MTLs are involved in encoding and

recalling declarative memories (for review see Schacter and Wagner, 1999;

Eldridge et al., 2000; Davachi and Wagner, 2002; Zeineh et al., 2003; Law et al.,

2005).

Specifically, in learning paradigms the hippocampus is believed to play a

role in encoding episodes (see Gluck et al., 2003 for review), stimulus-stimulus

7

representations (Gluck and Myers, 1993; Bunsey and Eichenbaum, 1995;

Eichenbaum and Bunsey, 1995), as well as flexibly adapting information to use in

novel contexts (Myers et al., 2003; Shohamy and Wagner, 2008). In category

learning, it has been suggested by Gluck and Myers (Gluck and Myers, 1993;

Gluck et al., 2003) that the hippocampus functions to compress overlapping

stimulus information and differentiate between distinct stimulus information.

These compressed and differentiated representations in the hippocampus are

formed over time through exposure to many trials during training, and are

subsequently communicated to cortical regions. It has also been suggested that

the connections of the MTL with the cortex are designed for rapid learning of

individual events (O'Reilly and Munakata, 2000). This type of learning is

particularly important for learning arbitrary category associations, for example in

unstructured categorization tasks which do not have an exemplar, do not involve

information integration, do not use prototype distortion, nor require learning of a

specific rule (Seger and Miller, 2010). Such unstructured categorization tasks

have been previously shown to engage the MTL (Poldrack et al., 1999; Poldrack

et al., 2001; Seger and Cincotta, 2005; Foerde et al., 2006). In non-human

primates, neurons in the hippocampus and temporal cortex have been shown to

respond to category-specific information (Hampson et al., 2004), suggesting that

category-specific information may be encoded within this region. In accord with

this theory, the hippocampus is thought to be involved in encoding episodes,

which may occur in as little as one learning instance. It is believed that the

hippocampus encodes a conjunction of many features which comprise the

8

episode, rather than encoding the individual features distinctly (Gluck et al.,

2003).

1.2.2 Basal ganglia and nondeclarative memory: In contrast,

nondeclarative memory is thought to recruit a host of distinct brain regions

including the basal ganglia (procedural or habit memory), the amygdala

(emotional memory), and the neocortex (priming and perceptual memory) (Squire

et al., 2004). For the purposes of this dissertation, the focus was on the role of

the basal ganglia in human probabilistic learning. Research using both animal

models and human studies demonstrates that the basal ganglia are involved in

several aspects of nondeclarative learning and memory. In animal models,

rodent maze studies reveal that lesions of the caudate nucleus, part of the

striatum, impair performance on a win-stay version of the radial maze task. This

version of the task requires animals to approach only certain cued arms of the

maze to retrieve a food reward, which is thought to be akin to stimulus-response

or habit learning (Packard et al., 1989; Packard and McGaugh, 1992).

Accumulating evidence suggests that the basal ganglia are also involved in many

aspects of cognitive learning (see Middleton and Strick, 2000; and Packard and

Knowlton, 2002 for review). In human based studies, recent fMRI research has

implicated this area in several other learning related functions including: habit

learning (see Yin and Knowlton, 2006 for review), skill learning (Atallah et al.,

2007), instrumental conditioning (O'Doherty et al., 2004), as well as reward

learning and decision-making (Tricomi et al., 2004; Balleine et al., 2007;

Delgado, 2007). It has become well-documented in the literature that the basal

9

ganglia are not solely a structure important for motor learning, but that they are

also critical for many types of nondeclarative memory and cognitive functioning.

The basal ganglia system has also been implicated in playing a

fundamental role in category learning in humans (Knowlton et al., 1994; Knowlton

et al., 1996; Poldrack et al., 1999; Poldrack et al., 2001; Shohamy et al., 2004b;

Shohamy et al., 2004a; Seger and Cincotta, 2005; Cincotta and Seger, 2007). It

has been suggested that the basal ganglia may be involved in encoding goal

outcomes (Tricomi and Fiez, 2008; Carter et al., 2009), error correction learning

(O'Doherty et al., 2004; O'Doherty, 2004; Abler et al., 2006; Pessiglione et al.,

2006), stimulus-response associations (Rolls, 1994; Packard and Knowlton,

2002; Ashby and Spiering, 2004), as well as mediating action selection (Seger,

2008). Work by Seger has suggested the involvement of multiple corticostriatal

loops, the visual (caudate nucleus body/tail), motor (putamen), executive (head

of caudate nucleus), and motivational (ventral striatum) loop, which play distinct

roles in category learning (2008). The author has proposed that the visual

corticostriatal loop may be involved in action selection, while the motor loop may

be involved in action selection as well as automatizing sequences. The

executive and motivational corticostriatal loops are engaged in feedback

processing, with the executive loop encompassing the head of the caudate

nucleus which may be involved in updating behavior via error correction.

Consistent with the actor-critic model, it has been theorized that the dorsal

striatum (head of caudate nucleus) acts as the actor and may be involved in

deciding which action to take, while the ventral striatum acts as the critic tracking

10

whether or not an excepted reward has been received (Joel et al., 2002;

O'Doherty et al., 2004).

1.2.3 Anatomical circuitry of the MTL and BG: It is important to consider

the anatomical circuitry of the MTL and BG individually before considering how

they may interact. Both the MTL and BG are large structures, consisting of

multiple subregions. The MTL is comprised of the parahippocampal cortex

(PHC), perirhinal cortex (PRC), entorhinal cortex (EC), and the hippocampal

complex (HC) (Squire et al., 2004). The parahippocampal and perirhinal cortices

receive information from unimodal and polymodal association areas and send

this information to the EC. The entorhinal cortex has multiple sublayers and

serves as the input unit to the hippocampus. The hippocampus itself also

consists of multiple layers, including the dentate gyrus, and the Cornu Ammonis

(CA) fields, referred to as CA1, CA2, CA3 and CA4 (Andersen, 1971; Amaral,

1993). The predominant loop arises as input from layer two of the EC, which

sends projections to the mossy fibers of the dentate gyrus and layers CA2 and

CA3, referred to as the perforant path (there is also a projection from the EC

layer 3 to CA1 and the subiculum). The mossy fibers of the dentate gyrus then

project to the pyramidal cells of the CA3 field, which project via the Schaffer

collaterals to the pyramidal cells in CA1 in a unilateral circuit. CA1 then projects

to the subiculum and deep layers of the EC. This flow of information from the

perforant path → dentate gyrus → CA3 → CA1 is referred to as the trisynaptic

circuit. The CA1 field is a sparse network, which functions well in pattern

separation, while CA3 has many interconnections, and functions well in pattern

11

completion. The output unit of the hippocampal unit is the subiculum and fimbria-

fornix fiber bundle.

Turning to basal ganglia circuitry, the basal ganglia system is comprised of

the striatum, the globus pallidus, the subthalamic nucleus, and the substantia

nigra (Packard and Knowlton, 2002; Yin and Knowlton, 2006). The striatum

consists of the caudate nucleus, putamen, and nucleus accumbens. The

caudate nucleus and putamen are often referred to collectively as the dorsal

striatum, and are implicated in learning tasks; while the nucleus accumbens is

referred to as the ventral striatum, and is more heavily involved in reward and

conditioning tasks (O'Doherty et al., 2004; Balleine et al., 2007; Delgado, 2007).

The BG also contains the globus pallidus, which is divided into an internal and

external component, as well as the subthalamic nucleus. In addition, the

substantia nigra (SN), a midbrain structure, is involved in basal ganglia circuitry.

The SN is divided into two main components, the SN reticulata (SNr) and the SN

compacta (SNc), the latter of which contains dopamine neurons that project to

the striatum. As a note, dopaminergic neurons in the ventral tegmental area

(VTA) also project to the striatum (Haber, 2003). The BG is generally

categorized into containing two main circuits, referred to as the direct pathway

and the indirect pathway. The direct pathway is as follows:

Cortex (glutamatergic) → striatum (GABAergic) → SNr-GPi (GABAergic) →

thalamus (glutamatergic) → cortex, forming a functional loop. Strong excitatory

input from the cortex results in strong inhibitory input from the striatum to the

SNr-GPi complex, which then weakly inhibits the thalamus, resulting in a stronger

12

excitatory projection to the cortex. The end result of this loop therefore is

excitation of the cortex.

The indirect circuit is as such:

Cortex (glutamatergic) → striatum (GABAergic) → GPe (GABAergic) →

subthalamic nucleus (glutamatergic) → SNr-GPi (GABAergic) → thalamus

(glutamatergic) → cortex. Excitatory input from the cortex results in strong

inhibition of the GPe from the striatum. The GPe then weakly inhibits the STN,

which then strongly excites the SNr-GPi complex, inhibiting the thalamus, which

can then only weakly excite the cortex. The end result of this loop therefore is

weak excitation of the cortex. Consequently, the direct and indirect pathways

have antagonistic functions in the brain.

1.2.4 Neuroanatomical connections between memory systems: In order to

better understand how two major learning and memory systems operate in the

human brain, much recent research has focused on examining the interactions

between the medial temporal lobes and the basal ganglia. Anatomically, these

regions are connected by a unilateral projection from the hippocampus to the

nucleus accumbens (Kelley and Domesick, 1982) and the medial, ventral, rostral,

and caudal aspects of the caudate nucleus/putamen (Krayniak et al., 1981;

Sorensen and Witter, 1983; Swanson and Kohler, 1986; Groenewegen et al.,

1987). Lisman and Grace (2005) formulated a theory of how the striatum and

hippocampus may interact via a loop with the ventral tegmental area. This model

is as follows:

13

Layer II of the EC → dentate gyrus and CA3 (glutamatergic) → CA1 (which also

receives input from EC layer III; glutamatergic) → subiculum (glutamatergic) →

nucleus accumbens (GABAergic) → ventral pallidum (GABAergic) → VTA

(dopaminergic) → CA1, layer III of the EC, and the nucleus accumbens.

Lisman and Grace propose that this model is involved in novelty detection

and encoding of information into long term memory. Newly detected information

in the hippocampus is sent as a novelty signal via the subiculum, nucleus

accumbens and ventral pallidum to the VTA in the midbrain. Subsequent DA

release from the VTA projects back up to the hippocampus, where it is involved

in facilitating long-term potentiation. Of critical importance for the purposes of

this dissertation, is the fact that the VTA also projects to the nucleus accumbens,

thereby effectively communicating with both the hippocampus and the ventral

striatum.

In addition to the Lisman and Grace model, Devan and White (1999) have

presented four anatomical links between the hippocampal complex and the

dorsomedial striatum:

1) CA1 and subiculum via the fimbria/fornix → ventral striatum and

caudate nucleus/putamen (Groenewegen et al., 1987; Brog et al.,

1993)

2) Perirhinal cortex → dorsal CA1, EC, and NAcc/medial caudate

nucleus/putamen (Krayniak et al., 1981; Sorensen and Witter, 1983;

Swanson and Kohler, 1986; McGeorge and Faull, 1989; Burwell et al.,

1995; Liu and Bilkey, 1996)

14

3) Hippocampal output → posterior cingulate cortex → medial caudate

nucleus/putamen (Domesick, 1969; McGeorge and Faull, 1989)

4) Hippocampal output → medial prefrontal cortex → ventral and

dorsomedial striatum (Swanson, 1981; Gerfen, 1984; Swanson and

Kohler, 1986; Jay et al., 1989)

Therefore it is possible that the hippocampus and striatum may also

communicate via a third region, such as the prefrontal cortex (Poldrack and

Rodriguez, 2004) or even the cingulate cortex (Devan and White, 1999).

Evidence from animal literature indicates that lesions to the cingulate cortex

(Sutherland et al., 1988) and medial prefrontal cortex (Sutherland et al., 1982;

Kolb et al., 1994) impair performance on water maze tasks in rodents, suggesting

that the connections between these regions is functionally important. Recent

research in particular has emphasized that connections with the prefrontal cortex

may be facilitating interactions between the MTL and the BG. In one study using

dynamic causal modeling, the dorsolateral prefrontal cortex (DLPFC) was found

to be the sole structure involved in receiving reward-related information directly,

which subsequently influenced activity in the midbrain (VTA) and ventral striatum

(NAcc) (Ballard et al., 2011). The authors suggest that the DLPFC integrates

information about reward and then sends this information to the ventral striatum

and midbrain dopaminergic regions, resulting in the initiation of motivated

behavior. As the midbrain projects to both the MTL and ventral striatum, it is

possible that these areas may interact via the PFC or the midbrain. The work of

15

Ballard and colleagues is consistent with a previous model posited by Poldrack

and Rodriguez (2004) which implicated the prefrontal cortex in mediating

interactions between the BG and MTL. In this theory, Poldrack and Rodriguez

posit that the MTL and BG most likely do not interact in a direct manner, but may

interact via the prefrontal cortex or via the action of neuromodulators. Both the

BG and MTL form anatomical loops with frontal cortical areas (Middleton and

Strick, 2000; Thierry et al., 2000; Seger, 2006). In terms of potential

neuromodulators, the authors nominate a couple of candidates, one of which is

dopamine. In addition to playing a role in reward learning and memory

functioning, dopamine also has an important influence on successful

classification learning (Beninger et al., 2003). Beninger and colleagues

conducted an experiment where schizophrenic patients were administered either

typical or atypical antipsychotic medications and asked to perform a probabilistic

classification learning task. Those on typical antipsychotic medication, which is

believed to block D2 receptors in the dorsal striatum, were impaired at

classification, whereas those on atypical medication, which does not block

dopamine receptors in the dorsal striatum, and healthy control participants,

performed normally. This study highlights the role dopamine plays in normal

acquisition of category learning information.

In accord with this general theory, a distinct experiment examining the

neural correlates of instructed knowledge observed that regions of the

ventromedial prefrontal cortex (vmPFC), ventral striatum, and MTL were involved

in reward-related learning and were functionally negatively correlated with a

16

region of the DLPFC during the instructed learning session when positive

outcomes were received (Li et al., 2011). The authors concluded that DLPFC

may be modulating action-outcome responses in reward-related structures

dependent on the presence and usefulness of explicit knowledge (instruction)

compared with only action-outcome information (no instruction). Lastly,

computational models exploring interactions among distinct neural substrates

mediating learning have suggested that the PFC may work together with the

hippocampus during rule-governance. In particular, Doll and colleagues (2009)

posit two hypotheses regarding how the MTL and BG interaction. The first

hypothesis suggests that the PFC and hippocampus (HC) may bias what the

striatum learns (bias model), while a second hypothesis posits that the BG learns

contingencies independently from the PFC/HC, but later the PFC/HC overrides

these learned contingencies (override model). Both of these models have been

shown to be valid (Doll et al., 2009); therefore further research is required to

explore how these systems interact. Thus, converging evidence suggests that

one manner by which the MTL and BG may interact is via the PFC.

1.3 Interactions between Multiple Memory Systems – Functional Significance

Summarizing the recent work of many researchers suggests three main

conflicting hypotheses regarding the nature of interactions between the basal

ganglia and the medial temporal lobe: 1) memory systems compete during

learning, 2) memory systems cooperate during learning, or 3) memory systems

17

operate in parallel during learning. Evidence in favor of each of these

hypotheses is described below:

1.3.1 Evidence supporting competitive interactions: Initial evidence

suggested that these memory systems interact in a competitive manner, such

that when one region is engaged, it suppresses or decreases activity in the other

region via either direct or indirect connections (Poldrack et al., 2001; Packard

and Knowlton, 2002; Foerde et al., 2006; Lee et al., 2008). Such studies have

ranged across species, from rodent-based lesion studies to human-based fMRI

experiments involving probabilistic category learning. Evidence of competitive

interactions in animal-based work has been observed during lesion experiments

when knocking out or pharmacologically modulating one region, for example the

caudate nucleus of the striatum results in an enhancement in performance on a

MTL dependent task and vice versa (Mitchell and Hall, 1988; Packard et al.,

1989; McDonald and White, 1993; Packard, 1999; Lee et al., 2008). Most

recently, Lee and colleagues showed a double dissociation between

hippocampal and striatal-based learning paradigms in mice. Disrupting striatal

functioning, via either lesions or inhibiting a transcription factor impaired cued-

learning (dependent on the striatum) and enhanced spatial-learning

(hippocampus-dependent). Likewise, lesions to the hippocampus impaired

spatial-learning while enhancing cued-learning. Based on these findings, the

authors concluded that the memory systems interact in a competitive manner.

In human fMRI studies, competition has been shown using probabilistic

category learning tasks (Poldrack et al., 2001; Foerde et al., 2006; Seger and

18

Cincotta, 2006). For example, Poldrack and colleagues (2001) employed a

between-subjects, blocked-fMRI design and a popular category learning task, the

weather prediction task. In the weather prediction task, participants are required

to learn to predict the outcome of an event (rainy or sunny weather) based on the

presentation of 1-4 cue cards (modified from the original medical symptom

classification task of Gluck and Bower, 1988). Each card is associated with a

certain probability of predicting either rain or sun (e.g., 20, 40, 60 or 80% rain).

This version of the task contained two different variations. The first was a

feedback task, designed to be analogous to nondeclarative learning, as the

probabilistic nature of the cards and trial and error approach to learning would

presumably make it difficult for participants to outwardly memorize the

associations (Knowlton et al., 1996). The second version was an observational

task, similar to declarative learning, as the cue card and outcome were presented

simultaneously, presumably making it easier for participants to directly memorize

the associations. One group of participants completed the feedback task and a

separate group completed the observational task. The authors found the

feedback version produced activation in the basal ganglia and deactivation in the

MTL. The MTL exhibited greater BOLD responses to the observational

compared with the feedback version of the task. Furthermore, when the authors

conducted a functional connectivity analysis, they found a negative correlation

between the MTL and a region in the right caudate nucleus across participants.

Additionally, in a separate event-related fMRI experiment using only the feedback

version of the task, the authors reported that the MTL was initially activated and

19

the caudate nucleus deactivated during the task. Over time, the MTL became

deactivated and the caudate nucleus became activated, suggesting a temporal

element to the engagement of the regions. Based on this evidence, the authors

concluded that the memory systems interact in a competitive manner during

learning.

A recent study replicated Poldrack and colleagues’ 2001 result by

reporting negative correlations between the hippocampus and specifically the

head of the caudate nucleus during a rule-learning task (Seger and Cincotta,

2006). The authors concluded that the antagonistic interaction between these

two regions is probably not direct, but rather is mediated by frontal cortical areas.

1.3.2 Evidence supporting cooperative interactions: A second hypothesis

regarding how these regions interact during learning suggests they cooperate,

such that they work together to facilitate learning (Dagher et al., 2001; Voermans

et al., 2004; Cincotta and Seger, 2007; Sadeh et al., 2011). Human-based fMRI

evidence supporting this theory was observed in a relatively recent experiment

which examined specifically how responses in the striatum are modified when

learning via feedback versus learning via observation (Cincotta and Seger,

2007). In this block-design study, the authors used an information integration

task and reported that BOLD responses in the caudate nucleus and the

hippocampus were analogous in nature. Contrary to Poldrack and colleagues’

2001 results where MTL and BG activation negatively correlated and the regions

were active at alternating time points, this observation suggests that the two

regions are at times active simultaneously and show similar patterns of activity

20

during learning. Similar patterns of activity within distinct memory regions may

be interpreted as cooperative engagement of these areas during learning.

Additionally, a recent study demonstrated cooperation between the hippocampus

and putamen during a declarative memory encoding task (Sadeh et al., 2011).

Activation within the hippocampus and putamen was associated with successful

memory encoding and critically, activity within these two regions was correlated

in participants for later remembered, but not forgotten words. Lastly, the strength

of the correlation between the putamen and hippocampus predicted successful

memory, such that the stronger the correlation across regions, the more words

participants subsequently remembered.

This observation is further supported by fMRI studies which posit that the

MTL and BG may cooperate in order to maintain normal functioning in

compromised brain states (Dagher et al., 2001; Voermans et al., 2004). Dagher

and colleagues (2001) investigated planning in Parkinson’s disease patients and

healthy control patients using the Tower of London task in a PET study. Results

indicated that Parkinson’s patients performed as well as healthy control

participants, but engaged a different network of brain regions during the task.

Healthy control participants primarily engaged areas of the PFC and the caudate

nucleus, while PD patients engaged similar prefrontal areas, but no caudate

nucleus activation was observed. Critically, activation within the hippocampus

was observed in PD patients, but was diminished in healthy control participants.

These results suggest that within PD patients, where striatal functioning is

compromised, hippocampal activation increased in order to maintain overall

21

successful planning in an executive function task. A similar result was observed

in a different disease population using a separate learning paradigm. In a study

by Voermans and colleagues (2004), Huntington’s disease (HD) patients were

required to perform a route navigation task. Results indicated that increasing

MTL BOLD responses helped compensate for decreased striatal functioning in

order to maintain successful route recognition in HD patients. In healthy control

participants, no such increase in MTL activity was observed. In fact, healthy

control participants engaged greater caudate nucleus activation, while HD

patients elicited greater hippocampus activation during route recognition.

Furthermore, healthy control participants exhibited greater functional connectivity

between the hippocampus and caudate nucleus than HD patients. The authors

suggest that in HD patients, this connectivity may change such that the MTL may

adapt to have a more independent and compensatory role due to compromised

striatal function. Such cooperation between learning systems suggests they may

work together to facilitate a normally functioning brain state. Converging

evidence across multiple disease states (PD, HD) employing distinct

experimental tasks (Tower of London, route navigation) provides strong evidence

that the medial temporal lobe and striatum do cooperate during some learning

scenarios, specifically when such cooperation maintains overall successful

learning and performance levels.

1.3.3 Evidence supporting parallel engagement: Lastly, a third hypothesis

describing how the MTL and BG interact posits that the memory systems operate

in parallel, complementing each other’s function in the brain, and therefore have

22

the ability to either cooperate or compete during learning (McDonald and White,

1994; White and McDonald, 2002; Albouy et al., 2008; Doeller et al., 2008;

Tricomi and Fiez, 2008). White and McDonald (2002) outline their theory of three

parallel memory systems in the rodent brain: the hippocampus, the dorsal

striatum, and the amygdala. They suggest that in a normal brain, these regions

process and store information simultaneously and in parallel. The authors also

state that the memory systems at times cooperate and at times compete during

learning depending on the specific learning situation. Evidence supporting this

theory in animal studies is seen in radial maze experiments which demonstrate

that similar behaviors, such as certain types of place learning, may be acquired

via either the dorsal striatum, particularly the medial dorsal section, or the

hippocampus (McDonald and White, 1995; Devan and White, 1999).

Furthermore, recent fMRI studies involving various, distinct paradigms

support parallel engagement of the medial temporal lobe and striatum. Doeller

and colleagues (2008) employed an environmental learning paradigm, adapted

from traditional animal maze tasks, to investigate interactions between the

hippocampus and striatum. Participants were required to learn the location of

several objects in a virtual environment, using a distinct environmental landmark

or a boundary for orientation purposes. Results indicated that the posterior

hippocampus was engaged in learning and remembering locations via

environmental boundaries, whereas the caudate nucleus was engaged when

learning and remembering locations via single, intramaze landmarks. These

results support the recruitment of the dorsal striatum during trial and error,

23

associative learning via distinct landmarks, and the engagement of the

hippocampus during boundary-based, incidental learning. Furthermore, the

authors employed a dynamic causal modeling analysis to investigate interactions

between these regions during learning. The results from this connectivity

analysis supported independent and parallel engagement of the hippocampus

and striatum, rather than direct interactions between the regions of interest. The

hippocampus and striatum therefore supported distinct types of learning and

were engaged independently and in parallel during learning of the environment in

this paradigm.

A second, unrelated study examined engagement of the striatum and

hippocampus during a declarative word pair learning task (Tricomi and Fiez,

2008). Participants were required to learn arbitrary word pairs via feedback, with

the same pairs of words being presented over three rounds. Importantly,

feedback was administered in each round, but on the first round, as participants

were randomly guessing the answer, feedback simply indicated a correct guess

and not a correct memory, as occurred in rounds 2 and 3. The authors reported

that both the caudate nucleus and the hippocampus were engaged during

learning, but exhibited distinct patterns of activity. The caudate nucleus

displayed an interaction of round by time, showing greater activation in the last 2

rounds, during the feedback presentation period. Furthermore, the caudate

nucleus differentiated between correct and incorrect feedback in the later rounds,

when feedback was meaningful. The hippocampus was also engaged, exhibiting

a main effect of time, initially decreasing after trial onset and subsequently

24

increasing after feedback presentation. These results are inconsistent with

purely competitive interactions between the BG and MTL as the two regions were

simultaneously active and did not negatively correlate. Furthermore the results

support parallel engagement of the MTL and BG as these areas were engaged

during the learning process, playing distinct roles during the acquisition of

declarative information. The authors concluded that the caudate nucleus may be

representing goal achievement during declarative learning.

Lastly, parallel engagement between the BG and MTL has been observed

in a fMRI motor sequence learning and memory task (Albouy et al., 2008). Both

the hippocampus and regions of the striatum, specifically the caudate nucleus

and putamen, were involved in initial motor sequence learning. Interestingly,

hippocampal activation during training predicted behavioral performance

improvement observed the next day. Additionally, a functional connectivity

analysis indicated competitive interactions between the hippocampus and the

putamen during learning. However, this interaction became cooperative 24

hours following sequence training in participants who were fast learners. This

result elegantly illustrates the parallel nature of the interactions between the MTL

and BG, showing that the two regions may both compete and cooperate during

learning and memory processes.

To summarize therefore, there are three conflicting hypotheses regarding

the nature of interactions between multiple memory systems. Evidence

supporting each theory is summarized in the table below:

25

Table 1.1 Evidence Supporting Interactions between Multiple Memory SystemsCompetitive Interactions Cooperative Interactions Parallel Engagement Mitchell and Hall, 1988 Dagher et al., 2001 McDonald and White, 1994Packard et al., 1989 Voermans et al., 2004 Devan and White, 1999 Packard 1999 Cincotta and Seger, 2007 Atallah et al., 2008 McDonald and White, 1993 Sadeh et al., 2011 Albouy et al., 2008 Poldrack et al., 2001 Tricomi and Fiez, 2008 Foerde et al., 2006 Doeller et al., 2008 Seger and Cincotta, 2006 Lee et al., 2008 For review see Poldrack and Packard 2003

1.4 Description of Dissertation Experiments

Thus it is clear there is an ongoing debate regarding the nature of

interactions between the medial temporal lobes and basal ganglia. The purpose

of this dissertation therefore was to investigate how these two memory systems

interact during learning. Based on the evolutionary theory of multiple memory

systems as well as anatomical and functional connectivity with dopaminergic

midbrain centers (SN/VTA), it was hypothesized that these distinct neural regions

will act in parallel during probabilistic learning, exhibiting signs of parallel

engagement, as well as competitive and cooperative interactions. Furthermore,

based on the compilation of evidence documenting dopamine’s involvement in

learning and memory, it was hypothesized that dopamine may play a role in

facilitating interactions between the MTL and BG during reward-related learning.

The goal of the first experiment was to investigate the relative

engagement and interactions between the MTL and the BG during probabilistic

learning using a within-subjects, event-related fMRI design and a novel, simple

probabilistic learning task. The purpose of this experiment was to explore in a

26

basic manner how each system, the BG and MTL, is engaged individually and

how these distinct systems interact during multiple types of probabilistic learning

(feedback and observation). It was hypothesized that the MTL and BG will

operate in parallel during learning, exhibiting parallel engagement as well as

potentially both cooperative and competitive interactions, which were measured

by relative activation within each region as well as functional and effective

connectivity between these regions during learning. It was further hypothesized

that such parallel engagement may be mediated by midbrain dopaminergic

connectivity, as indirectly assessed by the application of a reinforcement learning

model postulated to reflect dopaminergic activity during trial and error learning

(Schultz, 1997; O'Doherty et al., 2002). The results of this experiment provided a

fundamental understanding of the involvement of the MTL and BG during

different types of probabilistic learning.

The goal of the second experiment was to examine interactions between

the MTL and BG during re-learning or updating of probabilistic information

caused by a reversal in learning contingencies. The purpose of this experiment

was to examine how the MTL and BG interact during learning, using a novel

manner of addressing this question, via the employment of multiple types of

learning and a reversal in probabilistic cue contingencies. It was hypothesized

that immediately following reversal (cue contingencies switch) both the MTL and

BG will act in parallel to process new contingencies. Specifically, it was

predicted that initial inconsistencies between the old and new information will

diminish as the old contingencies are updated, regardless of whether updating

27

occurs within the same learning type (e.g., feedback-feedback or observation-

observation) or across learning types (e.g., feedback-observation or observation-

feedback). It was theorized that both competitive and cooperate interactions may

be observed during the updating process in accord with the parallel theory of

engagement among multiple memory systems. These interactions were

measured by the relative engagement of each system during initial learning and

reversal as well as functional and effective connectivity across the brain during

reversal. Lastly, in order to examine putative dopaminergic involvement in

mediating these interactions, a reinforcement learning model was applied. The

results of this experiment provided evidence of how multiple memory systems

interact when learning types evolve and learning contingencies reverse.

The goal of the third experiment was to probe a specific learning condition

in which the MTL and BG may exhibit cooperative interactions. The purpose of

this experiment was to test a direct hypothesis that was developed by observing

the engagement of the MTL and BG in experiments 1 and 2. Specifically, a

declarative memory recognition task was employed in order to engage the medial

temporal lobe (akin to a “behavioral knockout” of one region of interest, the MTL).

Concurrent with this interfering memory task, participants performed a

probabilistic feedback learning task, which contained both an easy/consistent as

well as a hard/inconsistent learning condition. Additionally, control conditions

were employed which did not contain the interfering declarative memory task,

and thereby did not aim to engage the MTL. The effect of the interfering memory

task was observed on behavior as well as brain regions engaged during

28

probabilistic feedback learning. It was hypothesized that multiple memory

systems will cooperate when information is inconsistent and therefore

challenging, specifically when both systems are available to be involved in

feedback probabilistic learning (control conditions), while memory systems will

operate more independently when information is more consistent and less

challenging, in support of a parallel processing model of learning. The relative

engagement of distinct memory regions was examined across learning sessions

(control sessions and declarative memory interference session) and interactions

across regions were examined by the application of simple correlation analyses.

The results of this experiment provided evidence of the engagement and

interactions between multiple memory systems in “normal” (single-task) versus

dual-tasking learning scenarios.

The experiments presented in this document were event-related and

mostly within-subjects fMRI designs which employed a novel probabilistic

learning task adapted from Delgado et al., 2005. Event-related fMRI designs

have certain advantages over block designs previously used (e.g., Poldrack et

al., 2001; Cincotta and Seger, 2007) in that they allow the experimenter to track

the BOLD signal changes to specific events. Additionally, when it is feasible,

within-subject designs are also preferable as they eliminate possible sources of

variance (across-subject comparisons). However, sometimes between-subjects

comparisons are necessary (experiment 2 for example) based on the theoretical

question and experimental design. While much important work has been

conducted using the popular weather prediction task (WPT) a novel probabilistic

29

categorization task was chosen for these experiments for the following two

reasons. First, the tasks designed in this thesis are simpler than the weather

prediction task, in that they contained only 1 learnable cue per stimulus, as

opposed to multiple cue cards. Second, the task is based on a card guessing

game, which has been previously shown to robustly engage the striatum in a trial

and error learning variant – one of the key regions of interest for these

experiments (Delgado et al., 2005). Other than these two distinctions, the tasks

employed in this thesis and the WPT are similar and therefore results from the

experiments presented here may be somewhat comparable to studies that

employ the weather prediction task.

1.5 Significance

The results from the experiments outlined in this dissertation aimed to

provide a richer understanding of how the MTL and BG interact during human

probabilistic learning. By better understanding how these systems are engaged

during learning and memory, and how they interact, a more complete picture of

how healthy humans solve probabilistic learning tasks was obtained.

Understanding interactions among human memory systems is critical, as these

neural areas are implicated in many neurodegenerative and neuropsychiatric

diseases. Therefore, these results may eventually aide research efforts directed

towards understanding and improving treatment for Parkinson’s disease,

Huntington’s disease, Alzheimer’s disease, MTL amnesia, and schizophrenia.

Additionally results from these experiments may shed some light on individual

30

differences in learning, such as individuals who learn better via observation via

those who learn better via trial and error experiences.

Chapter Two: General Materials, Methods, & Data Analyses

2.1 General Materials & Methods

This section provides a brief overview of the common elements in

experiments 1-3. Details containing information specific to individual

experiments are presented in the materials and methods section of the

appropriate chapter.

2.1.1 Experimental designs: Each task was programmed using E-Prime

Version 2.0 (PST, Pittsburgh). Participants inside the MRI machine saw the task

via use of a back projection system using a mirror affixed to the head coil. A MRI

compatible button box was used to record behavioral responses during the

experiments. For all experiments, participants were given detailed instructions

and completed a short practice version outside of the MRI prior to beginning the

actual task (different stimuli were used during practice sessions). In experiment

2, which was a between-subjects design, additional instructions were

administered while the participants remained inside the MRI machine after the

first test session was completed in order to train them on the type of learning they

did not experience during the learning phase (e.g. trained on feedback learning if

the participant was in the observation learning group). Monetary compensation

consisted of a minimum of $25 per hour upon arrival at the imaging center for

completion of the study, with additional incentives based on participants’

31

performance during the task for experiments 1 and 2. Specifically, participants

were awarded additional money based on a paper test administered after

completion of the MRI portion of the experiment (see Appendix 3 for the

questionnaires administered in experiments 1-3). In experiment 1, total

compensation ranged between $50 and $65; in experiment 2, total compensation

ranged between $50 and $56. In experiment 3, participants were told they would

be paid additional money based on how well they learned the cues and how well

they discriminated between old and new natural scenes. However, all

participants who completed the study were paid $60 and were told they

performed well in the task, irrespective of how well they actually performed in the

experiment. Participants were paid the same amount in experiment 3 solely to

simplify the payment process.

2.1.2 Participants: All participants were right handed adults over the age

of 18 years. Every participant was screened for head injury and a history of

psychiatric or neurological impairments as well as contraindications to MRI

(metal implants, claustrophobia). All females were administered a pregnancy

test, which was verified to be negative, prior to beginning the MRI session.

Informed consent was obtained from each participant before beginning the

experiment. All studies were approved by the Institutional Review Boards of

Rutgers University and the University of Medicine and Dentistry of New Jersey

(UMDNJ). Experiment 1 contained seventeen participants (nine female, mean

(M) age 24 years, standard deviation (SD) 4.1); experiment 2 had fifty-two

participants (26 female, M age 22 years, SD 4.7); and experiment 3 had twenty-

32

eight participants (15 female, M age 22 years, SD 5.0). Final analysis in

experiment 1 included 16 participants (9 female), as one person was excluded

due to a scanner malfunction in the middle of the MRI session. In experiment 2,

final analysis consisted of forty-one participants (20 female), as 6 people missed

an excessive amount of trials (greater than 2 SDs from the M; some due to falling

asleep), 3 people were excluded due to excessive motion (movement beyond 5

millimeters or systematic spikes), 1 person was excluded due to a scanner

malfunction, and 1 person was excluded due to failure to learn the task (chance

accuracy, 50%, in the learning phase). Experiment 2 was a between-subjects

design consisting of a feedback group and an observation group: final data

analysis included 21 participants in the feedback group and 20 in the observation

group. The complete data set for experiment 3 consisted of twenty-four

participants (12 female), as two participants were excluded due to malfunctions

with the MRI overheating, one person did not return for the second day (fMRI

session), and one person was excluded due to failure to attend during the day 1

encoding session.

2.1.3 fMRI acquisition: All MRI experiments were conducted at the

UMDNJ Advanced Imaging Center. The MRI machine at this facility is a 3-Tesla

Siemens Allegra, which was used to collect the structural (T1-weighted

MPRAGE: 256 x 256 matrix; FOV = 256 mm; 176 1-mm sagittal slices) and

functional images (single-shot echo EPI sequence; TR = 2000 ms, TE = 25ms;

FOV = 192 cm; flip angle = 80°; matrix = 64 x 64; slice thickness = 3 mm). Forty

contiguous oblique-axial slices (3 x 3 x 3 mm voxels) were acquired parallel to

33

the anterior commissure – posterior commissure line. Functional data

preprocessing and analysis was completed using Brain Voyager QX software

(Versions 1.10; 2.0, and 2.2 Brain Innovation, Maastricht, The Netherlands).

Preprocessing consisted of three dimensional motion correction (six-parameters),

spatial smoothing (4 mm, FWHM), voxel-wise linear detrending, high-pass

filtering of frequencies (3 cycles per time course), and normalizing the data to

Talairach sterotaxic space (Talairach and Tournoux, 1988). In addition, the

mean intensity of the BOLD signal was examined in experiments 2 and 3, but

was not included in the preprocessing of the data. A canonical two gamma

hemodynamic response function was used in all experiments to convolve the

events of interest.

2.2 General Data Analyses

This section provides descriptions and explanations of the common

analyses conducted in experiments 1-3. Study specific information for each

experiment is provided in the data analysis section of the appropriate chapter.

Following fMRI preprocessing, functional data analysis was completed using a

whole brain analysis technique for all experiments, as well as region of interest

(ROI) analyses for experiment 3.

2.2.1 General linear model: In all phases of each experiment, random-

effects general linear models (GLM) were performed. Predictors of interest

(which varied based on the experiment and are described in the relevant

chapters) were included in the model as well six motion parameters which were

34

included as regressors of no interest in order to account for potential motion

related issues. In experiments 1 and 2, data analysis was conducted across the

4 second time period where the stimulus, response, and feedback occurred. The

analysis was performed in this time period in order to most appropriately

compare the feedback and observation trials. Relevant information for the

feedback trials occurred in the last 2 seconds, where feedback was provided,

whereas relevant information for the observation trials was present in the first 2

seconds, where participants observed the cue and association (informative arrow

indicating the cue value). Therefore, to most accurately compare these distinct

trial types, the stimulus/response and feedback periods were combined for the

analyses performed in experiments 1 and 2. In experiment 3, analyses were

conducted over several time points. The first analysis examined BOLD activation

during the stimulus presentation period, where participants viewed the natural

scenes/probabilistic cues and made a choice regarding the probabilistic cue

value. The second analysis examined the BOLD signal during the feedback

period, where participants received feedback contingent on their response in the

cue period, which indicated the outcome of their probabilistic cue value choice

(correct, incorrect, missed trial). The third and final analysis was a region of

interest analysis examining BOLD signals during the stimulus presentation period

in 7 a priori ROIs: the bilateral midbrain (centered on the ventral tegmental area),

bilateral ventral striatum, bilateral hippocampus, and the left caudate nucleus.

Details about the ROIs used are provided in Chapter 5.

35

2.2.2 Granger causality analyses: In addition to the basic analyses

performed, a granger causality analysis (GCA) was conducted in order to

examine functional and effective connectivity in the brain during various phases

of the experiments (details provided in the methods sections of experiments 1

and 2). This analysis is especially informative as the experiments conducted in

this dissertation were designed to examine interactions between the MTL and

BG, which are difficult to probe using traditional neuroimaging analyses alone.

Granger causality is advantageous as it examines functional and effective

connectivity among multiple brain regions during a designated time period. GCA

operates by looking for relationships between time course data from a seed

region (x) with the rest of the brain (y). Using vector autoregression, Geweke

(1982) proposed a measure of linear dependence, Fx,y between two hypothetical

times series of data, x[n] and y[n]. Fx,y is the sum of three components:

Fx,y = Fx→y + Fy→x + Fx⋅y

Fx→y measures the directed influence from x to y; probing if past values of x

improve the current predicted value of y.

Fy→x measures the directed influence from y to x; probing if past values of y

improve the current predicted value of x.

Fx⋅y measures the undirected instantaneous influence of x and y. This measure

adds in the current value of x or y to the model already containing the past values

of x and y.

Therefore, GCA examines both undirected instantaneous influence (Fx⋅y) and

directed (Fx→y and/or Fy→x) influences between a designated seed region (x) and

36

the rest of the brain (see Goebel et al., 2003; Roebroeck et al., 2005 for more

details). For the analyses performed in the experiments in this dissertation,

connectivity was examined across the entire run (first to last volume/TR) and an

autoregressive model order of 1 TR (2 seconds) was used, consistent with the

BrainVoyager plugin and with previous studies (Goebel et al., 2003; Roebroeck

et al., 2005). Therefore Granger causality between distinct brain regions was

conducted by examining 2 seconds into past time course data. Of special

interest in this set of experiments was examining connectivity between the MTL

and BG, with potential involvement of the midbrain.

2.2.3 Prediction error analyses: Reinforcement learning models were

applied to learning trials in experiments 1 and 2. The purpose of applying the

reinforcement learning models was to gain a better understanding of not only

what regions were engaged during learning, but also how mechanistically

participants were solving the tasks. Specifically, it was hypothesized that

dopaminergic influences may play a role in facilitating interactions between the

MTL and BG. In addition to examining BOLD activation in the dopaminergic

midbrain, which is an anatomically small region whose exact location is difficult to

determine using standard neuroimaging techniques, it was theorized that the

application of a reinforcement learning model would aid in examining putative

dopaminergic influences. Dopamine neurons are known to be involved in

feedback-based learning and prediction error encoding and project to both the

striatum and hippocampus (Scatton et al., 1980; Lynd-Balta and Haber, 1994;

Schultz et al., 1997; Pagnoni et al., 2002; McClure et al., 2003; Abler et al.,

37

2006). Therefore, one way of examining putative dopaminergic influences during

learning is to apply a prediction error learning model to the behavioral accuracy

data, with the aim of examining which brain areas track prediction error signals

during learning. If the BG and/or MTL vary parametrically with a PE signal, it is

possible that these regions are receiving similar information via midbrain

dopaminergic neurons.

Regressors for the prediction error models used in experiments 1 and 2

were calculated based on a Q-learning model (Watkins, 1989). The Q learning

model was chosen to model behavioral choices as the literature supports this

type of learning as an effective means of predicting instrumental choice in

probabilistic reward learning tasks (Samejima et al., 2005; Pessiglione et al.,

2006; Burke et al., 2010). A Q learning model operates by estimating the

expected value of choosing action A and action B based on the individual

sequence of choices and outcomes obtained by participants. Q models use

prediction error signals to update action values according to the Rescorla-

Wagner learning rule. In the context of experiments 1 and 2, this choice is

making a button press to indicate whether a probabilistic cue has either a higher

or lower than 5 value (e.g., choosing a high value). The expected value of

making a cue-action choice is termed a Q value and represents the expected

reward obtained by taking a certain action, in this case receiving feedback when

making an individual choice. In order to model behavior, the following equations

were used:

38

1. Qj, i+1 ← Qj, i + λδ

2. δ = r – Qj, i

The model contains the following parameters: 1) initial Q values (Qj), 2)

learning rate, λ, and 3) the softmax function temperature, m (akin to slope). A

maximum likelihood estimation algorithm was used to estimate optimal values for

these three parameters. The cue-action values (Qs) were then updated

according the Rescorla-Wagner learning rule. Specifically, following each trial,

the value of choosing higher than 5 and lower than 5 was updated according to

Equation 1: Qj, i+1 ← Qj, i + λδ. Equation 2, which calculates the prediction error,

δ = r – Qj, i is simply the difference between the expected (Qj,i) and the actual

outcome (r) of a trial. The probability of choosing each action was estimated by

using the softmax rule: for example if choosing high, the softmax function is:

P(Q↑,i) = exp(Q↑i /m)/(exp(Q↑i /m)+exp(Q↓i /m). The variable m is a temperature

parameter which measures variability, such that for low temperatures, the

probability of choosing the action which has the highest expected reward

approaches 1 (steep slope, exploitative behavior), where as for high

temperatures, all actions have almost the same probability (moderate slope,

explorative behavior). In the model, prediction error is represented by δ; r equals

the reward amount, which in the feedback trials was 1 for correct trials or 0 for

incorrect trials; i = trial number and j represents the type of action: participants’

choice of high (↑) or low (↓). A single set of free parameters was used for all

39

participants when generating the prediction error regressors which were later

used in the neuroimaging analyses in experiments 1 and 2.

For experiment 2, the prediction error model described above was

modified for the observation trials, based on the model presented by Burke and

colleagues (2010). Specifically, an action prediction error signal was generated

for the observation trials. In order to do so, it was conceived that in the

observation learning condition, participants could simply view the cue/value

associations passively (although they were instructed not to), and therefore

theoretically think of each cue’s value as being assigned or chosen by the

computer. Therefore, for the participant, the observation trials could be thought

of as similar to observing another agent, in this case the computer, play to learn

the cue/value associations, while the participant observed and subsequently

learned the cue/value associations as well. An action prediction error is defined

as δaction = action(H,L) - p(H, L) where action ∈ (H, L; H: higher than five cue

value; L: lower than five cue value). The probability values, p(H, L) of observing

each cue-action pair were calculated by the Q(↑) and Q(↓) values used in the

softmax function. For example, for trial i, if the model predicted that the

probability of choosing H (higher than 5 cue value) is 0.7 and the participant

observed that choosing H is the correct choice, according the model: δaction =

actioni(H,L) - p(H, L), δaction = 1-0.7 = 0.3. Hence a low surprise or saliency signal

is generated. If the participant observed that in fact L (lower than 5 cue value)

was the correct choice, then δaction = 1-0.3 = 0.7, therefore a greater

surprise/salience signal. Due to the way δaction is constructed, an action

40

prediction error value is always positive (as action always sums to 1), and can be

used as an indicator for how “salient” the computer’s choice was to the

participant for that particular trial.

As a note, a prediction error analysis was not conducted in experiment 3.

The predominant reason for this was that the third experiment was much more

complex in design than experiments 1 and 2 and as a result would be difficult to

model. In experiment 3, during the stimulus presentation period, there was also

a natural scene present in all learning sessions and presumably participants

were trying to 1) ignore this information, 2) make a perceptual decision, or 3)

remember the scene, based on the session type. Therefore, this was not a pure

feedback learning environment as there were multiple cognitive demands

present. Even though prediction errors were most likely being generated in

response to the cue outcomes, since this was a much more complex learning

environment, it was decided not to conduct a prediction error analysis in this

experiment.

2.2.4 Cluster level statistical threshold estimator: The cluster level

statistical threshold estimator is a plugin included in the BrainVoyager analysis

package which provides a correction for multiple comparisons (Forman et al.,

1995; Goebel et al., 2006). This plugin operates by employing Monte Carlo

simulations in order to determine the likelihood of observing clusters of various

sizes in a given map at a set threshold. In order to use the plugin, the statistical

parametric map of interest is first thresholded at the desired level (e.g., p<0.005,

uncorrected). Proceeding the initial thresholding, the plugin estimates the spatial

41

smoothness of the map and performs a whole brain correction using Monte Carlo

simulations which estimate the rate of false positives in the present map at the

cluster level. In all analyses reported, 1,000 iterations (the recommended

number of the plugin) were chosen. After the simulations, the selected map

automatically applies the minimum cluster size threshold which produces the

desired cluster-level false-positive alpha rate (5% was chosen in all analyses).

All remaining active clusters in the corrected map are used in creating a table

that summarizes the number of clusters above the chosen threshold (5%) for

each cluster size. When the analysis is complete, each cluster size is assigned

an alpha value determined by the frequency of its occurrence in the map. This

analysis therefore corrects for multiple cluster tests in the map.

2.2.5 Sequential Bonferroni correction: In all experiments, post-hoc

analyses conducted on the behavior and neuroimaging data consisting of more

than two t-tests within a family of comparisons were corrected for multiple

comparisons with the sequential Bonferroni technique (Holm, 1979; Rice, 1989).

This analysis was developed in 1979 by Holm as a less strict correction for

multiple comparisons, compared to the traditional Bonferroni correction. In this

analysis, the desired alpha level (e.g., 0.05) is divided by the number of t-tests to

be performed, k (e.g., 3). The table-wide alpha level then becomes that value

(e.g., 0.05/3 = 0.0167). The p values of all three t-tests are then ranked in order

from most to least significant (P1 to Pk, or 0.004, 0.02, 0.04 for example). If the

most significant p value (0.004) is less than alpha/k (e.g., 0.0167), then it can be

considered significant at the “table-wide” alpha level and the experimenter may

42

proceed in examining the next test. If the most significant p value is greater than

alpha/k, then all resulting p values are termed insignificant at the “table-wide”

level. In the given example, 0.004 is less than 0.0167, so the experimenter may

examine the next test. The alpha level is then divided by k-1, in this example (3-

1=2). The new alpha level is now 0.05/2 or 0.025 and the next most significant p

value is 0.02, so it is also considered significant at the table-wide level. The

experimenter may proceed on to the last test, using an alpha level of 0.05/(3-2,

or 1). Therefore in this example, all tests are significant at the table-wide level.

2.2.6 Note regarding test phase neuroimaging results: All of the test

phase neuroimaging results are presented in Appendix 1. The test phases were

not primary phases of interest in experiments 1-3 but rather were performed as

secondary analyses in order to ensure that participants successfully acquired cue

contingencies in all learning sessions. These phases were also performed to

examine the engagement of memory regions when recalling previously learned

cue values in an exploratory fashion. Because the experimental sessions of

interest were long (learning and update phases), the test sessions were short;

especially the tests administered in experiments 1 and 2, which contained a

limited number of trials (30 trials in the experiment 1 test – 3 presentations of

each cue; 32 trials in the experiment 2 tests– 4 presentations of each cue). For

these reasons, the results were not of primary interest and are presented in

Appendix 1.

43

Chapter Three: Experiment 1

3.1 Introduction: Background & Rationale

It is theorized that memory is not a unitary process, but rather consists of

multiple systems which rely on distinct neural substrates (Sherry and Schacter,

1987; Squire, 1992; Squire and Zola, 1996). A significant body of research,

spanning across species, supports the existence of multiple memory systems in

the mammalian brain. A major division is between the declarative memory

system, dependent on the medial temporal lobes, and a nondeclarative memory

system, which engages several brain structures, one of which is the basal

ganglia system. While decades of research have investigated the individual roles

of the aforementioned areas during multiple types of learning, a key remaining

question of interest is examining how the medial temporal lobes and basal

ganglia interact during learning and memory formation. This question has

aroused great debate in the literature, with some evidence supporting

competitive interactions (Poldrack et al., 2001; Poldrack and Packard, 2003;

Foerde et al., 2006; Lee et al., 2008), other research supporting cooperative

interactions (Dagher et al., 2001; Voermans et al., 2004; Cincotta and Seger,

2007; Sadeh et al., 2011) and a third body of evidence supporting parallel

engagement of the medial temporal lobes and basal ganglia system (White and

McDonald, 2002; Albouy et al., 2008; Atallah et al., 2008). For the purposes of

this thesis, McDonald and White’s definition of parallel memory systems is

employed (2002), which suggests that these distinct systems may be online

simultaneously and involved in distinct aspects of learning. The authors posit

44

that information passes independently through each system, such that each

system receives the same information, but specializes in representing different

aspects of the information. Furthermore, it is theorized that these systems may

either interact directly in a cooperative or a competitive manner or may

simultaneously influence behavior in parallel.

While examining the nature of the interactions between the medial

temporal lobes and the basal ganglia, it is important to consider how these

distinct regions are functionally and anatomically connected. Research suggests

that there are direct anatomical projections from the hippocampal formation to

the nucleus accumbens and the caudate nucleus/putamen in the rodent (Jung et

al., 2003). Furthermore, some researchers have proposed models describing

how these regions may interact during learning. One key model of interest,

developed by Lisman and Grace (2005) proposes that the MTL and BG may

interact via anatomical connectivity with the dopaminergic midbrain. In the

model, the hippocampus functions to detect the entrance of novel information

and sends this novelty signal to the ventral tegmental area in the midbrain via

projections through the subiculum, nucleus accumbens (ventral striatum), and

ventral pallidum. The dopaminergic neurons within the VTA fire in response to

this novelty signal, subsequently releasing dopamine into the hippocampus

where it enhances long term potentiation. Importantly, the VTA also sends a

projection back to the nucleus accumbens. Therefore, this model suggests a

functional loop between the hippocampus, nucleus accumbens, and midbrain,

which specializes in novelty detection and the entrance of information into long-

45

term memory. Dopamine is a logical candidate for facilitating interactions

between these regions as it is believed to play a critical role in learning and

memory processes (Wise, 2004; Shohamy and Adcock, 2010). Dopamine is

implicated in reward-related learning, which is typically associated with striatal

function (for review see Schultz, 2002) as well as facilitating long-term memory

formation in humans, which is more traditionally associated with medial temporal

lobe activation (Wittmann et al., 2005; Adcock et al., 2006; Shohamy and

Wagner, 2008; Shohamy and Adcock, 2010). Therefore, anatomical as well as

functional evidence supports a role for dopamine in facilitating interactions

between the basal ganglia and medial temporal lobe.

Given the debate regarding the nature of interactions between multiple

memory systems, further research is required to examine multiple memory

systems and how they interact during learning and memory formation. The

purpose of the first experiment of this dissertation was to investigate in a broad

manner, the relative engagement and interactions between the MTL and the BG

during a simple learning paradigm using a within-subjects fMRI design.

Probabilistic learning was chosen because a major portion of the literature has

employed probabilistic learning tasks to examine both the BG’s and the MTL’s

involvement in learning. It has been suggested to be an effective manner of

engaging both nondeclarative/implicit and declarative/explicit learning

mechanisms (Knowlton et al., 1996; Shohamy et al., 2004b), although this theory

is debated (Lagnado et al., 2006). A novel paradigm was developed for this set

of experiments based on a card guessing task (Delgado et al., 2005), which was

46

previously shown to engage the striatum. It was decided to modify this task in

particular due to the fact that it strongly engaged one of the key systems of

interest (the basal ganglia). Modifications were performed to the task design in

order to try to engage the MTL as well; specifically an observation version of the

task was developed, modified from the task design used by Poldrack and

colleagues (2001). In the experiment, participants were required to learn the

value of easy and hard probabilistic cues either through feedback (trial and error)

or via observation (paired-association). Based on the synthesis of current

research in the field, three main points were considered while formulating a

hypothesis regarding the nature of interactions between the MTL and BG: 1) both

cooperative and competitive interactions between these regions have been

reported in the literature, 2) evolutionary considerations suggest these two

systems play fundamentally different roles in the brain, and 3) White and

McDonald’s theory of parallel engagement among memory systems in the brain

posits that the MTL and BG may operate in parallel or interact directly in a

cooperative or competitive manner. Based on the above evidence from the

literature, it was hypothesized that the MTL and BG will operate in a parallel

manner, potentially exhibiting both cooperative and competitive interactions.

This hypothesis was tested by examining the relative activation within both the

MTL and the BG during feedback and observation learning as well as functional

and effective connectivity between these regions during learning. It was further

hypothesized that such parallel engagement may be mediated by midbrain

dopaminergic connectivity, based on dopamine’s involvement in reward-related

47

learning and long-term memory formation as well as the known neuroanatomical

projections from the midbrain to both the striatum and hippocampus (Shohamy

and Adcock, 2010). Dopamine’s influence in mediating interactions between the

medial temporal lobe and the basal ganglia was indirectly assessed by the

application of a reinforcement learning model postulated to reflect dopaminergic

activity during feedback learning.

3.2 Materials & Methods

3.2.1 Experimental design

The probabilistic learning task used in this study was a modification of a

previously used feedback-based card learning paradigm (Delgado et al., 2005).

In this adapted version of the task, participants were asked to perform a

categorization task that involved determining the numerical value of several

probabilistic visual cues (e.g., circle is higher than the number 5; Figure 3.1).

There were two distinct phases of the task, a learning phase, where participants

acquired the values of the visual cues, followed by a test phase, where

participants were asked to remember the cue associations they learned in the

previous phase. Participants were instructed that each visual cue had been

assigned a numerical value on the scale from 1-9 and could be thought of as

having a value of either higher (6-9) or lower (1-4) than the number 5. They were

instructed that they did not need to learn the exact value of each cue (e.g. circle

is 7), but rather group the shapes into those with a value of higher than five and

those with a value of lower than five.

48

Two independent variables were manipulated in the task. The first was

the type of learning, or how the participants acquired the cue associations. In the

learning phase, there were two distinct ways in which participants learned about

the values of 8 cues. One manner was via feedback (4 cues), where participants

guessed and received feedback regarding the cue’s value, akin to trial-and-error

or nondeclarative learning (Figure 3.1A). The second manner was via

observation (4 cues) where participants observed the cue paired with its value,

more similar to declarative learning. During feedback trials, participants saw a

visual cue and made a button press indicating their belief regarding the value of

the cue (button 1 = higher than 5; button 2 = lower than 5). Visual cue

presentation and response (2 seconds total) were followed by feedback

contingent on the participants’ response (check mark for correct trials, X mark for

incorrect trials, and # symbol for missed trials; 2 seconds). Feedback was

followed by a jittered inter-trial-interval (ITI; 2-14 seconds) to allow for the

hemodynamic response to return to baseline. During observation trials,

participants observed the cue paired with an informative arrow indicating the

value of the cue (upward-facing arrow denotes a higher than 5 value; downward-

facing arrow denotes a lower than 5 value) and made a response indicating their

belief regarding the value of the cue (button 1 = higher than 5; button 2 = lower

than 5; Figure 3.1B). The visual cue and response period (2 seconds) was

followed by a message informing the participants if their response had or had not

been recorded (2 seconds). As long as the participants made a button press

within the 2 second visual cue/response period, they received a message that

49

their answer was recorded, if they failed to respond within 2 seconds for any

reason, they received a message that their answer was not recorded (missed

trial). This message was followed by a jittered ITI (2-14 seconds) before

presentation of the next visual cue. The button response in the observation trials

served as a motor control for the button response made during the feedback

trials, equated the duration of each trial type (4 seconds total), and was an

attempt to equate the difficulty of the trial types. During the learning phase,

participants completed 4 blocks, alternating between feedback and observation

blocks (2 of each type). Each block contained 40 trials, for a total of 160 trials in

the learning phase. Feedback and observation blocks differed in terms of the

cues presented as well as cue color (e.g., feedback cues were pink, observation

cues were blue). Block order and cue color were counterbalanced, and within

each block, cue trial order was randomized.

The second independent variable manipulated in the paradigm was the

probability of the cues (predictive outcome of the numerical value for each cue),

referred to as cue difficulty. Four cues (2 feedback and 2 observation) were 85%

predictive of the outcome (higher or lower than 5) and are referred to as “easy

cues” and the remaining four cues (2 feedback and 2 observation) were 65%

predictive of the outcome and are referred to as “hard cues”. Cue difficulty was

included as a variable as both the BG and MTL are known to be modulated by

the difficulty of material to-be-learned. During the learning phase, participants

were specifically told to optimize their responses to the cues. They were told that

as the value of the cues was probabilistic, for the observation trials, the direction

50

of the arrows would change. For example, if a cue is 85% higher than five it is

thereby 15% lower than five. Participants were instructed to pay careful attention

to the direction of the arrows and to try to determine what the value of each cue

was most of the time. On all trials, they were instructed to press the button

corresponding to the value that cue was associated with for the majority of the

trials in the task. Therefore, on the inconsistent trials, (cue is lower than five)

participants could choose to push the button indicating “higher”, if that is what

they believed the value of the cue to be most of the time. As a result of these

instructions, participants were actively making a choice regarding each cue’s

value and were not passively pushing the button that always matched the

direction of the arrow. Because participants were instructed to optimize their

responding, behavioral accuracy was scored according to participants’ actual

choices. That is if they always pushed the button indicating a higher than five

response, for a cue which was 85% or 65% higher than 5, they would receive a

score of 100% correct; if they followed the arrows exactly they would probability

match and be 85% or 65% correct respectively. As a point of clarification,

participants were not told the exact probabilities (e.g., 85%) but were simply

introduced to the concept of probabilistic information. The test phase was

presented immediately following completion of the learning phase, while the

participants remained inside the scanner (Figure 3.1C). The test phase

contained all eight visual cues presented in the learning phase, in addition to two

novel cues presented for the first time in the test phase. The novel cues were

presented to provide a control for comparing previously learned information with

51

novel information. There were 30 trials total in the test phase, three

presentations of each of the ten cues. Trials consisted of a self-timed cue

presentation/response period followed by a jittered ITI (6-14 seconds) before

onset of the next cue. Critically, feedback (from the feedback trials) and

informative arrows (from the observation trials) were not presented in this phase.

Participants saw only the cue by itself (e.g., circle) and made a button press to

indicate its value, based on what they learned in the learning phase. A second

test phase was administered a couple of days following completion of the MRI

study in a behavioral-only test session completed in the laboratory (M = 2.63

days, SD = 1.63 days).

3.2.2 Data analysis

GLM: A random-effects GLM analysis was conducted in the learning

phase to examine BOLD activity during the 4-second cue presentation +

feedback period, using the following five predictors: learning type (feedback and

observation), cue difficulty (easy and hard), and missed trials. From this GLM

statistical parametric maps (SPM) were generated and thresholded at p<0.005

with a cluster threshold correction of 5 contiguous voxels (135mm3 in 1x1x1mm

units). A repeated-measures ANOVA was then conducted within BrainVoyager

using learning type (feedback and observation) and cue difficulty (easy and hard)

as within-subjects factors. The primary SPM of interest investigated a main

effect of cue difficulty. This analysis was of interest as it provided a non-biased

examination of the relative engagement and pattern of activity within the BG and

the MTL during learning. Functional ROIs were defined based on the resulting

52

map and the BOLD signal (characterized by beta weights or mean parameter

estimates) was extracted in order to examine potential similarities and

differences between feedback and observation learning in post-hoc analyses. In

addition, potential effects of learning changes over time were examined. In order

to model time, the learning phase was divided according to run/block. The BOLD

signal was examined in the functional regions of interest (e.g., caudate nucleus

and hippocampus) for the first learning run/block of each learning type (e.g.,

feedback trials 1-40; observation trials 1-40) and similarly for the second learning

run/block of each learning type (e.g., feedback trials 41-80; observation trials 41-

80). Subsequently, the mean BOLD signal from the first or “early” learning run

was then compared with the mean BOLD signal from the second or “late”

learning run for both learning types in both regions. The second SPM of interest

directly investigated a main effect of learning type. The third SPM of interest

examined potential interactions of cue difficulty and learning type.

During the test phase, BOLD activity was examined during the stimulus

onset and participant response period. A random effects GLM was conducted

with observation (easy and hard), feedback (easy and hard), and novel cues as

predictors, in addition to 6 motion parameters included as regressors of no

interest. As the GLM is unbalanced, a 2x2 (learning session x cue difficulty)

within-subjects ANOVA could not be performed in this phase. Therefore, a

contrast of previously studied (observation and feedback cues collapsed across

difficulty) vs. non studied (novel) cues was performed, thresholded at p <0.005.

The cluster level statistical threshold estimator plugin was used on the resulting

53

SPM. Parameter estimates were extracted from the resulting region and

examined with post-hoc analyses. Any reported regions not withstanding

correction are labeled clearly and should be interpreted with caution (Poldrack et

al., 2008).

GCA: A Granger causality analysis was performed specifically to probe

potential interactions between the hippocampus and striatum during learning.

The functionally defined hippocampus ROI from the main effect of cue difficulty

ANOVA in the learning phase was used as the principal seed region for this

analysis. This ROI was chosen as the primary seed region given the direct

anatomical projections from the hippocampus to the basal ganglia (Kelley and

Domesick, 1982). Functional and effective connectivity maps were generated

separately for the feedback and observation runs. At the individual subject level,

the two feedback blocks were combined to form one feedback map and the two

observation blocks were combined to form one observation map. Time course

values entered into the analysis contained the entire run, from the first to the last

time point (240 volumes per run). At the multi-subject level, each participant’s

feedback map was combined to make one group feedback map and each

participant’s observation map was combined to make one group observation

map. Maps were given a threshold of p<0.0001 with a cluster threshold

correction of 7 continuous voxels for the observation maps and 8 continuous

voxels for the feedback maps (cluster-level false-positive rate of 5%). Standard

second level statistics were performed on the group maps; specifically a t-test

was calculated. Reported results do not contain effective connectivity (Fy→x

54

and/or Fx→y), as no effective connectivity was observed between the regions of

interest (MTL and BG). Therefore the results reported represent functional

connectivity (instantaneous influence with no directionality information; Fx⋅y).

Lastly, a second Granger causality analysis was performed in the same manner

as a control analysis, using the caudate nucleus region obtained from the

learning phase ANOVA main effect of cue difficulty analysis. The resulting maps

were thresholded at p<0.0005 with a cluster threshold correction of 12

continuous voxels (cluster-level false-positive rate of 5%) unless otherwise noted.

PE: The prediction error values generated from the Q learning model

presented in the general methods chapter were then used as a regressor in the

neuroimaging GLM analysis. Eight additional predictors were included in the

GLM: trial event and missed responses, as well as six motion parameters.

Within the GLM, the PE values were coded during the two second feedback

presentation time period for only the feedback learning trials. A SPM which

probed regions of the brain that varied parametrically with the PE values during

feedback learning was generated, thresholded at p<0.005 with a voxel contiguity

of 5 continuous voxels (cluster-level false-positive rate of 5%).

3.3 Results

3.3.1 Behavioral results

Learning phase: Accuracy

The primary analysis probed accuracy differences between the learning

types and levels of cue difficulty. A 2 (learning type: feedback vs. observation) x

55

2 (cue difficulty: easy vs. hard) repeated measures ANOVA was performed. No

main effect of learning type (F(1,15) = 1.47; p > 0.05), a main effect of cue

difficulty, (F(1,15) = 16.04; p< 0.01), and no significant interaction (F(1,15) = 0.85; p

> 0.05) were observed (Figure 3.2A). In order to examine the main effect of cue

difficulty within and across conditions, post-hoc t-tests were performed.

Participants’ accuracy was significantly better on easy compared to hard cues for

both the feedback (t(15) = 2.45; p < 0.05) and observation (t(15) = 2.54; p < 0.025)

trials. There was marginally better performance for easy cues during observation

compared with feedback learning (t(15) = 2.24; p = 0.04; trend after sequential

Bonferroni correction), with no performance differences for hard cues between

learning types (t(15) = 0.23; p > 0.05).

A secondary analysis examined changes in learning accuracy over time in

feedback and observation trials by comparing performance early (first block)

versus late (second block) during the learning phase. A 2 (learning type:

feedback vs. observation) x 2 (cue difficulty: easy vs. hard) x 2 (time: early vs.

late) repeated measures ANOVA indicated a no main effect of learning type

(F(1,15) = 1.38; p > 0.05), a main effect of cue difficulty (F(1,15) = 16.09; p < 0.01), a

main effect of time (F(1,15) = 35.07; p < 0.01), and no significant interactions. The

significant main effect of time was driven by better performance for the hard but

not the easy trials over time.

Test phase: Accuracy

The first analyses examined differences between learning sessions and

levels of cue difficulty in each individual test session (immediate and follow-up).

56

During the individual test phases, no differences between feedback and

observation learning or between easy and hard cues were observed. In the

immediate test phase, a 2 (learning type: observation vs. feedback) x 2 (cue

difficulty: easy vs. hard) repeated measures ANOVA (excluding the novel

information and examining only the previously studied material) revealed no

significant main effect of learning type (F(1,15) = 0.06; p > 0.05), no significant

main effect of cue difficulty (F(1,15) = 0.23; p > 0.05), and no significant interaction

(F(1,15) = 1.92; p > 0.05). These results suggest that participants successfully

learned contingencies independent of learning type and/or level of cue difficulty.

The same 2 x 2 repeated measures ANOVA was performed for the follow up test

phase. No significant main effect of learning type (F(1,15) = 0.51; p > 0.05), no

significant main effect of cue difficulty (F(1,15) = 0.37; p > 0.05), and no significant

interaction (F(1,15) = 0.00, p > 0.05) were observed.

The second analysis probed for differences in accuracy between learning

material over time. In order to test this, one analysis examined differences

across the two test sessions. A 3 (learning material: observation, feedback,

novel) x 2 (time: immediate vs. follow up test session) repeated measures

ANOVA was performed. A main effect of learning material (F(1.90,28.43) = 14.04; p

< 0.01), no main effect of time (F(1,15) = 0.11; p > 0.05), and a marginally

significant interaction (F(1.36,20.36) = 3.33; p = 0.07) were observed (all factors were

Greenhouse-Geisser corrected; Figure 3.2B). The main effect of learning

material indicated greater accuracy for studied (feedback and observation)

compared with non-studied (novel) material. Post-hoc t-tests revealed that

57

participants’ performance in the feedback version of the task marginally declined

in the follow-up test session (t(15) = 2.13; p = 0.05; trend after sequential

Bonferroni correction), but no differences were observed for observation

accuracy over time (t(15) = 1.04; p > 0.05). As anticipated, participants’ accuracy

in the novel condition did not change across time (t(15) = 1.01; p > 0.05).

3.3.2 Neuroimaging results

Learning phase

Main Effect of Cue Difficulty: The primary analysis of interest in the

learning phase was to examine a main effect of cue difficulty, as that was the

significant result in the behavioral data and would provide for a non-biased

examination of the MTL and BG’s activation during learning. A 2 (learning type:

feedback vs. observation) x 2 (cue difficulty: easy vs. hard) within-subjects

ANOVA was performed within BrainVoyager and a main effect of cue difficulty

was examined (Table 3.1). A region of the left caudate nucleus was identified as

processing a main effect of cue difficulty (Figure 3.3), along with an area in the

left hippocampus that did not survive cluster threshold correction. Mean

parameter estimates from these two functionally defined ROIs were then

extracted for further analyses. In the left caudate nucleus (x, y, z = -15, 20, 7;

Figure 3.3A and Figure 3.3B), the pattern of BOLD responses was similar for

both feedback and observation learning, with no differences observed between

type of learning when collapsed across cue difficulty (t(15) = 1.00; p > 0.05). Post-

hoc t-tests indicated a greater BOLD response for easy compared with hard cues

in both types of learning [feedback: (t(15) = 5.02; p < 0.025); observation: (t(15) =

58

3.99; p < 0.05]. In the left hippocampus (x, y, z = -36, -28, -8; Figure 3.3C and

Figure 3.3D) a marginally significant effect was observed when comparing mean

parameter estimates from the feedback and observation sessions (t(15) = 2.01; p

= 0.06), with a marginally greater BOLD response for the observation trials. This

difference was driven mostly by performance for the hard trials (t(15) = 2.40; p =

0.03; trend after sequential Bonferroni correction). Post-hoc t-tests indicated a

greater BOLD response for easy compared with hard cues for the feedback

learning (t(15) = 3.73; p < 0.025) and a marginally greater response for easy than

hard cues in the observation trials: (t(15) = 1.79; p = 0.09).

As a secondary analysis, changes in the BOLD response during feedback

and observation sessions over time were examined by comparing the BOLD

signal early (first block) versus late (second block) during the learning phase. A

2 (learning type: feedback vs. observation) x 2 (cue difficulty: easy vs. hard) x 2

(time: early vs. late) repeated measures ANOVA conducted on the mean BOLD

responses extracted from the caudate nucleus revealed no main effect of

learning type (F(1,15) = 0.82; p > 0.05), a main effect of cue difficulty (F(1,15) =

95.47; p < 0.01), no main effect of time (F(1,15) = 1.97; p > 0.05), and no

significant interactions. A nearly significant increase in mean parameter

estimates over time was observed for the observation hard cues in the caudate

nucleus (t(15) = 2.25; p = 0.04; trend after sequential Bonferroni correction). The

same 2 x 2 x 2 repeated measures ANOVA was conducted on the BOLD

responses extracted from the hippocampus ROI. A nearly significant main effect

of learning type (F(1,15) = 3.36; p = 0.09), a main effect of cue difficulty (F(1,15) =

59

20.69; p < 0.01), a main effect of time (F(1,15) = 5.35; p < 0.05), and a nearly

significant interaction between learning type and time (F(1,15) = 3.76; p < 0.07)

were observed. Post-hoc comparisons indicated a nearly significant increase in

the hippocampus for the feedback hard cues as learning progressed (t(15) = 2.38;

p = 0.03; trend after sequential Bonferroni correction).

Main Effect of Learning Type: The second analysis examined a potential

main effect of learning type, generated from the same learning type x cue

difficulty ANOVA described previously (Table 3.2). Several regions of the basal

ganglia were engaged including the right ventral portion of the head of the

caudate nucleus (x, y, z = 6, 3, 4) and the left ventral caudate nucleus extending

into the globus pallidus (x, y, z = -12, 2, 4). No voxels within the medial temporal

lobe were observed at a threshold of p < 0.005. These results indicate that some

regions of the BG are involved in primarily processing feedback information more

so than observation information, supporting a parallel model of engagement.

Interaction of Cue Difficulty and Learning Type: The last main analysis

from the learning phase ANOVA examined a potential interaction between

learning type and cue difficulty. Results revealed activation in an area of the left

medial prefrontal cortex. Post-hoc t-tests conducted on the mean parameter

estimates extracted from this region indicated greater activity for observation

easy than hard trials (t(15) = 4.63; p < 0.025); but no difficulty differences for the

feedback trials (t(15) = 1.26; p > 0.05). The interaction was driven by a greater

BOLD response to observation easy compared with feedback easy cues (t(15) =

2.72; p < 0.025) and a marginally greater response to feedback than observation

60

hard cues (t(15) = 1.94; p = 0.07). These results indicate that this region of the

prefrontal cortex is involved in processing feedback and observation information,

modified by cue difficulty.

Learning phase: Correlations between striatum and medial temporal lobe

As a first pass at exploring the relationship between the caudate nucleus

and hippocampus’s BOLD responses during learning, a series of Pearson’s

correlations were performed. A significant positive correlation during the

observation learning session was observed between mean parameter estimates

extracted from the hippocampus and the caudate nucleus identified in the main

effect of cue difficulty analysis (r = 0.498, p = 0.05). There was no relationship

between the two regions BOLD responses overall in the feedback session.

However, exploratory analyses revealed a marginally significant positive

correlation between the caudate nucleus and hippocampus during the last block

of feedback learning, specifically for easy cue trials when participants’

expectations were violated by the receipt of incorrect feedback (r = 0.530, p =

0.08).

Granger Causality Analysis

To more thoroughly assess the level of connectivity and potential

interactions between the BG and MTL during learning, a Granger causality

analysis was performed. The left hippocampus from the main effect of cue

difficulty learning phase analysis was used as the primary seed region. The

resulting Granger causality maps highlight instantaneous correlations between

the hippocampus (seed region) and regions of the striatum during both

61

observation and feedback probabilistic learning sessions. In particular,

instantaneous influence between the hippocampus and two regions of the right

caudate nucleus were observed during feedback learning (x, y, z = 14, 18, 13

and x, y, z = 14, 12, 19; not shown), as well as nearly the identical ROIs in the

caudate nucleus (x, y, z = 14, 18, 13 and x, y, z = 14, 10, 18; not shown; Figure

3.4A) and one region of the right ventral putamen in observation learning (x, y, z

= 22, 3, -4; not shown) (Figure 3.4B). A second Granger causality analysis using

the caudate nucleus as the seed region was performed as a control analysis

because the hippocampus seed was taken from an uncorrected region in the

learning session. The results yielded instantaneous influence between the

caudate nucleus and bilateral hippocampal regions in observation learning

sessions (x, y, z = -31, -38, -3 and x, y, z = 32, -29, -12; not shown) corrected to

a cluster level false positive rate of 5%, as well as an area near the right

hippocampus in feedback learning sessions (x, y, z = 29, -8, -15; uncorrected for

multiple comparisons).

Prediction Error Analysis

The last analysis conducted examined a putative influence of midbrain

dopaminergic neurons in learning, which are known to project to both the

striatum and the hippocampus (Scatton et al., 1980; Lynd-Balta and Haber,

1994) and are involved in learning and memory processes (Wise, 2004; Lisman

and Grace, 2005; Shohamy and Adcock, 2010). The prediction error learning

signal is believed to be a correlate of physiological firing of dopamine neurons

during reward-related learning (Schultz, 1997; Schultz et al., 1997; Schultz,

62

2002). Based on the observation of activation of both the striatum and

hippocampus during the learning phase - processing differences in cue difficulty,

a reinforcement learning algorithm was applied to the behavioral data. The

results yielded an area of the left putamen (x, y, z = -30, 2, 4; Figure 3.5A) and

the right hippocampus (x, y, z = 27, -28, -14; Figure 3.5B) whose activation

parametrically varied with the PE signals generated from the model. Thus, the

use of a reinforcement learning model during feedback learning engaged the

striatum, a typically reported region in prediction error encoding (Pagnoni et al.,

2002; McClure et al., 2003; Abler et al., 2006) as well as the hippocampus, a

novel finding in an area which has more often been associated with mismatch

signals (Ploghaus et al., 2000; Chen et al., 2011).

3.4 Discussion

The purpose of this experiment was to investigate the relative

engagement and interactions of multiple memory systems during probabilistic

learning. The nature of how multiple memory systems interact is debated, with

several studies suggesting that the BG and MTL compete during learning

(Poldrack et al., 2001; Foerde et al., 2006; Lee et al., 2008). The data collected

in this experiment, however, are inconsistent with the hypothesis of purely

competitive interactions, and support an alternative hypothesis that these

memory systems operate in parallel during learning (White and McDonald, 2002;

Albouy et al., 2008; Atallah et al., 2008). Using an event-related probabilistic

task and within-subjects measures of learning allowed for a simple and controlled

63

investigation of the engagement and interactions between multiple memory

systems. Behavioral results indicated that as learning progressed, accuracy was

modulated by cue difficulty (easy and hard) across learning types (feedback or

observation). Regions within the BG and MTL, specifically the caudate nucleus

of the striatum and the hippocampus, tracked these behavioral changes as they

were modulated by cue difficulty, displaying similar patterns of activity in

response to both feedback and observation information during learning.

Furthermore, BOLD signals within these regions were positively correlated during

learning, as assessed by simple correlations and a more sophisticated functional

connectivity analysis using both the hippocampus and the caudate nucleus as

reference regions. The functional connectivity observed between these areas

may potentially be explained by modulation of midbrain dopaminergic neurons

during reward-related learning (Lisman and Grace, 2005; Shohamy et al., 2008;

Shohamy and Adcock, 2010) as loci within both the striatum and MTL were found

to vary parametrically with a prediction error signal. The results of the prediction

error analysis further corroborate the hypothesis that these distinct memory

systems may operate in parallel while processing probabilistic information as

both the striatum and hippocampus were actively engaged in learning scenarios

where violations of expectations or mismatches occurred, which may function as

useful updating signals during the learning process.

The results from the present experiment complement the visuomotor and

simple association learning literature, which supports the theory that the basal

ganglia and medial temporal lobes may be engaged simultaneously in learning

64

arbitrary visuomotor associations (Toni et al., 2001; Amso et al., 2005; Law et al.,

2005; Haruno and Kawato, 2006). The current results are consistent in particular

with a recent study by Mattfeld and Stark (2010) in which the interaction of the

BG and MTL was investigated using an arbitrary visuomotor association task.

Several regions of the BG and MTL exhibited increases in BOLD signal as the

strength of memory increased during a trial and error task, suggesting that

regions of the BG and MTL are engaged when learning arbitrary associations.

The observation in the present experiment of the caudate nucleus and

hippocampus demonstrating larger BOLD responses to easy compared to hard

cues complements Mattfeld and Stark’s result. The authors also employed a

functional connectivity analysis, results of which indicated functional connectivity

between both the ventral (nucleus accumbens) and dorsal (caudate nucleus)

striatum with the hippocampus during learning, corroborating the interactive

nature of the hippocampus and striatum during learning and memory processes.

One key distinction between the task employed in the current experiment and

more traditional visuomotor tasks is the inclusion and examination of different

learning types (observation in addition to feedback).

The current experiment’s results enhance recent neuroimaging findings

demonstrating cooperative interactions between the BG and MTL during

category learning (Voermans et al., 2004; Cincotta and Seger, 2007). In the

present experiment, an event-related design allowed for the decoupling of factors

such as cue difficulty and examining changes in learning over time to lend further

support to the hypothesis that parallel processing in the BG and MTL contributes

65

to overall learning during probabilistic paradigms. A significant positive

correlation between the BG (caudate nucleus) and MTL (hippocampus) during

the observation learning session and a marginally significant positive correlation

during the feedback learning session were observed in the present experiment.

Whereas negative correlations have been interpreted as competition between

memory systems (Poldrack et al., 2001), positive correlations may suggest

cooperative, perhaps synergistic interactions, leading to the interpretation that

the systems may communicate with each other in specific contexts to facilitate

probabilistic learning. This interactive nature between the MTL and BG is

illustrated by Granger causality results in the present study which indicated that

loci within the striatum and the hippocampus were correlated at simultaneous

time points during the learning phase when using both the caudate nucleus and

hippocampus as reference regions.

Based on the Lisman and Grace model (2005), it may have been

predicted that ventromedial regions of the striatum would be engaged during the

present paradigm, correlating with hippocampus activation, as opposed to more

dorsal regions. Consequently, it may be surprising that dorsal regions of the

striatum were observed during learning. However, the literature supports a role

for the dorsal striatum during instrumental tasks (O'Doherty et al., 2004),

particularly when learning is action contingent (Tricomi et al., 2004). Moreover,

the dorsal striatum, especially the caudate nucleus, has been shown to be

involved in cognitive tasks involving feedback (Poldrack et al., 2001; Seger and

Cincotta, 2005; Seger, 2008; Tricomi and Fiez, 2008). Considering this

66

evidence, it is logical that the strongest loci of activation within the striatum were

observed in the dorsal rather than ventral striatum in the current cognitive

learning paradigm. Importantly, there is also a direct projection from the

hippocampus to the caudate nucleus in the rodent, albeit a ventral region of the

caudate nucleus (Jung et al., 2003). Lastly, it has been posited that the ventral

striatum may communicate with the dorsal striatum via multiple spiral loops

connecting the striatum with dopaminergic midbrain centers (Heimer et al., 1997;

Haber, 2003). Therefore, it is possible that interactions between the

hippocampus, midbrain DA areas, and more dorsal regions of the striatum exist,

via a ventromedial to dorsolateral movement of information through the

aforementioned spiral loops.

In addition to the similarities in the activation observed in the hippocampus

and regions of the basal ganglia during the task, two differences emerged. First,

regions within the basal ganglia were modulated by a main effect of learning

type, while no voxels were identified in the MTL showing such differentiation.

Several neuroimaging papers have shown that reward processing and feedback

recruit regions of the ventromedial striatum (for review see Delgado, 2007); as

such, it is not surprising that this region was recruited more strongly during the

feedback learning trials. It may have been expected that the MTL would be

selectively modulated by observation trials given previous findings (Poldrack et

al., 2001); however, this result was not observed in the current experiment.

While a null result in neuroimaging studies does not indicate any particular

finding, and the context and experimental design of the present paradigm differ

67

from previous probabilistic learning studies, it is possible that the MTL is recruited

during both feedback and observation learning in the present task – as

suggested by the main effect of difficulty analysis. A second difference that was

observed during learning between the hippocampus and caudate nucleus was

that the hippocampus showed a main effect of time (early x late learning) during

the learning session, whereas responses within the caudate nucleus were not

significantly modulated across time. This effect was driven by changes in activity

during feedback learning (primarily for the hard cues), which increased during

later compared to early stages of learning in the hippocampus. This result may

suggest that the hippocampus is recruited more heavily for difficult feedback

trials later in the learning process.

The observation of both the hippocampus and striatum processing

differences in cue difficulty, but only the ventral striatum exhibiting stronger

engagement during feedback learning is indicative of parallel engagement of

these regions during probabilistic learning. These results suggest that the

striatum plays a dynamic role during probabilistic learning, processing both

learning type as well as cue difficulty. Furthermore, the results suggest that the

striatum may be processing similar or distinct information as the hippocampus –

suggesting they may operate independently and in parallel. Neither purely

cooperative interactions (similar pattern of activity in the BG and MTL in all

learning scenarios) nor purely competitive interactions (opposite patterns of

activity during learning or negative correlations) were observed in the present

paradigm.

68

As a caveat for the present study, it should be noted that the hippocampus

locus obtained from the main effect of cue difficulty analysis was reported at a

threshold of p<0.005, uncorrected for multiple comparisons. Additionally, two

areas in the test phase analysis, a region adjacent to the hippocampus and the

caudate nucleus, were also reported at p<0.005 uncorrected. Although this

information may be useful for studies in the future, regions reported at an

uncorrected threshold should be regarded with caution due to the increased

likelihood of observing a Type I error (Poldrack et al., 2008).

There are several open questions remaining based on the results of

experiment 1. The primary question of interest is addressing in a more direct

manner how the BG and MTL interact during probabilistic learning. One manner

of addressing this question is by employing multiple types of learning as well as

by examining how systems interact during the reversal of probabilistic

contingencies. The present study examined the relative engagement of distinct

regions during different types of learning and probed interactions between these

regions using correlations, functional connectivity analyses, and reinforcement

learning models. However, because all participants completed both learning

types in the same learning session, it is possible that the current results are

biased by the presentation of both learning types (feedback and observation)

within the same learning phase. Therefore, in order to address this concern, it is

necessary to examine the engagement of these discrete regions in observation

and feedback learning in a distinct manner. Experiment 2 attempts to address

these unresolved issues by 1) employing a between-subjects design – such that

69

participants initially learn via only one type of learning and 2) by reversing

contingencies in the middle of the learning session to probe in a novel manner

how the memory systems interact during probabilistic learning.

Chapter Four: Experiment 2


The nature of interactions between multiple memory systems is currently

debated in the literature. Evidence supporting cooperative (Dagher et al., 2001;

Voermans et al., 2004; Cincotta and Seger, 2007; Sadeh et al., 2011),

competitive (Poldrack et al., 2001; Foerde et al., 2006; Seger and Cincotta, 2006;

Lee et al., 2008), and parallel (McDonald and White, 1994; White and McDonald,

2002; Albouy et al., 2008; Doeller et al., 2008; Tricomi and Fiez, 2008)

interactions has been documented across species using several different types

of learning paradigms. In experiments specifically involving human participants

probabilistic category learning (Knowlton et al., 1994; Knowlton et al., 1996;

Poldrack et al., 1999; Poldrack et al., 2001; Hopkins et al., 2004; Shohamy et al.,

2004b; Shohamy et al., 2004a; Shohamy et al., 2008), information integration

category learning (Ashby et al., 2002; Cincotta and Seger, 2007), visuomotor

learning (Seger and Cincotta, 2005; Mattfeld and Stark, 2010), spatial navigation

(Voermans et al., 2004), and declarative memory tasks (Tricomi and Fiez, 2008;

Sadeh et al., 2011) have all been employed when investigating this question.

Some studies have examined interactions between memory systems by

employing distinct types of learning, in particular feedback and observation

70

learning (Poldrack et al., 2001; Ashby et al., 2002; Shohamy et al., 2004b;

Cincotta and Seger, 2007; Schmitt-Eliassen et al., 2007; Djonlagic et al., 2009; Li

et al., 2011). Feedback learning is traditionally associated with engagement of

the basal ganglia, in particular the striatum (see Delgado, 2007 for review),

whereas human observation learning (or instructed learning) has been

associated with medial temporal lobe (Poldrack et al., 2001; Li et al., 2011),

striatal (Cincotta and Seger, 2007; Li et al., 2011), as well as dorsolateral and

ventromedial prefrontal cortex activation (Burke et al., 2010; Li et al., 2011).

Results from fMRI experiments that specifically employed feedback and

observation learning have supported both competitive (Poldrack et al., 2001) and

cooperative interactions (Cincotta and Seger, 2007) between the medial temporal

lobes and basal ganglia. Given that the nature of these interactions is debated,

further research is required to probe how multiple memory systems interact

during probabilistic learning. The goal of this experiment was to address this

question by employing multiple types of learning and examining interactions

across two stages of the learning process: initial acquisition and later updating

following a reversal in cue contingencies (e.g., cue value reversal).

Several studies have demonstrated a role for the basal ganglia during

reversal learning (Annett et al., 1989; Schoenbaum and Setlow, 2003; Setlow et

al., 2003; Frank and Claus, 2006). One study of interest examined the neural

correlates of probabilistic reversal learning and demonstrated activation of

frontostriatal regions during the key time point of behavioral reversal (Cools et al.,

2002). In this probabilistic reversal learning task, participants initially acquired

71

cue contingencies via trial and error. After a predetermined number of correct

responses, cue contingencies reversed, unbeknownst to the participants. The

participants were then required to update the new cue associations and change

their responding in order to receive correct feedback. Cools and colleagues

demonstrated that a frontostriatal network was engaged when processing the

critical, final error preceding participants’ reversing their behavioral choices in

order to receive correct feedback. This study illustrates the involvement of the

frontostriatal network during the updating of probabilistic cue contingencies. This

result is relevant as the task employed in the present experiment contained one

reversal of probabilistic cue contingencies during learning. Furthermore, the

striatum is one of the key areas of interest in the studies presented in this

dissertation, and it is therefore pertinent to know that it may be mediating the

reversal of probabilistic cue contingencies.

In addition, the medial temporal lobes have also been implicated in playing

a fundamental role in reversal learning. Both research using non-human animals

and humans has documented the involvement of the hippocampus during

reversal learning. Lesions to the hippocampus in animals impair reversal

learning (Zola and Mahut, 1973; Berger, 1983; Fagan and Olton, 1986; Marston

et al., 1993). In human studies, patients with bilateral damage to the

hippocampus are impaired at reversing previously acquired stimulus-outcome

associations (Myers, 2000; Carrillo et al., 2001; Myers et al., 2006) and tend to

perseverate by choosing previously a correct association. A study recently

confirmed a role for the basal ganglia and the medial temporal lobes during

72

probabilistic reversal learning using a feedback-based task with Parkinson’s

patients and MTL amnesic patients (Shohamy et al., 2009). Shohamy and

colleagues modified the weather prediction task, making a simpler task in which

each cue predicted the correct outcome 80% of the time and each cue pattern

(every trial consisted of a pattern of 3 cue cards on the screen) predicted the

outcome 100% of the time. Results indicated that while both PD and MTL

amnesic patients were able to initially acquire cue contingencies, both patient

groups demonstrated impairments with reversal. PD patients did successfully

reverse cue contingencies, but did so by choosing to respond to a new cue

among the pattern, thereby learning a new cue-outcome association, rather than

updating the old cue-outcome association and thus “opting out” of the reversal.

MTL amnesic patients failed to successfully reverse cue contingencies

completely and demonstrated perseverative behavior to previously correct

associations. The authors concluded that the basal ganglia and medial temporal

lobes therefore both contribute to probabilistic reversal learning, but in unique

ways. The basal ganglia may be required for updating old cue-outcome

associations, whereas the medial temporal lobes may play a broader role in

flexibly modifying cue-outcome associations. These data are in accord with the

parallel model of engagement between multiple memory systems.

Of additional interest, Swainson and colleagues (2000) demonstrated that

probabilistic reversal learning was impaired in mild Parkinson’s patients (PD),

who were receiving dopaminergic medication as a part of disease treatment.

The authors concluded that their results were in agreement with the dopamine

73

“overdosing hypothesis” (Gotham et al., 1988; Frank et al., 2004; Frank, 2005;

Cools, 2006). The overdosing hypothesis suggests that dopaminergic medicine,

while augmenting depleted dopamine levels in the dorsal striatum in the early

stages of PD, overdoses the still intact ventral striatum’s dopamine levels, one of

the consequences of which is washing out critical prediction error signals thus

preventing them from influencing behavior. This elegant study therefore

demonstrated a critical role for dopamine during the acquisition of probabilistic

information. This result is relevant to the present experiment, and to this

dissertation as a whole, as it is hypothesized that dopamine may be facilitating

interactions among multiple memory systems during probabilistic learning.

The task in experiment 2 combines the use of multiple types of learning

(feedback and observation) with probabilistic reversal learning in order to probe

how multiple memory systems interact during a dynamic learning environment. It

may be informative to examine the engagement of memory systems when

learning types evolve and when learning contingencies change, as this may

provide a novel way of examining how distinct neural substrates interact during

the learning process. The purpose of experiment 2 therefore was to examine in

a novel manner, how multiple memory systems interact, by probing these

interactions when learning types evolve. In particular, the experimental task

investigated changes in the engagement and interactions between multiple

memory systems when probabilistic cue contingencies were initially acquired via

one learning type and subsequently updated via the same as well as a distinct

learning type. Specifically, the experiment was a between-subjects, event-

74

related fMRI design involving two groups of participants. While undergoing

functional neuroimaging, one group initially acquired all probabilistic cue

contingencies via feedback learning, and subsequently updated the cue

contingencies via the same learning type, feedback, as well as a new, different

learning type, observation (group 1/feedback group). A second group of

participants initially acquired cue contingencies via observation learning and

subsequently updated contingencies via both the same, observation, and a new,

different learning type, feedback (group 2/observation group). This design

therefore allowed for probing a key question of interest, that being the

investigation of the relative engagement and interactions between memory

systems when updating probabilistic cue information via the same learning type

as initial cue acquisition (feedback-feedback/group1 and observation-

observation/group2) and a different learning type as initial cue acquisition

(feedback to observation/group1 and observation to feedback/group2).

Addressing the question in this manner allowed for the examination of neural

substrates in initial feedback learning versus observation learning, as well as the

investigation of what systems were engaged and potential interactions between

these systems when the learning type changed (that being either feedback-

observation or observation-feedback) versus when the learning type remained

the same (feedback-feedback or observation-observation).

Additionally, by employing this task design, an examination of pure

feedback learning (group 1) as well as pure observation learning (group 2) was

initially examined. This was necessary as the results from experiment 1

75

indicated involvement of both the striatum and hippocampus during feedback and

observation learning. It is possible that this result was observed, however, not

due to the fact that the striatum and hippocampus were both involved in both

learning types, but rather that each region was engaged in one learning type

(e.g., striatum during feedback and hippocampus during observation) and

activation was simply contaminated in the areas due to the presence of both

learning types in the same learning session. Therefore, a between-subjects

design was employed in experiment 2 in order to address this confounding issue

in experiment 1. While between-subjects designs do contain increased

variability, as comparisons are made across subjects, at times such designs are

deemed necessary. Following the initial acquisition period, a test was

administered in both groups to ensure participants had fully acquired the cue

contingencies. After the test, the update phase was administered, in which

participants in both groups 1 and 2 learned about cue contingencies via both

learning types (feedback and observation). Critically, cue contingencies were

reversed in this session, unbeknownst to the participants, requiring them to

update previously correct answers in order to receive correct feedback and

perform well in the task. Lastly, a second test was administered in order to test

whether participants had correctly updated the new cue contingencies.

The following behavioral hypotheses were formulated: in the learning

phase, it was hypothesized that participants would initially acquire cue

contingencies well, irrespective of how contingencies were initially acquired

(feedback/group1 vs. observation/group2 learning). In the update phase, it was

76

hypothesized that participants’ accuracy would initially decrease following

probabilistic cue reversal for both learning types (feedback and observation trials)

in both groups (groups 1 and 2), and subsequently improve over time as

participants updated old cue contingencies. Lastly, two additional hypotheses

were made regarding behavioral accuracy in the update phase: 1) participants

would be more efficient at updating cue associations within the same learning

type (observation-observation or feedback-feedback) or 2) participants would be

more efficient at updating cue associations by employing a new learning type

(observation-feedback or feedback-observation).

The following neuroimaging hypotheses were formulated: based on the

results of experiment 1, it was hypothesized that the basal ganglia and medial

temporal lobes would be engaged during the initial acquisition (learning phase) of

both feedback (group 1) and observation (group 2) information. During the

update (reversal) phase, it was hypothesized that both memory systems would

be engaged and involved in updating cue contingencies in parallel, potentially

exhibiting both cooperative as well as competitive interactions during the

updating process. Interactions between distinct memory systems during

updating were measured by the relative engagement of each region during the

update phase, the application of simple correlation analyses, as well as the

application of more sophisticated functional and effective connectivity analyses.

Lastly, reinforcement learning models were applied to the learning and update

phases in order to examine the role of putative dopaminergic influences during

the initial acquisition of feedback (group 1) and observation (group 2) information,

77

as well as during the updating of cue contingencies via both feedback and

observation learning in both groups.



This experiment was a modification of the task used in experiment 1

(Figure 4.1). In this between-subjects version of the probabilistic learning task,

participants were asked to learn the numerical value of six different visual cues,

which ranged on a numeric scale from 1-9 and could be categorized as either

higher or lower than 5. There were five phases of the task: learning phase

(Figure 4.1A), test phase 1 (Figure 4.1B), update phase (Figure 4.1C), test phase

2 (Figure 4.1D), and a follow-up test (not shown) administered on a subsequent

day in the laboratory. The same two independent variables were manipulated as

those in experiment 1, with some modifications: learning type (feedback and

observation) and cue difficulty (easy and hard, with the addition of a new type

which was deterministic). In the learning phase, participants acquired the value

of six cues in only one manner, via either feedback or observation. Those who

completed the feedback learning session are referred to as group1/feedback

group and those who completed the observation learning session are referred to

as group2/observation group. In the update phase, all participants learned the

new cue values via both feedback and observation trials (half of the cues via

feedback; half of the cues via observation). In terms of cue difficulty, there were

three types of cues: 100% predictive of the outcome (2 cues referred to as

78

deterministic), 87.5% predictive of the outcome (2 cues referred to as easy), and

62.5% predictive of the outcome (2 cues referred to as hard).

The learning phase consisted of 2 blocks of 48 trials each, all of which

were either feedback or observation (feedback shown in Figure 4.1A). Feedback

and observation trial structure and timing in this version of the task were identical

to that used in experiment 1 (2 second stimulus/response phase, 2 second

feedback period, 2-14 second jittered ITI). All cues in this task were blue in color.

As in experiment 1, participants were instructed to optimize their responding, in

both the learning and update phases. Following completion of the learning

phase, participants advanced to the first test session, which contained the 6 cues

learned in the learning phase, as well as two novel cues (Figure 4.1B). There

were 32 trials total in the test phase, four presentations of each of the eight cues,

presented in random order. The cues from the learning phase were blue, and

the novel cues were white in color. Trial structure and timing in this test phase

were identical to that used in the test phase from experiment 1 (self-timed

stimulus/response phase, no feedback or observation information provided, 6-14

second jittered ITI).

Following completion of the first test phase, participants were instructed

on the type of learning they had not experienced in the learning phase (feedback

learning for the observation group; observation learning for the feedback group).

After completion of a short practice session while inside the MRI, they began the

update phase. Participants were informed that they were going to continue

learning about the value of the cues, now via both types of learning. Participants

79

were also instructed that some things may have changed since the first part of

the experiment (the learning phase), but they were not given any specific

information other than the vague indication that something may be different. The

structure of the update phase was very similar to that of the learning phase and

consisted of 2 blocks of 48 trials each (Figure 4.1C). However, there were two

critical manipulations in the update phase. The first was that half of the trials

were feedback (3 cues: 1 deterministic, 1 easy, and 1 hard) and the remaining

half were observation (3 cues: 1 deterministic, 1 easy, and 1 hard). The second

key manipulation was that all of the cue contingencies switched, (e.g., circle was

higher than 5 in the learning phase, but lower than 5 in the update phase);

however the probabilities of the cues remained the same (e.g., circle was 100%

higher than five in the learning phase and 100% lower than five in the update

phase). Trial structure and timing were identical to that used in the learning

phase (2 second stimulus/response phase, 2 second feedback period, 2-14

second jittered ITI). Trial type (feedback or observation) and trial order were

randomized; therefore feedback and observation trials were intermixed during

this phase. Participants were not given any indication of the nature of the

upcoming trial type; they simply determined the trial type based on the cue

presentation/response period. If it was a feedback trial, they saw only the cue, if

it was an observation trial, they saw a cue paired with an arrow. Participants

were instructed that the arrow was an informative clue indicating the value of the

cue (higher or lower than 5). Following completion of the update phase, a

second test phase was administered that was identical to the first test phase (six

80

blue cues shown in the update phase and the same 2 white novel cues that were

shown in the first test phase; Figure 4.1D). All four phases were completed while

the participants remained in the MRI. Lastly, a follow-up behavioral-only test

session was administered a couple of days following completion of the MRI

portion of the experiment in the laboratory (M = 2.24 days, SD = 1.82 days).

4.2.2 Data analysis

Behavioral Data Analysis:

In order to examine the behavioral accuracy data, a series of ANOVAs

were performed. The primary question of interest was to examine accuracy in

each phase of the experimental task (learning phase, test 1, update phase, test

2, follow-up test) as it varied according to learning type (feedback and

observation for the learning and update phases, with the addition of novel for the

test phases) and cue difficulty (deterministic vs. easy vs. hard). It was also a

question of interest to examine behavioral performance across groups, to

determine if accuracy was significantly greater for one group than the other. In

order to do so, group was used as between-subjects factor in all of the analyses

presented in the behavioral results section and learning type and cue difficulty

were used as within-subjects factors. Details regarding specific ANOVAs

performed in each session are presented in the appropriate behavioral results

sections. As a note, one subject’s data from the first test session was lost;

therefore the feedback group test 1 data set consists of 20 subjects.

Neuroimaging Data Analysis:

81

GLM: In the learning phase participants who completed the feedback and

observation learning sessions were analyzed separately in order to examine the

engagement of the basal ganglia and medial temporal lobes during feedback

learning (group 1) distinct from observation learning (group 2). However, the

same analyses were performed on each group’s data; specifically two random-

effects GLMs were conducted to examine BOLD activity during the 4-second cue

presentation + feedback period using the following predictors: learning type

(feedback for the feedback group and observation for the observation group) and

cue difficulty (deterministic, easy, and hard). Missed trials, six motion

parameters, and the mean intensity BOLD signal were included as regressors of

no interest. Mean intensity was included in this experiment only as a regressor

of no interest as an examination of the mean intensity of all participants indicated

fairly substantial deviations from the mean (M = 5 SDs away from the average

intensity signal). Two one-way ANOVAs using cue difficulty as a within-subjects

factor were conducted within BrainVoyager. Resulting statistical parametric

maps were set at a threshold of p<0.005 and a cluster threshold level of 6

contiguous voxels or 136mm3 (corrected to a cluster-level false-positive rate of

5%). Functionally defined regions were identified from the resulting maps (group

1/feedback and group 2/observation) and the BOLD signal was extracted for

further post-hoc analyses.

In the update phase, the comparison of greatest interest was to examine

which regions were engaged when updating information via the same (feedback-

feedback for group 1 and observation-observation for group 2) or a different

82

(feedback-observation for group 1 and observation-feedback for group 2) type of

learning. In order to examine this, all 41 participants were combined into one

analysis. One random-effects GLM was performed to examine BOLD activity

during the 4-second cue presentation + feedback period using the following

predictors: learning type (same and different) and cue difficulty (deterministic,

easy, and hard). In addition, missed trials, six motion parameters, and the mean

intensity signal were included as regressors of no interest. A 2 (learning type:

same vs. different) x 3 (cue difficulty: deterministic vs. easy vs. hard cues) x 2

(group: feedback vs. observation) repeated-measures ANOVA was performed

within BrainVoyager, using learning type and cue difficulty as within-subjects

factors and group as a between-subjects factor. Based on this ANOVA a main

effect of learning type, main effect cue difficulty, main effect of group, as well as

interactions were examined within BrainVoyager. SPMs were generated and

corrected to a cluster-level false-positive rate of 5% which was calculated

individually for each main effect and interaction: 1) main effect of learning type:

p<0.005, cluster threshold = 6 or 137mm3; 2) main effect of cue difficulty:

p<0.005, cluster threshold = 5 or 129mm3; 3) main effect of group: p<0.005,

cluster threshold = 6 or 136mm3; 4) interaction of learning type x group:

p<0.00001, and an assigned cluster threshold of 10 (the cluster threshold plugin

was unable to run as the p value was so significant; therefore a cluster threshold

of 10 was chosen by the experimenter). Functional ROIs were identified and the

mean parameter estimates were extracted for further post-hoc comparisons.

83

For the test phases, BOLD activity was examined across the entire trial

(stimulus onset and participant response). Four random-effects GLMs were

performed, one per test session per group. All GLMs included learning type

(which varied based on the analysis and is described below) and cue difficulty

(deterministic, easy, and hard) as factors: 1) feedback group test 1 - learning

material (feedback and novel) and cue difficulty; 2) observation group test 1 -

learning material (observation and novel) and cue difficulty; 3) feedback group

test 2 - learning material (feedback, observation, and novel) and cue difficulty; 4)

observation group test 2 - learning material (feedback, observation, and novel)

and cue difficulty. SPMs were generated and in every test session a contrast of

studied versus non-studied (novel) cues was performed. Mean parameter

estimates were extracted from resulting functional ROIs and used for further

post-hoc analyses.

GCA: A granger causality analysis was performed in the update phase

using the midbrain as the seed region. The midbrain was chosen as the principal

seed region in order to examine potential functional and effective connectivity

between dopaminergic regions and the BG and MTL (Shohamy and Adcock,

2010). Previously published midbrain regions (Adcock et al., 2006),

encompassing the bilateral ventral tegmental area were used as the seed

regions (x, y, z = -4, -15, -9 and x, y, z = 5, -14, -8). Functional and effective

connectivity maps were generated at the individual subject level for both runs of

the update phase. Time course values over the entire run, from the first to last

time point (290 volumes per run) were entered into the analysis. At the multi-

84

subject level, each participant’s map was combined to make one group map

(which contained both feedback and observation trial information, as the trial

types were intermixed in this phase). For the feedback group, a technical error

prevented two subjects’ data from being included; therefore, the feedback group

consisted of 19 subjects in the GCA analysis. For this group, functional

connectivity maps were given a threshold of p<0.0001 with a cluster threshold

correction of 3 voxels (cluster-level false-positive rate of 5%) and observation

group maps were thresholded at p<0.0005 and a cluster threshold correction of 3

voxels (cluster-level false-positive rate of 5%). Effective connectivity maps were

thresholded at p<0.05 for both groups and corrected to a cluster-level false-

positive rate of 5% (FB group: right VTA cluster threshold = 37 or 998mm3 and

left VTA cluster threshold = 38 or 1005mm3; for the OB group: right VTA cluster

threshold = 39 or 998mm3 and left VTA cluster threshold = 37 or 965mm3).

Standard second level statistics were performed on the group maps; specifically

a t-test was calculated.

PE: Four GLMs were created for the prediction error analyses: 1)

feedback group learning phase; 2) observation group learning phase; 3)

feedback group update phase; 4) observation group update phase. The

prediction error values generated from the Q learning model presented in the

general methods chapter were used as a regressor for the feedback trials in GLM

1 (feedback learning phase). The action prediction error values generated for the

observation trials were used as a regressor for the observation trials in GLM 2

(observation learning phase). In GLM 3 and 4, feedback prediction errors and

85

action prediction errors were generated for the feedback update phase trials and

the observation update phase trials respectively. In GLMs 3 and 4, one

regressor contained the feedback prediction error signal values (feedback trials)

and a separate regressor in the same model contained the action prediction error

signal values (observation trials). GLM 3 contained both of these regressors for

group1/feedback group, while GLM 4 contained both of these regressors for

group2/observation group. In addition, in all four of the GLMs, eight additional

predictors were included: trial event and missed responses, as well as six motion

parameters. Within GLMs 1, 3, and 4, for the feedback trials, the feedback

prediction error values were coded during the two second feedback presentation

period (indicating a correct, incorrect, or missed trial). In GLMs, 2, 3, and 4, for

the observation trials, the action prediction error values were coded during the

two second cue presentation period (where the cue and informative arrow,

indicating the cue’s value, were presented simultaneously). Four SPMs which

probed regions of the brain that varied parametrically with either feedback

prediction error signals or action prediction error signals (depending on the GLM)

were generated, set to a threshold of p<0.005 with an appropriate voxel

contiguity that corrected to a cluster-level false-positive rate of 5% (FB group:

learning phase cluster threshold = 6 or 138 mm3; update phase cluster threshold

= 5 or 136 mm3 for both the feedback and observation trial models; OB group:

learning phase cluster threshold = 5 or 135 mm3; update phase cluster threshold

= 5 or 131 mm3 for both the feedback and observation trial models).

86

4.3 Results

4.3.1 Behavioral results

Learning phase: Accuracy

The analysis of interest in the learning phase was to investigate how

participants’ accuracy varied according to cue difficulty (deterministic vs. easy vs.

hard) across groups (group1/feedback vs. group2/observation). In order to

address this question a 3 (cue difficulty: deterministic vs. easy vs. hard) x 2

(group: feedback vs. observation) repeated measures ANOVA was conducted

using cue difficulty as a within-subjects measure and group as a between-

subjects measure. A main effect of cue difficulty (F(1.730,67.49) = 77.13; p <

0.0001), a main effect of group (F(1,39) = 18.29; p < 0.0001), and a marginally

significant interaction of cue difficulty and group (F(1.730,67.49) = 77.13; p = 0.085)

were observed. Post-hoc t-tests were conducted to probe the main effect of cue

difficulty and main effect of group results. As expected, behavioral accuracy

scaled according to difficulty: deterministic > easy > hard (paired-samples t-tests,

collapsed across groups: deterministic > easy t(40) = 4.26; p < 0.001; easy > hard

t(40) = 7.59; p < 0.001). An independent samples, post-hoc t-test was performed

to examine the main effect of group and indicated that participants in the

observation group (mean accuracy = 85%) performed significantly better than

participants in the feedback group (mean accuracy = 75%; t(27.12) = 4.36; p <

0.001). In order to examine the marginally significant interaction of cue difficulty

by group, independent samples t-tests were performed and indicated that

participants in the observation group performed significantly better than

87

participants in the feedback group for both deterministic and easy cues (t(28.61) =

4.89; p < 0.001 and t(26.87) = 4.20; p < 0.001, respectively), with no performance

difference exhibited for the hard cues (t(39) = 0.94; p > 0.05; Figure 4.2A and B).

This result suggests that the two learning types are perhaps best equated in the

hardest cue condition (62.5% probability).

Test 1: Accuracy

In order to confirm that participants acquired all cue contingencies

successfully in the learning phase, a test phase was administered which

contained the probabilistic cues that were acquired in the learning phase, now

presented without any cue value information (e.g., no feedback in the trials for

group1/feedback group; no observational cue information in the trials for

group2/observation group). Additionally, novel information was presented in the

test phase as a control in order to compare accuracy for previously learned with

novel information presented for the first time in the test session. Within each

group, a paired-samples t-test was performed to compare accuracy for previously

learned information with novel information (collapsed across levels of cue

difficulty). In the feedback group a t-test of feedback versus novel information

was performed and for the observation group, a t-test of observation versus

novel information was conducted. Participants in both groups performed

significantly better on previously learned information compared to novel material

(FB group: feedback > novel t(19) = 3.94; p < 0.001; OB group: observation >

novel t(19) = 4.95; p < 0.001).

88

In addition to examining differences within each group comparing

previously studied with novel information, potential differences in the test phase

among cue difficulty were also investigated. This question was examined across

groups to determine if participants in one group were more accurate than the

other group in the test phase. A 3 (cue difficulty: deterministic vs. easy vs. hard)

x 2 (group: feedback vs. observation) repeated measures ANOVA was

conducted using cue difficulty as a within-subjects measure and group as a

between-subjects measure. A main effect of cue difficulty (F(1.609,61.15) = 14.60; p

< 0.001), no main effect of group (F(1,38) = 1.65; p > 0.05), and no significant

interaction between cue difficulty and group (F(1.609,61.15) = 1.40; p > 0.05) were

observed. Post-hoc paired-samples t-tests were performed to explore the main

effect of cue difficulty result, collapsed across group. The results indicated that

accuracy nearly scaled according to difficulty with marginally better performance

for deterministic compared with easy cues (t(39) = 1.80; p = 0.08), and greater

accuracy for both deterministic and easy cues compared with hard cues (t(39) =

5.16; p < 0.001 and t(39) = 3.20; p < 0.005 respectively). To summarize, in the

first test phase, participants’ accuracy nearly scaled according to cue difficulty,

with no accuracy differences observed across the feedback and observation

groups, indicating that both groups acquired initial cue contingencies well.

Update phase: Accuracy

Two analyses were performed examining participants’ behavioral

accuracy during the update (reversal) phase. The first probed potential

differences across different types of learning (feedback vs. observation trials) as

89

well as the various levels of cue difficulty (deterministic vs. easy vs. hard) across

groups (group1/feedback vs. group2/observation). Therefore, a 2 (learning type:

feedback vs. observation) x 3 (cue difficulty: deterministic vs. easy vs. hard) x 2

(group: feedback vs. observation) repeated-measures ANOVA, using learning

type and cue difficulty as within-subjects factors and group as a between-

subjects factor was conducted. A main effect of learning type (F(1,39) = 5.19; p <

0.05), a main effect of cue difficulty (F(2,78) = 31.81; p < 0.001), no main effect of

group (F(1,39) = 0.226; p > 0.05), and no significant interactions (all p values >

0.05) were observed. A post-hoc paired-samples t-test was performed to

examine the main effect of learning type (collapsed across group), with results

indicating greater accuracy for observation compared with feedback cues (t(40) =

2.31; p < 0.05). The main effect of cue difficulty was caused by significantly

greater accuracy for deterministic and easy compared with hard cues (paired-

samples t-tests: t(40) = 6.93; p < 0.001 and t(40) = 5.73; p < 0.001 respectively),

with no significant difference observed between deterministic and easy cue

performance (t(40) = 1.35; p > 0.05). To summarize, participants were more

accurate for observation than feedback cues in the update phase, as occurred in

the learning phase, with accuracy modulated by difficulty such that participants

performed better on the “easier cues” (deterministic and easy cues) compared

with the hard cues.

Because there was a reversal in cue contingencies in the beginning of the

update phase, a second analysis examined changes in accuracy at multiple time

points throughout this phase. It was expected that participants’ performance

90

would reflect this reversal by exhibiting initial poor accuracy, followed by

increases in performance as participants updated previously learned

contingencies over time. In order to do this, trials were binned into 12-trial

increments and participants’ accuracy for those 12 trials was averaged. Mean

accuracy was examined at trials: 1-12 (time point 1), 13-24 (time point 2), 25-36

(time point 3), 37-48 (time point 4) for block 1 and the same time points: 1-12, 13-

24, 25-36, 37-48 for block 2 (which can also be thought of as trials 49-60, 61-72,

73-84, and 85-96 of the task).

To examine behavioral accuracy changes across the aforementioned time

points, a 2 (learning type: feedback vs. observation trials) x 4 [time points: 1

(trials 1-12) vs. 2 (13-24) vs. 3 (25-36) vs. 4 (37-48)] x 2 (group: group1/feedback

vs. group2/observation) x 2 (block: early vs. late) repeated-measures ANOVA

was performed using learning type, time point, and block as within-subjects

factors and group as a between-subjects factor. Results revealed a significant

main effect of learning type (F(1,39) = 8.74; p < 0.01), a main effect of time point

(F(3,117) = 27.70; p < 0.001), a main effect of block (F(1,39) = 33.70; p < 0.001), and

no main effect of group (F(1,39) = 0.35; p > 0.05; Figure 4.2C and D). Significant

interactions included an interaction of learning type by time point (F(3,117) = 7.66; p

< 0.001), learning type by block (F(1,39) = 13.21; p < 0.005), time point by block

(F(3,117) = 8.91; p < 0.001), and a three way interaction of learning type by time

point by block (F(3,117) = 6.49; p < 0.001), with no other significant interactions

observed (all p values > 0.05). Post-hoc t-tests were completed to examine the

significant main effects and interactions. The main effect of learning type

91

indicated greater accuracy for observation compared with feedback trials (t(40) =

2.31; p < 0.05) as previously reported in the first ANOVA performed in the update

phase. The main effect of time point revealed that as expected, participants’

accuracy increased across trial time points, time points 4 (81%) > 3 (76%) > 2

(72%) > 1 (67%) [time point 4 > time point 3 (t(40) = 2.59; p < 0.05); time point 3 >

time point 2 (t(40) = 2.67; p < 0.05); time point 2 > time point 1 (t(40) = 2.67; p <

0.05)]. As expected, the main effect of block was driven by greater accuracy in

the second (80%) compared with the first block (68%; t(40) = 5.96; p < 0.001).

The interaction of learning type by time point was driven by an increase in

accuracy across time points for the feedback trials [time points 4 (81%) > 3 (73%)

(t(40) = 3.04; p < 0.005); time point 3 (73%) marginally > 2 (68%) (t(40) = 1.80; p =

0.08); time point 2 (68%) > 1 (59%) (t(40) = 3.28; p < 0.005)], but not for the

observation trials (all p values > 0.05). The significant interaction of learning type

by block was caused by significantly greater accuracy for observation (74%) than

feedback (62%) cues in the first block (t(40) = 3.37; p < 0.005) with no difference

between learning types observed in the second block (FB = 79%; OB = 81%; t(40)

= 0.57; p > 0.05). This was most likely due to the fact that participants

significantly increased their accuracy for the feedback trials throughout the

course of the first block in the update phase (as seen in the learning type x time

point x block interaction). The significant interaction of time point by block was

driven by increases across time points in block 1 [time point 4 (79%) marginally

greater than 3 (73%) (t(40) = 1.78; p = 0.08); time point 3 (73%) > 2 (66%) (t(40) =

2.80; p < 0.01); time point 2 (66%) > 1 (55%) (t(40) = 3.26; p < 0.005)], but no

92

significant increases in accuracy across time points in block 2 [only marginally

greater accuracy for time point 4 (84%) > (79%) (t(40) = 1.85; p = 0.07), all other p

values > 0.05], suggesting that participants’ accuracy was stabilized in the

second block of the update phase, demonstrating that participants updated cue

contingencies successfully by the end of the first block. Lastly, the three way

interaction of learning type by time point by block was driven by a significant

increase in only feedback trial accuracy in the first block only [time point 4 (79%)

> 3 (69%) (t(40) = 2.38; p < 0.05); time point 3 (69%) > 2 (59%) (t(40) = 2.63; p <

0.05); time point 2 (59%) > 1 (38%) (t(40) = 4.43; p < 0.001)], with no significant

changes in the second block for feedback trials (all p values > 0.05), and no

significant changes in either block for the observation trials (all p values > 0.05;

Figure 4.2 C and D). To summarize, these results suggest that participants in

both groups were initially worse at specifically feedback cues following

probabilistic cue value reversal, but rapidly updated their responses such that by

the end of the first update block, there was no significant difference between

feedback and observation cue performance. Interestingly, neither a significant

main effect of group nor any significant interactions with group were observed,

suggesting that both groups updated cue contingencies in a similar fashion.

Test 2: Accuracy

The second test phase was administered to determine how well

participants learned the new cue contingencies, after the reversal in the update

phase. Critically, as in all test sessions, no informative information about the cue

was provided (no feedback for the feedback trials; no observational information

93

for the observation trials). In this test, feedback, observation, and novel trials

were administered for both group 1 and group 2. The first question of interest

examined if participants in both groups performed better on information learned

in the update phase compared with novel information and compared accuracy

across the two groups to determine if one group performed better than the other

in this test session. To address this question, a 3 (learning material: feedback

vs. observation vs. novel) x 2 (group: feedback vs. observation) repeated-

measures ANOVA was conducted using learning material as a within-subjects

factor and group as a between-subjects factor. A main effect of learning material

(F(1.67,65.10) = 26.28; p < 0.001), no main effect of group (F(1,39) = 0.003; p > 0.05),

and no significant interaction between learning material and group (F(1.67,65.10) =

0.42; p > 0.05) were observed. Post-hoc paired-samples t-tests were performed

to examine the main effect of learning material, collapsed across group. As

expected, results indicated no accuracy differences between previously learned

material in the update phase (feedback vs. observation trials; t(40) = 1.36; p >

0.05), and greater accuracy for both feedback and observation cues compared to

novel cues (t(40) = 6.28; p < 0.001 and t(40) = 5.13; p < 0.001, respectively).

The second analysis probed for potential differences between levels of

cue difficulty and learning types, across groups, excluding novel information. In

order to do so, a 2 (learning type: feedback vs. observation) x 3 (cue difficulty:

deterministic vs. easy vs. hard) x 2 (group: group1/feedback vs.

group2/observation) repeated-measures ANOVA was conducted using learning


94

subjects factor. No significant results were observed (no main effect of learning

type (F(1,39) = 1.80; p > 0.05); no main effect of cue difficulty (F(1.55,60.57) = 1.31; p

> 0.05); no main effect of group (F(1,39) = 0.007; p > 0.05); and no significant

interactions - all p values greater than 0.05). These results suggest that

irrespective of group or learning type, participants updated cue contingencies

well.

Follow-up Test: Accuracy

A final test was conducted behaviorally in the laboratory to examine

changes in accuracy approximately 2 days after participation in the fMRI portion

of the experiment. The same two analyses were performed to examine potential

performance differences in this phase as were performed in the second test

phase administered inside the MRI. First, to examine potential differences within

learning material across groups, a 3 (learning material: feedback vs. observation

vs. novel) x 2 (group: group1/feedback vs. group2/observation) repeated-

measures ANOVA was conducted using learning material as a within-subjects

factor and group as a between-subjects factor. A main effect of learning material

(F(1.67,64.97) = 15.33; p < 0.001), no main effect of group (F(1,39) = 0.219; p > 0.05),

and no significant interaction between learning material and group (F(1.67,64.97) =

1.16; p > 0.05) were observed. Post-hoc paired-samples t-tests probing the main

effect of learning material (collapsed across group) indicated no difference

between previously learned material (feedback vs. observation cues; t(40) = 1.00;

p > 0.05) and significantly greater accuracy for both feedback and observation

cues compared to novel cues (t(40) = 4.96; p < 0.001 and t(40) = 6.51; p < 0.001,

95

respectively) as was expected based on the results from the previous test

phases.

Second, to probe for potential cue difficulty differences across groups in

this follow-up test phase, a 2 (learning type: feedback vs. observation) x 3 (cue

difficulty: deterministic vs. easy vs. hard) x 2 (group: group1/feedback vs.

group2/observation) repeated-measures ANOVA was conducted using learning


subjects factor. As anticipated, based on the results from the second test

administered on day 1, there were no significant findings. Specifically, no main

effect of learning type (F(1,39) = 1.34; p > 0.05); no main effect of cue difficulty

(F(2,78) = 0.376; p > 0.05); no main effect of group (F(1,39) = 1.05; p > 0.05); and no

significant interactions (all p values greater than 0.05) were observed. These

results therefore confirm the results of the second test administered on day 1,

suggesting that participants updated cue contingencies well, irrespective of how

they initially acquired the cue associations (the group they were assigned to:

group1/feedback vs. group2/observation), the trial learning type, or the difficulty

of the probabilistic cues.


Learning phase

Main Effect of Cue Difficulty: One unresolved question from the results of

experiment 1 was to determine if the MTL and BG are engaged in pure feedback

learning as well as pure observation learning (nonbiased by both learning types

presented in the same learning session). In order to examine the engagement of

96

these areas during distinct types of learning, the feedback groups’ and

observation groups’ neuroimaging data were analyzed separately. Therefore,

two independent within-subjects, repeated-measures ANOVAs investigating a

main effect of cue difficulty were performed within BrainVoyager: one for the

feedback learning participants (group 1) and a separate ANOVA for the

observation learning participants (group 2). Results indicated that a network of

similar regions was engaged in both the feedback learning group (Table 4.1) and

observation learning group (Table 4.2), including loci within the striatum, medial

temporal lobe, insula, cingulate cortex, and prefrontal areas.

Feedback learning group: A region of the right caudate nucleus (x, y, z =

5, 1, 12; Figure 4.3A) and left medial temporal lobe, encompassing the amygdala

and hippocampus (x, y, z = -22, -8, -15; Figure 4.3C) were engaged during

feedback learning. Mean parameter estimates were extracted from these

regions of interest for further analysis. Activity within the caudate nucleus was

greatest in response to hard cues and decreased according to cue difficulty (hard

> easy > deterministic) (hard cues marginally greater than easy cues, t(20) = 2.90;

p = 0.07; easy > deterministic t(20) = 4.30; p < 0.0005; Figure 4.3B), while the left

MTL showed the opposite pattern (deterministic > easy > hard) (deterministic

marginally greater than easy cues, t(20) = 1.96; p = 0.06; easy > hard t(20) = 3.10;

p < 0.01; Figure 4.3D).

Observation learning group: Within the observation group, a region of the

right caudate nucleus (x, y, z = 5, -2, 15), the left caudate nucleus extending into

the left putamen (x, y, z = -13, 1, 9) (Figure 4.4 A), and left hippocampus (x, y, z

97

= -25, -20, -12; Figure 4.4C) were engaged during learning. Mean parameter

estimates were extracted from these regions for further examination. A similar

pattern of activity appeared within the right caudate nucleus, left caudate nucleus

extending into the putamen, and the hippocampus as was observed in the

feedback learning group. Specifically in the right caudate nucleus, hard cues

elicited stronger activation compared with easy and deterministic cues (t(19) =

4.77; p < 0.0005 and t(19) = 4.26; p < 0.0005, respectively; Figure 4.4B), with a

similar pattern observed in the left caudate/putamen region (hard > easy and

deterministic; t(19) = 4.11; p < 0.001 and t(19) = 4.55; p < 0.0005, respectively; not

depicted in the figure). The left hippocampus showed the opposite pattern with

deterministic and easy cues eliciting stronger activation than hard cues (t(19) =

3.62; p < 0.005 and t(19) = 4.28; p < 0.0005, respectively; Figure 4.4D).

These results therefore are a partial replication of the results observed in

experiment 1. Loci within both the caudate nucleus and hippocampus were

involved in feedback learning, as well as observation learning and were

modulated by cue difficulty. The hippocampus exhibited the same pattern of

results as reported in experiment 1, however the caudate nucleus, was engaged

differentially in the learning task employed in experiment 1 versus the task

employed in experiment 2; an interpretation of this differentiation is provided in

the discussion section.

Learning phase: Correlations between striatum and medial temporal lobe

As a rudimentary manner of examining interactions between the striatum

and medial temporal lobe during learning, a series of Pearson’s correlations were

98

performed on mean parameter estimates extracted from the striatal and medial

temporal lobe regions in the feedback and observation learning groups.

Specifically, the areas functionally defined by the main effect of cue difficulty

analyses were used in the analyses.

Feedback learning group: In the feedback group, no significant positive or

negative correlations were observed between the mean BOLD signal extracted

from the right caudate nucleus (x, y, z = 5, 1, 12) and left medial temporal lobe (x,

y, z = -22, -8, -15).

Observation learning group: In the observation group, a significant

positive correlation was observed between the right caudate nucleus (x, y, z = 5,

-2, 15) and the left MTL (x, y, z = -25, -20, -12) for observation deterministic cues

(r = 0.467, p = 0.04) with a nearly significant positive correlation observed

between these same regions for easy cues (r = 0.434, p = 0.056). No other

significant positive or negative correlations were observed.

Update phase

The key question of interest in the update phases was to examine what

brain regions were engaged during the updating of probabilistic learning, when

learning types remained the same (feedback-feedback and observation-

observation) versus when learning types evolved (feedback-observation and

observation-feedback). It was hypothesized that this novel manner of examining

how multiple memory systems interact during a dynamic learning environment

may provide new insight to the debate regarding the nature of interactions among

memory regions. In order to address this question of interest, one analysis was

99

performed which included all forty-one participants’ data – combining groups 1

(feedback) and 2 (observation). A 2 (learning type: same vs. different) x 3 (cue

difficulty: deterministic vs. easy vs. hard) x 2 (group: group1/feedback vs.

group2/observation) repeated-measures ANOVA was performed within

BrainVoyager using learning type and cue difficulty as within-subjects measures

and group as a between-subjects measure. The results presented below

highlight the most relevant findings.

Main Effect of Cue Difficulty: A similar network of regions engaged in

processing differences in cue difficulty was observed during the update phase as

was engaged during the learning phases, including loci within the striatum, MTL,

insula, cingulate cortex, as well as several frontal and parietal regions (Table

4.5). Mean parameter estimates were extracted from key areas of interest,

specifically two regions bordering the right putamen/insula (x, y, z = 26, 13, 3 and

x, y, z = 29, -14, 12; Figure 4.5A and C) and bilateral parahippocampus (x, y, z =

20, -20, -18, not shown in the figure, and x, y, z = -22, -14, -21; Figure 4.5E).

Interestingly, the mean BOLD signal in the more anterior region of the lateral

putamen/insula (x, y, z = 26, 13, 3) scaled according to difficulty with the greatest

response elicited for hard cues (irrespective of learning type): hard > easy >

deterministic (hard > easy t(40) = 3.12; p < 0.005; easy > deterministic t(40) = 2.13;

p < 0.05; Figure 4.5B). However, the more lateral and posterior region of the

putamen/insula identified (x, y, z = 29, -14, 12) showed the opposing pattern of

activity, with the greatest response elicited to deterministic cues (irrespective of

learning type): deterministic > easy > hard (deterministic > easy t(40) = 2.28; p <

100

0.05; easy > hard t(40) = 2.39; p < 0.05; Figure 4.5D). Furthermore, within the

parahippocampus , the BOLD signal also scaled according to difficulty with both

regions showing the greatest response to deterministic cues: deterministic > easy

> hard (irrespective of learning type; right parahippocampus: deterministic >

easy: t(40) = 2.30; p < 0.05; easy > hard: t(40) = 2.72; p < 0.01; left

parahippocampus: deterministic marginally > easy: t(40) = 1.85; p = 0.07; easy >

hard: t(40) = 2.67; p < 0.05; Figure 4.5F). This result of both similar and opposing

patterns of BOLD responses within regions of the putamen and

parahippocampus when updating cue contingencies following a reversal in cue

values supports parallel engagement of these regions during the updating

process.

Main Effect of Learning Type: Brain regions which processed differences

in learning type, that being variations in BOLD responses to the same versus a

different learning type as was employed in the learning phase, were also

investigated. A group of areas involved in processing differences in learning type

included a locus within the cingulate cortex (x, y, z = -10, -5, 36), a locus within

the insula (x, y, z = 41, 7, 6), as well as several parietal and temporal regions

(Table 4.6). Interestingly, no areas within the medial temporal lobes or basal

ganglia exhibited a main effect of learning type, indicating that these regions did

not differentiate between whether the learning type remained the same or

changed compared with initial cue acquisition.

Main Effect of Group: Areas processing differential activity for participants

in the feedback group versus those in the observation group were also

101

examined. The main effect of group analysis revealed activation within two areas

which survived correction, the precentral gyrus and the thalamus (Table 4.7).

Interestingly, no engagement of the basal ganglia, MTL, or midbrain was

observed in processing a main effect of group. While null findings in

neuroimaging studies are difficulty to interpret, and must be done with caution,

one possible interpretation is that this null result may be indicative of a parallel

mode of operations between multiple memory regions, as no key region of

interest was engaged in processing greater activation to the feedback versus the

observation group, or vice versa.

Interaction of Learning Type and Group: Several regions exhibited an

interaction of learning type and group, including loci within the striatum and

midbrain, as well as the insula, and cingulate cortex. Specifically the bilateral

caudate nucleus (right: x, y, z = 8, 3, 11; and left x, y, z = -8, 3, 8) as well as the

bilateral midbrain, centered on the VTA (right: x, y, z = 7, -15, -9; and left: x, y, z

= -4, -15, -9) exhibited an interaction of learning type by group. Mean parameter

estimates were extracted from these regions of interest for further examination.

In all regions examined, the interaction was driven by a larger response to same

than different learning trials for the feedback group (e.g., feedback trials), and a

larger response to different than same learning trials for the observation group

(e.g., feedback trials) [right caudate: FB group: same > different t(20) = 6.85; p <

0.0001; OB group: different > same: t(19) = 8.34; p < 0.0001; left caudate: FB

group: same > different t(20) = 8.77; p < 0.0001; OB group: different > same: t(19) =

5.14; p < 0.0001; right midbrain: FB group: same > different t(20) = 8.28; p <

102

0.0001; OB group: different > same: t(19) = 4.95; p < 0.0001; left midbrain: FB

group: same > different t(20) = 7.90; p < 0.0001; OB group: different > same: t(19) =

5.92; p < 0.0001]. Therefore, within both the feedback and observation groups, a

greater response to feedback trials was observed in loci within the caudate

nucleus and midbrain. This result is consistent with the literature suggesting a

role for the striatum and midbrain in processing feedback information (Packard

and Knowlton, 2002; Wise, 2004; Delgado, 2007).

Update phase: Correlations between striatum and medial temporal lobe

Correlations were performed on mean parameter estimates extracted from

striatal and medial temporal lobe regions in the update phase, functionally

defined by the main effect of cue difficulty analysis. Only correlations across

regions, which were matched according to cue difficulty (deterministic-

deterministic, easy-easy, hard-hard) were examined. The only significant across-

region correlation observed was between the right parahippocampus (x, y, z =

20, -20, -18) and right putamen/insula (x, y, z = 29, -14, 12) for deterministic cues

in the same learning condition (r = 0.308, p = 0.050).

Granger Causality Analysis

Granger causality analyses were conducted for the feedback and

observation groups separately during the update phase. The purpose of this

analysis was to examine functional and effective connectivity during the dynamic

portion of the task when participants were updating previous cue contingencies

via both feedback and observation trials. The midbrain (centered on the VTA)

was chosen as the seen region given its anatomical projections to both the

103

striatum and the hippocampus and its critical role in reward-related learning and

memory processes (Figure 4.6A).

1) Functional connectivity:

Feedback group: Within the feedback group, using the right VTA as the

seed region resulted in functional connectivity with the bilateral caudate nucleus

extending into the putamen (x, y, z = 6, 4, 6 and x, y, z = -17, 1, 5) as well as the

right hippocampus (x, y, z = 24, -20, -6). Functional connectivity maps using the

left VTA seed revealed very similar instantaneous correlations with the bilateral

caudate nucleus extending into the putamen/globus pallidus (x, y, z = 8, 1, 12

and x, y, z = -12, -2, 4) and the bilateral hippocampus (x, y, z = 24, -22, -8 and x,

y, z = -32, -15, -13).

Observation group: Functional connectivity in the observation group using

the right and left VTA seeds occurred with similar regions of the basal ganglia,

namely loci within the bilateral globus pallidus. Two loci of the globus pallidus,

(x, y, z = 8, 1, -1 and x, y, z = -15, 1, -2) correlated with the right VTA, while two

similar areas of the globus pallidus (x, y, z = -8, 1, -3 and x, y, z = 18, -2, -6)

correlated with the left VTA. In addition, the right VTA exhibited functional

connectivity with the bilateral parahippocampus (x, y, z = 16, -23, -12 and x, y, z

= -19, -28, -9) while the left VTA exhibited connectivity with the bilateral posterior

entorhinal cortex (x, y, z = 17, -23, -9 and x, y, z = -20, -25, -9).

To summarize the functional connectivity results, instantaneous

correlations were observed between the bilateral VTA, the caudate nucleus

extending into the globus pallidus, and the hippocampus in the feedback group

104

while somewhat overlapping regions in the globus pallidus and medial temporal

lobes were functionally connected with the bilateral VTA in the observation

group. Therefore, similar functional connectivity in known reward-related and

memory regions (midbrain, basal ganglia, and medial temporal lobes) was

observed across groups, suggesting interactions between these areas during the

updating of previously learned cue contingencies.

2) Effective connectivity:

Feedback group: Maps using the left VTA as a seed region revealed

directed connectivity originating from the bilateral putamen (x, y, z = 15, 2, 5 and

x, y, z = -19, 0, 5) to the left VTA (Figure 4.6B). Using the right VTA seed

revealed directed influences from the right VTA to the right hippocampus (x, y, z

= 26, -20, -7; Figure 4.6C) and right parahippocampus (x, y, z = 16, -31, -12; not

shown).

Observation group: Effective connectivity using the left VTA as a

reference region indicated no regions of interest surviving cluster threshold

correction. Using the right VTA as a reference region revealed directed

influences from the right VTA going to the left hippocampus (x, y, z = -30, -21, -

9), the left parahippocampal gyrus (x, y, z = -25, -29, -9) (Figure 4.6D), as well as

the posterior cingulate and thalamus.

In sum, a unilateral flow of information was observed in which directed

influences from the putamen were sent to the left VTA, and directed influences

from the right VTA were sent to the hippocampus and parahippocampus in the

feedback group. The observation group exhibited a segment of this arc, that

105

from the right VTA to the left hippocampus and parahippocampus [a region of the

left putamen (x, y, z = -24, -3, 6) did send directed influences to the left VTA, but

did not survive cluster threshold correction].

Prediction Error Analysis

Learning phase:

Feedback learning group: A traditional Q learning model was used to

estimate prediction error values for the feedback trials in group 1 (feedback

group). Regions that positively correlated with this prediction error signal

included three loci within the basal ganglia, as well as an area within the anterior

cingulate (x, y, z = -1, 34, -6) and the inferior frontal gyrus (x, y, z = -49, 25, 15).

Within the basal ganglia, the ventral striatum (nucleus accumbens: x, y, z = 5, 4,

-3), the internal segment of the globus pallidus (x, y, z = -13, 1, -3) (Figure 4.7A),

and the head of the caudate nucleus (x, y, z = 14, 22, 9; not shown) all positively

correlated with the prediction error signal.

Observation learning group: An action prediction error, which is a non-

signed measure of participants’ surprise at viewing a given cue-value association

(saliency signal), was calculated for the observation learning trials for group 2

(observation group). Areas of interest which positively correlated with this action

prediction error signal included the medial frontal gyrus (x, y, z = 5, -2, 48), the

midbrain (x, y, z = 2, -26, -6), and the bilateral insula (x, y, z = 29, 16, 0 and x, y,

z = -28, 16, 6) (not depicted in the figure).

To summarize the learning phase prediction error results, areas of the

basal ganglia were involved in encoding feedback prediction errors, including the

106

ventral striatum, globus pallidus, and caudate nucleus, consistent with previously

reported results in the literature (Ashby et al., 2002; O'Doherty et al., 2004;

Tanaka et al., 2004; Pessiglione et al., 2006). For the observation trials, the

action prediction error signal engaged distinct regions including the medial frontal

gyrus, the midbrain, and the insula, which did not replicate previous findings in

the literature (Burke et al., 2010).

Update phase: Both feedback prediction errors (for the feedback trials) and

action prediction errors (for the observation trials) were examined in the update

phase, for both group 1 (feedback) and group 2 (observation).

Feedback group: Areas of interest which positively correlated with the

prediction error signal for the feedback trials included the right ventral caudate

nucleus (x, y, z = 2, 10, 0) and the external component of the left globus pallidus

(x, y, z = -19, -2, -6), regions fairly similar to those engaged during the learning

phase while encoding the prediction error signal for this group (Figure 4.7B). For

the observation trials, an area of interest that positively correlated with the action

prediction error signal included the superior frontal gyrus (x, y, z = -22, 13, 48) a

region distinct from the medial frontal gyrus which encoded this signal in the

observation group leaning phase (not depicted in the figure).

Observation group: Loci within the ventral striatum (bilateral nucleus

accumbens: x, y, z = 5, 4, -3 and x, y, z = -10, 4, -3; Figure 4.7C), the medial

temporal lobe (encompassing the hippocampus/amygdala: x, y, z = -25, -5, -21;

Figure 4.7D), and the medial frontal gyrus (x, y, z = -1, 22, -15; not shown)

positively correlated with the prediction error signal for the feedback trials. For

107

the observation trials, the cingulate gyrus (x, y, z = 11, -29, 36) was the only area

that positively correlated with the action prediction error signal (not shown in the

figure).

To summarize, in the update phase, regions of the basal ganglia

correlated with feedback prediction errors in both groups, with the observation

group additionally showing correlations with the medial temporal lobe and medial

frontal gyrus. The action prediction error signal correlated with distinct regions in

the feedback (superior frontal gyrus) and observation groups (cingulate gyrus).

4.3.3 Exploratory analyses

Two additional analyses were performed for exploratory purposes in order

to examine potential effects of cue accuracy and cue consistency on the BOLD

responses in the BG and MTL, as well as to probe for potential differences in

these regions for participants who learned to optimize their responses versus

those who did not. The first analysis investigated potential differences in the BG

and MTL when accounting for cue accuracy (correct versus incorrect) in the

feedback trials and cue consistency (consistent versus inconsistent) in the

observation trials. This analysis was performed as the BG and MTL are known

to be modulated by both accuracy and consistency during learning (Delgado et

al., 2000; Delgado, 2007; Chen et al., 2011; Li et al., 2011). For each region

described below, BOLD responses were extracted from the previously defined

functional regions of interest processing a main effect of cue difficulty (FB group:

right caudate nucleus and left MTL). A within-subjects ANOVA was performed

examining potential differences in cue difficulty and cue accuracy for the

108

feedback trials in each ROI. To summarize the results briefly, no significant main

effects of cue accuracy or significant interactions were observed (right caudate

nucleus: main effect of cue difficulty F(2,38) = 11.69; p < 0.001; marginally

significant main effect of cue accuracy F(1,19) = 3.19; p = 0.09; no significant

interaction F(2,38) = 0.29; p > 0.05; left MTL: main effect of cue difficulty F(2,38) =

9.10; p < 0.005; no main effect of cue accuracy F(1,19) = 0.55; p > 0.05; no

significant interaction F(2,38) = 0.43; p > 0.05). However, for exploratory purposes,

Student’s t-tests were performed in order to examine potential differences across

levels of difficulty according to accuracy (corrected for multiple comparisons) as it

was desired to examine if the responses in the BG and MTL were driven by

responses to correct rather than incorrect trials (e.g., easy correct > hard correct,

but no difference for incorrect trials). Note one person was excluded from this

analysis as he/she experienced no incorrect trials in the deterministic condition.

In the learning phase, for the feedback group, the right caudate nucleus

and the left MTL exhibited differences in cue difficulty for both correct and

incorrect trials. In the right caudate nucleus, BOLD response for hard correct

trials was greater than deterministic correct trials (t(19) = 2.89; p < 0.01) while a

marginally greater response for hard than easy correct trials (t(19) = 1.88; p =

0.08) was observed. The same pattern was observed for incorrect trials (hard >

deterministic t(19) = 3.06; p < 0.01 and marginally greater for hard than easy t(19) =

2.26; p = 0.04; trend after sequential Bonferroni correction), suggesting that the

pattern of activity in the caudate was not driven by one type of feedback alone.

The left MTL however, exhibited greater responses for deterministic than hard

109

correct trials (t(19) = 3.17; p < 0.005) and marginally greater responses for easy

than hard correct trials (t(19) = 1.63; p = 0.12). Marginally greater BOLD signals

were observed for deterministic than hard incorrect trials (t(19) = 2.26; p = 0.04;

trend after sequential Bonferroni correction) and significantly greater for easy

than hard incorrect trials (t(19) = 2.65; p < 0.05). This pattern of results suggests

that both the caudate nucleus and MTL were not primarily modulated by correct

versus incorrect trials. Furthermore, accounting for accuracy did not change the

pattern of results observed in these areas; that is, in the striatum the greatest

BOLD response was elicited in response to hard cues, while in the MTL the

largest BOLD response occurred for deterministic cues.

A similar analysis was performed in the update phase, including both the

feedback and observation group as these groups were combined in this phase.

Note five people were excluded from this analysis as they experienced no

incorrect trials for at least one condition. Four repeated-measures ANOVAs were

performed using cue difficulty and feedback cue accuracy as within-subjects

measures and group as a between-subjects measure (feedback group and

observation group; one ANOVA per ROI). To summarize the results briefly, no

significant main effect of cue accuracy was observed (anterior putamen/insula:

F(1,34) = 0.58; p > 0.05; posterior putamen/insula F(1,34) = 1.79; p > 0.05; right

parahippocampus: F(1,34) = 0.89; p > 0.05; left parahippocampus: F(1,34) = 2.47; p

> 0.05). The anterior putamen/insula exhibited a significant three way interaction

of cue difficulty, cue accuracy, and group (F(2,68) = 3.60; p < 0.05) and the

posterior putamen/insula exhibited a significant interaction of cue difficulty by cue

110

accuracy (F(1.66,56.40) = 3.59; p < 0.05). For exploratory purposes, Student’s t-

tests were performed in order to examine potential differences across levels of

difficulty according to accuracy (corrected for multiple comparisons).

Interestingly, differences across levels of cue difficulty in these ROIs occurred in

correct, but not incorrect trials. The anterior putamen exhibited greater

responses for hard than deterministic (t(35) = 3.39; p < 0.0005) and marginally

greater responses for hard than easy correct trials (t(35) = 1.89; p = 0.07) and

easy than deterministic correct trials (t(35) = 2.02; p = 0.05). The more posterior

region of the putamen exhibited the opposing pattern of activity, with greater

responses for deterministic compared with easy (t(35) = 3.04; p < 0.005) and hard

(t(35) = 4.29; p < 0.001) correct trials. In the right parahippocampus marginally

greater responses were observed for deterministic compared with hard correct

trials (t(35) = 1.82; p = 0.07). In the left parahippocampus, greater BOLD

responses were elicited for both deterministic and easy compared hard correct

trials (t(35) = 3.41; p < 0.005 and t(35) = 2.89; p < 0.01 respectively). No

differences were observed across levels of cue difficulty for the incorrect trials in

any ROI (all p values > 0.05). Therefore, in the update phase, the putamen and

parahippocampus exhibited differences in BOLD responses across levels of

difficulty in correct but not incorrect trials. However, accounting for cue accuracy

did not change the pattern of results observed in these regions.

Lastly, the pattern of the putamen and parahippocampal ROIs in the

update phase observation trials was examined, when accounting for cue

consistency. Note one person was excluded from this analysis as he/she

111

experienced no easy inconsistent trials (due to missed trials). Four repeated-

measures ANOVAs were performed using cue difficulty and cue consistency as

within-subjects measures and group as a between-subjects measure (one

ANOVA per ROI). To summarize briefly, the putamen ROIs exhibited no effects

of cue consistency (anterior putamen/insula: F(1,38) = 1.17; p > 0.05; posterior

putamen/insula F(1,38) = 0.29; p > 0.05), while the parahippocampal ROIs were

modulated by cue consistency (right parahippocampus: F(1,38) = 11.32; p < 0.005

and a marginally significant interaction of cue difficulty and cue consistency F(1,38)

= 3.96; p = 0.05; left parahippocampus: F(1,38) = 5.13; p < 0.05). The right

parahippocampus indicated a main effect of cue consistency, with greater BOLD

responses observed for inconsistent (e.g., circle is lower than 5) than consistent

trials (e.g., circle is higher than 5 – value of circle is 87.5% higher than 5; t(39) =

2.75; p < 0.01) while the left parahippocampus exhibited a marginally greater

response for inconsistent trials (t(39) = 1.80; p = 0.08). In both regions, this effect

was driven by differences in easy, but not hard, cue consistency (right

parahippocampus: easy inconsistent > consistent; t(39) = 3.27; p < 0.005; left

parahippocampus: easy inconsistent > consistent; t(39) = 2.80; p < 0.01).

The second exploratory analysis probed for potential differences between

participants who learned to optimize their responses, termed optimizers, versus

those who did not optimize their responses, termed non-optimizers, during the

learning and update phases. It was theorized that the responses in the MTL and

BG may be modulated by the ability to optimize during the task (e.g., optimizers

may exhibit greater MTL activation but reduced BG recruitment compared with

112

non-optimizers). In the learning phase, behavioral accuracy was examined in the

last 24 trials of the second block of learning (trials 72-96). Participants who were

100% correct for either all the easy cues or all the easy and hard cues presented

in the last 24 trials were termed optimizers, whereas participants who were less

than 100% correct were categorized as non-optimizers. In the feedback group, 9

participants were categorized as optimizers while the remaining 12 were

categorized as non-optimizers. In the observation group, 8 participants were

categorized as optimizers while the remaining 12 were categorized as non-

optimizers. Two repeated-measures ANOVAs using cue difficulty (deterministic,

easy, and hard) as a within-subjects factor and optimizer group (optimizer and

non-optimizer) as a between-subjects factor were performed (one for the FB and

one for the OB group) in order to examine potential differences in accuracy

throughout learning for participants who optimized their responding at the end of

the learning phase versus those who did not.

In the feedback group, results indicated a main effect of cue difficulty

(F(2,38) = 25.95; p < 0.001), a main effect of optimizer group (F(1,19) = 15.38; p <

0.005), and a significant interaction (F(2,38) = 4.06; p < 0.05). Optimizers’

accuracy (82%) in the entire learning phase was significantly better than non-

optimizers (70%; t(19) = 3.92; p < 0.005). This was driven by significantly greater

accuracy for easy cues (t(16.99) = 7.14; p < 0.001) and marginally greater accuracy

for deterministic cues (t(19) = 1.83; p = 0.08) for optimizers compared with non-

optimizers. For the observation group, results indicated a main effect of cue

113

difficulty (F(1.39,25.05) = 66.07; p < 0.001), no main effect of optimizer group (F(1,18)

= 0.38; p > 0.05), and no significant interaction (F(1.39,25.05) = 0.67; p > 0.05).

The next analyses examined whether the BOLD responses in the

functionally defined ROIs in the learning phase were modulated by optimizer

group. Specifically, the regions of the caudate nucleus and MTL which exhibited

a main effect of cue difficulty were examined. A repeated-measures ANOVA was

performed on the mean BOLD responses extracted from each ROI using cue

difficulty as within-subjects factor and optimizer group as a between-subjects

factor. In the FB group, the right caudate nucleus exhibited a main effect of cue

difficulty (F(2,38) = 14.71; p < 0.001), no main effect of optimizer group (F(1,19) =

1.86; p > 0.05), and a significant interaction of cue difficulty by optimizer group

(F(2,38) = 7.09; p < 0.005). The interaction was driven by marginally greater BOLD

responses for easy cues in the non-optimizers than the optimizers (t(16.19) = 2.37;

p = 0.03; trend after sequential Bonferroni correction). This result is consistent

with the critical role of the striatum in acquiring cue contingencies and then

decreasing activation once contingencies are acquired (Tricomi et al., 2004;

Delgado et al., 2005). The left MTL however exhibited a main effect of cue

difficulty (F(2,38) = 11.98; p < 0.001), but no main effect of optimizer group (F(1,19) =

0.97; p > 0.05), and no significant interaction (F(2,38) = 0.22; p > 0.05).

The same ANOVA was performed on the regions engaged in learning in

the observation group. The right and left caudate nucleus and left MTL exhibited

main effects of cue difficulty (right caudate nucleus: F(2,36) = 13.38; p < 0.001; left

caudate nucleus: F(2,36) = 12.88; p < 0.001; left MTL: F(2,36) = 8.83; p < 0.005), but

114

not optimizer group (right caudate nucleus: F(1,18) = 0.11; p > 0.05; left caudate

nucleus: F(1,18) = 1.35; p > 0.05; left MTL: F(1,18) = 0.49; p > 0.05), and no

significant interaction (right caudate nucleus: F(2,36) = 2.46; p > 0.05; left caudate

nucleus: F(2,36) = 0.00; p > 0.05; left MTL: F(2,36) = 3.20; p = 0.05). To summarize

therefore, only the right caudate nucleus in the feedback group was modulated

by optimizer group, no other region’s BOLD responses were modulated by

whether or not participants optimized their responding at the end of the learning

phase.

In the update phase, in order to classify optimizer versus non-optimizers

different criteria were used than the criterion used in the learning phase. In this

phase, behavioral accuracy was examined in the second block of the update

phase as a whole, rather than examining only the last 24 trials. This was done

because when examining the last 24 trials of the second block, nearly every

participant optimized their responses and since participants were relearning cue

contingencies, they had more experience with the task. Therefore, any

participant who was 100% correct for the easy cues was categorized as an easy

cue optimizer, any participant who was 100% correct for the easy and hard cues

was categorized as an easy/hard cue optimizer, and those who did not optimize

for either easy or easy and hard cues were termed non-optimizers. Using this

criterion, 15 participants were easy cue optimizers, 13 participants were

easy/hard cue optimizers, and the remaining 13 participants were non-

optimizers. A repeated-measures ANOVA was performed on the behavioral data

using learning type (same or different) and cue difficulty (deterministic, easy, and

115

hard) as within-subjects measures and learning phase group (feedback or

observation) and optimization group (easy cue optimizers, easy/hard cue

optimizers, and non-optimizers) as between-subjects factors. Results indicated

no main effect of learning type (F(1,35) = 0.08; p > 0.05), a main effect of cue

difficulty (F(2,70) = 33.73; p < 0.001), no main effect of learning phase group (F(1,35)

= 0.08; p > 0.05), and a main effect of optimization group (F(2,35) = 7.05; p <

0.005). Significant interactions included learning type by learning phase group

(F(1,35) = 6.05; p < 0.05), and a three way interaction of cue difficulty, learning

phase group, and optimization group (F(4,70) = 3.07; p < 0.05). Of interest was the

observation that accuracy overall was modulated by optimization group with the

easy/hard cue optimizers (83%) significantly more accurate than easy cue

optimizers (73%;t(21.04) = 3.09; p < 0.01) and non-optimizers (67%; t(24) = 4.10; p <

0.001).

Lastly, the ROIs functionally defined by the main effect of cue difficulty

analysis were examined to probe for potential differences between optimizers

and non-optimizers in the update phase. The same ANOVA performed on the

behavioral data was conducted on the neuroimaging data, one ANOVA per ROI.

To summarize the findings briefly, no region exhibited a main effect of optimizer

group [anterior putamen: (F(2,35) = 0.35; p > 0.05); posterior putamen: (F(2,35) =

1.96; p > 0.05); right parahippocampus: (F(2,35) = 1.16; p > 0.05); left

parahippocampus: (F(2,35) = 0.41; p > 0.05)], nor any significant interactions with

optimizer group (all p values > 0.05). Therefore, the BOLD responses during the

update phase in the putamen and parahippocampus were not significantly

116

different for participants who optimized their responses in this phase versus

those who did not.

4.4 Discussion

The purpose of this experiment was to examine in a novel manner how

the medial temporal lobe and basal ganglia interact during probabilistic learning,

by employing multiple types of learning (feedback and observation trials) and a

reversal of probabilistic cue contingencies. Behavioral results indicated that

participants in the observation group initially acquired cue contingencies better

than participants in the feedback group. In the update phase, interestingly, all

participants, irrespective of group, were initially more efficient at updating old cue

contingencies with new cue values via observation learning. Over the course of

the update phase, however, updating via feedback improved for both groups,

such that by the end of the update phase and in the test phases, no learning type

differences were observed. Neuroimaging results revealed the engagement of

the MTL and BG during both feedback (group 1) and observation (group 2)

learning, supporting the hypothesis that these regions are engaged

simultaneously and may operate in parallel during probabilistic learning. In

further support of this hypothesis, while both of these regions were engaged

during learning, they exhibited differential patterns of activity, with the basal

ganglia exhibiting greatest BOLD responses to hard cues and the medial

temporal lobe exhibiting greatest BOLD responses to deterministic cues. Results

from the update phase corroborate this finding as loci within the basal ganglia

117

and medial temporal lobe were engaged in the update phase, processing

differences in cue difficulty. In the update phase, interestingly, distinct loci within

the basal ganglia exhibited both differential (BOLD signal greatest for hard cues)

as well as similar (BOLD responses greatest for deterministic cues) patterns of

activity as those observed in the medial temporal lobe (BOLD signal greatest for

deterministic cues). The application of functional and effective connectivity

allowed for a more sophisticated examination of interactions among these

regions during the reversal of probabilistic cue contingencies. Results revealed

both functional (instantaneous correlations) and effective (directed influences)

connectivity among key reward-related and memory structures, specifically the

midbrain, loci within the basal ganglia (caudate nucleus, putamen, and globus

pallidus), and loci within the medial temporal lobe (hippocampus and

parahippocampus). Lastly, the application of reinforcement learning models

allowed for the examination of putative dopaminergic influences during both initial

acquisition as well as updating of cue contingencies. Feedback trial prediction

error signals were encoded within regions of the basal ganglia (ventral striatum,

caudate nucleus, and globus pallidus) during both learning and updating of

probabilistic cues, replicating findings previously reported in the literature

(O'Doherty et al., 2004; Tanaka et al., 2004; Pessiglione et al., 2006). An action

prediction error, representing an unsigned surprise signal generated during

participants’ viewing cue/association pairs, exhibited recruitment of several

distinct regions, including the superior frontal gyrus, medial frontal gyrus,

midbrain, as well as the cingulate gyrus. Taken as a whole, these results

118

implicate a critical role for the basal ganglia and medial temporal lobes during

initial acquisition of feedback and observation information (Poldrack et al., 1999;

Poldrack et al., 2001; Foerde et al., 2006; Cincotta and Seger, 2007) as well as

reversal learning in accord with the literature (Annett et al., 1989; Myers, 2000;

Carrillo et al., 2001; Cools et al., 2002; Schoenbaum and Setlow, 2003; Setlow et

al., 2003; Frank and Claus, 2006; Myers et al., 2006; Shohamy et al., 2009) and

in support of the parallel model of engagement of these distinct regions.

It was hypothesized that participants would initially acquire cue

contingencies well, irrespective of how cue values were initially acquired

(feedback vs. observation learning). However, behavioral results indicated that

participants in the observation group acquired cue contingencies in the learning

phase better than feedback group participants, in accord with a recent study

which demonstrated greater behavioral accuracy for instructed compared with

feedback learning (Li et al., 2011). However, this result was not hypothesized,

based on the results of experiment 1, and suggests that observation learning

may have be easier for participants than learning via feedback in this specific

learning environment, which included deterministic as well as probabilistic cues.

The presence of deterministic cues may have helped improve observation

learning, while not facilitating feedback learning to the same extent. In support of

this theory, accuracy differences were greatest across learning types for

deterministic cues while accuracy was most equated across learning types for

the hard probabilistic cues. It was further hypothesized that participants’

accuracy would initially decrease in the update phase following probabilistic cue

119

reversal for both learning types in both groups, and subsequently improve over

time as participants updated old cue contingencies. This hypothesis was partially

supported, with a significant improvement in feedback trial accuracy within the

first block of the update phase, but no significant increase in observation trial

performance over time (as participants were already performing quite well, mean

= 74% in block 1). Lastly, two hypotheses were made regarding behavioral

accuracy in the update phase: 1) participants may have been more efficient at

updating cue values within the same learning type (observation-observation or

feedback-feedback) as they would be accustomed to this manner of learning and

perhaps could readily employ neural structures already involved in initial learning

to update contingencies. 2) Alternatively, participants may have exhibited more

efficient updating of cue values when employing a new learning type

(observation-feedback or feedback-observation) as perhaps engaging a new

learning mechanism would facilitate faster updating (particularly for feedback

learning, which can be habit-driven and at times impervious to changes in

stimulus-outcome associations – for review see (Packard and Knowlton, 2002;

Yin and Knowlton, 2006). Neither of these two hypotheses were supported,

however, as interestingly, all participants, irrespective of group, initially were

more efficient at updating cue contingencies via observation learning. This result

suggests that employing observation learning may be a faster and more effective

manner of updating information early in learning. This observation learning type

advantage was not permanent however, as feedback trial accuracy steadily

improved throughout the first update block, resulting in no learning type

120

differences in the second update block. Therefore, while updating cue

contingencies via observation information was initially beneficial, both learning

types were equally effective in the long term, consistent with the behavioral

results of experiment 1 and reports in the literature of equivalent accuracy for

information acquired via feedback and observation (Poldrack et al., 2001;

Shohamy et al., 2004b).

The pattern of activity reported within the striatal regions in the learning

phases of the current experiment, showing greater responses to hard compared

with easy and deterministic cues, was not expected based on the results of

experiment 1. However, such a finding is not uncommon in the literature.

Delgado and colleagues (2005) suggest that the striatum is involved in the initial

learning of cue contingencies and decreases activation once these contingencies

have been acquired. In a probabilistic card guessing task, from which the

experiments in this dissertation were modified, participants were required to learn

the value of several cues: two cues were deterministic (100%), two cues were

probabilistic (67%), and one cue was random (50%). The authors discovered

that the striatum was initially engaged for all three cue types relatively equally,

early in the task when participants were initially learning the cue outcomes and

therefore feedback was meaningful (e.g., feedback indicated a correct choice for

the 100% and 67% cues). However, as time progressed and participants

acquired the cue contingencies, the striatum was differentially engaged

depending on the cue probability, exhibiting decreased activation in the feedback

period of the trial for the 100% cues and greater BOLD signal responses during

121

the feedback period for the unlearnable cue (50%). The authors theorized that

the striatum plays a pivotal role in learning cue associations, and becomes less

engaged once these associations have been successfully acquired. The cue

probabilities employed in the current task were 100%, 87.5%, and 62.5%. It is

possible that the addition of the deterministic cue changed the learning

environment, effectively making the hard cues seem harder. Therefore, the

striatum was most engaged in learning the value of the 62.5% cue. An

environment where everything is probabilistic (task employed in experiment 1)

compared with an environment where only some things are probabilistic [tasks

employed in experiment 2 and (Delgado et al., 2005)] is most likely a very

different experience for the participants. In the task employed in experiment 1,

there were only two probabilities, 85% and 65% which most likely made for a

simpler learning environment. Greater striatal BOLD activation in response to

the 85% cue in experiment 1 was most likely driven by more correct feedback for

the easy compared with hard cues.

The learning phase results support the hypothesis that the basal ganglia

and medial temporal lobes operate in parallel during learning, by providing

evidence of parallel engagement, as well as competitive and cooperative

interactions between these regions. The fact that these two distinct regions were

engaged simultaneously, suggests that they may be online and involved in

learning at the same time. This ability to be engaged during the same time

period indicates parallel processing of these areas during learning. Evidence of

cooperative interactions was observed from positive correlations exhibited

122

between the hippocampus and caudate nucleus in the observation group for the

deterministic and easy cues. It is thought provoking that despite the fact that

these two regions displayed opposing patterns of activity the BOLD signal was

still positively correlated between the areas for only the observation group.

Perhaps acquiring the information in an observational manner, possibly by

employing a declarative-like strategy, modified the interactions of these regions

despite their opposing pattern of BOLD signals. This result may also have been

driven by individual variations in mean BOLD responses in the caudate nucleus

and hippocampus. This is an interesting result which merits further exploration in

order to fully understand its significance. Lastly, the finding that the pattern of

activity within the hippocampus and caudate nucleus/putamen was opposing,

suggests that the striatum and hippocampus may encode distinct information in

the learning phase of the current task. Opposing patterns of activity and

differential activation within the hippocampus and striatum may be interpreted as

competitive interactions, as has been previously done in the literature (Poldrack

et al., 1999; Poldrack et al., 2001; Foerde et al., 2006; Seger and Cincotta,

2006). Furthermore, this finding partially replicates Poldrack and colleagues

2001 study, which reported activation of the striatum and deactivation of the MTL

during feedback learning. However, the authors also observed greater activation

in the MTL during an observation version of their task (WPT) compared with the

feedback version; whereas in the present study BOLD responses below baseline

occurred for both feedback and observation trials. Mean BOLD signal below

baseline in the MTL has been reported in several probabilistic learning tasks

123

(Poldrack et al., 1999; Poldrack et al., 2001; Aron et al., 2004; Foerde et al.,

2006). It has been suggested that “rest” in neuroimaging experiments may

engage the MTL, and therefore BOLD responses below baseline during task

periods may not be indicative of active suppression of this region during learning

(Foerde et al., 2006). Additionally, a seminal study by Stark and Squire

demonstrated that BOLD responses within the MTL may appear as either

activation or deactivation in the same task, dependent solely on the rest/baseline

condition with which MTL BOLD responses are compared (Stark and Squire,

2001).

Results from the update phase also support parallel engagement of the

basal ganglia and medial temporal lobes. Compared with the learning phase,

distinct regions within the basal ganglia and MTL were engaged in the update

phase, processing differences in cue difficulty, specifically loci within the left

putamen and bilateral parahippocampal gyri. The right and left

parahippocampus displayed the same pattern of activation as the hippocampus

region in the learning phase, while only one putamen area exhibited greater

responses to hard than easy and deterministic cues. The more posterior

putamen area displayed the same pattern of activity as seen within the MTL; that

is greater responses to deterministic and easy compared with hard cues. This

result also supports parallel engagement between the MTL and basal ganglia, as

different striatal areas processed probabilistic information in opposing manners,

making obvious the fact that the dorsal striatum plays several roles during the

learning process. There are multiple loops within the striatum (Haber, 2003),

124

which connect to distinct sections of both the midbrain and cortex. It is possible

that discrete regions of the putamen, one being more posterior in the putamen,

and one located more anterior and medially, may be engaged in distinct

functional loops and therefore convey different information to both the midbrain

and the cortex. The observation of activation bordering the putamen/insula

makes interpreting these results slightly more challenging however, as this result

could be driven more by insula than putamen BOLD responses. This caveat

should be taken into consideration as it is possible that more medial regions of

the striatum may display a different pattern of results; however give the learning

phase results this seems unlikely.

The null findings within the main effect of learning type and main effect of

group analyses in the update phase may possibly be interpreted as evidence of

parallel engagement of the MTL and basal ganglia. Neither the striatum nor any

regions within the MTL were more strongly engaged in updating information that

was the same (feedback-feedback or observation-observation) or different

(feedback-observation or observation-feedback) as the manner in which cue-

value information was initially acquired when both groups were combined into

one analysis. However, regions of the cingulate and insular cortex were

engaged and exhibited greater BOLD responses to the same than a different

learning type. Neither were loci within either the MTL or basal ganglia more

strongly engaged for either the feedback or the observation group in the update

phase. It could have been theorized, that if the striatum was selectively engaged

in feedback learning, it may have been recruited in initial feedback acquisition

125

(group 1), and as a result of this strong early activation, subsequently online

during the reversal phase to a greater degree in group 1 compared with group 2

participants. Likewise, it could have been theorized that if the MTL was strongly

recruited during initial observation learning (group 2) it may have been online and

more strongly engaged throughout the reversal phase for group 2 compared with

group 1. This result was not observed; however, null findings in neuroimaging

studies are difficult to interpret, and not necessarily indicative of any finding per

se. This null result therefore should be interpreted with caution and may be an

area of interest for further pursuit in future studies.

The interaction of learning type by group observed in the update phase

revealed that regions of the striatum and midbrain were more strongly involved in

processing feedback compared with observation trials. It is not surprising that

these areas were more strongly engaged during feedback trials, which contained

outcome relevant information, as compared with observational trials, which did

not carry outcome relevant information. This result is consistent with a large

body of research implicating both the striatum and the midbrain in feedback

processing and outcome monitoring (Delgado et al., 2000; Knutson et al., 2001;

Delgado et al., 2003; Aron et al., 2004; Tricomi et al., 2004; Marco-Pallares et al.,

2007; Carter et al., 2009).

The application of reinforcement learning models to the behavioral data

provided the opportunity to probe how neural substrates are engaged during

initial acquisition and updating of probabilistic information. Furthermore, it

allowed for the investigation of putative dopaminergic influences during learning

126

and reversal of cue contingencies. The results implicated the traditional network

of regions commonly reported to encode reward prediction errors, namely the

nucleus accumbens, the globus pallidus, and the caudate nucleus (O'Doherty et

al., 2004; Tanaka et al., 2004; Pessiglione et al., 2006). Interestingly, for the

observation learners, the medial temporal lobe (hippocampus/amygdala) and

medial frontal gyrus were also engaged in processing feedback trial prediction

errors in the update phase. One possible explanation for this result is that

participants in the observation group employed a more declarative-like learning

strategy, and thus engaged the medial temporal lobe, which then tracked

outcomes that violated participants’ expectations when updating old cue

contingencies. This finding is similar to the observation of the hippocampus

tracking feedback trial prediction error signals in experiment 1. Furthermore, a

recent study (Chen et al., 2011) observed mismatch signals in the CA1 region of

the hippocampus and the perirhinal cortex during associative retrieval when

previously viewed face-house pairs where mismatched. Additionally, the

amygdala has previously been reported in processing prediction error signals

(Bray and O'Doherty, 2007) and integrating prediction error signals with other

learning signals (Belova et al., 2007); therefore this result is not completely

unexpected. However, it is interesting that the MTL was engaged in encoding

prediction errors for the feedback trials, only within the observation group.

Results from the action prediction error analyses, which measured the

level of surprise or saliency when participants viewed cue/value associations

during the observation trials, revealed the engagement of two prefrontal areas

127

including the medial frontal gyrus and superior frontal gyrus as well as the

midbrain, insula, and cingulate gyrus. This network of regions, in particular the

midbrain, insula, and cingulate gyrus, was commonly activated in both the

learning and update phases of the task. Therefore it is reasonable that this

network also positively correlated with a saliency signal during observation trials.

It may have been expected that the dorsal lateral prefrontal cortex (DLPFC)

would be engaged during encoding of the action prediction error signal in the

learning and update phases, based on the findings of Burke and colleagues

(2010). The authors developed an observational-based action prediction error

model in order to examine the neural correlates of observational learning. Burke

and colleagues reported that a signal correlating with this model was encoded in

the DLPFC. The task employed in this experiment, however, is dramatically

different from the experiment utilized by Burke et al., which may explain the

discrepancy in the results. In the current experiment, participants, although

observing the cue and outcome on the screen, were instructed to be active

participants in the task. They were informed to optimize their responding and

told that the amount of money they would be paid depended on their

performance. In contrast, Burke and colleagues employed both an active and a

passive (observer) condition for the participants. In the observer condition,

participants were told they were viewing the actual choice and outcome of

another human player’s choice. On the observer trials, the participants were not

required to make choices, but rather simply observe the other player’s choices

and outcomes. In this learning environment, perhaps a more realistic

128

observational learning situation, action prediction errors were encoded in the

DLPFC. Burke et al.’s task therefore is fundamentally different from the task

employed in the current experiment, and as such the result of distinct areas of

activation involved in encoding the action prediction error signal employed in this

experiment may not be so surprising.

Granger causality analyses using the ventral tegmental area as a

reference region support the theory that the MTL and BG operate in parallel

during probabilistic learning. Functional connectivity results revealed a relatively

similar network of regions was engaged when updating cue contingencies,

including the caudate nucleus, putamen, globus pallidus, hippocampus, and

parahippocampus, across the observation and feedback groups. Some

distinctions arose however, with the feedback group correlating primarily with the

caudate nucleus, putamen and hippocampus, while the observation group

exhibited instantaneous correlations with the globus pallidus and

parahippocampus. The functional connectivity results implicate that key reward

learning and memory structures are correlated during probabilistic reversal

learning. This finding is in accord with a broad literature linking the involvement

of the BG and MTL during reversal learning and enhances previous results by

suggesting that these regions may not act independently during the reversal

process, but rather are functionally connected (Annett et al., 1989; Myers, 2000;

Carrillo et al., 2001; Cools et al., 2002; Frank and Claus, 2006; Myers et al.,

2006; Shohamy et al., 2009). Furthermore, directed connectivity results revealed

that the putamen influenced the VTA, and the VTA influenced the hippocampus

129

and parahippocampus during probabilistic reversal. This result supports the

theory that the MTL and BG may not interact directly, but rather may

communicate via midbrain dopaminergic centers (Poldrack and Rodriguez, 2004;

Shohamy and Adcock, 2010). Regions of the putamen do exhibit reciprocal

anatomical projections with the VTA/SN (Haber and Knutson, 2010), and the

VTA does send dopaminergic projections to the hippocampus (Haber and

Knutson, 2010). Therefore the effective connectivity results observed during the

updating of probabilistic cue contingencies in the present data set are

anatomically possible. Subsequent studies exploring the potential flow of

information from the putamen to VTA and from the VTA to the hippocampus

during probabilistic reversal learning may be an area of future research.

Lastly, the results from the learning phase provided an important

clarification regarding the results from experiment 1. In the learning phase of the

task employed in experiment 1, both a region of the caudate nucleus and the

hippocampus were engaged in processing differences in cue difficulty. However,

in that experiment, both feedback and observation learning trials were completed

in the same phase and therefore the results may have been skewed by the

presence of both trial types within the same phase. As a consequence, it was

important to examine the engagement of the hippocampus and striatum in

feedback and observation learning independently. The learning phase of

experiment 2 provided such a distinct examination as one group of participants

acquired cue contingencies via feedback and a separate group acquired cue

contingencies via observation. Importantly, in both the feedback group and the

130

observation group, regions of the MTL, specifically the hippocampus extending

into the amygdala, and the striatum, including the caudate nucleus and the

putamen, were engaged in processing cue difficulty. Perhaps equally interesting

was the result that in this experiment the striatal and hippocampal regions

displayed opposing patterns of activity. Both regions were involved in processing

cue difficulty, but whereas the MTL (activity encompassing the

hippocampal/amygdala regions) showed greatest activity to deterministic cues

and decreased in activation according to difficulty, the striatal regions displayed

greatest activity to hard cues and decreased with respect to difficulty. This

finding therefore partially replicates the result of experiment 1, where the

hippocampus and the caudate nucleus displayed stronger activation to easy

compared with hard cues. It is theorized that this hippocampal pattern of activity

(easy > hard) represents a learning signal, as the hippocampus is known to be

involved in encoding and recalling information. Several studies have shown

greater hippocampal activation is elicited for information that is subsequently

remembered, compared with information that is subsequently forgotten (Brewer

et al., 1998; Wagner et al., 1998; Law et al., 2005; Adcock et al., 2006; Shrager

et al., 2008). While follow up tests did not support this effect in this study, it is

possible that if a test had been administered one week following completion of

the task, participants would have better remembered the easier cues.

To summarize, results from experiment 2 suggest that regions within MTL

and BG are involved in the initial acquisition and updating of both feedback and

observation probabilistic information. Furthermore, prediction error and Granger

131

causality analyses lend support to the theory that these distinct memory systems

operate in parallel during learning and may interact via midbrain dopaminergic

regions. However, it is still unknown what specific learning contexts drive

memory systems to operate in parallel or interact directly via competition or

cooperation. Experiment 3 was designed to address this question directly.

Chapter Five: Experiment 3


One manner of investigating the nature of interactions between multiple

memory systems in non-human animals is to lesion, or temporarily knockout, one

brain region and observe the effect on behavior. In humans, this question has

been addressed by studying multiple memory systems in patients who have

abnormal functioning of one system (e.g., Parkinson’s disease or MTL amnesia).

Examining this question in healthy human participants, however, is challenging.

One manner of doing so is to employ behavioral interference tasks, whereby one

region of interest (e.g., the MTL) is recruited during learning in order to observe

the subsequent effect on behavior and memory system interactions. The goal of

this experiment was to engage the MTL during learning of a secondary task in

order to examine interactions between multiple memory systems. It was

theorized, given the rich animal lesion and human patient literature that new

insights would be gained into the nature of how multiple memory systems interact

during behavioral interference. A specific hypothesis, based on the results of

experiments 1 and 2, and described in detail below, was formulated in which

132

memory systems would cooperate when information to-be-learned was

challenging and operate more independently when information to-be-learned was

easy, in accordance with the parallel processing model of multiple memory

systems.

The results of animal lesion studies in rodents have demonstrated a clear

double dissociation between the function of the basal ganglia and the medial

temporal lobe in learning, with the MTL supporting spatial learning and the BG

supporting stimulus-response instrumental learning (Packard et al., 1989;

Packard and McGaugh, 1992, 1996). One relatively recent exemplary study

demonstrated bidirectional competitive interactions between these distinct

regions, as knocking out one region, the dorsal striatum, actually improved

hippocampal dependent learning, and likewise, knocking out the dorsal

hippocampus, improved striatal dependent learning in mice (Lee et al., 2008). In

addition, the application of neurotransmitters, in particular glutamate and

dopamine has been shown to influence the expression of hippocampal and

striatal-based learning (Packard and White, 1991; Packard, 1999). In a plus

maze task, rodents were always placed in the same start location in the maze

(south arm) and required to navigate to one arm which was baited with a food

reward (west arm) (Packard, 1999). Rodents naturally solve this task in one of

two ways, and a probe trial is administered in order to determine how each

individual rodent solves the task. On the probe trial, rodents are placed in a new

starting location in the maze (north arm). Rodents that navigate to the correct

arm (west arm) are labeled place learners, as they remembered the correct

133

location of the food reward. Rodents which make the previously correct motor

response (turn left) to navigate to the previously correct location (west arm), will

now end up in the opposite arm (east arm), and are termed response learners

because they learned the correct motor response in order to navigate to the food

location. However, when rodents are over-trained on this task, all rodents

typically engage in response learning, suggesting a shift in the engagement of

memory systems over time. Interestingly, intrahippocampal injections of the

neurotransmitter glutamate bias rodents towards place learning both early and

late in the training process, indicating a blockade by the hippocampus in shifting

towards response learning. Likewise, intracaudate injections of glutamate

produce response learning both early and late in the task, suggesting

accelerated response learning.

In a related study using an eight-arm radial maze task, the affects of

intracerebral injections of dopamine agonists in both the hippocampus and the

caudate nucleus were examined in rodents (Packard and White, 1991). In this

version of the win-shift radial maze task, animals obtained food from 4 randomly

selected arms, then a delay period was administered, after which rodents were

returned to the maze and the other 4 arms, which previously did not contain food,

now held the food reward. In the win-stay version of this task, animals learned to

approach 4 randomly lit arms, twice within a trial, in order to obtain food. Post-

training intrahippocampal injections of three types of dopamine agonists (indirect,

D1, and D2) improved retention on the win-shift version of the task, compared

with control rodents (received saline injections), while intracaudate injections of

134

these dopamine agonists had no affect on the win-shift version. Likewise,

intracaudate injections of all three dopamine agonists improved acquisition of the

win-stay version of the task, compared with control rodents, while

intrahippocampal injections had no effect on this task version. This important

animal work demonstrates two key points: one, both the hippocampus and

striatum are involved in spatial learning tasks, are online simultaneously, and

contribute to solving these tasks in unique ways; two, how these memory

systems are engaged and the relative degree of their engagement can be

manipulated by the administration of neurotransmitters to bias the engagement of

one memory system over the other, thereby affecting how learning tasks place.

In studies involving human participants, the question of the medial

temporal lobes’ and basal ganglia’s contribution to learning and how these

multiple memory systems interact has been addressed in a multitude of manners

including behavioral and functional neuroimaging experiments using healthy

human participants (Poldrack et al., 1999; Poldrack et al., 2001; Seger and

Cincotta, 2005; Cincotta and Seger, 2007; Shohamy and Wagner, 2008; Mattfeld

and Stark, 2010), patient populations (Knowlton et al., 1994; Knowlton et al.,

1996; Dagher et al., 2001; Myers et al., 2003; Hopkins et al., 2004; Voermans et

al., 2004; Shohamy et al., 2005; Shohamy et al., 2006; Schmitt-Eliassen et al.,

2007) as well as by employing tasks involving cognitive interference (Foerde et

al., 2006; Brown and Robertson, 2007; Foerde et al., 2007). Studies examining

probabilistic learning in human patient populations have provided evidence that

the medial temporal lobes play a crucial role in the flexible transfer of information,

135

while the basal ganglia is involved in the initial acquisition of stimulus

associations (Myers et al., 2003) as well as the formulation of sequences of

stimulus outcome associations (Nagy et al., 2007).

One manner of examining the involvement of distinct neural substrates in

healthy human participants is to employ tasks of cognitive interference in order to

engage one neural substrate and observe the outcome on behavior, akin to a

behavioral “knockout” of one region of interest. One study examined the

interaction of multiple memory systems using fMRI of healthy human participants

who completed a feedback-based probabilistic learning task while simultaneously

engaging in a second cognitive demanding task (tone counting task) (Foerde et

al., 2006). Specifically, participants completed a feedback-based version of the

weather prediction task, where they were required to learn the associations of

various cues via feedback. The task consisted of two conditions: one condition

was a control condition in which tones were played in the background and

participants were instructed to ignore the tones (single-task); the second

condition required participants to simultaneously attend to and count the number

of high tones (high and low pitch tones were played; dual-task). The authors

hypothesized that the engagement of the MTL and BG could be modulated by

distraction, such that the addition of a cognitively demanding task (tone counting)

would occupy working memory, thereby decreasing declarative memory

encoding and biasing the engagement of the basal ganglia system. Results

revealed that behavioral accuracy was only moderately diminished in the dual-

task compared with the single-task condition. In a subsequent probe session (no

136

feedback administered, no tones played, cues learned under single and dual-task

conditions were intermixed), no accuracy differences were observed between

cues learned under single versus dual-task conditions. Furthermore, in the probe

session, BOLD signal within the medial temporal lobe correlated with behavioral

accuracy for associations made under the single-task condition only, while BOLD

signal within a region of the striatum, specifically the putamen, correlated with

behavioral accuracy for those associations made under the dual-task condition

only. Lastly, BOLD activity in the MTL during the probe session positively

correlated with declarative cue knowledge for cues learned under the single-task

condition only. The authors concluded that this probabilistic classification task

can be successfully learned via the MTL or the BG, and that the engagement of

these regions depends on the task demands – such that the presence of a

distracting working memory task biased the engagement of the BG during

learning, while learning under “normal” conditions engaged the MTL.

Importantly, while the MTL and the striatum were both able to support successful

performance on the weather prediction task, the nature of the knowledge that

was acquired differed, with only the MTL supporting declarative knowledge about

learned cue associations.

A second, unrelated behavioral study examined interactions between

declarative and procedural memory during the consolidation process, which

consisted of either a period of participants staying awake versus a period of

sleep (Brown and Robertson, 2007). This study consisted of two experiments: in

experiment 1 the influence of declarative learning (word list learning) was

137

measured on procedural consolidation (serial reaction time task/SRTT); in

experiment 2 the influence of procedural learning was measured on declarative

consolidation (using the same tasks). In experiment 1 participants first

performed the SRTT, then performed the word list learning task, and after a 12

hour period of either wakefulness or sleep, were retested on the SRTT.

Interestingly, the authors observed that following a period of wakefulness,

declarative learning interfered with subsequent improvements on the procedural

skill (SRTT). Furthermore, a negative correlation was observed between

subsequent skill performance and declarative list learning, such that the more

words participants learned, the worse they performed on the procedural skill.

However, following a period of sleep, declarative learning had no interference on

skill performance. The authors concluded that during periods of wakefulness,

declarative learning interfered with “off-line” processing, or the consolidation, of

the procedural skill. The mirror result was observed when declarative learning

was followed by procedural skill learning. Procedural skill learning disrupted off-

line processing of declarative information following a period of wakefulness, and

a negative correlation was observed such that greater motor skill was associated

with worse word recall. Additionally, no interfering effect of procedural skill

learning was observed on declarative learning following a period of sleep. The

authors proposed several models by which multiple memory systems may

interact, 1) potentially exhibiting interconnected or overlapping neural circuits

during the awake state thereby interfering with each other’s consolidation

process during wakefulness, while 2) exhibiting functional uncoupling or the

138

engagement of independent systems during the sleep state such that

consolidation processes among distinct brain regions are independent during

sleep.

To summarize, new insights into how multiple memory systems interact

during learning can be achieved by employing tasks of cognitive interference,

such as the studies employed by Foerde and colleagues (2006) as well as Brown

and Robertson (2007). Foerde et al. demonstrated that the relative engagement

of multiple memory systems can be biased by task demands, while Brown and

Robertson demonstrated that the exact nature of interactions between multiple

memory systems is dynamic, still yet to be completely characterized, and may

change depending on whether consolidation of declarative and procedural

information occurs during an awake versus a sleep state.

One remaining open question is characterizing under which specific

learning scenarios memory systems may cooperate versus compete. The

purpose of the present experiment was to address this question by employing a

cognitive interfering task, in order to engage the medial temporal lobe and

examine the effects of such a behavioral “knockout” of the MTL on a feedback-

based probabilistic learning task. The aim of employing such a design was to

gain new insights into the relative engagement and interactions between the MTL

and BG during feedback probabilistic learning by probing one learning scenario

in which multiple memory systems may cooperate.

The following hypothesis was formulated based on the results of

experiments 1 and 2: the MTL and BG may cooperate during

139

challenging/inconsistent learning scenarios and may operate independently

during easy/consistent learning scenarios. Results from experiment 1 suggested

that the BG and MTL may cooperate when learning is inconsistent as the

following results were observed: 1) positive correlations between the

hippocampus and caudate nucleus’s BOLD signal during probabilistic

observation learning; 2) a positive correlation between the hippocampus and

caudate nucleus’s BOLD signal for feedback easy trials, late in learning when

participants received incorrect feedback (prediction error-like situation); 3)

marginally greater hippocampal BOLD signal for feedback hard cues, later in

learning and marginally greater caudate nucleus BOLD signal for observation

hard cues, later in learning; and 4) encoding of a feedback prediction error signal

within both the striatum and hippocampus. Furthermore, results from experiment

2 corroborate the theory that these memory systems have the ability to both

compete and cooperate during learning, as mutual engagement of these regions

was observed during the learning and update phases, at times exhibiting similar

patterns of activation (potential cooperation) and at other times exhibiting

opposing patterns of activity (potential competition). Positive correlations were

observed in both the learning and update phases across the striatum and MTL

for deterministic cues and a feedback prediction error signal was encoded within

loci in the striatum and the MTL (hippocampus/amygdala). Additionally, it has

been theorized that depending on the learning scenario, memory systems may

operate in parallel or may interact directly via competition or cooperation (White

and McDonald, 2002). However, the exact learning scenarios when cooperation

140

or competition occur remain uncharacterized. Therefore, it was hypothesized,

based on the aforementioned results and theoretical work that the BG and MTL

operate in parallel and may interact directly via cooperation when the learning

scenario is particularly challenging/inconsistent. The rationale for this hypothesis

is that in inconsistent and therefore challenging learning scenarios, when cue

value contingencies are low, e.g., 70% predictive of the value, it is possible that

neither the MTL nor the BG system will learn the cue values well. Therefore, if

both systems independently are not able to acquire the values, it is theorized that

they may cooperate, by communicating and sharing information each system has

individually acquired in order to learn the cue values in a synergistic manner.

However, when the contingencies are consistent, and therefore less challenging

to learn, e.g., 90%, one system alone may successfully acquire the cue values or

both systems independently may learn the contingencies and therefore the

systems do not need to cooperate.

In order to test this hypothesis, a within-subjects fMRI study was

conducted using a feedback probabilistic learning task, which contained both

easy (90%) and hard (70%) to learn information. The hypothesis was that the

MTL and BG may cooperate during acquisition of difficult (70% cues), but not

easy (90% cues) information. Furthermore, in order to demonstrate that both the

MTL and BG are required to learn challenging/inconsistent information, multiple

learning sessions were utilized, one of which was designed to engage the MTL in

an unrelated task (behavioral “knockout” of the MTL). This was a two-day

experiment, in which participants came to the laboratory one day and performed

141

a declarative memory scene encoding task and returned the following day to

complete a feedback probabilistic learning task while undergoing functional

neuroimaging. Specifically probabilistic cue learning occurred via feedback in

three distinct sessions. In each session, a probabilistic cue was presented on

the screen along with a natural scene (landscape image). The three sessions

were: 1) a feedback control session, in which participants were required to learn

only the value of probabilistic cues and ignored the natural scenes. This session

provided for a baseline examination of multiple memory systems during feedback

learning; 2) a perceptual decision session, in which participants were required to

learn the value of the probabilistic cues as well as attend to the natural scenes

and make a perceptual decision about each scene. This session was employed

as a control for the cognitive demands elicited in the declarative memory

interference condition (dual-tasking); 3) a declarative memory interference

condition, in which participants were required to learn the value of the

probabilistic cues while simultaneously engaging in a declarative memory

recognition task (determining whether each natural scene was old or new). Of

key importance was the declarative memory interference condition, which was

designed to behaviorally sequester or “knock out” the medial temporal lobe by

engaging it in the recognition of previously viewed scenes. This task design

therefore allowed for an examination of the relative engagement of the MTL and

BG and interactions between these two regions when the MTL is available to

assist in learning (control conditions – feedback control and perceptual decision

session) and when it may not be available to assist in learning (experimental

142

condition – declarative memory interference condition).



This study involved two days of participation. On day one, participants

completed a behavioral declarative memory scene encoding task in the

laboratory. Participants viewed 40 natural landscape scenes on a computer and

were asked to determine whether or not each scene contained water (sixty

percent of the scenes contained water, e.g., a lake, stream, waterfall; each scene

was presented one time; Figure 5.1A). Every scene was a unique landscape

image, borrowed from a database from the Center for Cognitive Neuroscience at

Duke University. The purpose of the water detection question was to ensure that

participants paid attention to the scenes during the encoding session.

Participants were told that they should try their best to learn the scenes as they

would be tested on them the following day and their payment on the subsequent

day would be determined based on how well they correctly discriminated

between old scenes (shown on day 1) and new scenes (to be shown on day 2).

During the encoding session, each scene was displayed for 4 seconds during

which time the participants studied the scenes and looked for water. This was

followed by a 2-4 second jittered inter-stimulus-interval (ISI), followed by a 4

second period in which participants made a choice regarding the presence or

absence of water in the scene they had just viewed. Participants were instructed

to press 1 if there was definitely water in the scene, 2 if there was probably water

143

in the scene, 3 if there was probably no water in the scene, and 4 if there was

definitely no water in the scene. They were instructed to press 2 or 3 if they had

trouble remembering the scene or if it was ambiguous as to whether the scene

contained water. A blue box highlighted participants’ choice and no penalty was

assigned if the participant missed a trial. The water response period was

followed by a 4-10 second jittered ITI before the onset of the next scene.

Participants were paid $10 for completion of the scene encoding session on day

1.

On the second day of the experiment, participants completed the MRI

portion of the study, which consisted of a learning and memory task that was a

modification of the tasks employed in experiments 1 and 2. Participants were

required to learn the numerical value (range 1-9) of six different visual cues. The

value of each cue was either higher or lower than the number five. There were

two phases in the experiment, a learning phase and a test phase. Two

independent variables were manipulated in the learning phase: the learning

session type and the difficulty of the probabilistic cues. There were three distinct

learning sessions, all of which contained feedback learning: the feedback control

session (Figure 5.1B), the perception decision session (Figure 5.1C), and the

declarative memory interference session (Figure 5.1D). The second independent

variable was the probabilistic value of the cues (predictive outcome of the

numerical value of each cue): 3 cues were 90% predictive of their value, termed

“easy cues” and 3 cues were 70% predictive of their value termed “hard cues”.

Participants were instructed on all three session types and performed a short

144

practice version with different probabilistic cues and different natural scenes prior

to beginning the task inside the MRI. The practice session natural scenes were

drawn from the Duke University scene database as well as the McGill University

Calibrated Colour Image Database (Olmos and Kingdom, 2004).

In the learning phase participants’ primary goals were to learn the value of

six cues and correctly identify old versus new natural scenes. The learning

phase consisted of six blocks (two presentations of each of the three session

types) of 20 trials each for a total of 120 trials. In all sessions, participants were

required to learn the numerical value of two cues (one 90% cue and one 70%

cue per session). Each session contained 10 presentations of each of the two

cues; all trials were feedback based. The trial format in every session consisted

of a 4 second natural scene/cue presentation period, followed by a 2-4 second

jittered ISI, proceeded by feedback presentation (contingent on participant’s

response for the cue value; check mark for correct, X for incorrect, and # for

missed trials), followed by a response screen which varied by session type

(details provided below), and lastly a jittered ITI (4-10 seconds) before onset of

the next stimulus. In all learning sessions, the stimulus presentation period

consisted of a natural scene displayed at the top of the screen and a probabilistic

cue presented at the bottom of the screen. Depending on the session type, the

natural scene was either new (presented for the first time) or old (previously

shown on the day prior during the encoding session). Each natural scene

presented in the experiment was shown only one time; therefore the scenes did

not overlap between sessions, and participants were made explicitly aware of

145

this fact. Participants were instructed to make a button press in all sessions

during the stimulus presentation period regarding the value of the probabilistic

cue (button 1= higher than 5; button 2 = lower than 5). Similar to the feedback

trials in experiments 1 and 2, participants were instructed to optimize their

responses. Instructions for the natural scenes varied by session type as

described below.

In the feedback control session, participants were instructed to focus on

learning the value of the probabilistic cues and were told to ignore the natural

scenes, which were all new (no old scenes were presented from the day prior)

and unimportant (Figure 5.1B). Participants were informed that the natural

scenes in the feedback control session would never be shown again and that

they would not be tested on these scenes in the future. Participants viewed the

natural scene/cue and made a button press indicating the value of the

probabilistic cue during the 4 second stimulus presentation/response period.

After the ISI, participants received feedback contingent on their response.

Feedback presentation was proceeded by a response screen, where participants

made a button press (button 1, button 2, button 3, button 4) which they were

informed was not meaningful and had no associated value. They were told they

could know why the button press was necessary in this session at the end of the

experiment if they desired (it served as a motor control). The purpose of this

session was to provide a baseline level of feedback learning in the task.

In the perception decision session, participants were instructed to pay

attention to the natural scenes and to detect water in them while simultaneously

146

learning the value of the probabilistic cues (Figure 5.1C). They were informed

that in this session all of the scenes were new. They were also told that they did

not need to learn the scenes, as they would not be tested on them, but were

simply required to detect water in each scene. The purpose of this session was

to serve as a cognitive-demand control for the declarative memory interference

session. While looking for water may not be as mentally taxing as trying to

remember a scene, it does require some attentional processing and dual-tasking.

In the perception decision session participants viewed the natural scene/cue and

made a button press indicating the value of the probabilistic cue during the

stimulus presentation period. Following the ISI and feedback presentation, the

response screen appeared and participants were instructed to make a response

regarding the presence or absence of water in the natural scene they had viewed

in the stimulus presentation period. This response screen was identical to that

presented during the day 1 encoding session. They were instructed to press 1 if

the scene definitely contained water, 2 if it probably contained water, 3 if it

probably contained no water, and 4 if it definitely contained no water.

In the declarative memory interference session, participants were required

to determine whether the natural scene presented in each trial was old or new

while simultaneously learning the value of the probabilistic cues (Figure 5.1D).

During the stimulus presentation period they viewed a natural scene and

probabilistic cue and were instructed to try to remember if the scene was old or

determine if it was new while learning the value of the probabilistic cue. As in all

other sessions, they were informed to make a button press indicating the value of

147

the probabilistic cue in the stimulus presentation period. Following the ISI and

feedback presentation period, the response screen appeared. In this session

participants were told to press 1 if they thought the scene was definitely old, 2 if it

was probably old, 3 if it was probably new, and 4 if it was definitely new.

Additionally, participants were instructed to press 1 only if they had a strong

memory of seeing the scene on the day prior (I remember) and 2 if they

recognized the scene to be old, but did not have a strong memory of seeing it the

day before (I know). They were explicitly informed that half of the scenes would

be old and half would be new in this session and that this was the only session in

which they would see natural scenes that were shown on the day prior (old

scenes). They were reminded that the total amount of money they would be paid

would be determined by how well they correctly identified old versus new natural

scenes and how well they learned the value of the feedback cues in all three of

the learning sessions.

Participants completed the sessions in one of two orders: order 1) block 1:

feedback control, block 2: declarative memory interference, block 3: perception

decision, block 4: feedback control, block 5: declarative memory interference,

block 6: perception decision; order 2) block 1: feedback control, block 2:

perception decision, block 3: declarative memory interference, block 4: feedback

control, block 5: perception decision, block 6: declarative memory interference.

Half of the participants completed one session order and the other half

completed the other session order. Trial order in each block was pseudo-

randomized to ensure that the trial types were balanced throughout each block

148

(e.g., there were an equal number of old and new scenes paired with easy and

hard cues in trials 1-10 and 11-20). Different trial orders were created for the two

blocks of each session type (e.g., trial order in feedback control block 1 differed

from trial order in feedback control block 2); all participants completed the

identical trial order.

Following completion of the learning sessions, participants completed the

test session (not depicted in the figure). The test phase contained the

probabilistic cues learned in all three sessions, as well as two novel cues

presented for the first time. The cues from the learning phase were blue, and the

novel cues were white in color. Critically, in the test session, no feedback was

provided and no natural scenes were presented. Participants were simply

required to indicate the value of the probabilistic cues they had learned in the

learning sessions. The test session contained 80 trials total, 10 presentations of

each cue, presented in random order. Trial structure and timing in this test phase

were identical to that used in the test phases in experiments 1 and 2 (self-timed

stimulus/response phase followed by a 6-14 second jittered ITI before onset of

the next cue).

5.2.2 Data analysis

Behavioral Data Analysis:

D Prime: In order to examine participants’ ability to correctly identify

scenes which contained water versus those that did not in the day 1 and day 2

sessions as well as scenes which were old versus new in the day 2 task, D prime

(d’) analyses were performed (Macmillan and Creelman, 1991). D prime

149

analyses are informative as they account for a participant’s response bias (e.g.,

more likely to indicate water versus no water; or more likely to choose old versus

new) when calculating a participant’s ability to correctly identify a stimulus

(sensitivity).

Neuroimaging Data Analysis:

GLM: The primary analysis of interest was to examine activation within

the brain during the three distinct learning sessions (feedback control, perceptual

decision, and declarative memory interference). Comparing brain activation

across learning sessions was most desired as the key question of interest was

examining how multiple memory systems were engaged during pure feedback

learning compared with feedback learning concurrent with declarative memory

interference. Therefore, three random-effects GLM analyses were conducted in

the learning phase; one GLM per learning session. Each GLM contained the

following two predictors: session type (which varied by session: feedback control,

perception decision, and declarative memory interference) and cue difficulty

(easy and hard). Missed trials and six motion parameters were also included as

regressors of no interest. In order to gain the most complete picture of the

engagement of brain regions during learning and memory processes, the mean

BOLD signal was examined during two periods: 1) the stimulus presentation

period and 2) the feedback presentation period. The stimulus presentation

period of the trial occurred when the probabilistic cue and natural scene were

presented on the screen. This time period was examined in order to probe for

differences in cue difficulty and memory processing within the brain, with the

150

primary focus on reward-related learning and memory structures (e.g., striatum,

medial temporal lobe, and midbrain). It was during this time period that

participants were making a choice regarding the value of the probabilistic cue

and presumably either ignoring or attending to the natural scene depending on

the session (ignore the scene in the feedback control session; look for water in

the perceptual decision session; or try to remember the scene in the declarative

memory interference session). During the feedback presentation period,

participants received feedback regarding their choice of the probabilistic cue

value (correct, incorrect, missed trial).

In the GLM created for the feedback control session, a contrast of easy vs.

hard cues was performed to examine activity within the brain during both the

stimulus presentation as well as the feedback presentation periods. In the

perception decision GLM, a 2 (water: water vs. no water) x 2 (cue difficulty: easy

vs. hard) within-subjects ANOVA was performed using water (water-scene and

no water-scene) and cue difficulty (easy and hard) as within-subjects factors

when examining activity during both the stimulus presentation as well as the

feedback presentation periods. The primary SPM of interest investigated a main

effect of cue difficulty. The water condition was included as a factor because

participants attended to the presence or absence of water in the scenes during

the day 1 encoding session, and discussions with participants after the

experiment indicated that some participants used the presence or absence of

water in the natural scenes as a memory cue (way to learn and subsequently

remember the scenes). In the declarative memory GLM, a 2 (scene: old vs. new)

151

x 2 (cue difficulty: easy vs. hard) within-subjects ANOVA was performed using

memory (old scenes and new scenes) and cue difficulty (easy and hard) as

within-subjects factors when examining activity during both the stimulus

presentation as well as the feedback presentation periods. SPMs of interest

investigated a main effect of memory, a main effect of cue difficulty, and an

interaction of memory and cue difficulty. SPMs were thresholded at p<0.005 with

an appropriate cluster threshold correction, correcting to a cluster-level false-

positive rate of 5% when possible. As noted in the results section, these

analyses did not produce activation within any a priori regions of interest

(striatum, MTL, midbrain) at cluster threshold corrected levels in the control

conditions. Therefore, the hypothesis could not be tested and results from these

SPMs are not reported.

For the test phase, BOLD activity was examined across the entire trial

(stimulus onset and participant response). A random-effects GLM was

performed using the following predictors: learning material (feedback control,

perception decision, declarative memory interference, and novel) and cue

difficulty (easy and hard). SPMs were generated from this GLM with the contrast

of interest as previously studied (feedback control, perception decision,

declarative memory interference) versus non-studied (novel) cues. Resulting

SPMs were set to a threshold of p<0.005 and a cluster threshold level of 5

contiguous voxels or 132mm3. As a note, three participants’ data was excluded

from the test phase neuroimaging analysis due to technical errors (e.g., Eprime

quitting during the scanning session).

152

Region of Interest Analyses: Independent regions of interest were drawn

in order to examine activity within a priori brain regions in an unbiased manner.

Specifically, the following seven regions were examined: bilateral hippocampus

(x, y, z = -20, -11, -17 and x, y, z = 21, -12, -18); bilateral ventral striatum (x, y, z

= -11, 4, 0 and x, y, z = 10, 3, -6); bilateral midbrain, centered on the ventral

tegmental area (x, y, z = -4, -15, -9 and x, y, z = 5, -14, -8); and the left caudate

nucleus (x, y, z = -15, 20, 7). All but one of the ROIs (left caudate nucleus) were

drawn from a previously published paper (Adcock et al., 2006). The bilateral

hippocampus, bilateral ventral striatum, and bilateral midbrain (VTA) were

chosen from this study in particular as the experiment employed in Adcock and

colleagues examined the effects of reward-related motivation on memory. It was

theorized that the areas reported by the authors would be relevant to examine in

the current experiment, which investigated the effect of declarative memory

interference on reward-related learning. The left caudate nucleus ROI was taken

from experiment 1 of this dissertation. The caudate nucleus region was

investigated due to the observation that loci within the caudate nucleus were

engaged in both experiments 1 and 2 of this dissertation, suggesting this area

plays a role in feedback probabilistic learning. Mean parameter estimates were

extracted from the above regions in all three learning sessions (feedback control,

perception decision, and declarative memory interference) and mean BOLD

responses were examined during the stimulus presentation period.

153

5.3 Results

5.3.1 Behavioral Results

Day 1 scene encoding

Results from the d’ analysis revealed that participants’ hit rate for

accurately identifying scenes with water (0.97; responding water to a water-

scene) was significantly greater than the false alarm rate (0.16; responding water

to a no-water-scene; t(23) = 48.12; p < 1.35 x 10-24), with a bias towards labeling

scenes as containing water (d’=2.95; C=-0.41; Figure 5.2A). This result suggests

that participants discriminated very well between scenes that contained water

versus those that did not during the day 1 encoding session (97% accurate).

Day 2 scene water and memory discrimination

Participants’ ability to correctly discriminate between new scenes that

contained water versus those that did not (perception decision session) as well

as old and new scenes (declarative memory interference session) was

examined. In the perception decision session, participants’ hit rate (0.93) was

significantly greater than the false alarm rate (0.11; t(23) = 32.97; p < 7.24 x 10-21)

with a bias towards labeling the scenes as containing water (d’= 2.98; C=-0.14).

Participants’ ability to accurately detect water in the natural scenes on day 1 was

compared with their ability to accurately detect water in the natural scenes on

day 2. Results revealed that participants had a significantly higher hit rate during

the day 1 encoding task (97% > 93%; t(23) = 2.91; p < 0.01), with a significantly

greater false alarm rate observed in the day 1 encoding task as well (16% > 11%;

154

t(23) = 2.20; p < 0.05) suggesting that on a whole, participants performed better

during the day 1 encoding session.

In the declarative memory interference session, participants’ hit rate (0.61;

responding old to an old scene) was significantly greater than the false alarm rate

(0.24; responding old to a new scene; t(23) = 9.58; p < 1.76 x 10-9) with a bias

towards labeling scenes as new (d’=1.11; C=0.25), suggesting that participants

were able to successfully discriminate between old and new scenes.

Furthermore, participants’ hit rate was greater than chance (61% > 50%; t(23) =

3.12; p < 0.005). In addition, participants’ hit and false alarm rates were

compared across session types (perception decision vs. declarative memory

interference). Results revealed a significantly higher hit rate in the perception

decision session compared with the declarative memory interference session

(93% > 61%; t(23) = 9.39; p < 2.48 x 10-9) and a lower false alarm rate in the

perception decision session compared with the declarative memory interference

session (24% > 11%; t(23) = 3.12; p < 0.005; Figure 5.2B), suggesting that, as

expected, identifying scenes which contained water was an easier task than

discriminating between old versus new scenes.

Day 2 probabilistic cue accuracy

Behavioral accuracy was examined in all three cue learning sessions:

feedback control, perception decision, and declarative memory interference

(Figure 5.2C). One question of interest was to examine differences within each

session type as it varied according to cue difficulty. A second question of interest

was to examine behavioral accuracy across sessions in order to determine if

155

accuracy was greater for one session compared with another (e.g., feedback

control > declarative memory). Lastly, as this was a demanding task, it was

hypothesized that participants’ accuracy would improve over time (early/first

block vs. late/second block of learning). Therefore, the first block (trials 1-20) of

each session was used as the early time factor, while the second block (trials 21-

40) of each session was used as the late time factor. In order to examine these

questions, a 3 (learning session: feedback control vs. perception decision vs.

declarative memory interference) x 2 (cue difficulty: easy vs. hard) x 2 (time:

early vs. late) within-subjects, repeated-measures ANOVA was performed.

Results indicated a main effect of session type (F(1.65,37.89) = 16.60; p < 0.001), a

main effect of cue difficulty (F(1,23) = 9.56; p < 0.01), a marginally significant effect

of time (F(1,23) = 3.09; p = 0.09), a marginally significant interaction of cue

difficulty by time (F(1,23) = 3.89; p = 0.06), and a significant three way interaction

(session type x cue difficulty x time) (F(2,46) = 3.40; p < 0.05). Post-hoc t-tests

were performed in order to examine the significant main effects and interactions.

When examining the main effect of session type, results revealed better

performance for cues learned in the feedback control (83%) and perception

decision session (78%) compared with cues in the declarative memory

interference session (63%) (t(23) = 5.76; p < 0.0001 and t(23) = 3.54; p < 0.005,

respectively), with no accuracy differences observed between the feedback and

perception decision control sessions (t(23) = 1.60; p > 0.05). A post-hoc t-test was

also performed in order to examine the main effect of cue difficulty result, and as

expected performance was significantly greater for easy (80%) compared with

156

hard (70%) cues (t(23) = 3.13; p < 0.005). In addition, a post-hoc t-test was

performed to examine the marginally significant main effect of time, which as was

theorized, was caused by nearly better performance in the second block (76%)

compared with the first block (73%; t(23) = 1.71; p = 0.1). The marginally

significant interaction of cue difficulty by time was driven by significantly greater

performance for easy (80%) than hard (67%) cues in the first block (t(23) = 3.52; p

< 0.005), but only marginally better accuracy for easy (79%) compared with hard

(73%) cues in the second block (t(23) = 1.76; p = 0.09). This result was most

likely driven by the increase in accuracy for the hard cues over time (t(23) = 2.25;

p < 0.05), but not the easy cues (t(23) = 0.27; p > 0.05). Post-hoc t-tests were

also performed to examine the significant three way interaction (session type x

cue difficulty x time). This interaction was driven by significant changes in

performance from the first to the second learning block in the feedback control

learning session only (81% to 85%), with no changes observed over time in

either the perception decision (77% to 79%) or the declarative memory

interference sessions (61% to 65%; all p values > 0.05). In the feedback control

session, performance for hard cues increased over time (t(23) = 3.31; p < 0.005)

while performance for easy cues was marginally worse in the second compared

with the first block (t(23) = 2.14; p = 0.04; trend after sequential Bonferroni

correction).

Correlations between natural scene sensitivity (d’) and probabilistic cue accuracy

Two correlations were performed in order to examine potential

relationships between participants’ ability to perform the probabilistic cue learning

157

task concurrently with the scene discrimination tasks (both the perception

decision detection task and declarative memory interference task). Specifically,

one correlation was conducted to probe a potential relationship between

participants’ ability to differentiate between water/no-water scenes while learning

the probabilistic cue values in the perception decision session. Participants’ hit

rate, false alarm rate, and probabilistic cue accuracy for easy and hard cues

were entered into a Pearson’s correlation analysis. Results revealed a significant

positive correlation between participants’ hit rate and easy cue accuracy

(r=0.510, p = 0.01), but not hard cue accuracy (r=0.275, p = 0.19). A second

correlation was conducted to examine a potential relationship between

participants’ ability to discriminate between old and new scenes while learning

probabilistic cue values. No significant relationships were observed between

participants hit rate, false alarm rate, and probabilistic easy and hard cue

accuracy (all p values > 0.05).

Day 2 test phase

A test phase was performed while participants remained inside the MRI to

probe for behavioral performance differences on probabilistic cues (easy and

hard) that were acquired across the three session types. The first analysis

probed for potential differences in session type, irrespective of cue difficulty, by

conducting a repeated-measures, within-subjects ANOVA using session type as

a within-subjects factor (feedback control, perception decision, declarative

memory interference, and novel). A significant main effect of session type was

observed (F(2.11,48.62) = 13.21; p < 0.001). Post-hoc t-tests indicated significantly

158

greater performance for cues presented in all sessions compared with novel cues

(feedback control vs. novel: t(23) = 5.18; p < 0.0005; perception decision vs. novel:

t(23) = 4.54; p < 0.0005; declarative memory interference vs. novel: t(23) = 3.19; p

< 0.005). Marginally significant better accuracy was observed for cues previously

presented in the feedback control session (81%) compared with the declarative

memory interference session (64%; t(23) = 2.19; p = 0.04; trend after sequential

Bonferroni correction). No other significant differences were observed between

cues previously presented in the various learning sessions (all p values >0.05).

The second analysis examined differences across session types and

levels of cue difficulty, excluding novel cues. In order do so, a 3 (learning

session: feedback control vs. perception decision vs. declarative memory

interference) x 2 (cue difficulty: easy vs. hard) repeated measures, within-

subjects ANOVA was performed. Results revealed a main effect of session type

(F(1.21,27.87) = 3.88; p < 0.05), no main effect of cue difficulty (F(1,23) = 0.07; p >

0.05), and no interaction of cue difficulty and session type (F(2,46) = 0.60; p >

0.05). Post-hoc t-tests indicated marginally greater accuracy for the feedback

control compared with the declarative memory cues, collapsed across difficulty

(t(23) = 2.19; p = 0.04; trend after sequential Bonferroni correction).


Learning phase analyses

Region of Interest Analyses: In order to examine the engagement of

learning and memory regions in an unbiased manner during the learning

sessions, and to try to test the hypothesis that the MTL and BG correlate during

159

inconsistent learning conditions, independent regions of interest were drawn. It

was hypothesized that regions of the MTL and BG would correlate during the

control sessions (feedback and perception decision) most strongly for the

inconsistent learning condition (hard cues) as the MTL may be online and

available to assist in probabilistic cue learning. It was further hypothesized that

these regions would not correlate during the declarative memory interference

condition as the MTL would be engaged in the declarative memory task rather

than the probabilistic cue learning task. Specifically activity within the bilateral

hippocampus, bilateral ventral striatum, bilateral midbrain, and left caudate

nucleus was examined. Two sets of analyses were performed. The first analysis

examined activity within each ROI during all learning sessions independently. In

the feedback control session, this analysis therefore examined differences in

BOLD signal according to levels of cue difficulty; in the perception decision

session, this analysis examined differences in BOLD signal across levels of cue

difficulty and the presence or absence of water in the natural scenes; and in the

declarative memory session, this analysis examined differences in BOLD signal

across levels of cue difficulty and the declarative memory scene condition (old or

new scene).

The second analysis examined activity within each ROI across different

session types (feedback control vs. perception decision vs. declarative memory

interference) and levels of cue difficulty (easy vs. hard), collapsed across the

memory scene condition (water/no-water or old/new). Results are presented

below for the first analysis (activity in each learning session independently)

160

followed by the results from the second analysis (comparison of activity within

each ROI across learning sessions).

Feedback control session: In the feedback control session, paired-

samples t-tests were performed in each ROI to probe for differences in the BOLD

signal response to easy and hard probabilistic cues. The right ventral striatum

exhibited significantly greater responses to hard than easy cues (t(23) = 2.79; p <

0.05), while marginally greater responses for hard compared with easy cues was

observed in the left ventral striatum (t(23) = 1.82; p = 0.08). No other regions of

interest displayed any modulation in activation by cue difficulty.

Perception decision session: In the perception decision session, paired-

samples t-tests were performed in each ROI to explore potential differences in

the mean BOLD signal in response to easy versus hard cues; however no

significant differences were observed in any region of interest in this session.

Declarative memory interference session: In the declarative memory

session, modulation of the mean BOLD signal in response to levels of cue

difficulty as well as old versus new declarative memory scenes was examined.

The right midbrain exhibited greater BOLD responses to hard compared with

easy cues (t(23) = 2.18; p < 0.05) [a 2 (memory scene: old vs. new) x 2 (cue

difficulty: easy vs. hard) within-subjects, repeated-measures ANOVA was

conducted, results indicated no main effect of memory scene (F(1,23) = 0.87; p >

0.05); a main effect of cue difficulty (F(1,23) = 4.76; p < 0.05); and no interaction of

memory scene by cue difficulty (F(1,23) = 0.05; p > 0.05)]. No other significant

effects were observed.

161

Session type comparison: A comparison of the mean BOLD signal within

each ROI across learning sessions (feedback control vs. perception decision vs.

declarative memory interference) was also performed. In order to do so, seven

separate 3 (session type: feedback control vs. perception decision vs. declarative

memory interference) x 2 (cue difficulty: easy vs. hard) repeated-measures,

within-subjects, ANOVAs were conducted (one per ROI) using session type and

cue difficulty as within-subjects factors. Interestingly, every ROI except the left

caudate nucleus exhibited a significant main effect of session type and no

significant main effect of cue difficulty; only the right ventral striatum displayed an

interaction of session type by cue difficulty.

For each ROI, the main effect of session type was driven by greater BOLD

responses in the declarative memory interference session compared with both

the perception and feedback control sessions (except for the caudate nucleus,

which exhibited no main effect of session type; representative results are shown

in Figure 5.3 – beta plots are exhibited for the left ROIs as the pattern of activity

was similar across bilateral ROIs). The right hippocampus mean BOLD signal

scaled according to session type with declarative memory interference >

perception decision > feedback control. All other regions exhibited no differences

between the mean BOLD signal elicited in response to feedback versus the

perception decision control sessions. These results suggest that the

hippocampus, ventral striatum, and ventral tegmental area are all more strongly

engaged during the memory session than the control sessions, but interestingly

162

are not modulated by probabilistic cue difficulty. The results of this analysis are

presented below according to ROI:

1) Right hippocampus: main effect of session type (F(2,46) = 13.09; p <

0.001); no main effect of cue difficulty (F(1,23) = 0.59; p > 0.05); no

interaction of session type and cue difficulty (F(2,46) = 1.16; p > 0.05).

Declarative memory interference session > perception decision (t(23) =

2.26; p < 0.05); perception decision > and feedback control session

(t(23) = 3.00; p < 0.01).

2) Left hippocampus: main effect of session type (F(1.59,36.45) = 8.37; p <



Declarative memory interference session > perception decision and

feedback control session (t(23) = 3.52; p < 0.005 and t(23) = 4.48; p <

0.0005, respectively), with no differences observed between BOLD

response to the control conditions (t(23) = 0.95; p > 0.05; Figure 5.3A).

3) Right ventral striatum: main effect of session type (F(2,46) = 3.96; p <

0.05); no main effect of cue difficulty (F(1,23) = 2.06; p > 0.05); an

interaction of session type and cue difficulty (F(2,46) = 5.06; p < 0.05).

Declarative memory interference session marginally > perception

decision (t(23) = 2.32; p = 0.03; trend after sequential Bonferroni

correction); declarative memory interference session > feedback

control session (t(23) = 2.63; p < 0.05, respectively), with no differences

163

observed between BOLD response to the control conditions (t(23) =

0.72; p > 0.05).

4) Left ventral striatum: main effect of session type (F(2,46) = 9.01; p <



Declarative memory interference session > perception decision and

feedback control session (t(23) = 4.00; p < 0.001 and t(23) = 4.03; p <

0.001, respectively), with no differences observed between BOLD

response to the control conditions (t(23) = 0.24; p > 0.05; Figure 5.3B).

5) Right midbrain: main effect of session type (F(2,46) = 9.73; p < 0.001);

no main effect of cue difficulty (F(1,23) = 1.10; p > 0.05); no interaction of

session type and cue difficulty (F(1.57,36.01) = 1.29; p > 0.05). Declarative

memory interference session > perception decision and feedback

control session (t(23) = 3.12; p < 0.005 and t(23) = 4.47; p < 0.0005,

respectively), with no differences observed between BOLD response to

the control conditions (t(23) = 1.27; p > 0.05).

6) Left midbrain: main effect of session type (F(2,46) = 9.37; p < 0.001); no

main effect of cue difficulty (F(1,23) = 1.96; p > 0.05); no interaction of

session type and cue difficulty (F(1.55,35.74) = 0.42; p > 0.05). Declarative

memory interference session > perception decision and feedback

control session (t(23) = 2.67; p < 0.05 and t(23) = 4.73; p < 0.0005,

respectively), with no differences observed between BOLD response to

the control conditions (t(23) = 1.49; p > 0.05; Figure 5.3C).

164

7) Left caudate nucleus: no main effect of session type (F(2,46) = 1.13; p >


interaction of session type and cue difficulty (F(2,46) = 1.00; p > 0.05;

Figure 5.3D).

Independent ROI Correlation Analyses: In order to test the hypothesis

that the mean BOLD signal within the striatum and hippocampus would be

positively correlated during the control conditions, specifically for the inconsistent

learning situation (hard cues), with potential correlations observed between the

midbrain, a series of Pearson’s correlations were performed. Specifically three

correlations, one for each learning session, were conducted. An abundance of

correlations were observed across regions. Therefore, for simplicity of

presentation, a summary of the correlations observed is described below and the

details regarding the correlations (which areas correlated with each other and at

what statistical value) are presented in Appendix 2:

Feedback control session: five positive correlations were observed

across anatomical regions, four occurred in the hard cue condition and one

occurred in the easy cue condition. In the hard cue condition, the hippocampus

correlated with the ventral striatum and caudate nucleus, while the ventral

striatum additionally correlated with the midbrain. In the easy cue condition, the

midbrain correlated with the ventral striatum.

Perception decision session: seventeen positive correlations were

observed across anatomical regions, five occurred in the hard cue condition and

165

twelve occurred in the easy cue condition. In the hard cue condition the midbrain

correlated with the hippocampus and ventral striatum. In the easy cue condition,

the hippocampus correlated with the ventral striatum, the caudate nucleus, and

midbrain; the midbrain additionally correlated with both the ventral striatum and

the caudate nucleus.

Declarative memory interference session: fourteen positive correlations

were observed across anatomical regions, seven in the hard cue condition and

seven in the easy cue condition. In the hard cue condition, the hippocampus

correlated with the ventral striatum, caudate nucleus, and midbrain, while the

ventral striatum correlated with the midbrain. In the easy cue condition, the

hippocampus correlated with the ventral striatum and midbrain, and the ventral

striatum correlated with the midbrain.

To summarize, the correlation results indicate that while there were

significant positive correlations in the control conditions between the striatum,

hippocampus, and midbrain for the hard cues, positive correlations were also

observed in the declarative memory interference condition and in all conditions

for the easy cues. The observation of positive correlations in all learning

sessions for both easy and hard cues indicates that these regions did not

correlate exclusively when the learning condition was inconsistent (hard cue

condition). However, it is thought provoking that several more correlations were

observed across anatomical regions in the more challenging conditions

(perception decision and declarative memory interference) compared with the

166

feedback control condition. A possible interpretation of this result is presented in

the discussion.

Stimulus Presentation & Feedback Presentation Period Whole Brain

Analyses: As described in the methods section, the stimulus presentation period

of the trial, when the probabilistic cue and natural scene were presented on the

screen, was examined in order to probe areas of the brain that were involved in

processing differences in cue difficulty and memory information (old and new

scenes). Additionally, the feedback period of the trial was examined in order to

probe for differences in the brain while processing feedback regarding the

probabilistic cues. However, no BOLD activation was observed during either of

the control conditions (feedback or perception decision) in any area of interest

(MTL, BG, midbrain) at a threshold of p<0.005, uncorrected during the stimulus

or the feedback presentation periods. Therefore, this hypothesis could not be

tested by examining functional activation in these regions during either the

stimulus presentation or the feedback presentation period and as a

consequence, no results are reported during these time periods.

5.4 Discussion

The purpose of the current experiment was to investigate interactions

between memory systems during feedback-based probabilistic learning

concurrent with declarative memory interference. Behavioral sensitivity (d’) as

well as accuracy measures revealed that participants successfully encoded

natural scenes (day 1) and subsequently remembered them the following day

167

while performing a feedback-based probabilistic cue learning task. Participants’

accuracy during the feedback-based learning task was significantly modulated by

the presence of the concurrent declarative memory task, such that performance

during the control sessions was greater than the declarative memory interference

session. In addition, behavioral differences in levels of cue difficulty (easy cue

accuracy > hard cue accuracy) were only observed in the feedback control

condition, suggesting that session type had a stronger influence in modulating

behavior. Neuroimaging results revealed that within independent regions of

interest, BOLD activity was also modulated by session type, perhaps tracking

these behavioral measures. Likewise, only one region, namely the ventral

striatum exhibited differences in BOLD responses according to cue difficulty in

the feedback control condition. No ROIs were modulated by cue difficulty in the

perception decision session, and in the declarative memory interference session

only the right midbrain’s BOLD signal was modulated by cue difficulty. The right

ventral striatum and right midbrain both displayed greater BOLD responses to

hard compared with easy cues, replicating a similar result observed in

experiment 2. Lastly, a series of Pearson’s correlations revealed widespread

positive correlations across ROIs in all learning sessions, with the most

correlations observed in the dual-tasking sessions. Interestingly, no negative

correlations were observed across these independently defined regions. The

significance of these results is postulated below.

Behavioral results indicated that participants successfully attended to the

presentation of natural scenes during the day 1 encoding session, as

168

demonstrated by the high hit rate (97%) on a perceptual detection task during

encoding. Furthermore, participants performed the perception decision task

(water detection) well, while simultaneously performing feedback-based

probabilistic cue learning on day 2. Behavioral sensitivity (d’) was significantly

worse on the water detection task on the second day, when learning the

probabilistic cues; however, a hit rate of 93% is still indicative of participants’

ability to successfully detect water in the new natural scenes despite having to

dual-task. Participants’ sensitivity at detecting old versus new natural scenes in

the declarative memory interference session was significantly worse compared

with the perception decision detection task (hit rate: 61% versus 93%), indicating

that the tasks were not equated in their difficulty. This result complicates

interpreting the neuroimaging results, as described below. However, participants

did discriminate between old versus new scenes at above chance levels,

suggesting that the task was not impossible to complete.

An examination of behavioral probabilistic cue accuracy revealed that, as

expected, participants were better at learning cue values in the feedback control

condition (83%) compared with the declarative memory interference condition

(63%). However, it was hypothesized that accuracy would scale according to

session type, such that feedback control > perception decision > declarative

memory interference. Contrary to this hypothesis however, no significant

difference was observed between participants’ cue accuracy in the feedback

control compared with the perception decision session (83% ≅ 78%; p = 0.12). It

appears that this form of dual-tasking did not significantly affect participants’

169

ability to learn the probabilistic cues. In fact, a correlation analysis conducted on

participants’ hit rate and probabilistic cue accuracy revealed a significant positive

relationship for probabilistic easy cues only, in the perception decision session.

No such positive relationship was observed for the hard cues. This result

indicates that participants’ ability to accurately perform these distinct tasks

simultaneously was related. A similar correlation performed for the declarative

memory interference session produced no significant relationships between

participants’ hit rate, false alarm rate, or cue accuracy, for neither easy nor hard

cues. This result may suggest that perhaps participants solved these distinct

tasks in different ways and that their ability to perform each task: old/new scene

discrimination versus probabilistic cue learning was unrelated.

The main effect of session type observed in the bilateral hippocampus,

ventral striatum, and midbrain ROIs is an interesting and unexpected finding.

These reward-related learning and memory regions displayed a global increase

in the mean BOLD signal in the declarative memory interference session

compared with the control sessions. Two theories were formulated to interpret

this finding. The first theory is that participants’ motivation to gain increased

monetary payment modulated the BOLD signal in key learning and memory

ROIs. It is possible that participants were motivated to perform well, especially in

the declarative memory interference condition, and as a result of this motivation,

ramping up of activation in these areas occurred during this session. In support

of this theory, Adcock and colleagues (2006) demonstrated modulated BOLD

signal in the exact regions examined in this experiment (ROIs, except for the

170

caudate nucleus, were drawn from this paper), in a declarative memory encoding

task. BOLD signal within these areas varied depending on the reward value

associated with to-be-learned-stimuli. In Adcock and colleagues study,

participants performed a declarative memory encoding task (viewed natural

scenes) while undergoing functional neuroimaging. Critically, the presentation of

each scene was preceded by a monetary value, either $5 or 10 cents which

indicated to the participant how much they would be paid if they subsequently

remembered the scene correctly on a follow up test. Interestingly, only high-

reward scenes ($5) that were subsequently remembered on a behavioral test

administered the next day elicited greater BOLD signal in the hippocampus,

ventral striatum, and midbrain. High-reward scenes that were subsequently

forgotten as well as low-reward scenes (that were both remembered and

forgotten) did not elicit increased engagement of these learning and memory

regions preceding encoding. Furthermore, greater correlation between the

hippocampus and the right midbrain (VTA) for high-reward compared with low-

reward scenes, and for high-reward scenes that were subsequently remembered

was observed. This data provides strong evidence that reward-related

motivation influences learning by specifically predicting the information which is

subsequently remembered. The authors concluded that their results are in

accord with the hypothesis that reward-motivated dopamine release in the

hippocampus prior to learning promotes memory formation (Lisman and Grace,

2005; Shohamy and Adcock, 2010).

171

In accord with Adcock et al.’s result, Wittman and colleagues (2005)

demonstrated that pictures which predicted monetary reward versus those that

did not were associated with greater BOLD activation in the substantia nigra.

Furthermore, on a test administered three weeks later, participants were more

accurate at remembering the pictures which predicted reward versus those that

did not. Additionally midbrain and hippocampal BOLD activation was greater for

subsequently recognized reward-predicting pictures versus those that were

forgotten. Therefore, this study also highlights a critical relationship between

reward-related motivation and learning and memory. An additional line of

research examining genetic polymorphisms in dopamine clearance pathways

among human individuals supports the hypothesis that dopamine is involved in

human memory formation (Meyer-Lindenberg et al., 2005; Schott et al., 2006). A

relatively recent study demonstrated differential degrees of engagement of

reward-related memory structures depending on individual polymorphisms in

dopamine clearance pathways (Schott et al., 2006). In this experiment,

polymorphisms in the dopamine transporter (DAT1) gene affected levels of

midbrain activation in an episodic encoding task. Additionally, catechol-O-methyl

transferase (COMT) polymorphisms modulated activity in the right prefrontal

cortex, such that participants with low COMT activity displayed stronger coupling

between the hippocampus and PFC during the encoding task. The authors

concluded that individual genetic variations in dopamine clearance pathways in

the brain effect both prefrontal cortex and midbrain BOLD activation in an

172

episodic encoding task, implicating a strong role for dopamine in human memory

formation.

Therefore based on the literature, it is possible that the engagement of

reward-related learning and memory structures was modulated by participants’

motivation to earn monetary compensation. Perhaps participants were highly

motivated to perform well in especially the declarative memory interference

condition as they were aware that their payment would be determined based on

how well they learned the probabilistic cues in all conditions and how accurately

they discriminated between old and new natural scenes. Unfortunately, no

questionnaire addressed this theory directly by asking participants how motivated

they were to perform in each session individually and is therefore a limitation of

this study. The potential role of dopamine, in particular, in influencing this global

BOLD signal increase in reward-related learning structures is only moderately

supported by the observation that a region of the dorsal striatum, namely the

caudate nucleus, did not display such a global increase in BOLD responses

during the declarative memory interference session. Given that midbrain

projections do not reach the dorsal components of the caudate nucleus (Devan

and White, 1999; Haber and Knutson, 2010), it is possible that a lack of

dopamine modulation in this area resulted in no differentiation of the BOLD signal

by learning session. However, this theory is impossible to test in the current

paradigm and therefore should be regarded with caution, especially given that

null findings in neuroimaging studies are not indicative of any particular finding.

This theory may represent an area of future studies, namely exploring differential

173

BOLD activation within regions of the brain that receive direct dopamine

modulation versus those that do not in reward-related learning and memory

tasks.

The second possible explanation for the observed increased BOLD

activation during the declarative memory interference session is that it was the

most cognitively demanding session. Because it was the hardest condition, as

demonstrated by behavioral sensitivity and accuracy measures, this session may

have recruited the greatest neural activation, eliciting the largest metabolic

demand, and causing the most substantial increase in BOLD responses in

reward-related learning structures. This explanation is in accord with one theory

regarding the nature of the BOLD signal (Heeger and Ress, 2002). Additionally,

the literature supports the notion that some brain regions do correlate with

increased task demand, in particular several prefrontal regions, including the

posterior dorsolateral prefrontal cortex BA44/46 (see Buckner and Wheeler, 2001

for review). However, such an increase was not observed in the caudate

nucleus, which may be an indication that this increased BOLD signal was not

uniform throughout the brain in the current paradigm. Given that the perceptual

decision and declarative memory interference sessions were not equated in their

difficulty makes it challenging to disentangle these two interpretations of the

neuroimaging results, and is a limitation of this study. If the two tasks had been

equated in difficulty, it would have strengthened the argument that learning and

memory regions are ramping up activation due to reward-related motivational

influences.

174

The results from the independent ROI analyses therefore, did not support

the hypothesis that the MTL and BG cooperate, as measured by simple

correlations, during inconsistent learning conditions (hard probabilistic cue

condition) and operate independently during more consistent learning conditions

(easy probabilistic cue condition). Examining the mean BOLD signal extracted

from these a priori regions of interest revealed that rather than tracking

differences in cue difficulty in individual sessions, all ROIs, with the exception of

the caudate nucleus, exhibited increased BOLD signal in the declarative memory

interference session compared with both the control sessions (perception

decision and feedback learning). Furthermore, an abundance of correlations

were observed across the ROIs for both the easy and hard cues in all learning

sessions. Therefore, while these data do not support the exclusivity of the

hypothesis generated (correlations for only inconsistent learning condition), the

data do not completely refute it either, as there were positive correlations across

regions for hard cues in the control sessions. Additionally, it is possible that in

the present task design, the declarative memory interference condition itself

functioned as the challenging learning condition, rather than the hard probabilistic

cue condition. Evidence supporting this theory comes from a post-experimental

questionnaire in which participants were asked to indicate which session was the

most difficult for them. The majority of participants indicated that the declarative

memory interference session was the hardest (14 participants), although others

indicated that the perceptual decision session was the most challenging (9

participants; 1 person stated that the feedback control session was the most

175

difficult). If in fact the declarative memory interference session was the most

difficult, then the data do somewhat support the theory that the MTL and BG

cooperate when the learning environment is challenging, as demonstrated by

worse behavioral performance in the declarative memory interference session as

well as an increased number of correlations in this session compared with the

feedback control condition. However, the lack of behavioral interference and

large number of positive correlations across regions in the perception decision

session make this theory weak.

To summarize, these results suggest that distinct learning and memory

regions are more globally engaged when simultaneously learning the value of

probabilistic cues via feedback and performing a declarative memory recognition

task than when performing less cognitively demanding tasks (control conditions).

Although the data in this experiment did not support the proposed hypothesis,

employing a behavioral “knockout” of the MTL did provide new insights into the

engagement and interactions of multiple memory systems (modulation of

behavior and BOLD activation across multiple memory regions, as well as the

number of positive correlations across regions, by learning session type) and

may be an area of further research in the future. In particular it is still unknown

under what conditions multiple memory systems compete versus cooperate

during learning.

176

Chapter Six: General Discussion

6.1 Purpose & Summary of Experiments

The purpose of the experiments performed in this dissertation was to

examine the engagement and interactions between multiple human memory

systems during probabilistic learning. Specifically the engagement and

interactions between the medial temporal lobe, with focus on the hippocampus,

and the basal ganglia, with emphasis on the striatum was examined during

feedback and observation probabilistic learning. The tasks employed

investigated this question in young healthy human participants using both within

and between-subjects, event-related designs which allowed for the examination

of BOLD activity within regions to discrete events across the dynamic learning

process. Multiple types of learning as well as variations in cue difficulty were

employed in order to probe how distinct brain substrates, namely the MTL and

BG, were modulated in multiple learning environments. The key hypothesis for

the experiments performed in this dissertation was that multiple memory

systems, specifically the MTL and BG, operate in parallel – they are online

simultaneously, engaged in learning and may operate independently by encoding

unique information or may compete or cooperate directly. Furthermore, it was

hypothesized that interactions between these anatomical regions may be

facilitated by dopaminergic midbrain activity, as investigated by the application of

reinforcement learning models which are postulated to represent a dopaminergic

learning signal in experiments 1 and 2.

177

Nature of representations in the MTL and BG: It is evident that the BG

and the MTL are involved in probabilistic learning as demonstrated by the results

in the current studies as well as an abundance of evidence in the literature.

However, important questions to consider include what is the nature of

representations in each of these regions? Is it the same or distinct? While these

are difficult questions to illuminate completely in the experiments utilized in this

dissertation, one may theorize about the representations encoded in the MTL

and BG during the feedback and observation tasks in the present experiments.

As the MTL is often implicated in encoding stimulus-stimulus associations and

episodic information (Gluck and Myers, 1993; Bunsey and Eichenbaum, 1995;

Eichenbaum and Bunsey, 1995) as well as recently implicated in tracking reward-

related information (Li et al., 2011), it is theorized that for the observation trials in

experiments 1 and 2, the MTL encoded the stimulus-stimulus associations of the

cue (stimulus 1) and the informative arrow (stimulus 2). Over the course of

learning, as these two stimuli are repeatedly paired (e.g., circle and upward-

facing arrow) the MTL may have encoded the value for the circle as higher than

5, binding these two associations together. For the feedback trials, since the trial

structure is distinct, it is possible that the MTL encoded either the episodic

experiences of incrementally learning the cue-outcome associations over time

(participant remembers previous episodes of receiving correct feedback for a

cue, which then informs the current cue-value choice) or that the hippocampus

was tracking the reward value of the probabilistic cues (differentiates between

correct and incorrect feedback; consistent with Li and colleagues 2011). The

178

basal ganglia however, may have encoded distinct representations during both

feedback and observation learning trials than the MTL. For the feedback trials,

one hypothesis is that the basal ganglia, in particular the striatum, was involved

in feedback learning tracking goal outcomes and updating cue contingencies via

error correction, consistent with the literature. During observation learning, the

striatum may have been involved in action selection as well as encoding cue-

arrow associations, in accord with Seger’s theory of the role of the BG during

category learning (2008).

One manner of testing the above hypotheses and examining whether or

not the MTL and BG were encoding similar or distinct information may be to

examine their ability to flexibly use previously acquired cue-outcome associations

in a novel manner. It has been suggested that the hippocampus is involved in

flexibly using knowledge in novel scenarios whereas the BG is not (Myers et al.,

2003). Therefore, a probe test could have been administered for both trial types

in order to determine in if a new context, where participants would need to

flexibly use previously acquired cue-value information in a novel manner, the

hippocampus would be engaged in the flexible usage of the knowledge, while the

BG would not. An alternative idea would be to track the BOLD responses over

the learning phase in a trial by trial basis. BOLD responses in the striatum may

initially have been driven by responses to the feedback period, but as learning

progressed it may have shifted to the cue period. Examining these phases (cue

versus feedback) distinctly could have allowed for decoupling of such activation.

However, the task was not designed to address this question and there was no

179

inter-stimulus-interval placed between the cue and feedback periods. As a

consequence, examining the cue and feedback periods separately would be

challenging and specifically the results obtained by examining the feedback

period may have been skewed by the BOLD responses in the cue period.

Some evidence supporting the theory that the MTL may have been

encoding stimulus-stimulus associations was observed in the BOLD responses to

consistent versus inconsistent information. In both experiments 1 and 2, the

striatum (caudate nucleus and putamen) was not modulated by cue consistency,

whereas the MTL (hippocampus and parahippocampus) was modulated by the

consistency of the observation cues. In both studies, the MTL exhibited greater

BOLD responses (sometimes manifested as the least negative BOLD responses)

to inconsistent compared with consistent cue values, perhaps encoding a

mismatch signal between the expected (e.g., circle paired with upward-facing

arrow) and actual (e.g., circle paired with downward-facing arrow) cue-values

observed during learning. It is interesting that only the MTL regions were

modulated by cue consistency, and not the striatum, suggesting that perhaps

these areas may have encoded distinct representations of cue values, and that

the MTL was involved in binding stimulus-stimulus associations and encoding

mismatch signals, consistent with the literature (Gluck and Myers, 1993; Bunsey

and Eichenbaum, 1995; Eichenbaum and Bunsey, 1995; Ploghaus et al., 2000;

Chen et al., 2011).

The main findings from experiments 1-3 support the hypothesis that the

MTL and BG are engaged simultaneously during feedback (experiments 1-3) and

180

observation (experiments 1 and 2) probabilistic learning and operate in parallel,

by at times exhibiting parallel processing, and at times interacting directly via

cooperation or competition. Evidence supporting each manner of interaction is

summarized below:

Evidence of parallel engagement: Parallel processing for the purpose of

this dissertation is defined as the engagement of multiple memory regions in the

same learning session, while displaying unique patterns of BOLD activation.

This result suggests that both areas have the capability of being engaged

simultaneously during the learning process, but are encoding unique information,

in accord with White and McDonald’s theory (2002). The examination of the

brain substrates involved in feedback versus observation learning revealed areas

of the ventral striatum which exhibited greater BOLD signal to feedback learning

trials than observation learning trials in experiment 1 (as shown by a main effect

of learning type in the learning phase). A similar effect was observed in

experiment 2, in which regions of the ventral striatum and midbrain exhibited

stronger engagement during feedback than observation learning trials (as shown

by an interaction of learning type by group in the update phase). Interestingly,

only regions within the basal ganglia were modified by a main effect of learning

type, while no areas were identified in the MTL exhibiting such a differentiation.

Several neuroimaging papers have shown that feedback and reward processing

engages the ventral striatum (for review see Delgado, 2007), thus, it is not

surprising that this area was recruited more strongly during feedback learning,

which contained outcome-relevant information. It may have been expected that

181

the MTL would be selectively modulated during observation learning, as

previously reported (Poldrack et al., 2001); however this effect was not observed.

While a null result in neuroimaging is not indicative of any particular finding and

the details of the paradigms employed in this dissertation differ from previous

probabilistic learning studies, it is possible that the MTL BOLD signals observed

in experiments 1 and 2 indicated mutual activation during both feedback and

observation learning – as suggested by the main effect of cue difficulty analyses.

Therefore, this result suggests that at times, memory systems may operate

independently and encode unique information.

In further support of parallel engagement during learning, Granger

causality analyses, which allowed for an investigation of both functional and

effective connectivity across the brain during feedback and observation learning

distinctly (experiment 1) and concurrently (experiment 2), revealed recruitment of

a common network of neural substrates during learning. Results from both

experiments supported the engagement of a similar network of regions, including

the striatum, medial temporal lobe, and midbrain. In experiment 1, using the

hippocampus as the seed region revealed functional connectivity with the

caudate nucleus, while using the caudate nucleus as a seed region revealed

connectivity with the hippocampus. In experiment 2, using the midbrain as a

reference region elicited functional connectivity with a larger network of regions

including the caudate nucleus/putamen, globus pallidus, hippocampus, and

parahippocampus. Furthermore, effective connectivity results implicated a

potential unilateral flow of information whereby the putamen sent directed

182

influences to the midbrain (VTA) and the midbrain sent directed influences to the

hippocampus and parahippocampus. This observation that key learning and

memory regions were functionally and effectively connected during probabilistic

learning is strengthened by the fact that three reference regions employed in the

analyses: the caudate nucleus, the hippocampus, and midbrain, all produced

predominately converging results. The theoretical implications of these results

suggest that rather than the MTL, BG, and midbrain functioning independently

during learning, instead dynamic interactions may be occurring between these

areas such that they influence each other during both initial acquisition and

reversal of probabilistic information. Effective connectivity results suggest that

this interaction may be occurring via connectivity with midbrain dopaminergic

areas.

Evidence of cooperative interactions: Cooperative interactions for the

purpose of this dissertation are defined as the exhibition of similar patterns of

BOLD signal in both the MTL and BG during learning as well as positive

correlations across these distinct regions. In experiment 1, a region of the

caudate nucleus and hippocampus both displayed greater BOLD signal to easy

compared with hard probabilistic cues for feedback and observation trials. In

experiment 2, one region of the basal ganglia, the putamen, as well as multiple

areas of the MTL, including the hippocampus and parahippocampus, exhibited

greater BOLD signal in response to both deterministic and easy compared with

hard cues. This result is in agreement with studies in the literature which have

also documented similar patterns of activity in the MTL and BG during simple

183

visuomotor learning tasks (Toni et al., 2001; Amso et al., 2005; Law et al., 2005;

Haruno and Kawato, 2006) as well as information integration category learning

tasks (Cincotta and Seger, 2007). Furthermore, multiple positive correlations

across distinct regions (MTL, BG, and midbrain) were observed in all three

experiments. In fact negative correlations, which have previously been used as

evidence of competition (Poldrack et al., 2001; Seger and Cincotta, 2006), were

never observed across these distinct learning and memory structures, even when

the BOLD signal displayed opposite patterns of activity across regions

(experiment 2) as well as when the BOLD signal was investigated using

independent regions of interest (experiment 3). Lastly, both loci within the MTL

and BG were involved in encoding a feedback-based prediction error signal in

experiments 1 and 2. This result suggests three things, one that a feedback

learning signal is encoded in the MTL, in addition to the basal ganglia, which has

previously been shown in the literature (O'Doherty et al., 2004; Tanaka et al.,

2004; Pessiglione et al., 2006); second that distinct memory regions may at times

encode the same learning signal; and third, that dopamine may be facilitating

interactions among discrete regions during learning in accord with the effective

connectivity results observed in this dissertation as well as the theory that

dopamine modulates reward-related learning (Lisman and Grace, 2005;

Shohamy and Adcock, 2010).

Evidence of competitive interactions: Competitive interactions in this

thesis are defined as the exhibition of opposing patterns of BOLD signal in both

the MTL and BG during learning and negative correlations across distinct

184

regions. In experiment 2, multiple regions of the basal ganglia, including the

ventral striatum, caudate nucleus, and putamen, as well as multiple areas of the

MTL, including the hippocampus and parahippocampus, exhibited opposing

patterns of activation in response to varying levels of probabilistic cue difficulty.

Specifically, the MTL displayed greater responses to easy than hard cues,

whereas the loci within the basal ganglia exhibited greater activation in response

to hard than easy cues. Opposing patterns of BOLD signal across distinct brain

regions has been interpreted as competitive interactions in the literature

(Poldrack et al., 2001; Foerde et al., 2006).

6.2 Limitations

Perhaps the largest limitation of this dissertation is that it is very difficult to

examine interactions between multiple memory systems using functional

magnetic resonance imaging. While this question is an active area of research in

the functional neuroimaging field, it is one that contains rather significant

challenges. The primary concern is that using BOLD as a dependent measure, it

is difficult to examine if distinct regions are actually functionally interacting.

Simply because the pattern of BOLD responses observed in discrete regions is

similar or distinct, does not necessarily indicate that the regions are interacting.

Therefore, the definitions of cooperative and competitive interactions during

learning used in this dissertation, while consistent with the literature, should be

considered with some amount of caution. Specifically because of this limitation,

more sophisticated analyses were employed including a Granger causality

185

analysis as well as a prediction error analysis in order to explore in a more in

depth manner how distinct memory systems interact during learning.

The major differences between the feedback and observation versions of

tasks employed in this dissertation were outlined in the materials and methods

sections of experiments 1 and 2. Despite their differences, however, the two

learning sessions share the common goal of learning the value of probabilistic

cues. As such, participants may have engaged in a variety of cognitive

strategies in order to successfully complete the task. For example, as learning

progressed over time in the feedback session it is possible that during the

stimulus presentation period participants may have employed a more declarative-

based cued-recall strategy. Participants may have also engaged in verbal

rehearsal strategies, irrespective of the task version, during the learning phase.

Research investigating how participants solve a distinct probabilistic learning

task, the WPT, may provide insight into possible declarative and nondeclarative

components of category learning in addition to exploring the declarative

knowledge that participants posses during these types of learning tasks (Gluck et

al., 2002; Meeter et al., 2006). Newell and colleagues (2007) recently reported

that participants had comparable declarative knowledge on a feedback and

observation version of the WPT. As a result, the authors suggested that the

feedback version of the WPT may not be an exclusively nondeclarative task.

Corroborating this theory, Meeter and colleagues (2008) have suggested that

participants may solve the WPT via several distinct strategies including

engagement of rule-learning, incremental learning (both of which are thought to

186

engage the BG), memorization techniques (MTL dependent), or some

combination of these three strategies. Furthermore, Shohamy and colleagues

(2004b) suggest that in order to solve probabilistic categorization tasks

participants most likely recruit multiple parallel learning systems. Therefore, it is

quite possible that both the observation and feedback versions of the tasks

employed in experiments 1 and 2 contain some declarative and nondeclarative

components. As a result, the use of the terms declarative and nondeclarative in

this context are meant as a reference, and are not meant to indicate the sole

manner in which participants may solve the tasks.

The possibility that the tasks used in experiments 1 and 2 contain

elements of declarative and nondeclarative learning may have contributed to the

observation that the hippocampus and striatum were involved in both observation

and feedback learning – primarily modified by cue difficulty rather than learning

type. The involvement of multiple cognitive operations (e.g., cued recall or

rehearsal strategies) therefore, may have facilitated the mutual engagement of

these regions. One limitation of the studies employed in this dissertation is that

this possibility cannot be eliminated definitively. Future studies may be better

able to parse out the distinct contributions that multiple cognitive processes may

have on these tasks and the subsequent BOLD signals observed in the MTL and

BG.

In addition, some limitations of the prediction error analyses employed in

experiments 1 and 2 exist. Namely, it is difficult to assess at which time point in

the trial (given the task designs – specifically no jittered ISI between the cue and

187

feedback presentation periods) the individual correlations with the PE signals

occurred within the MTL and BG. Further, it is possible that the trial structure

employed did not allow separation of the prediction error signal from an

uncertainty signal (between cue and outcome) that has been shown to be linked

with local field potentials in the anterior hippocampus (Vanni-Mercier et al.,

2009). However, PE signals carry valence information (i.e., an unexpected

negative outcome produces a negative PE signal), while a saliency, novelty, or

uncertainty signal may be positive irrespective of the valence of the outcome

(more akin to the action prediction error used for the observation trials in

experiment 2). The finding of feedback trial PE signals in the MTL is not

commonly reported in the literature and may point to an area of future research

investigating the nature of these signals in the MTL and potential interactions

with the striatum that may underlie parallel processing in these distinct regions

during probabilistic learning.

6.3 Conclusions & Future Directions

To conclude, the set of experiments performed in this dissertation support

the model of parallel processing of the medial temporal lobe and basal ganglia

during probabilistic learning. However, several questions remain unaddressed.

For example, it is unknown under what circumstances memory systems

cooperate versus compete. Experiment 3 of this dissertation was designed to

address this question, but the results were somewhat inconclusive. A second

open question is probing how exactly dopamine modulates motivated learning in

188

the hippocampus and other learning structures (for review see Shohamy and

Adcock, 2010). Lastly, other neurotransmitters are involved in learning and

memory functions, in addition to dopamine, for example acetylcholine (Hasselmo,

2006). The role that multiple neurotransmitters play during human probabilistic

learning also remains to be fully characterized. These are key questions which

will help to illustrate how two major learning and memory systems interact, which

has implications not only for healthy humans, but may also impact our

understanding and treatment of several neuropsychiatric disorders that target

these structures, including Parkinson’s disease, schizophrenia, and MTL

amnesia.

189

References

Abler B, Walter H, Erk S, Kammerer H, Spitzer M (2006) Prediction error as a

linear function of reward probability is coded in human nucleus accumbens. Neuroimage 31:790-795.

Adcock RA, Thangavel A, Whitfield-Gabrieli S, Knutson B, Gabrieli JD (2006) Reward-motivated learning: mesolimbic activation precedes memory formation. Neuron 50:507-517.

Albouy G, Sterpenich V, Balteau E, Vandewalle G, Desseilles M, Dang-Vu T, Darsaud A, Ruby P, Luppi PH, Degueldre C, Peigneux P, Luxen A, Maquet P (2008) Both the hippocampus and striatum are involved in consolidation of motor sequence memory. Neuron 58:261-272.

Amaral DG (1993) Emerging principles of intrinsic hippocampal organization. Curr Opin Neurobiol 3:225-229.

Amso D, Davidson MC, Johnson SP, Glover G, Casey BJ (2005) Contributions of the hippocampus and the striatum to simple association and frequency-based learning. Neuroimage 27:291-298.

Andersen P, Bliss, T.V.P., Skrede, K.K. (1971) Lamellar organization of hippocampal excitatory pathways. Exp Brain Res 13:222-238.

Annett LE, McGregor A, Robbins TW (1989) The effects of ibotenic acid lesions of the nucleus accumbens on spatial learning and extinction in the rat. Behav Brain Res 31:231-242.

Aron AR, Shohamy D, Clark J, Myers C, Gluck MA, Poldrack RA (2004) Human midbrain sensitivity to cognitive feedback and uncertainty during classification learning. J Neurophysiol 92:1144-1152.

Ashby FG, Spiering BJ (2004) The Neurobiology of Category Learning. Behavioral and Cognitive Neuroscience Reviews.3:pp.

190

Ashby FG, Maddox WT, Bohil CJ (2002) Observational versus feedback training in rule-based and information-integration category learning. Mem Cognit 30:666-677.

Atallah HE, Rudy JW, O'Reilly RC (2008) The role of the dorsal striatum and dorsal hippocampus in probabilistic and deterministic odor discrimination tasks. Learn Mem 15:294-298.

Atallah HE, Lopez-Paniagua D, Rudy JW, O'Reilly RC (2007) Separate neural substrates for skill learning and performance in the ventral and dorsal striatum. Nat Neurosci 10:126-131.

Ballard IC, Murty VP, Carter RM, MacInnes JJ, Huettel SA, Adcock RA (2011) Dorsolateral prefrontal cortex drives mesolimbic dopaminergic regions to initiate motivated behavior. J Neurosci 31:10340-10346.

Balleine BW, Delgado MR, Hikosaka O (2007) The role of the dorsal striatum in reward and decision-making. J Neurosci 27:8161-8165.

Belova MA, Paton JJ, Morrison SE, Salzman CD (2007) Expectation modulates neural responses to pleasant and aversive stimuli in primate amygdala. Neuron 55:970-984.

Beninger RJ, Wasserman J, Zanibbi K, Charbonneau D, Mangels J, Beninger BV (2003) Typical and atypical antipsychotic medications differentially affect two nondeclarative memory tasks in schizophrenic patients: A double dissociation. Schizophrenia Research.61:pp.

Berger TW, and Orr, W.B. (1983) Hippocampectomy selectively disrupts discrimination reversal conditioning of the rabbit nictitating membrane response. Behavioral Brain Research 8:49-68.

Bliss TV, Collingridge GL (1993) A synaptic model of memory: Long-term potentiation in the hippocampus. Jan 1993. Nature.361:pp.

Bolam JP, Hanley JJ, Booth PA, Bevan MD (2000) Synaptic organisation of the basal ganglia. J Anat 196 ( Pt 4):527-542.

191

Bray S, O'Doherty J (2007) Neural coding of reward-prediction error signals during classical conditioning with attractive faces. J Neurophysiol 97:3036-3045.

Brewer JB, Zhao Z, Desmond JE, Glover GH, Gabrieli JD (1998) Making memories: brain activity that predicts how well visual experience will be remembered. Science 281:1185-1187.

Brog JS, Salyapongse A, Deutch AY, Zahm DS (1993) The patterns of afferent innervation of the core and shell in the "accumbens" part of the rat ventral striatum: immunohistochemical detection of retrogradely transported fluoro-gold. J Comp Neurol 338:255-278.

Brown RM, Robertson EM (2007) Off-line processing: Reciprocal interactions between declarative and procedural memories. Journal of Neuroscience 27:10468-10475.

Buckner RL, Wheeler ME (2001) The cognitive neuroscience of remembering. Nat Rev Neurosci 2:624-634.

Bunsey M, Eichenbaum H (1995) Selective damage to the hippocampal region blocks long-term retention of a natural and nonspatial stimulus-stimulus association. Hippocampus 5:546-556.

Burke CJ, Tobler PN, Baddeley M, Schultz W (2010) Neural mechanisms of observational learning. Proc Natl Acad Sci U S A 107:14431-14436.

Burwell RD, Witter MP, Amaral DG (1995) Perirhinal and postrhinal cortices of the rat: a review of the neuroanatomical literature and comparison with findings from the monkey brain. Hippocampus 5:390-408.

Carrillo MC, Gabrieli JD, Hopkins RO, McGlinchey-Berroth R, Fortier CB, Kesner RP, Disterhoft JF (2001) Spared discrimination and impaired reversal eyeblink conditioning in patients with temporal lobe amnesia. Behav Neurosci 115:1171-1179.

Carter RM, Macinnes JJ, Huettel SA, Adcock RA (2009) Activation in the VTA and nucleus accumbens increases in anticipation of both gains and losses. Front Behav Neurosci 3:21.

192

Chen J, Olsen RK, Preston AR, Glover GH, Wagner AD (2011) Associative retrieval processes in the human medial temporal lobe: Hippocampal retrieval success and CA1 mismatch detection. Learn Mem 18:523-528.

Cincotta CM, Seger CA (2007) Dissociation between striatal regions while learning to categorize via feedback and via observation. J Cogn Neurosci 19:249-265.

Cools R (2006) Dopaminergic modulation of cognitive function-implications for L-DOPA treatment in Parkinson's disease. Neurosci Biobehav Rev 30:1-23.

Cools R, Clark L, Owen AM, Robbins TW (2002) Defining the neural mechanisms of probabilistic reversal learning using event-related functional magnetic resonance imaging. J Neurosci 22:4563-4567.

Corkin S, Sullivan EV, Carr FA (1984) Prognostic factors for life expectancy after penetrating head injury. Arch Neurol 41:975-977.

Dagher A, Owen AM, Boecker H, Brooks DJ (2001) The role of the striatum and hippocampus in planning: a PET activation study in Parkinson's disease. Brain 124:1020-1032.

Davachi L, Wagner AD (2002) Hippocampal contributions to episodic encoding: insights from relational and item-based learning. J Neurophysiol 88:982-990.

Delgado MR (2007) Reward-related responses in the human striatum. Ann N Y Acad Sci 1104:70-88.

Delgado MR, Locke HM, Stenger VA, Fiez JA (2003) Dorsal striatum responses to reward and punishment: effects of valence and magnitude manipulations. Cogn Affect Behav Neurosci 3:27-38.

Delgado MR, Miller MM, Inati S, Phelps EA (2005) An fMRI study of reward-related probability learning. Neuroimage 24:862-873.

Delgado MR, Nystrom LE, Fissell C, Noll DC, Fiez JA (2000) Tracking the hemodynamic responses to reward and punishment in the striatum. J Neurophysiol 84:3072-3077.

193

Devan BD, White NM (1999) Parallel information processing in the dorsal striatum: Relation to hippocampal function. Apr 1999. The Journal of Neuroscience.19:pp.

Djonlagic I, Rosenfeld A, Shohamy D, Myers C, Gluck M, Stickgold R (2009) Sleep enhances category learning. Learn Mem 16:751-755.

Doeller CF, King JA, Burgess N (2008) Parallel striatal and hippocampal systems for landmarks and boundaries in spatial memory. Proc Natl Acad Sci U S A 105:5915-5920.

Doll BB, Jacobs WJ, Sanfey AG, Frank MJ (2009) Instructional control of reinforcement learning: a behavioral and neurocomputational investigation. Brain Res 1299:74-94.

Domesick VB (1969) Projections from the cingulate cortex in the rat. Brain Res 12:296-320.

Eichenbaum H, Bunsey M (1995) On the binding of associations in memory: Clues from studies on the role of the hippocampal region in paired-associate learning. Feb 1995. Current Directions in Psychological Science.4:pp.

Eldridge LL, Knowlton BJ, Furmanski CS, Bookheimer SY, Engel SA (2000) Remembering episodes: a selective role for the hippocampus during retrieval. Nat Neurosci 3:1149-1152.

Fagan AM, Olton DS (1986) Learning sets, discrimination reversal, and hippocampal function. Behav Brain Res 21:13-20.

Foerde K, Knowlton BJ, Poldrack RA (2006) Modulation of competing memory systems by distraction. Proc Natl Acad Sci U S A 103:11778-11783.

Foerde K, Poldrack RA, Knowlton BJ (2007) Secondary-task effects on classification learning. Mem Cognit 35:864-874.

Forman SD, Cohen JD, Fitzgerald M, Eddy WF, Mintun MA, Noll DC (1995) Improved assessment of significant activation in functional magnetic

194

resonance imaging (fMRI): use of a cluster-size threshold. Magn Reson Med 33:636-647.

Frank MJ (2005) Dynamic dopamine modulation in the basal ganglia: a neurocomputational account of cognitive deficits in medicated and nonmedicated Parkinsonism. J Cogn Neurosci 17:51-72.

Frank MJ, Claus ED (2006) Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. Psychol Rev 113:300-326.

Frank MJ, Seeberger LC, O'Reilly R C (2004) By carrot or by stick: cognitive reinforcement learning in parkinsonism. Science 306:1940-1943.

Frey U, Schroeder H, Matthies H (1990) Dopaminergic antagonists prevent long-term maintenance of posttetanic LTP in the CA1 region of rat hippocampal slices. Brain Res 522:69-75.

Frey U, Matthies H, Reymann KG (1991) The effect of dopaminergic D1 receptor blockade during tetanization on the expression of long-term potentiation in the rat CA1 region in vitro. Neurosci Lett 129:111-114.

Gerfen CR (1984) The neostriatal mosaic: compartmentalization of corticostriatal input and striatonigral output systems. Nature 311:461-464.

Geweke J (1982) Measurement of Linear Dependence and Feedback between Multiple Time Series. J Am Stat Assoc 77:304-313.

Gluck MA, Bower GH (1988) From conditioning to category learning: an adaptive network model. J Exp Psychol Gen 117:227-247.

Gluck MA, Myers CE (1993) Hippocampal mediation of stimulus representation: a computational theory. Hippocampus 3:491-516.

Gluck MA, Shohamy D, Myers C (2002) How do people solve the "weather prediction" task?: individual variability in strategies for probabilistic category learning. Learn Mem 9:408-418.

195

Gluck MA, Meeter M, Myers CE (2003) Computational models of the hippocampal region: linking incremental learning and episodic memory. Trends Cogn Sci 7:269-276.

Goebel R, Esposito F, Formisano E (2006) Analysis of functional image analysis contest (FIAC) data with brainvoyager QX: From single-subject to cortically aligned group general linear model analysis and self-organizing group independent component analysis. Hum Brain Mapp 27:392-401.

Goebel R, Roebroeck A, Kim DS, Formisano E (2003) Investigating directed cortical interactions in time-resolved fMRI data using vector autoregressive modeling and Granger causality mapping. Magn Reson Imaging 21:1251-1261.

Gotham AM, Brown RG, Marsden CD (1988) 'Frontal' cognitive function in patients with Parkinson's disease 'on' and 'off' levodopa. Brain 111 ( Pt 2):299-321.

Groenewegen HJ, Vermeulen-Van der Zee E, te Kortschot A, Witter MP (1987) Organization of the projections from the subiculum to the ventral striatum in the rat. A study using anterograde transport of Phaseolus vulgaris leucoagglutinin. Neuroscience 23:103-120.

Haber SN (2003) The primate basal ganglia: parallel and integrative networks. J Chem Neuroanat 26:317-330.

Haber SN, Knutson B (2010) The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology 35:4-26.

Hampson RE, Pons TP, Stanford TR, Deadwyler SA (2004) Categorization in the monkey hippocampus: a possible mechanism for encoding information into memory. Proc Natl Acad Sci U S A 101:3184-3189.

Haruno M, Kawato M (2006) Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning. J Neurophysiol 95:948-959.

Hasselmo ME (2006) The role of acetylcholine in learning and memory. Curr Opin Neurobiol 16:710-715.

196

Heeger DJ, Ress D (2002) What does fMRI tell us about neuronal activity? Nat Rev Neurosci 3:142-151.

Heimer L, Alheid GF, de Olmos JS, Groenewegen HJ, Haber SN, Harlan RE, Zahm DS (1997) The accumbens: beyond the core-shell dichotomy. J Neuropsychiatry Clin Neurosci 9:354-381.

Holm S (1979) A simple sequentially rejective multiple test procedure. Scand J Stat 6:65-70.

Hopkins RO, Myers CE, Shohamy D, Grossman S, Gluck M (2004) Impaired probabilistic category learning in hypoxic subjects with hippocampal damage. Neuropsychologia 42:524-535.

Jay TM, Glowinski J, Thierry AM (1989) Selectivity of the hippocampal projection to the prelimbic area of the prefrontal cortex in the rat. Brain Res 505:337-340.

Joel D, Niv Y, Ruppin E (2002) Actor-critic models of basal ganglia: New anatomical and computational perspectives. Neural Networks.15:pp.

Jung Y, Hong S, Haber SN (2003) Organization of Direct Hippocampal Projections to the Different Regions of the Ventral Striatum in primate. The Korean J Anat 36:67-76.

Kelley AE, Domesick VB (1982) The distribution of the projection from the hippocampal formation to the nucleus accumbens in the rat: an anterograde- and retrograde-horseradish peroxidase study. Neuroscience 7:2321-2335.

Knowlton BJ, Squire LR, Gluck MA (1994) Probabilistic classification learning in amnesia. Learn Mem 1:106-120.

Knowlton BJ, Mangels JA, Squire LR (1996) A neostriatal habit learning system in humans. Science 273:1399-1402.

Knutson B, Adams CM, Fong GW, Hommer D (2001) Anticipation of increasing monetary reward selectively recruits nucleus accumbens. The Journal of Neuroscience.21:pp.

197

Kolb B, Buhrmann K, McDonald R, Sutherland RJ (1994) Dissociation of the medial prefrontal, posterior parietal, and posterior temporal cortex for spatial navigation and recognition memory in the rat. Cereb Cortex 4:664-680.

Krayniak PF, Meibach RC, Siegel A (1981) A projection from the entorhinal cortex to the nucleus accumbens in the rat. Brain Res 209:427-431.

Lagnado DA, Newell BR, Kahan S, Shanks DR (2006) Insight and strategy in multiple-cue learning. J Exp Psychol Gen 135:162-183.

Law JR, Flanery MA, Wirth S, Yanike M, Smith AC, Frank LM, Suzuki WA, Brown EN, Stark CE (2005) Functional magnetic resonance imaging activity during the gradual acquisition and expression of paired-associate memory. J Neurosci 25:5720-5729.

Lee AS, Duman RS, Pittenger C (2008) A double dissociation revealing bidirectional competition between striatum and hippocampus during learning. Proc Natl Acad Sci U S A 105:17163-17168.

Li J, Delgado MR, Phelps EA (2011) How instructed knowledge modulates the neural systems of reward learning. Proc Natl Acad Sci U S A 108:55-60.

Li S, Cullen WK, Anwyl R, Rowan MJ (2003) Dopamine-dependent facilitation of LTP induction in hippocampal CA1 by exposure to spatial novelty. Nature Neuroscience.6:pp.

Lisman JE, Grace AA (2005) The hippocampal-VTA loop: controlling the entry of information into long-term memory. Neuron 46:703-713.

Liu P, Bilkey DK (1996) Direct connection between perirhinal cortex and hippocampus is a major constituent of the lateral perforant path. Hippocampus 6:125-135.

Lynd-Balta E, Haber SN (1994) The organization of midbrain projections to the ventral striatum in the primate. Neuroscience 59:609-623.

Macmillan NA, Creelman CD (1991) Detection Theory: A User's Guide. New York: Cambridge University Press.

198

Marco-Pallares J, Muller SV, Munte TF (2007) Learning by doing: an fMRI study of feedback-related brain activations. Neuroreport 18:1423-1426.

Marston HM, Everitt BJ, Robbins TW (1993) Comparative effects of excitotoxic lesions of the hippocampus and septum/diagonal band on conditional visual discrimination and spatial learning. Neuropsychologia 31:1099-1118.

Mattfeld AT, Stark CE (2010) Striatal and Medial Temporal Lobe Functional Interactions during Visuomotor Associative Learning. Cereb Cortex.

McClure SM, Berns GS, Montague PR (2003) Temporal prediction errors in a passive learning task activate human striatum. Neuron 38:339-346.

McDonald RJ, White NM (1993) A triple dissociation of memory systems: hippocampus, amygdala, and dorsal striatum. Behav Neurosci 107:3-22.

McDonald RJ, White NM (1994) Parallel information processing in the water maze: evidence for independent memory systems involving dorsal striatum and hippocampus. Behav Neural Biol 61:260-270.

McDonald RJ, White NM (1995) Hippocampal and nonhippocampal contributions to place learning in rats. Behavioral Neuroscience.109:pp.

McGeorge AJ, Faull RL (1989) The organization of the projection from the cerebral cortex to the striatum in the rat. Neuroscience 29:503-537.

Meeter M, Myers CE, Shohamy D, Hopkins RO, Gluck MA (2006) Strategies in probabilistic categorization: results from a new way of analyzing performance. Learn Mem 13:230-239.

Meeter M, Radics G, Myers CE, Gluck MA, Hopkins RO (2008) Probabilistic categorization: how do normal participants and amnesic patients do it? Neurosci Biobehav Rev 32:237-248.

Meyer-Lindenberg A, Kohn PD, Kolachana B, Kippenhan S, McInerney-Leo A, Nussbaum R, Weinberger DR, Berman KF (2005) Midbrain dopamine and

199

prefrontal function in humans: interaction and modulation by COMT genotype. Nat Neurosci 8:594-596.

Middleton FA, Strick PL (2000) Basal ganglia output and cognition: evidence from anatomical, behavioral, and clinical studies. Brain Cogn 42:183-200.

Mitchell JA, Hall G (1988) Learning in rats with caudate-putamen lesions: unimpaired classical conditioning and beneficial effects of redundant stimulus cues on instrumental and spatial learning deficits. Behav Neurosci 102:504-514.

Myers CE, Deluca J, Hopkins RO, Gluck MA (2006) Conditional discrimination and reversal in amnesia subsequent to hypoxic brain injury or anterior communicating artery aneurysm rupture. Neuropsychologia 44:130-139.

Myers CE, Shohamy D, Gluck MA, Grossman S, Kluger A, Ferris S, Golomb J, Schnirman G, Schwartz R (2003) Dissociating hippocampal versus basal ganglia contributions to learning and transfer. J Cogn Neurosci 15:185-193.

Myers CE, Hopkins, R.O., Kesner, R.P, Monti, L., Gluck, M.A. (2000) Conditional spatial discrimination in humans with hypoxic brain injury. Psychobiology 28:275-282.

Nagy H, Keri S, Myers CE, Benedek G, Shohamy D, Gluck MA (2007) Cognitive sequence learning in Parkinson's disease and amnestic mild cognitive impairment: Dissociation between sequential and non-sequential learning of associations. Neuropsychologia 45:1386-1392.

Newell BR, Lagnado DA, Shanks DR (2007) Challenging the role of implicit processes in probabilistic category learning. Psychon Bull Rev 14:505-511.

O'Doherty J, Dayan P, Schultz J, Deichmann R, Friston K, Dolan RJ (2004) Dissociable roles of ventral and dorsal striatum in instrumental conditioning. Science 304:452-454.

O'Doherty JP (2004) Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr Opin Neurobiol 14:769-776.

200

O'Doherty JP, Deichmann R, Critchley HD, Dolan RJ (2002) Neural responses during anticipation of a primary taste reward. Neuron 33:815-826.

O'Reilly R, C., Munakata Y (2000) Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain. Cambridge, MA: MIT Press.

Olmos A, Kingdom FA (2004) A biologically inspired algorithm for the recovery of shading and reflectance images. Perception 33:1463-1473.

Packard MG (1999) Glutamate infused posttraining into the hippocampus or caudate-putamen differentially strengthens place and response learning. Proc Natl Acad Sci U S A 96:12881-12886.

Packard MG, White NM (1989) Memory facilitation produced by dopamine agonists: Role of receptor subtype and mnemonic requirements. Jul 1989. Pharmacology, Biochemistry and Behavior.33:pp.

Packard MG, White NM (1991) Dissociation of hippocampus and caudate nucleus memory systems by posttraining intracerebral injection of dopamine agonists. Behav Neurosci 105:295-306.

Packard MG, McGaugh JL (1992) Double dissociation of fornix and caudate nucleus lesions on acquisition of two water maze tasks: Further evidence for multiple memory systems. Behavioral Neuroscience.106: 439-446.

Packard MG, McGaugh JL (1994) Quinpirole and d-amphetamine administration posttraining enhances memory on spatial and cued discriminations in a water maze. Mar 1994. Psychobiology.22:pp.

Packard MG, McGaugh JL (1996) Inactivation of hippocampus or caudate nucleus with lidocaine differentially affects expression of place and response learning. Neurobiol Learn Mem 65:65-72.

Packard MG, Knowlton BJ (2002) Learning and memory functions of the Basal Ganglia. Annu Rev Neurosci 25:563-593.

201

Packard MG, Hirsh R, White NM (1989) Differential effects of fornix and caudate nucleus lesions on two radial maze tasks: evidence for multiple memory systems. J Neurosci 9:1465-1472.

Pagnoni G, Zink CF, Montague PR, Berns GS (2002) Activity in human ventral striatum locked to errors of reward prediction. Nat Neurosci 5:97-98.

Penfield W, Milner B (1958) Memory deficit produced by bilateral lesions in the hippocampal zone. AMA Arch Neurol Psychiatry 79:475-497.

Pessiglione M, Seymour B, Flandin G, Dolan RJ, Frith CD (2006) Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442:1042-1045.

Ploghaus A, Tracey I, Clare S, Gati JS, Rawlins JN, Matthews PM (2000) Learning about pain: the neural substrate of the prediction error for aversive events. Proc Natl Acad Sci U S A 97:9281-9286.

Poldrack RA, Packard MG (2003) Competition among multiple memory systems: converging evidence from animal and human brain studies. Neuropsychologia 41:245-251.

Poldrack RA, Rodriguez P (2004) How do memory systems interact? Evidence from human classification learning. Neurobiol Learn Mem 82:324-332.

Poldrack RA, Prabhakaran V, Seger CA, Gabrieli JD (1999) Striatal activation during acquisition of a cognitive skill. Neuropsychology 13:564-574.

Poldrack RA, Fletcher PC, Henson RN, Worsley KJ, Brett M, Nichols TE (2008) Guidelines for reporting an fMRI study. Neuroimage 40:409-414.

Poldrack RA, Clark J, Pare-Blagoev EJ, Shohamy D, Creso Moyano J, Myers C, Gluck MA (2001) Interactive memory systems in the human brain. Nature 414:546-550.

Reber PJ, Knowlton BJ, Squire LR (1996) Dissociable properties of memory systems: differences in the flexibility of declarative and nondeclarative knowledge. Behav Neurosci 110:861-871.

202

Rempel-Clower NL, Zola SM, Squire LR, Amaral DG (1996) Three cases of enduring memory impairment after bilateral damage limited to the hippocampal formation. J Neurosci 16:5233-5255.

Rice WR (1989) Analyzing Tables of Statistical Tests. Evolution 43:223-225.

Roebroeck A, Formisano E, Goebel R (2005) Mapping directed influence over the brain using Granger causality and fMRI. Neuroimage 25:230-242.

Rolls ET (1994) Neurophysiology and cognitive functions of the striatum. Aug-Sep 1994. Revue Neurologique.150:pp.

Sadeh T, Shohamy D, Levy DR, Reggev N, Maril A (2011) Cooperation between the hippocampus and the striatum during episodic encoding. J Cogn Neurosci 23:1597-1608.

Samejima K, Ueda Y, Doya K, Kimura M (2005) Representation of action-specific reward values in the striatum. Science 310:1337-1340.

Scatton B, Simon H, Le Moal M, Bischoff S (1980) Origin of dopaminergic innervation of the rat hippocampal formation. Neurosci Lett 18:125-131.

Schacter DL, Wagner AD (1999) Medial temporal lobe activations in fMRI and PET studies of episodic encoding and retrieval. Hippocampus 9:7-24.

Schmitt-Eliassen J, Ferstl R, Wiesner C, Deuschl G, Witt K (2007) Feedback-based versus observational classification learning in healthy aging and Parkinson's disease. Brain Res 1142:178-188.

Schoenbaum G, Setlow B (2003) Lesions of nucleus accumbens disrupt learning about aversive outcomes. J Neurosci 23:9833-9841.

Schott BH, Seidenbecher CI, Fenker DB, Lauer CJ, Bunzeck N, Bernstein HG, Tischmeyer W, Gundelfinger ED, Heinze HJ, Duzel E (2006) The dopaminergic midbrain participates in human episodic memory formation: evidence from genetic imaging. J Neurosci 26:1407-1417.

203

Schultz W (1997) Dopamine neurons and their role in reward mechanisms. Curr Opin Neurobiol 7:191-197.

Schultz W (2002) Getting formal with dopamine and reward. Neuron 36:241-263.

Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593-1599.

Scoville WB, Milner B (1957) Loss of recent memory after bilateral hippocampal lesions. J Neurol Neurosurg Psychiatry 20:11-21.

Seger CA (2006) The basal ganglia in human learning. Neuroscientist 12:285-290.

Seger CA (2008) How do the basal ganglia contribute to categorization? Their roles in generalization, response selection, and learning via feedback. Neurosci Biobehav Rev 32:265-278.

Seger CA, Cincotta CM (2005) The roles of the caudate nucleus in human classification learning. J Neurosci 25:2941-2951.

Seger CA, Cincotta CM (2006) Dynamics of frontal, striatal, and hippocampal systems during rule learning. Cereb Cortex 16:1546-1555.

Seger CA, Miller EK (2010) Category learning in the brain. Annu Rev Neurosci 33:203-219.

Setlow B, Schoenbaum G, Gallagher M (2003) Neural encoding in ventral striatum during olfactory discrimination learning. Neuron 38:625-636.

Sherry DF, Schacter DL (1987) The evolution of multiple memory systems. Psychological Review 94:439-454.

Shohamy D, Wagner AD (2008) Integrating memories in the human brain: hippocampal-midbrain encoding of overlapping events. Neuron 60:378-389.

204

Shohamy D, Adcock RA (2010) Dopamine and adaptive memory. Trends Cogn Sci 14.

Shohamy D, Myers CE, Onlaor S, Gluck MA (2004a) Role of the basal ganglia in category learning: how do patients with Parkinson's disease learn? Behav Neurosci 118:676-686.

Shohamy D, Myers CE, Kalanithi J, Gluck MA (2008) Basal ganglia and dopamine contributions to probabilistic category learning. Neurosci Biobehav Rev 32:219-236.

Shohamy D, Myers CE, Grossman S, Sage J, Gluck MA (2005) The role of dopamine in cognitive sequence learning: evidence from Parkinson's disease. Behav Brain Res 156:191-199.

Shohamy D, Myers CE, Geghman KD, Sage J, Gluck MA (2006) L-dopa impairs learning, but spares generalization, in Parkinson's disease. Neuropsychologia 44:774-784.

Shohamy D, Myers CE, Hopkins RO, Sage J, Gluck MA (2009) Distinct hippocampal and basal ganglia contributions to probabilistic learning and reversal. J Cogn Neurosci 21:1821-1833.

Shohamy D, Myers CE, Grossman S, Sage J, Gluck MA, Poldrack RA (2004b) Cortico-striatal contributions to feedback-based learning: converging data from neuroimaging and neuropsychology. Brain 127:851-859.

Shrager Y, Kirwan CB, Squire LR (2008) Activity in both hippocampus and perirhinal cortex predicts the memory strength of subsequently remembered information. Neuron 59:547-553.

Sorensen KE, Witter MP (1983) Entorhinal efferents reach the caudato-putamen. Neurosci Lett 35:259-264.

Squire LR (1992a) Memory and the hippocampus: a synthesis from findings with rats, monkeys, and humans. Psychol Rev 99:195-231.

205

Squire LR (1992b) Declarative and nondeclarative memory: Multiple brain systems supporting learning and memory. Journal of Cognitive Neuroscience 4:232-243.

Squire LR, Zola SM (1996) Structure and function of declarative and nondeclarative memory systems. Proc Natl Acad Sci U S A 93:13515-13522.

Squire LR, Stark CE, Clark RE (2004) The medial temporal lobe. Annu Rev Neurosci 27:279-306.

Stark CE, Squire LR (2001) When zero is not zero: the problem of ambiguous baseline conditions in fMRI. Proc Natl Acad Sci U S A 98:12760-12766.

Stubley-Weatherly L, Harding JW, Wright JW (1996) Effects of discrete kainic acid-induced hippocampal lesions on spatial and contextual learning and memory in rats. Brain Res 716:29-38.

Sutherland RJ, Kolb B, Whishaw IQ (1982) Spatial mapping: definitive disruption by hippocampal or medial frontal cortical damage in the rat. Neurosci Lett 31:271-276.

Sutherland RJ, Whishaw IQ, Kolb B (1988) Contributions of cingulate cortex to two forms of spatial learning and memory. J Neurosci 8:1863-1872.

Swainson R, Rogers RD, Sahakian BJ, Summers BA, Polkey CE, Robbins TW (2000) Probabilistic learning and reversal deficits in patients with Parkinson's disease or frontal or temporal lobe lesions: possible adverse effects of dopaminergic medication. Neuropsychologia 38:596-612.

Swanson LW (1981) A direct projection from Ammon's horn to prefrontal cortex in the rat. Brain Res 217:150-154.

Swanson LW (1982) The projections of the ventral tegmental area and adjacent regions: a combined fluorescent retrograde tracer and immunofluorescence study in the rat. Brain Res Bull 9:321-353.

206

Swanson LW, Kohler C (1986) Anatomical evidence for direct projections from the entorhinal area to the entire cortical mantle in the rat. J Neurosci 6:3010-3023.

Talairach J, Tournoux P (1988) Co-Planar Stereotaxic Atlas of the Human Brain. New York: Thieme Medical Publishers, Inc.

Tanaka SC, Doya K, Okada G, Ueda K, Okamoto Y, Yamawaki S (2004) Prediction of immediate and future rewards differentially recruits cortico-basal ganglia loops. Nat Neurosci 7:887-893.

Thierry AM, Gioanni Y, Degenetais E, Glowinski J (2000) Hippocampo-prefrontal cortex pathway: anatomical and electrophysiological characteristics. Hippocampus 10:411-419.

Toni I, Ramnani N, Josephs O, Ashburner J, Passingham RE (2001) Learning arbitrary visuomotor associations: temporal dynamic of brain activity. Neuroimage 14:1048-1057.

Tricomi E, Fiez JA (2008) Feedback signals in the caudate reflect goal achievement on a declarative memory task. Neuroimage 41:1154-1167.

Tricomi EM, Delgado MR, Fiez JA (2004) Modulation of caudate activity by action contingency. Neuron 41:281-292.

Vanni-Mercier G, Mauguiere F, Isnard J, Dreher JC (2009) The hippocampus codes the uncertainty of cue-outcome associations: an intracranial electrophysiological study in humans. J Neurosci 29:5287-5294.

Voermans NC, Petersson KM, Daudey L, Weber B, Van Spaendonck KP, Kremer HP, Fernandez G (2004) Interaction between the human hippocampus and the caudate nucleus during route recognition. Neuron 43:427-435.

Wagner AD, Schacter DL, Rotte M, Koutstaal W, Maril A, Dale AM, Rosen BR, Buckner RL (1998) Building memories: remembering and forgetting of verbal experiences as predicted by brain activity. Science 281:1188-1191.

207

Watkins CJCH (1989) Learning from delayed rewards. In. England: University of Cambridge.

White NM, McDonald RJ (2002) Multiple parallel memory systems in the brain of the rat. Neurobiol Learn Mem 77:125-184.

Wise RA (2004) Dopamine, learning and motivation. Nat Rev Neurosci 5:483-494.

Wittmann BC, Schott BH, Guderian S, Frey JU, Heinze HJ, Duzel E (2005) Reward-related FMRI activation of dopaminergic midbrain is associated with enhanced hippocampus-dependent long-term memory formation. Neuron 45:459-467.

Yin HH, Knowlton BJ (2006) The role of the basal ganglia in habit formation. Nat Rev Neurosci 7:464-476.

Zeineh MM, Engel SA, Thompson PM, Bookheimer SY (2003) Dynamics of the hippocampus during encoding and retrieval of face-name pairs. Science 299:577-580.

Zola-Morgan S, Squire LR (1984) Preserved learning in monkeys with medial temporal lesions: sparing of motor and cognitive skills. J Neurosci 4:1072-1085.

Zola-Morgan S, Squire LR (1985) Medial temporal lesions in monkeys impair memory on a variety of tasks sensitive to human amnesia. Behav Neurosci 99:22-34.

Zola-Morgan S, Squire LR (1986) Memory impairment in monkeys following lesions limited to the hippocampus. Behavioral Neuroscience 100:155-160.

Zola SM, Mahut H (1973) Paradoxical facilitation of object reversal learning after transection of the fornix in monkeys. Neuropsychologia 11:271-284.

208

Figures Figure 3.1 Schematic of the Task used in the Experiment 1

209

Figure 3.2 Experiment 1 Behavioral Results

210

Figure 3.3 Experiment 1 Neuroimaging Results: Main Effect of Cue Difficulty in the Striatum and Hippocampus

211

Figure 3.4 Experiment 1 Neuroimaging Results: Granger Causality Analysis – Functional Connectivity in the Caudate Nucleus using the Hippocampus as the Seed Region

212

Figure 3.5 Experiment 1 Neuroimaging Results: Prediction Error Analysis in the Hippocampus and Striatum

213

Figure 4.1 Schematic of the Task used in Experiment 2

214


215

Figure 4.3 Experiment 2 Neuroimaging Results: Learning Phase Main Effect of Cue Difficulty in the Striatum and MTL – Feedback Group

216

Figure 4.4 Experiment 2 Neuroimaging Results: Learning Phase Main Effect of Cue Difficulty in the Striatum and MTL – Observation Group

217

Figure 4.5 Experiment 2 Neuroimaging Results: Update Phase Main Effect of Cue Difficulty in the Striatum and MTL – Feedback and Observation Groups

218

Figure 4.6 Neuroimaging Results: Granger Causality Analysis – Effective Connectivity using the Midbrain as the Seed Region

219

Figure 4.7 Experiment 2 Neuroimaging Results: Prediction Error Analysis

220

Figure 5.1 Schematic of the Task used in Experiment 3

221


222

Figure 5.3 Experiment 3 Neuroimaging Results: ROI Analysis during the Stimulus Presentation Period

223

Tables

Table 3.1 Learning Phase ANOVA: Main Effect of Cue Difficulty Talairach Coordinates Region of

Activation Brodmann Area (BA)

Laterality x y z

# Voxels FStat

Postcentral gyrus BA 5 Left -27 -37 61 257 21.84Superior frontal gyrus BA 6 Right 6 14 52 169 29.79Inferior parietal lobe BA 40 Left -33 -37 46 170 34.36

Postcentral gyrus BA 40 Right 42 -28 43 162 21.32

Precuneus BA 7 Right 3 -43 43 242 20.16

Precentral gyrus BA 6 Right 45 -1 43 161 19.80

Precentral gyrus Right 30 17 37 334 25.42

Medial frontal gyrus BA 32 Left -3 20 37 319 21.72

Middle frontal gyrus BA 9 Left -48 5 37 162 17.04

Cingulate gyrus BA 24 Right 21 17 25 209 39.44

Middle frontal gyrus BA 10 Right 36 44 22 758 56.88

Insula BA 13 Right 27 -34 22 239 24.35Superior temporal lobe BA 22 Left -21 -40 22 408 53.63Claustrum Right 22 -13 22 136 29.74

Insula Right 42 -4 19 49 22.32Inferior frontal gyrus BA 44 Left -60 14 19 219 31.51Superior temporal lobe Right 36 -46 13 125 22.02

Precentral gyrus BA 44 Left -42 14 10 76 17.75Caudate nucleus (body) Left -15 20 7 198 35.40

Lingual gyrus Right 9 -52 1 335 32.75

Insula BA 13 Right 45 14 1 218 27.25

Hippocampus Left -36 -28 -8 96 24.66Middle temporal gyrus BA 20 Right 54 -31 -11 166 23.69

Cerebellum BA 37 Left -21 -40 -14 570 49.03

224

Table 3.2 Learning Phase ANOVA: Main Effect of Learning Type

Talairach Coordinates Region of Activation

Brodmann Area (BA)

Laterality x y z

# Voxels FStat

Superior parietal lobe BA 7 Left -18 -70 58 175 22.45Middle frontal gyrus BA 6 Left -27 8 58 153 20.61Superior parietal lobe BA 7 Left -24 -52 55 245 28.86Superior frontal gyrus BA 8 Right 6 26 52 206 27.13Frontal medial gyrus BA 6 Right 12 17 46 215 34.85Inferior parietal lobe BA 40 Left -39 -37 43 122 21.09Cuneus BA 18 Left -3 -82 16 225 17.93

Thalamus Right 3 -7 13 623 42.32Middle frontal gyrus BA 10 Right 33 50 10 505 34.07Middle occipital gyrus BA 18 Right 27 -85 4 250 19.62

Caudate Right 6 3 4 354 35.41

Globus pallidus Left -12 2 4 571 35.44

Lingual gyrus BA 18 Left -9 -73 1 502 26.84Superior temporal gyrus BA 22 Left -48 -7 1 153 33.19Superior temporal gyrus BA 21 Left -67 -10 1 262 33.11Superior temporal gyrus BA 22 Right 54 2 -2 186 21.03Middle temporal gyrus BA 21 Left -63 -37 -2 349 25.40Claustrum Right 30 11 -2 464 28.67

Insula BA 13 Left -33 14 -2 347 31.27Inferior temporal gyrus BA 37 Right 48 -61 -8 287 37.79

Fusiform gyrus BA 19 Right 30 -64 -11 148 21.50Parahippocampal gyrus Right 18 -37 -11 242 18.04

Cerebellum Right 27 -46 -17 663 27.78

Cerebellum Left -30 -64 -17 269 23.91

225

3.3 Test Phase Contrast of Previously Studied vs. Novel Material

Talairach Coordinates

Region of Activation Brodmann Area (BA)

Laterality

x y z

# Voxels tstat

Previously Studied > Novel

Paracentral lobe BA 4/5 Left -12 -37 61 140 4.30 Medial frontal gyrus BA 6 Left -9 -7 49 77 4.41 Postcentral gyrus BA 1 Left -51 -19 49 422 5.29 Inferior parietal lobe BA 40 Right 51 -28 46 62 3.95 Postcentral gyrus BA 2 Right 45 -16 46 126 4.15 Precuneus BA 7 Left -12 -49 46 69 4.78 Cingulate gyrus BA 31 Left -15 -31 40 127 4.64 Precentral gyrus BA 4 Right 45 -16 34 106 4.35 Inferior parietal lobe BA 40 Left -51 -31 31 148 5.53 Cingulate gyrus BA 23 Right 6 -34 28 71 4.15 Insula BA 13 Left -33 -31 25 56 4.67 Precentral gyrus BA 4 Left -51 -7 22 204 4.35 Postcentral gyrus BA 40 Left -54 -22 19 125 4.36 Superior temporal gyrus BA 41 Right 45 -34 16 221 4.38 Superior temporal gyrus BA 22 Right 39 -46 16 51 4.07 Caudate nucleus Right 6 5 16 88 4.75 Superior temporal gyrus BA 22 Right 51 -46 13 176 4.77 Thalamus Left -18 -25 13 241 5.60 Middle occipital gyrus BA 18 Right 33 -88 10 80 3.84 Superior temporal gyrus BA 42 Left -60 -31 10 247 4.47 Caudate nucleus (tail) Right 27 -40 7 120 4.73 Insula Left -36 -13 7 64 4.24 Thalamus Right 0 -16 4 57 4.42 Middle temporal gyrus BA 21 Left -60 -25 1 124 4.58 Medial temporal lobe BA 28 Left -24 -22 -5 101 4.14 Parahippocampal gyrus BA 36 Left -21 -40 -8 233 4.78 Cerebellum Right 12 -52 -14 758 7.93 Middle occipital gyrus BA 19 Right 30 -79 -17 217 4.25 Cerebellum Left -15 -91 -17 182 4.70 Cerebellum Right 36 -70 -23 126 4.43 Cerebellum Right 15 -22 -23 189 5.12

226

Table 4.1 Learning Phase ANOVA: Main Effect of Cue Difficulty - Feedback Group

Talairach Coordinates Region of Activation

Brodmann Area (BA)

Laterality x y z

# Voxels FStat

Superior Parietal Lobule BA 7 Right 16 -65 60 206 10.42Superior Parietal Lobule BA 7 Left -10 -71 57 251 9.63 Paracentral Lobule BA 5 Left -13 -32 54 223 11.19Superior Parietal Lobule BA 7 Left -31 -53 54 213 11.38Precuneus BA 7 Left -19 -56 51 139 9.58 Precentral Gyrus BA 6 Right 44 4 45 1657 12.97Inferior Parietal Lobule BA 40 Right 41 -50 39 509 12.28Precuneus BA 19 Right 32 -65 39 1025 11.43Inferior Parietal Lobule BA 40 Left -40 -59 39 676 15.47Precentral Gyrus BA 6 Right 38 1 36 152 10.70


Cingulate Gyrus BA 32 Right 1 22 30 4355 19.08Middle Frontal Gyrus BA 9 Left -43 19 30 751 16.81Middle Frontal Gyrus BA 10 Right 29 58 21 388 12.82Medial Frontal Gyrus BA 9 Right 2 49 21 206 9.43 Middle Temporal Gyrus BA 19 Left -49 -80 19 2468 18.79Superior Temporal Gyrus BA 39 Right 41 -56 18 261 9.07

Insula Right 41 -23 18 158 12.09Medial Frontal Gyrus BA 10 Left -7 55 18 2897 12.43

Caudate Nucleus Right 5 1 12 243 9.39 Posterior Cingulate BA 23 Left -13 -56 9 6733 20.07Middle Frontal Gyrus BA 46 Right 38 49 6 617 12.71

Insula Right 29 16 3 1954 19.17

Insula Left -34 10 3 2287 25.01Inferior Temporal Gyrus BA 37 Left -34 -65 0 146 11.85MTL/Hippo/Amygdala Left -22 -8 -15 567 12.96

Cerebellum Left -37 -59 -33 156 10.32

Brainstem Left -1 -35 -36 207 8.51

227

Table 4.2 Learning Phase ANOVA: Main Effect of Cue Difficulty - Observation

Group Talairach

Coordinates Region of Activation

Brodmann Area (BA)

Laterality

x y z

# Voxels FStat

Medial Frontal Gyrus BA 6 Left -4 -23 66 195 9.19 Superior Parietal Lobule BA 7 Right 17 -59 57 237 10.23

Precuneus BA 7 Right 5 -53 57 713 10.76Middle Frontal Gyrus BA 6 Right 32 -2 54 170 9.86 Inferior Parietal Lobule BA 40 Right 47 -53 48 1066 10.67

Precuneus BA 7 Left -10 -53 42 339 11.65

Cingulate Gyrus BA 32 Left -1 13 39 3922 19.60Middle Frontal Gyrus BA 9 Right 47 10 36 1384 14.44

Angular Gyrus BA 39 Right 35 -62 30 327 10.25Middle Frontal Gyrus BA 46 Right 38 52 15 328 19.49

Caudate Nucleus Right 5 -2 15 459 13.66Inferior Frontal Gyrus BA 46 Left -52 40 12 184 10.27Caudate Nucleus/Putamen Left -13 1 9 831 13.47Middle Frontal Gyrus BA 46 Left -34 49 9 872 12.37

Thalamus Right 9 -11 6 1402 15.54Inferior Frontal Gyrus BA 44 Right 29 43 0 715 13.41

Insula BA 13 Left -37 13 -3 198 11.52

Hippocampus Left -25 -20 -12 145 10.25

Cerebellum Right 17 -80 -18 287 12.67

228

Table 4.3 Test Phase 1: Contrast of Previously Studied vs. Novel - Feedback Group

Talairach Coordinates

Region of Activation Brodmann Area (BA)

Laterality

x y z

# Voxels tstat


Superior Parietal Lobule BA 7 Left -34 -47 60 438 4.61 Superior Parietal Lobule BA 7 Right 14 -62 57 724 4.85 Superior Parietal Lobule BA 7 Left -31 -44 48 441 7.24

Postcentral Gyrus BA 4 Right 47 -14 42 187 5.11


Cingulate Gyrus BA 31 Right 8 -38 39 368 5.02 Superior Temporal Gyrus BA 22 Right 56 -38 21 394 5.10 Superior Temporal Gyrus BA 22 Left -55 -44 15 154 4.75

Insula BA 13 Right 41 -14 3 337 5.59 Middle Temporal Gyrus BA 21 Left -37 -26 0 440 6.40

Novel > Previously

Studied

Medial Frontal Gyrus BA 8 Left -7 19 48 13762 -6.92 Superior Parietal Lobule BA 7 Right 38 -65 45 221 -4.41

Inferior Parietal Lobule BA 40 Left -46 -56 42 1997 -5.71

Precuneus BA 19 Left -37 -77 39 212 -3.90

Middle Frontal Gyrus BA 9 Right 44 22 33 141 -4.49 Caudate Nucleus/Putamen Left -16 7 12 474 -7.33 Inferior Frontal Gyrus BA 45 Left -46 40 3 1948 -5.89

Cerebellum Left -4 -38 -9 150 -4.50 Inferior Temporal Gyrus BA 37 Right 56 -44 -15 280 -4.42 Inferior Temporal Gyrus BA 20 Left -61 -32 -15 401 -5.47 Inferior Temporal Gyrus BA 21 Right 62 -8 -18 219 -5.49

Cerebellum Right 23 -77 -39 270 -5.48

229

Table 4.4 Test Phase 1: Contrast of Previously Studied vs. Novel - Observation Group

Talairach Coordinates Region of Activation Brodmann Area (BA)

Lateralityx y z

# Voxels tstat

Previously Studied > Novel Paracentral Lobule BA 4 Right 17 -41 57 196 4.12 Paracentral Lobule BA 5 Right 8 -35 51 286 5.30 Precuneus BA 7 Right 14 -53 48 225 5.01 Precuneus BA 7 Left -13 -50 48 512 5.84 Inferior Parietal Lobule BA 40 Right 50 -32 24 413 5.35 Cuneus BA 19 Left -13 -92 24 318 4.32 Cingulate Gyrus BA 20 Right 20 -50 18 209 4.72 Cuneus BA 18 Left -4 -83 15 500 4.14 Superior Temporal Gyrus BA 22 Left -49 -41 15 1382 6.53 Superior Temporal Gyrus BA 22 Left -61 -23 15 470 7.05 Lingual Gyrus BA 19 Left -10 -47 0 464 4.88 Lingual Gyrus BA 18 Left -13 -71 -6 170 4.91 Cerebellum Left -16 -50 -18 276 4.82 Novel > Previously Studied Superior Frontal Gyrus BA 8 Left -19 25 48 34672 -9.28 Precuneus BA 19 Right 35 -65 42 507 -5.10 Middle Frontal Gyrus BA 8 Left -31 25 39 608 -6.70 Angular Gyrus BA 39 Left -52 -68 30 407 -4.58 Thalamus Left -19 -11 18 140 -6.47 Middle Frontal Gyrus BA 46 Right 35 49 15 142 -4.12 Inferior Frontal Gyrus BA 44 Left -61 19 6 882 -5.30 Middle Frontal Gyrus BA 10 Left -40 49 3 550 -5.39 Insula BA 13 Right 38 19 0 941 -5.20 Claustrum Left -31 16 0 262 -5.20 Inferior Temporal Lobe BA 37 Right 50 -44 -3 889 -7.17 Middle Frontal Gyrus BA 47 Right 38 40 -3 1130 -5.67 Middle Frontal Gyrus BA 48 Left -49 43 -3 668 -4.77 Middle Temporal Gyrus BA 21 Right 44 10 -33 189 -5.14

230

Table 4.5 Update Phase ANOVA: Main Effect of Cue Difficulty

Feedback & Observation Groups Combined Talairach Coordinates Region of Activation Brodmann

Area (BA)Laterality

x y z # Voxels FStat

Precentral Gyrus BA 4 Left -31 -23 58 130 10.98Superior Parietal Lobule BA 7 Right 14 -63 54 493 11.19Precuneus BA 7 Left -16 -59 48 178 8.76 Inferior Parietal Lobule BA 7 Right 41 -62 43 339 9.25 Inferior Parietal Lobule BA 40 Right 26 -41 39 230 10.61Precuneus BA 7 Right 11 -50 39 378 10.28Cingulate Gyrus BA 31 Right 8 -29 39 225 10.10Cingulate Gyrus BA 32 Left -1 16 39 1581 15.57Precuneus BA 31 Left -19 -41 36 187 9.17 Inferior Parietal Lobule BA 40 Right 53 -23 33 237 8.22 Inferior Frontal Gyrus BA 44 Right 44 13 31 1881 17.54Inferior Parietal Lobule BA 40 Left -64 -29 30 9298 20.81Lateral Sulcus Right 44 -32 21 192 9.11 Insula BA 13 Right 32 -29 21 380 12.50Middle Occipital Gyrus BA 18 Left -16 -83 21 10268 16.91Inferior Frontal Gyrus BA 45 Right 53 -41 18 1638 11.66Middle Temporal Gyrus BA 39 Right 53 -62 15 3619 17.36Middle Occipital Gyrus BA 18 Right 11 -89 15 279 10.23Middle Occipital Gyrus BA 19 Left -37 -80 15 231 10.12Insula Left -37 -14 15 256 9.11 Superior Frontal Gyrus BA 10 Right 26 55 12 3461 13.09Putamen/Insula Right 29 -14 12 220 12.40Anterior Cingulate BA 32 Left -13 37 12 1604 10.98Middle Temporal Gyrus BA 39 Left -43 -71 12 361 10.57Middle Frontal Gyrus BA 10 Left -34 52 9 159 10.00Superior Temporal Gyrus BA 42 Left -43 -26 9 329 10.71Middle Temporal Gyrus BA 22 Left -61 -41 6 176 10.06Insula Right 35 -14 3 191 11.64Putamen/Insula Right 26 13 3 618 11.04Lingual Gyrus BA 18 Right 8 -50 3 492 14.92Insula Left -31 13 3 254 8.54 Middle Frontal Gyrus BA 10 Left -40 49 3 667 13.70Middle Temporal Gyrus BA 21 Right 47 -11 -6 2254 12.77Cerebellum Right 20 -50 -12 352 12.75Cerebellum Right 23 -65 -15 512 10.59Cerebellum Right 11 -71 -15 165 9.56 Fusiform Gyrus BA 20 Left -25 -35 -15 133 10.64Inferior Temporal Gyrus BA 21 Left -58 -11 -15 792 17.63Parahippocampal Gyrus Right 20 -20 -18 242 13.36Cerebellum Left -28 -53 -18 186 9.54 Parahippocampal Gyrus BA 28 Left -22 -14 -21 187 8.60

231

Table 4.6 Update Phase ANOVA: Main Effect of Learning Type

Feedback & Observation Groups Combined Talairach CoordinatesRegion of Activation Brodmann

Area (BA)Laterality

x y z # Voxels FStat

Postcentral Gyrus BA 2 Right 33 -23 48 275 19.13Paracentral Lobule Left -1 -13 48 149 16.86Cingulate Gyrus BA 24 Left -10 -5 36 221 19.45Angular Gyrus BA 39 Right 41 -62 35 523 24.68Cingulate Gyrus BA 31 Right 11 -26 33 248 28.32Angular Gyrus BA 39 Left -49 -62 30 229 15.48Middle Temporal Gyrus BA 39 Left -43 -71 28 142 22.55Orbital Gyrus BA 19 Left -34 -77 27 169 26.06Thalamus Left -22 -14 18 193 16.81Superior Temporal Gyrus BA 42 Right 56 -29 15 1134 23.84Insula Right 35 2 9 441 16.64Middle Temporal Gyrus BA 21 Left -64 -45 9 211 26.85Insula Right 41 7 6 318 19.61Lingual Gyrus BA 18 Left -22 -56 6 203 15.97Superior Temporal Gyrus BA 42 Left -48 -23 6 3049 37.20Middle Temporal Gyrus BA 22 Left -43 -38 4 270 15.19Superior Temporal Gyrus BA 21 Right 57 -11 0 213 16.09

232

Table 4.7 Update Phase ANOVA: Main Effect of Group


Lateralityx y z

# Voxels FStat

Precentral Gyrus BA 4 Right 14 -23 54 411 15.63Thalamus Left -4 -20 0 212 15.45

233

Table 4.8 Test Phase 2: Contrast of Previously Studied vs. Novel - Feedback Group


Lateralityx y z

# Voxels tstat

Previously Studied > Novel Precentral Gyrus BA 4 Left -28 -23 63 895 5.11 Inferior Parietal Lobule BA 40 Left -22 -38 54 300 6.28 Precuneus BA 7 Right 2 -56 51 4334 7.48 Inferior Parietal Lobule BA 40 Left -43 -50 51 139 3.90 Precentral Gyrus BA 4 Left -19 -20 48 211 6.43 Cingulate Gyrus BA 31 Left -4 -35 42 611 5.82 Inferior Parietal Lobule BA 40 Left -28 -44 39 370 6.24 Inferior Parietal Lobule BA 40 Right 56 -29 30 610 5.14 Postcentral Gyrus BA 2 Left -64 -20 30 317 4.64 Inferior Parietal Lobule BA 40 Left -64 -41 24 195 4.02 Superior Temporal Gyrus BA22 Left -61 -50 15 277 4.85 Superior Temporal Gyrus BA 22 Left -61 -32 12 182 4.61 Parahippocampal Gyrus Right 20 -50 3 149 4.25 Cerebellum Right 2 -44 -3 189 5.22 Novel > Previously Studied

Precentral Gyrus BA 9 Left -37 4 36 131 -4.54 Middle Frontal Gyrus BA 46 Left -40 43 18 838 -5.07 Middle Frontal Gyrus Left -28 46 3 138 -4.74 Insula BA 13 Left -37 13 0 353 -4.40

234

Table 4.9 Test Phase 2: Contrast of Previously Studied vs. Novel - Observation Group


Lateralityx y z

# Voxels tstat


Precuneus BA 31 Left -22 -41 33 526 5.90 Precuneus BA 7 Left -22 -56 30 260 4.77 Superior Temporal Gyrus BA 22 Left -34 -50 15 428 4.72 Cerebellum Right 5 -38 0 553 5.30 Cerebellum Right 5 -41 -39 282 5.05

Novel > Previously Studied Medial Frontal Gyrus BA 6 Right 2 4 51 276 -5.47Medial Frontal Gyrus BA 6 Right 17 10 48 198 -4.20Middle Frontal Gyrus BA 6 Right 32 -5 45 140 -4.08Precentral Gyrus BA 6 Left -37 -8 45 353 -4.94Middle Frontal Gyrus BA 9 Right 35 19 33 163 -4.20Precentral Gyrus BA 6 Left -49 7 12 295 -8.48Middle Temporal Gyrus BA 21 Left -52 -38 -9 292 -4.65

235

Table 5.1 Test Phase: Contrast of Previously Studied vs. Novel


Laterality x y z

# Voxels tstat

Novel > Previously Studied

Middle Frontal Gyrus BA 9 Right 41 16 36 436 -4.17 Middle Frontal Gyrus BA 9 Left -22 37 36 429 -5.15 Medial Frontal Gyrus BA 9 Left -7 34 33 4164 -6.08 Angular Gyrus BA 39 Left -43 -59 33 303 -3.78

236

Appendices

Appendix 1: Neuroimaging Test Phase Results for Experiments 1-3

Experiment 1 Neuroimaging Test Phase Analysis:

Test Phase: Contrast of previously studied versus novel information

In the test phase, the primary analysis examined differences in BOLD

responses to learned versus novel material. A contrast of studied (observation

and feedback) versus non-studied (novel) cues was used to identify areas

involved in memory processes (Table 3.3). This contrast produced one area of

activation within the medial temporal lobe – the parahippocampal gyrus (x, y, z =

-21, -40, -8). Post-hoc t-tests on the mean parameter estimates extracted from

the parahippocampal region revealed no significant difference between

observation and feedback information (t(15) = 0.37; p > 0.05). An area just

outside the hippocampus (x, y, z = -24, -22, -5), and a region within the right

caudate nucleus (x, y, z = 6, 5, 16) were also observed in this contrast, but did

not survive correction at the cluster level. This analysis revealed that the

parahippocampal gyrus was most active in response to previously learned

information compared to novel information.

Experiment 2 Neuroimaging Test Phase Analyses:

Test phase 1: Contrast of previously studied versus novel information

As in experiment 1, in the test phases in experiment 2, the analyses of

interest examined differences in BOLD responses to learned versus novel

237

material. Two separate analyses were performed, one for the feedback group

and one for the observation group.

Feedback group: a contrast of previously learned (feedback) versus novel

cues was performed. In this group, a network of regions was involved showing

greater activation to previously learned versus novel information, including a

region in the middle temporal gyrus, just dorsal to the hippocampus (x, y, z = -37,

-26, 0) as well as the insula (x, y, z = 41, -14, 3), and cingulate gyrus (x, y, z = 8,

-38, 39). Several regions also showed greater activation to novel cues, including

a region in-between the caudate nucleus and putamen (x, y, z = -16, 7, 12)

(Table 4.3).

Observation group: a contrast of previously learned (observation) versus

novel cues was performed. A somewhat similar pattern of results was observed

for the observation group, including a region of the cingulate gyrus (x, y, z = 20, -

50, 18) which exhibited greater responses to previously learned compared with

novel cues. Regions exhibiting greater BOLD responses to novel cues included

the insula (x, y, z = 38, 19, 0) (Table 4.4).

Test phase 2: Contrast of previously studied versus novel information

For both the feedback and observation groups, a contrast of previously

learned versus novel cues was performed.

Feedback group: a host of regions showed greater activation for

previously learned information compared with novel information including the

right parahippocampus (x, y, z = 20, -50, 3) and cingulate gyrus (x, y, z = -4, -35,

238

42). Only a few regions showed greater activation to novel stimuli including the

left insula (x, y, z = -37, 13, 0), precentral gyrus, and the middle frontal gyrus

(Table 4.8).

Observation group: the superior temporal gyrus and precuneus were

engaged more strongly for previously learned versus novel information, while

areas of the middle and medial frontal gyri were more engaged when processing

novel information (Table 4.9).

Experiment 3 Neuroimaging Test Phase Analysis:

Test phase: Contrast of previously studied versus novel information

As in experiments 1 and 2, in the test phase in experiment 3, the analysis

of interest examined differences in BOLD responses to learned versus novel

material. A contrast of previously learned (feedback control + perception

decision + declarative memory interference) versus novel cues was performed

(balancing for the unequal comparison). Results revealed activation within the

medial frontal gyrus, middle frontal gyrus, and angular gyrus exhibiting greater

responses to novel cues than previously studied information. These regions

were the only active areas observed in the SPM that survived cluster threshold

correction (Table 5.1).

Appendix 2: Experiment 3 Correlations Observed across the Independent

Region of Interest Analysis

239

1) In the feedback control session, the following positive correlations were

observed for the:

Hard cues:

- Left hippocampus – right ventral striatum (r = 0.416, p < 0.05) and left

caudate nucleus (r = 0.467, p < 0.05)

- Left ventral striatum – right hippocampus (r = 0.584, p < 0.005) and left

VTA (r = 0.469, p < 0.05)

Easy cues:

- Left VTA - left ventral striatum (r = 0.415, p < 0.05)

2) In the perception decision session the following positive correlations were

observed for the:

Hard cues:

- Right VTA – left hippocampus (r = 0.477, p < 0.05); right hippocampus (r =

0.430, p < 0.05); left ventral striatum (r = 0.507, p < 0.05); right ventral

striatum (r = 0.410, p < 0.05)

- Left VTA – left ventral striatum (r = 0.480, p < 0.05)

Easy cues:

- Left hippocampus – left ventral striatum (r = 0.429, p < 0.05); right ventral

striatum (r = 0.527, p < 0.01); left VTA (r = 0.589, p < 0.005); right VTA (r =

0.587, p < 0.005)

- Right hippocampus – right ventral striatum (r = 0.522, p < 0.01) and left

VTA (r = 0.448, p < 0.05)

240

- Left ventral striatum – left VTA (r = 0.606, p < 0.005) and right VTA (r =

0.698, p < 0.001)

- Right ventral striatum – left VTA (r = 0.561, p < 0.005) and right VTA (r =

0.538, p < 0.01)

- Left caudate nucleus – left hippocampus (r = 0.419, p < 0.05) and left VTA

(r = 0.447, p < 0.05)

3) In the memory session the following positive correlations were observed for

the:

Hard cues:

- Left hippocampus – left (r = 0.425, p < 0.05) and right ventral striatum (r =

0.529, p < 0.01) and left caudate nucleus (r = 0.530, p < 0.01)

- Right hippocampus – left (r = 0.621, p < 0.005) and right ventral striatum (r

= 0.517, p < 0.05); and the right VTA (r = 0.462, p < 0.05)

- Left ventral striatum – right VTA (r = 0.667, p < 0.001)

Easy cues:

- Right hippocampus – left (r = 0.483, p < 0.05) and right ventral striatum (r

= 0.584, p < 0.005); and the left VTA (r = 0.453, p < 0.05)

- Left ventral striatum – left (r = 0.677, p < 0.001) and right VTA (r = 0.672, p

< 0.001)

- Right ventral striatum – left (r = 0.671, p < 0.001) and right VTA (r = 0.647,

p < 0.005)

241

Appendix 3: Behavioral Questionnaires Administered in Experiments 1-3

Experiments 1 & 2: Pre-Experiment Motivation Questionnaire:

Now that you understand the game, I’d like to ask you a few questions before we begin:

1. How interested are you in playing this game? 1 2 3 4 5 6 7 Not interested, Extremely Here for the money Interested 2. How motivated are you to succeed at this game? 1 2 3 4 5 6 7 Not at all Extremely Motivated 3. How excited are you to play this game?

1 2 3 4 5 6 7 Not at all Extremely Excited 4. How excited are you about the money you could win based on your performance? 1 2 3 4 5 6 7 Not at all Extremely Excited

242

Experiment 1: Behavioral Questionnaire Administered Following MRI Task:

Object

H or L 1 2 3 4 5 6 7

ConfidenceValue

H or L

H or L

H or L

H or L

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

Very Somewhat Very unsure sure





H or L 1 2 3 4 5 6 7

H or L

H or L

H or L

H or L

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7






243


Object ConfidenceValue

H or L 1 2 3 4 5 6 7Very Somewhat Very unsure sure

H or L

H or L

H or L

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7Very Somewhat Very unsure sure



H or L 1 2 3 4 5 6 7

H or L

H or L

H or L 1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7





244

Object

Percent of time you thought this shape was H?________Percent of time you thought this shape was L?________

Percentage








245

Object Like Scale

1 2 3 4 5 6 7Dislike Neutral Like

very much very much


very much very much


very much very much


very much very much

1 2 3 4 5 6 7Neutral Like

very much


very much


very much


very much

Dislikevery much

Dislikevery much

Dislikevery much

Dislikevery much

246

• Did you use any strategies to solve this task? Please try to describe how

you played this game:

• Did you find it difficult to learn from both the arrows and the checks and Xs

in the second part? Which one was easier?

• When trying to determine the value of each object, did you rely more on

what you learned in the first or the second part of the game?

• What did you think happened in the second part of the game? Did you

think the values changed? If so, did you think they all changed or only

some of them?

247

Experiments 1 & 2: Post-Experiment Motivation Questionnaire:

Now that you’ve played the game, I’d like to ask you a few questions:

1. How interested were you when playing this game? 1 2 3 4 5 6 7 Not interested, Extremely Here for the money Interested 2. How motivated were you to succeed at this game? 1 2 3 4 5 6 7 Not at all Extremely Motivated 3. How excited were you to play this game?

1 2 3 4 5 6 7 Not at all Extremely Excited 4. How excited are you about the money you just won based on your performance? 1 2 3 4 5 6 7 Not at all Extremely Excited

248

Experiment 3: Pre-Experiment Motivation Questionnaires Administered on Days

1 &2:

Now that you understand the game, I’d like to ask you a few questions before we begin:

1. How interested are you in playing this game? 1 2 3 4 5 6 7 Not interested, Extremely Here for the money Interested 2. How motivated are you to succeed at this game? 1 2 3 4 5 6 7 Not at all Extremely Motivated 3. How excited are you to play this game?

1 2 3 4 5 6 7 Not at all Extremely Excited

249


ValueShape Confidence Scale

H or L1 2 3 4 5 6 7




H or L1 2 3 4 5 6 7



H or L1 2 3 4 5 6 7


H or L1 2 3 4 5 6 7



250

Shape Estimate of Percentage

Percent of time you thought this shape was high?_____Percent of time you thought this shape was low? _____








251

Shape Like Scale1 2 3 4 5 6 7Dislike Neutral Like very much very much

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

1 2 3 4 5 6 7

Dislike Neutral Like very much very much







252

• Did you use any strategies while trying to determine the value of the shapes?

Please try to describe how you played this game:

• Which learning scenario was most difficult for you: shape learning session,

remember the scenes session, or the perception decision session, and why?

• How difficult did you find it to try to simultaneously learn the value of the shape

while recalling whether or not the natural scene was old or new?

• How accurate do you think you were at remembering which natural scenes were

old and which are new?

253

Experiment 3: Post-Experiment Motivation Questionnaire Administered on Day 1 (Scene Encoding Session): Now that you’ve played the game, I’d like to ask you a few questions:


1 2 3 4 5 6 7 Not at all Extremely Excited 4. How successful do you think were you at attending to the natural scenes?

1 2 3 4 5 6 7 Not at all Extremely Successful 5. Did you use any strategies to try to learn and remember the natural scenes? If so

please describe what you did during the task to try to learn the scenes:

254

Experiment 3: Post-Experiment Motivation Questionnaire Administered on Day 2

(MRI Session):

Now that you’ve played the game, I’d like to ask you a few questions:


1 2 3 4 5 6 7 Not at all Extremely Excited 4. How successful do you think were you at attending to the natural scenes?

1 2 3 4 5 6 7 Not at all Extremely Successful 5. How successful do you think you were at remembering which scenes are old and

which scenes are new?

1 2 3 4 5 6 7 Not at all Extremely

Successful 6. Did you use any strategies to try to remember the natural scenes? If so please

describe what you did during the task to try to remember the scenes:

7. Since the shapes were presented to you in pairs ([circle square] [club triangle] [diamond sun]) did you assume that one shape in each of the pairs was higher than 5 and one was lower than 5? Or did you treat each shape independently?

255

Vita

Kathryn Cain Dickerson

1983 Born January 16th in Syracuse, New York.

2001 Graduated from Tully High School, Tully, New York.

2001-2006 Attended the University of Rochester, Rochester New York.

2005-2006 Take 5 Scholar at the University of Rochester, Rochester New

York.

2006 B.A. in Brain and Cognitive Sciences, University of Rochester,

Rochester, New York.

2006-2011 Graduate work in Behavioral and Neural Science, Rutgers

University, Newark, New Jersey.

2008 Article: “Distinct mechanisms of impairment in cognitive ageing and

Alzheimer’s disease,” Brain, vol. 131(6), p. 1618-1629.

2010 Article: “Parallel contributions of distinct human memory systems

during probabilistic learning,” NeuroImage, vol. 55(1), 266-276.

2011 Article: “Reward-related learning via multiple memory systems”,

Biological Psychiatry, submitted.

Documents

© 2011 Kathryn Cain Dickerson ALL RIGHTS RESERVED