MOVING THROUGH THE BRAIN: A STUDY OF MOVEMENT

PREPARATION IN THE OCULOMOTOR AND REACH SYSTEMS

A DISSERTATION

SUBMITTED TO THE NEUROSCIENCES PROGRAM

AND THE COMMITTEE ON GRADUATE STUDIES

OF STANFORD UNIVERSITY

IN PARTIAL FULFILLMENT OF THE REQUIREMENTS

FOR THE DEGREE OF

DOCTOR OF PHILOSOPHY

Rachel Stern Kalmar

August 2010

http://creativecommons.org/licenses/by-nc/3.0/us/

This dissertation is online at: http://purl.stanford.edu/sk937qn7081

© 2010 by Rachel Stern Kalmar. All Rights Reserved.

Re-distributed by Stanford University under license with the author.

This work is licensed under a Creative Commons Attribution-Noncommercial 3.0 United States License.

ii

I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.

William Newsome, Primary Adviser

I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.

Krishna Shenoy, Primary Adviser

I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.

Tirin Moore

I certify that I have read this dissertation and that, in my opinion, it is fully adequatein scope and quality as a dissertation for the degree of Doctor of Philosophy.

Brian Wandell

Approved for the Stanford University Committee on Graduate Studies.

Patricia J. Gumport, Vice Provost Graduate Education

This signature page was generated electronically upon submission of this dissertation in electronic format. An original signed hard copy of the signature page is on file inUniversity Archives.

iii

Abstract

Movement preparation allows the rapid and accurate execution of voluntary move-

ments, and can be influenced by factors that may change from moment to moment,

such as attention and differences in stimulus properties. Consequently, movement

preparation unfolds differently across many repetitions of the same movement. Av-

eraging neural responses across many repetitions is necessary to interpret single-cell

recordings, but diminishes our ability to characterize the dynamics of the underlying

process. A central question in neuroscience, and also of fundamental clinical impor-

tance, is to understand how these plans develop in the brain. Several research groups

are starting to build prosthetic devices that are controlled directly by neural activity

in motor areas of the brain (Nicolelis, 2001; Donoghue, 2002; Musallam et al., 2004;

Schwartz, 2004; Santhanam et al., 2006; Hochberg et al., 2006; Mulliken et al., 2008;

Andersen et al., 2010), but the extent to which these can be developed may hinge crit-

ically upon our understanding of the neural basis of motor preparation. Simultaneous

recording from populations of neurons allows dynamics of movement preparation to

be estimated on single trials. Our goal is to characterize these dynamics, to gain

insight into the process underlying movement preparation.

Here, we recorded peri-saccadic activity from ensembles of neurons in an oculo-

motor area, prearcuate cortex, in two monkeys. While monkeys performed visually-

guided eye movements, we measured firing rates of a population of neurons using a

96-electrode array. We studied (1) the responses from a heterogeneous population

of prearcuate cortex neurons involved in decision-making and movement preparation,

(2) the relationship between saccade initiation times and responses from the neural

population, and (3) how these responses compared to those recorded in PMd, a cor-

tical area involved in arm movements. The array data from prearcuate allowed us to

compare responses from individual neurons with previous findings, but also allowed

iv

us to analyze the population dynamics of movement planning, by using techniques

applied to the reach system.

We found that ensemble responses from diverse populations of prearcuate neurons

(1) can be visualized as trajectories moving through a low-dimensional neural state

space, (2) reflect visual, decision-, and movement-related aspects of the task, and

(3) correlate with a monkey’s reaction time on a trial-by-trial basis. Further, the

single-trial relationship between ensemble activity in prearcuate cortex and saccadic

reaction times was qualitatively and quantitatively very similar to the relationship

between ensemble activity in PMd and corresponding reach reaction times. This

framework for analyzing neural population activity and dynamics should permit new

extensions of single-neuron-level models, and may offer further insight into general

mechanisms of movement preparation across motor systems.

v

Acknowledgements

Having done neuroscience research as an undergraduate, I had some idea of what

graduate research would be like. However, what I learned during my six years at

Stanford was more multidimensional than I could have guessed, and it far surpassed

my expectations. I credit this to the amazing people who have been my mentors,

colleagues, classmates and friends during my graduate career.

First and foremost, I want to thank my advisors, Bill Newsome and Krishna

Shenoy. I appreciate their guidance and support, but also that they allowed me the

space to figure things out on my own, to follow my intellectual curiosity, and pushed

me to learn how to make and stick to a timeline. Not only were Krishna and Bill

excellent to work with individually, but working with both together was even better.

Each has a special talent for communicating complex concepts clearly, boundless

enthusiasm and courage for trying new things, and the critical insight needed to find

flaws in experimental design or analyses. I benefited greatly from our joint weekly

meetings. I thank my advisors for their collaborative mentorship, for their patience,

and for their friendship. My thesis committee members, Tirin Moore and Brian

Wandell, have also been valuable mentors, providing constructive feedback about my

research plan in its many, many iterations. I left each of my committee meetings

feeling energized and excited about science.

The Shenoy and Newsome labs are stimulating and encouraging environments,

and I feel privileged to have been a part of both. In particular, I owe thanks to

John Reppas in the Newsome Lab. John taught me how to do awake, behaving

monkey experiments, mentored me through my rotation and throughout my time in

lab, generously shared his array data with me, gave critical feedback when needed,

and always had time to chat about nitty gritty or big-picture questions. In the

Shenoy Lab, discussions with Mark Churchland about monkey training and the motor

vi

control literature, and conversations with Byron Yu, John Cunningham, Afsheen

Afshar and Zuley Rivera about dimensionality reduction techniques and data analyses

were incredibly useful. I am also grateful for the expert care and indispensable training

assistance of the technicians in both labs: Jessica Powell, Stacy Rosenbaum, Jamie

Sanders (Newsome Lab) and Mackenzie Risch (Shenoy Lab). I am indebted to Tex

and Vito for the data in this thesis, and Ham, Larry and Isaac not only for providing

data but also for teaching me how to work with non-human primates. The lessons I

learned about behavior have proved invaluable in working with primates, macaques

and humans alike.

The ‘Neurosci-X’ community in the Clark Center and the greater Stanford neuro-

science community have provided rich intellectual—and fun—environments in which

to learn and grow. During my time here, Bill Newsome and John Huguenard had the

leadership and vision to steer the Neurosciences Program to where it is today, and

Rebecca Wyse and Ross Colvin worked tirelessly to keep all the gears well-oiled and

running. In addition to the neuroscience community, I would like to acknowledge the

Bio-X community as well. I am thankful for the Bio-X Graduate Fellowship I was

awarded, and for the ample opportunities in the vibrant Bio-X community to learn

about research outside my field as well as to present my own work. Jill Sakata and

Heideh Fattey have been especially helpful coordinators of the fellowship program

and the Bio-X endeavor overall. I feel honored to have been a member of the Bio-X

and neuroscience communities at Stanford, and to have been surrounded by so many

interesting, well-rounded, happy and motivated colleagues.

Outside of my thesis work, I have been extremely fortunate to have had the

opportunity to follow other passions at Stanford. Taking classes through the Hasso

Plattner Institute of Design (better known as the d.school) was my first introduction

to “design thinking”, with which I instantly fell in love. Thanks to everyone I’ve

worked with at the d.school, and particularly to Jim Patell and Dave Beach for having

the vision to see how a neuroscientist could fit into the Entrepreneurial Design For

Extreme Affordability class. It was a life-changing experience. Thanks also to Dan

Klein, for putting improvisation in my creative toolbox. Since my first d.school class,

I have integrated design thinking into every aspect of my work and my life.

I also feel privileged to have had access to wonderful support resources at Stanford.

Without the Hume Writing Center, the Center for Career Development (especially

vii

Chris Pohalski), and programs supported by the Vice Provost for Graduate Edu-

cation like WISE (Women in Science and Engineering), my graduate career would

undoubtedly have been far less pleasant.

Several other people have played pivotal roles in my career. When I was an

impressionable college freshman, Ken Reisman and Matt Simon took me under their

wings, introducing me to scientific research and setting me on the path that ultimately

brought me to graduate school. I will be forever grateful to E.J. Chichilnisky, for

taking a chance and letting a completely inexperienced and naive undergraduate join

his lab. Working with E.J. completely changed the trajectory of my career. Over the

nearly 5 years I spent in his lab, E.J. was a remarkably supportive mentor, teacher

and friend.

I would also like to thank my friends and family for their tireless support and

encouragement. Whether collaborating on projects, traveling together to destinations

near and far, or sitting through yet another round of practice talks, it has been great

to have such an extraordinary network of people I can rely on. Of my friends, I owe

special thanks to Jim Sowers. Jim pushed me to achieve my goals in school and out

and also shared some of my favorite moments during my time at Stanford, including

unicycling across Nova Scotia! Finally, I would like to acknowledge the unconditional

love of my immediate and extended family, and in particular of my parents, my own

special forces team. I am very lucky to have such a caring, unique and interesting

family. It has been a joy and a privilege to share my graduate school journey with so

many wonderful people.

viii

Contents

Abstract iv

Acknowledgements vi

1 Introduction 1

1.1 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 The study of movement preparation . . . . . . . . . . . . . . . . . . . 2

1.2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1.3 Neural basis of movement preparation . . . . . . . . . . . . . . . . . . 4

1.3.1 Early work about movement centers in the brain . . . . . . . . 4

1.3.2 Evidence for movement preparation . . . . . . . . . . . . . . . 5

1.3.3 Movement centers of the frontal lobe . . . . . . . . . . . . . . 8

1.4 Somatic movement in the frontal lobe . . . . . . . . . . . . . . . . . . 9

1.5 Eye movement centers in the frontal lobe . . . . . . . . . . . . . . . . 10

1.5.1 Arcuate concavity . . . . . . . . . . . . . . . . . . . . . . . . . 15

1.5.2 FEF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

1.5.3 8Ar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

1.6 Linking movement preparation to behavior . . . . . . . . . . . . . . . 23

1.6.1 Models and paradigms for studying movement preparation in

the oculomotor system . . . . . . . . . . . . . . . . . . . . . . 23

1.6.2 Extending single-neuron models to a population . . . . . . . . 25

1.7 Dynamics of movement preparation . . . . . . . . . . . . . . . . . . . 26

1.7.1 Motivation to study dynamics . . . . . . . . . . . . . . . . . . 26

1.7.2 Open questions: using state-space approaches in the oculomo-

tor system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28

1.8 Questions addressed here . . . . . . . . . . . . . . . . . . . . . . . . . 29

ix

2 Response properties of prearcuate cortex 30

2.1 Background & Motivation . . . . . . . . . . . . . . . . . . . . . . . . 30

2.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

2.2.1 Subjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

2.2.2 Task design and training . . . . . . . . . . . . . . . . . . . . . 32

2.2.3 Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

2.2.4 Array locations and neural recordings . . . . . . . . . . . . . . 35

2.2.5 Dimensionality reduction of neural data . . . . . . . . . . . . 39

2.2.6 Computing the ROC predictive index . . . . . . . . . . . . . . 41

2.3 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

2.3.1 Tuning properties of neurons / cell classification . . . . . . . . 41

2.3.2 Diversity of PSTHs . . . . . . . . . . . . . . . . . . . . . . . . 45

2.3.3 Visualizing population activity . . . . . . . . . . . . . . . . . . 45

2.3.4 Decision-related activity . . . . . . . . . . . . . . . . . . . . . 46

2.3.4.1 Visualizing the effect of coherence on population activity 49

2.3.4.2 Dimensionality of data . . . . . . . . . . . . . . . . . 52

2.3.5 Effects of choice and reward history: this trial . . . . . . . . . 54

2.3.6 Effects of choice and reward history: subsequent trials . . . . . 56

2.4 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

2.4.1 Bias in population measures of choice predictivity . . . . . . . 59

2.4.2 Dimensionality vs. epoch . . . . . . . . . . . . . . . . . . . . . 61

2.4.3 History effects: rethinking the idea of ‘baseline’ . . . . . . . . 62

2.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

3 Population activity in prearcuate cortex accounts for variance in

saccadic latencies 65

3.1 Chapter Intro . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.2 Background & Motivation . . . . . . . . . . . . . . . . . . . . . . . . 66

3.3 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

3.3.1 Subjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

3.3.2 Task design and training . . . . . . . . . . . . . . . . . . . . . 68

3.3.3 Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

3.3.4 Array locations and neural recordings . . . . . . . . . . . . . . 69

3.3.5 Datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

x

3.3.6 Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

3.3.7 Dimensionality reduction of neural data . . . . . . . . . . . . 70

3.3.8 Analysis and presentation . . . . . . . . . . . . . . . . . . . . 70

3.4 Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

3.4.1 Diverse neural responses in prearcuate . . . . . . . . . . . . . 71

3.4.2 Neural trajectories evolve along a stereotyped path . . . . . . 72

3.4.3 Trial-by-trial relationship between neural dynamics and RT . . 74

3.4.4 RT correlation varies with a number of factors . . . . . . . . . 79

3.4.5 Does simultaneity matter? Comparing results for single cells

and populations . . . . . . . . . . . . . . . . . . . . . . . . . . 80

3.4.6 Dimensionality of neural data . . . . . . . . . . . . . . . . . . 83

3.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

3.5.1 Saccadic preparation in prearcuate . . . . . . . . . . . . . . . 89

3.5.2 Comparison between dynamics in prearcuate and PMd . . . . 90

3.5.3 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

4 PMd experiments 93

4.1 The speed limits of planning . . . . . . . . . . . . . . . . . . . . . . . 93

4.1.1 The prosthetic cursor task . . . . . . . . . . . . . . . . . . . . 93

4.1.1.1 Tuning curves are modulated by chain position . . . 96

4.1.1.2 Inter-monkey differences . . . . . . . . . . . . . . . . 98

4.1.1.3 Gain field analogy . . . . . . . . . . . . . . . . . . . 100

4.1.1.4 State-space model for investigating cortical differences 101

4.1.2 Future directions . . . . . . . . . . . . . . . . . . . . . . . . . 102

4.2 Planning with uncertainty . . . . . . . . . . . . . . . . . . . . . . . . 103

4.3 Neural dynamics and muscle activity . . . . . . . . . . . . . . . . . . 111

4.4 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

5 Conclusion 116

5.1 Summary & Contributions . . . . . . . . . . . . . . . . . . . . . . . . 116

5.2 Array recording . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

5.2.1 Pros and cons of array recording . . . . . . . . . . . . . . . . . 119

5.2.1.1 Advantages . . . . . . . . . . . . . . . . . . . . . . . 119

5.2.1.2 Challenges of array placement . . . . . . . . . . . . . 120

xi

5.2.1.3 Other challenges of array recording . . . . . . . . . . 122

5.2.2 What can and cannot be addressed by multielectrode recording

and state-space approaches? . . . . . . . . . . . . . . . . . . . 124

5.3 Open questions & Future directions . . . . . . . . . . . . . . . . . . . 125

5.4 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 127

xii

List of Tables

5.1 PMd arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

5.2 Prearcuate arrays . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

xiii

List of Figures

1.1 Mean RT vs. delay period duration (reaches) . . . . . . . . . . . . . . 3

1.2 Mean RT vs. delay period duration (saccades) . . . . . . . . . . . . . 4

1.3 Motoric areas of the brain . . . . . . . . . . . . . . . . . . . . . . . . 6

1.4 Organization of frontal cortex . . . . . . . . . . . . . . . . . . . . . . 9

1.5 The arcuate concavity contains FEF and 8Ar . . . . . . . . . . . . . 12

1.6 Prearcuate recordings . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

1.7 Eye movements evoked by microstimulation . . . . . . . . . . . . . . 20

1.8 FEF connectivity schematic . . . . . . . . . . . . . . . . . . . . . . . 21

1.9 Oculomotor connectivity schematic . . . . . . . . . . . . . . . . . . . 22

1.10 The rise-to-threshold hypothesis . . . . . . . . . . . . . . . . . . . . . 26

1.11 Optimal subspace hypothesis . . . . . . . . . . . . . . . . . . . . . . . 28

2.1 Behavioral tasks: delayed saccade and direction discrimination . . . . 33

2.2 Array location in prearcuate and PMd . . . . . . . . . . . . . . . . . 36

2.3 Array placement, with respect to arcuate and principal sulci . . . . . 36

2.4 Diversity of waveform signal-to-noise ratios . . . . . . . . . . . . . . . 38

2.5 Diversity of inter-spike interval statistics . . . . . . . . . . . . . . . . 39

2.6 Polar tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

2.7 Tuning during the delayed saccade task . . . . . . . . . . . . . . . . . 44

2.8 Average activity of population: low-dimensional representation . . . . 46

2.9 Coherence effects: individual unit PSTHs . . . . . . . . . . . . . . . . 47

2.10 Coherence effects: individual unit ROC . . . . . . . . . . . . . . . . . 48

2.11 ROC predictive index comparison between prearcuate and LIP . . . . 50

2.12 Low-dimensional representation of population activity during dots task. 51

2.13 Measuring distance between mean trajectories as a function of coher-

ence and time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

xiv

2.14 Relative proportion of shared variance explained varies across epochs 53

2.15 Behavioral task: direction discrimination, hold period . . . . . . . . . 54

2.16 Hold period effects represent previous action or previous choice . . . . 55

2.17 Low-dimensional representation of population activity during hold period 56

2.18 History effects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

3.1 Behavioral task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

3.2 PSTHs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3.3 Neural trajectories . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

3.4 Schematic of projection along mean trajectory . . . . . . . . . . . . . 75

3.5 Reaction time correlation . . . . . . . . . . . . . . . . . . . . . . . . . 77

3.6 Correlation coefficients vs. reference offset . . . . . . . . . . . . . . . 78

3.7 Correlation coefficients are influenced by many factors . . . . . . . . . 81

3.8 Measuring curvature . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

3.9 Does simultaneity matter? One example dataset . . . . . . . . . . . . 82

3.10 Does simultaneity matter? Looking across datasets . . . . . . . . . . 83

3.11 Time-courses of GPFA projections . . . . . . . . . . . . . . . . . . . . 85

3.12 Relative proportions of variance explained . . . . . . . . . . . . . . . 86

3.13 Hypotheses relating neural preparatory activity to reaction time . . . 87

3.14 Comparison of example prearcuate and FEF neurons . . . . . . . . . 89

4.1 The chain task . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

4.2 Tuning does not depend on position in chain . . . . . . . . . . . . . . 96

4.3 Chain position modulates firing rate . . . . . . . . . . . . . . . . . . . 97

4.4 Gain histograms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

4.5 Differences in placement of electrode arrays . . . . . . . . . . . . . . . 99

4.6 Trial structure and reward schedule . . . . . . . . . . . . . . . . . . . 100

4.7 Gain analogy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100

4.8 Gain in state space . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

4.9 Schematic of a proposed task for investigating neural dynamics of rapid

plan-switching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102

4.10 Delayed reach and distractor tasks . . . . . . . . . . . . . . . . . . . 105

4.11 Trajectories during the delay period, with and without a distractor

present . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

xv

4.12 State-space hypotheses for representing plans with uncertainty . . . . 107

4.13 Sample trajectories from the ambiguous period of the distractor task 108

4.14 Shuffling test and occupancy rate . . . . . . . . . . . . . . . . . . . . 109

4.15 Bias represented in trajectories . . . . . . . . . . . . . . . . . . . . . 110

4.16 Low-dimensional neural state-space related to muscle activity . . . . 112

4.17 Illustration of neural curvature axis (NCA) computation . . . . . . . 113

4.18 NCA projections predict EMG features . . . . . . . . . . . . . . . . . 114

5.1 Comparison of waveforms found using different spike-sorting techniques 124

xvi

Chapter 1

Introduction

1.1 Motivation

Fast, accurate movements are critically important to survival. An animal waiting

to catch its prey may only have one chance—a poorly executed pounce could mean

no dinner, or worse. Neural circuits in the brain must evaluate and choose between

multiple motor plans, using incoming sensory information to decide where and when

to move. The brain’s capacity to integrate sensory information with prior information

and statistics about the world allows us to prepare movements in advance. This

planning is critical for executing fast and accurate movements. A central question

in neuroscience, and also of fundamental clinical importance, is to understand how

these plans unfold in the brain. This is an active area of research both in the context

of eye and arm movements. As an example of the potential translational importance

of understanding planning in the arm system, several research groups are starting to

build prosthetic devices that are controlled directly by neural activity in motor areas

of the brain (Nicolelis, 2001; Donoghue, 2002; Musallam et al., 2004; Schwartz, 2004;

Santhanam et al., 2006; Hochberg et al., 2006; Mulliken et al., 2008; Andersen et

al., 2010). However, the extent to which these can be developed may hinge critically

upon our understanding of the neural basis of motor preparation. This thesis aims

to further our understanding of movement preparation on timescales relevant for

behavior by employing array recordings in prearcuate cortex, an oculomotor area.

1

2

1.2 The study of movement preparation

In early studies of motor control, researchers catalogued classes of standard move-

ments. Comparing the amount of time needed for movements in each class allowed

evaluation of different models of movement processing (Keele, 1968). From these

studies, researchers observed that subjects’ speed and accuracy depends on many

variables, including direction and extent of an instructed movement, and how far in

advance of a go cue the target was specified (Keele, 1968). Subsequent behavioral

studies provide strong evidence that voluntary movements are prepared before they

are executed (Rosenbaum, 1980; Riehle and Requin, 1989; Crammond and Kalaska,

2000; Churchland et al., 2006). The process of movement preparation takes time,

and without adequate time to plan (∼150-250 ms), subjects take longer to initiate

the movement once instructed to do so. With adequate time to plan, subjects can

initiate their movements 30-40 ms faster. One way to determine how long the motor

system needs to prepare an instructed movement is to measure the latency between

when subjects are given a ‘go cue’ and when they start moving. This latency before

movement initiation is what we will refer to as a subject’s reaction time (RT).

Instructed delay vs. RT tasks

Studies of movement preparation come in two general categories: reaction time tasks

and instructed delay tasks. In the first category of tasks, information about the target

location is presented simultaneously with the go cue. In these reaction time tasks,

subjects are allowed to move to the target as soon as they are ready, thus subjects’

reaction times include the time needed to plan a movement, in addition to how long

it takes to trigger the movement after it is fully planned. By contrast, instructed

delay tasks allow separation of the processes of movement preparation and movement

initiation. In these tasks, information about an upcoming movement is separated

from the go cue by a delay. Thus, subjects are able to use the delay period to plan

movements before they are allowed to move. In this paradigm, reaction times measure

to what extent the process of movement preparation was complete before the go cue,

giving us insight into how long it takes for the motor system to translate a plan into

a movement.

Several studies show that the length of the delay period between the instruction

3

cue and the go cue influences subjects’ reaction times. This effect is consistent across

variations in tasks, subjects, and target locations (Pare and Munoz, 1996). Figure 1.1

illustrates this trend for three monkeys performing a reaching task in our lab: the

shorter the delay period preceding the movement cue, the longer was the animals RT

(Churchland et al., 2006). Figure 1.2 shows the same relationship between saccadic

reaction times and delay period duration in a delayed saccade task (data from Mark

Churchland, unpublished). This effect has also been described in the saccade system

by Schall and Hanes (1993).

Subjects’ reaction times are faster not only when given more time to plan a move-

ment, but also when given more information about the movement to perform. Ambi-

guity or incomplete information about the movement instruction increased subjects’

reaction times (Posner et al., 1980; Rosenbaum, 1980). Rosenbaum (1980) presented

some, none, or all of the information about the required movement in advance of

the go cue, and measured the related changes in reaction time. Providing subjects

with information about the direction and extent of a movement in advance of a go

cue allowed them to initiate movements more rapidly. This suggested that reaction

time reflects the time needed to specify parameters of the upcoming movement that

were not known in advance. Churchland et al. (2006) use this evidence to support

their hypothesis that there is some time-consuming preparatory process that starts

happening during the delay.

30 430

220

260

monkey H

delay period (ms)

RT

(m

s)

Figure 1.1: Mean reach RT vs. delay period duration. Data from monkeys A and Bare from Churchland et al., 2006, Fig. 4. Reprinted with permission.

1.2.1 Overview

Building on this behavioral evidence for movement preparation, we will give an

overview of the motor systems of the frontal lobe, focusing on the role of specific areas

4

220

260

RT

(m

s)

0 300delay period (ms)

Figure 1.2: Mean saccadic RT vs. delay period duration. Data from Mark Churchland(unpublished).

in preparing and initiating movements. We will review the somatic motor system,

focusing on the role of premotor cortex in movement preparation. Drawing parallels

between the somatic and oculomotor systems, we will then describe the premotor

areas of the frontal lobe involved in saccade planning and execution.

A number of models have been proposed to explain variability in RT as it relates

to movement preparation. In the second half of this chapter, we will discuss some of

these models with respect to the neural mechanisms of movement preparation.

1.3 Neural basis of movement preparation

1.3.1 Early work about movement centers in the brain

The neural basis of how movements are prepared and executed has captured the

interest of scientists for well over a century. In 1870, Gustav Fritsch and Eduard

Hitzig pioneered the field of modern electrophysiology with their finding that electrical

stimulation of the motor cortex evoked contralateral movements (Taylor and Gross,

2003). Four years later David Ferrier built upon this work, more finely mapping

out the movement centers of the brain. Injecting current into different regions, he

catalogued the eye, limb, and body movements that were evoked (Ferrier, 1874).

The next century of research on the motor system built on these foundations,

using electrical stimulation and lesion studies to refine Ferrier’s maps of cortical and

subcortical motor areas. However, the first experiments that connected recordings

of individual neurons in these regions with voluntary movement of monkeys’ eyes or

limbs did not take place until nearly 100 years later.

5

Using the same microelectrode recording techniques that were being used to study

cells’ responses in anesthetized animals, Edward Evarts made the first connection be-

tween single-unit physiology and behavior in awake animals. Evarts measured the

activity of individual pyramidal tract neurons in precentral motor cortex while a

monkey made hand movements (Evarts, 1966). Evarts’ series of experiments demon-

strated that activity of pyramidal tract neurons preceded hand movements (Evarts,

1966) and was also related to the force and displacement of a load the monkey con-

trolled with a handle (Evarts, 1968). Around the same time, Emilio Bizzi (1967)

identified oculomotor frontal lobe neurons using antidromic stimulation, and mea-

sured the timing of these neurons’ responses with respect to different types of eye

movements.

Since these early electrophysiology studies, much more has been learned about

the areas of the brain involved in preparing and executing movements. Studies of

cytoarchitecture, connectivity, electrophysiological recordings, electrical stimulation,

and changes in behavior following lesions have helped identify which cortical and sub-

cortical areas contribute to which aspects of behavior (see comprehensive reviews by

Wise, 1985; Schall, 1997). In particular, electrophysiological recordings have allowed

researchers to probe, on fine spatial scales, how individual neurons respond to aspects

of stimuli and upcoming movements. Recording from different collections of individual

cells across the brain, a number of areas involved in eye movements and arm move-

ments have been discovered and characterized (Figure 1.3). Figure 1.3a illustrates

regions of the brain involved in arm movements. Figure 1.3b shows a schematic of

the eye movement areas, and their connections to other visual and oculomotor areas.

Thus, we have a wealth of empirical knowledge about the major movement centers of

the brain.

1.3.2 Evidence for movement preparation

Since much research about movement preparation has been done in the context of

reaching tasks (reviewed by Wise, 1985; Kalaska, 1991; Kalaska et al., 1997), we will

first focus on areas of the frontal lobe involved in arm movements. Then we will

move to a system where the effector dynamics are simpler and discuss the oculomotor

regions of the frontal lobe.

As described in Section 1.2, psychophysical evidence suggests that movements are

6

a

b

Figure 1.3: (a) Regions of the brain involved in arm movements (modified fromAchtman et al., 2007, with permission). AIP, anterior interparietal cortex; cPMd,caudal dorsal premotor cortex; cPMv, caudal ventral premotor cortex; FEF, frontaleye fields; LIP, lateral interparietal cortex; preSMA, pre-supplementary motor area;M1, primary motor cortex; MIP, medial interparietal cortex; rPMd, rostral dorsalpremotor cortex; rPMv, rostral ventral premotor cortex; S1, primary somatosensorycortex; SMA, supplementary motor area; VIP, ventral interparietal cortex. (b) Cor-tical and subcortical oculomotor areas (from Fecteau and Munoz, 2003, adapted bypermission from Macmillan Publishers Ltd: Nature). CN, caudate nucleus; DLPFC,dorsolateral prefrontal cortex; LIP, lateral intraparietal cortex; MRF, medullary retic-ular formation; PPRF, paramedian pontine reticular formation; SEF, supplementaryeye fields; SNp, substantia nigra pars reticulata

prepared before they can be executed. This suggests that movement preparation is a

process unfolding in the brain, that is necessary for the rapid and accurate execution

7

of movements. Where and how does this process occur in the brain? A number of

motor and premotor areas show signals correlated with an upcoming action or motor

plan. Using instructed delay tasks in which the instruction is separated from the go

cue by a variable delay period, collecting neural activity during this delay allows the

study of the neural basis of movement preparation.

One of the first pieces of physiological evidence supporting movement preparation

for specific movements came from Kutas and Donchin (1974). In this study, Kutas

and Donchin instructed subjects with which hand to perform their task, and found

that scalp potentials were greater over the hemisphere contralateral to the respond-

ing hand. The potential was associated with a specific movement, supporting the

hypothesis that movement preparation is specific for an upcoming action. Neural sig-

nals correlated with an upcoming action or motor plan have since been found in many

of the areas shown in Figure 1.3 (e.g., Snyder et al., 2006). This delay-period activity

reflecting motor preparation is common to frontal, premotor, and motor cortex, su-

perior colliculus, and basal ganglia (discussed in Snyder et al., 2006 and Churchland

et al., 2006).

Neural signals related to movement preparation are tuned for parameters of a

specific upcoming action (Tanji and Evarts, 1976; Stoet and Snyder, 2003; Church-

land et al., 2006; Snyder et al., 2006). In the supplementary motor area (SMA),

neural movement preparation signals reflect the dynamics, but not kinematics, of an

upcoming movement (Padoa-Schioppa et al., 2002). Movement preparation signals

can be predictive of accuracy and reaction time for arm movements (Lecas et al.,

1986; Riehle and Requin, 1993; Snyder et al., 2006) and signals related to movement

initiation also predict saccadic reaction time (Hanes and Schall, 1996; Dorris et al.,

1997; Everling and Munoz, 2000; Snyder et al., 2006).

Evidence for a causal role of delay period activity in movement initiation

in PMd

Behavioral and physiological studies provide strong evidence that voluntary move-

ments are prepared before they are executed, and that neural activity represents

advance preparation for specific actions. Churchland et al. (2006) show that, pre-

ceding movements, the variance of neural activity in dorsal premotor cortex (PMd)

decreases and stabilizes, across trials. This supports the hypothesis that the goal of

8

motor planning is to move neural activity into a prepared ‘state’, preceding a mon-

key’s reach in a given direction. In this hypothesis, the across-trial drop in variance

corresponds to the point in time when preparation for this upcoming movement is

complete. The similar timescales for movement preparation found from the time

taken for variance to decrease and the monkey’s RTs on a delayed reach task provide

additional support for this hypothesis. But does delay period activity play a causal

role in movement initiation? Churchland et al. tested this hypothesis by injecting

current into PMd during the delay period when the monkey was planning a reach.

When putative planning activity in these neurons was disrupted with microstimula-

tion, monkeys’ RTs were longer than would be predicted from Fig. 1.1 (Churchland

et al., 2006; Churchland and Shenoy, 2007). These two pieces of evidence, combined

with behavioral results, support the idea that movements are prepared before they

are executed, and that PMd is involved in this movement preparation.

1.3.3 Movement centers of the frontal lobe

To investigate more deeply the neural mechanisms underlying movement preparation,

we will focus on the somatotopic and oculomotor areas of the frontal lobe. The

confluence of visual, cognitive, and motoric signals make the motor and dorsolateral

regions of the frontal lobe a rich area in which to study movement preparation.

The primate frontal lobe can be separated into two general divisions: one motor

and one prefrontal. The motor division contains primary motor cortex (M1) and

several premotor areas, including premotor cortex and SMA (Preuss, 2007). The

prefrontal division is generally split into three parts: a medial region, an orbital

region, and a dorsolateral region. The medial region corresponds to anterior cingulate

cortex, and the orbital region contains orbital frontal cortex. The dorsolateral region,

which contains the area around the arcuate cortex, is involved in the highest levels of

cognitive function, and malfunctions in this area are tied to neuropsychiatric disorders

like schizophrenia (Preuss, 2007). This region is also involved in selective attention,

working memory, and programming behavior in novel tasks. The frontal eye field

(FEF), which is involved in controlling the direction of eye movements and attention,

and the principalis cortex, which is involved in spatial working memory are also

contained in this dorsolateral region (Preuss, 2007).

9

a

b

Figure 1.4: Organization of frontal cortex. (a) From Preuss and Goldman-Rakic(1991), with permission. (b) From Romanski (2004).

1.4 Somatic movement in the frontal lobe

The frontal lobe contains at least 6 premotor areas, each of which projects directly

to primary motor cortex (M1) and to the spinal cord, and each of which has some

degree of somatotopic organization (Dum and Strick, 2002). We will focus here on

premotor cortex, which contains neurons responsive to behaviorally-relevant stimuli

and are involved in movement preparation (Wise, 1985).

10

PMd

Dorsal premotor cortex (PMd) is involved in planning reaching movements (Weinrich

and Wise, 1982; Weinrich et al., 1984; Godschalk et al., 1985). Individual neurons in

PMd are active before movements are initiated, and are tuned for the direction (Geor-

gopoulos et al., 1982; Crammond and Kalaska, 1989), distance (Messier and Kalaska,

2000), and speed (Churchland et al., 2006) of an upcoming arm movement. Addi-

tionally, PMd projects directly to the spinal cord and primary motor cortex (M1),

receives input from a number of parietal areas, and is connected with frontal areas,

via primary, supplementary, and cingulate motor cortices (Wise et al., 1997). Impor-

tantly, neural activity in PMd is causally related to motor preparation (Churchland

and Shenoy, 2007), as well as lower-level aspects of motor control, such as distance

and speed. Being involved in both high level planning and lower-level motor con-

trol makes PMd an ideal place for studying how cognitive and behavioral signals are

integrated to form movement plans.

Many studies of population coding have been done in the somatotopic motor

system. However, it is not known to what extent the principles that govern movement

preparation in somatotopic areas will generalize to the oculomotor system. Eye and

arm movements differ in many ways (e.g., effector speed, weight, number of muscles

required), so one might expect oculomotor and reach systems to utilize different

strategies for motor preparation. By performing our experiments both systems, we

can compare our results in the reach system with populations of oculomotor neurons

with similar premotor properties (e.g., higher stimulation thresholds, fewer direct

connections to movement generators). This allows us to test whether the dynamics of

motor planning generalize to many movement preparation systems in the brain. The

next section will give an overview of the oculomotor signals found in the frontal lobe.

We place a particular emphasis on the motoric signals around the arcuate gyrus, the

area we targeted in our experiments.

1.5 Eye movement centers in the frontal lobe

The oculomotor region of the frontal lobe is involved in visual, motor, spatial and

cognitive functions. The general region around the arcuate concavity, 8A, is involved

in varying degrees of movement preparation (Fig. 1.5) (Bruce and Goldberg, 1985;

11

Seidemann et al., 2002). This area contains FEF and area 8Ar, as illustrated in

Figure 1.5. The functional and anatomical distinctions within the arcuate concavity

have evolved as technological advances have allowed finer mapping and enhanced

resolution of microstimulation, electrophysiological measurements and imaging. It is

clear that we are recording from the arcuate region with our electrode arrays. But

are our arrays in 8Ar or FEF? Understanding the historical and current literature

about this region is crucial for accurate interpretation and context of our results. We

first review what is known about the arcuate concavity and its general neighborhood,

then give an overview of the distinction between FEF and area 8Ar.

In this premotor region, FEF (also called 8Ac (Schall, 1995) or 8Ac and 8Am

(Preuss and Goldman-Rakic, 1991)) is the area most tightly connected to eye move-

ments and participates significantly in the network mediating gaze control. FEF is

located at the posterior aspect of the arcuate concavity and is involved in transform-

ing visual computations guided by cognitive processes into saccade motor commands

(reviewed by Schall, 1997). Just anterior, area 8Ar has some similar response prop-

erties to FEF. Microstimulation in 8Ar evokes eye movements, though with higher

stimulation thresholds than FEF (Bruce and Goldberg, 1985). Responses of neurons

in this area also reflect metrics of eye movements (Seidemann et al., 2002). How-

ever, anatomical and cytoarchitectural studies show that 8Ar is less connected to

direct motor output than FEF (Preuss and Goldman-Rakic, 1991; Schall, 1995; Ger-

bella et al., 2007). Like the distinction between premotor and primary motor cortex

(Wise, 1985), the boundary between 8Ar and FEF cannot be fully distinguished by

cytoarchitectural differences. Rather, FEF and 8Ar are distinguished by microstimu-

lation. Functional relationships between the premotor and motor functions may also

be analogous in these areas.

Anterior of area 8Ar, the cortex surrounding the principal sulcus (PS) plays a

role in spatial working memory (Fuster, 1973; Goldman-Rakic, 1987; Funahashi et

al., 1990). Dorsal to the PS, dorsolateral prefrontal cortex (DLPFC) is involved

in spatial aspects of working memory (Goldman-Rakic, 1987, 1988; Funahashi et

al., 1989; Petrides, 1994; Levy and Goldman-Rakic, 2000). Ventrolateral prefrontal

cortex, ventral to the PS, also plays a role in short-term memory but unlike DLPFC is

not spatially selective. The supplementary eye field (SEF) is another eye-movement

related region near the arcuate sulcus. The SEF is a second order motor area, and

12

Arcuate Concavity

8Ar

FEF

Figure 1.5: The arcuate concavity region contains FEF and 8Ar.

neurons are more responsive to stimuli in the context of behavior than when presented

with the stimulus alone. This area is less understood than FEF. Finally, posterior to

the arcuate sulcus, premotor cortex represents movement preparation or ‘set’, as well

as the visual aspects of the stimulus in the context of movement. Premotor cortex

is activated by sensory stimuli that guide movements, and contains both neurons

that respond to eye movements (more ventrally) and others that respond to arm

movements (more dorsally).

The area from the caudal terminus of the principal sulcus to the caudal bank of

the arcuate sulcus, known as the arcuate concavity (generally corresponding to area

8A, including both FEF/8Ac and 8Ar), is tightly linked to eye movement prepa-

ration. Microstimulating in this region, nearly all locations in principal sulcus and

all neurons in the arcuate sulcus elicit contralateral eye movements (Wagman et al.,

1961; Robinson and Fuchs, 1969; but see Boch and Goldberg, 1989). Additionally, a

high percentage of neurons in this area respond in relation to saccadic eye movements

(Wagman et al., 1961; Funahashi et al., 1991). Many respond before only purposive

saccades, with or without visual guidance (Wagman et al., 1961; Bruce and Goldberg,

1985; Funahashi et al., 1991).

13

14

Figure 1.6: 120 years of prearcuate studies. Overview of areas in frontal cortex relatedto eye movement generation (from Schall, 1997, with permission). (A) Original mapfrom Ferrier (1875). Eye movements were evoked from the shaded region. (B) Mapfrom Mott and Schaefer (1890). Zones from which different directions of eye move-ments were evoked are indicated. (C) Map from Levinsohn (1909). Eye movementswere evoked from the shaded region; the darker shading indicates the lower thresholdregion. (D) Map from Smith (1940). The regions from which the different types ofresponses were observed are indicated. (E) Map from Crosby et al. (1952). The dif-ferent directions of eye movements evoked are indicated. (F) Map from Wagman etal. (1961). Regions from which different directions of eye movements were evoked areindicated by different stipples and line widths. (G) Map from Robinson and Fuchs(1969). Zones are shown from which different amplitude saccades were evoked. (H)Map of visual response field eccentricity from Suzuki and Azuma (1983). (I) Map ofvisual response field size from Suzuki and Azuma (1983). (J) Map of frontal eye fieldfrom Bruce et al. (1985) and Gottlieb et al. (1994). The arcuate sulcus is representedas opened: the thick line is the lip, and the thin dashed line is the fundus. The low-threshold zone is indicated by the shaded region. (K) Location of supplementary eyefield from Schlag and Schlag-Rey (1987); Schall (1991); Parthasarathy et al. (1992).

Historically, the region considered the frontal eye field included the entire anterior

convexity of the arcuate, from the caudal end of the principalis sulcus to the arcu-

ate sulcus (Funahashi et al., 1991). In 1985, Bruce et al. systematically measured

the minimum current needed to evoke an eye movement. They found that, although

saccades could be evoked from this entire region, the region in which saccades were

elicited with small amounts of current (<50 µA) was restricted to the arcuate sulcus

and the adjacent arcuate lip. This smaller region characterized by microstimulation

thresholds is now agreed upon as ‘core’ FEF. The rostral aspect of the arcuate con-

cavity, on the cortical surface adjacent to the arcuate sulcus and bounded rostrally by

the principal sulcus was excluded from FEF, based on higher stimulation thresholds.

This part of the arcuate concavity with higher stimulation thresholds corresponds to

area 8Ar (Preuss and Goldman-Rakic, 1991) or 8r (Gerbella et al., 2009).

We will first review some of the older work that lays the groundwork for current

studies of FEF and 8Ar, which describes response properties from the arcuate con-

cavity (the pre-1985 definition of FEF, roughly area 8A), which includes both areas

(see Fig. 1.5). In the sections that follow, we will describe properties of “core” FEF

(post-1985 distinction) and 8Ar individually.

15

1.5.1 Arcuate concavity

Ferrier’s demonstration that electrical stimulation of the prearcuate gyrus causes con-

traversive eye movements has been reproduced in subsequent experiments in many

neocortical and subcortical sites, including the whole prefrontal convexity. Stimula-

tion in this area has also been shown to evoke movements of the ears, head and body

(Rosenkilde, 1979).

In more recent experiments, contralateral eye movements have been evoked by

stimulating in sites in nearly all of the principal sulcus and all arcuate sulcus (Robin-

son and Fuchs, 1969; Gottlieb et al., 1993). Robinson and Fuchs (1969) showed that

electrical stimulation in this area elicited fixed-vector saccadic eye movements that

were similar to natural, voluntary saccades. This suggests that these microstimula-

tion studies are activating the same circuitry for saccade generation that is used in

the context of voluntary behavior. Therefore, these microstimulation studies allow us

to learn about natural saccade maps.

The caudal prearcuate concavity is a source of polysynaptic projections to ex-

traocular muscles (Moschovakis et al., 2004; Gerbella et al., 2007). This area also has

a functional mapping onto superior colliculus. One piece of evidence for this comes

from anatomic studies of topographic connectivity (Schall, 1997). Also, simultane-

ous stimulation of the arcuate concavity and superior colliculus evokes saccades with

amplitudes and directions that are the vector sum of individual saccades at the site

of stimulation in either area, weighted by stimulation current (Schiller et al., 1979;

Schall, 1997).

Lesion studies have indicated that the arcuate concavity is sufficient but not nec-

essary for saccades (Schall and Thompson, 1999). In experiments where superior

colliculus is lesioned, eye movements can still be made (e.g. Schiller et al., 1980;

Collin et al., 1982). In other experiments, the effects of arcuate concavity lesions

were assessed. Bilateral lesions including the arcuate concavity caused slight visual

discrimination deficits when the whole sulcus was ablated (Rosenkilde, 1979). These

animals were not blind but showed decreased sensitivity to stimuli in the entire visual

field (Rosenkilde, 1979). Though the lesioned animals showed slight abnormalities

in fixation and needed more time to find a circular target among other geometrical

shapes, saccadic eye movements were not impaired (Rosenkilde, 1979). Importantly,

animals had time to recover after lesions, allowing time for alternate eye-movement

16

strategies to develop. This suggests a distributed or redundant mechanism of saccade

generation. This differs from the somatic motor system, where significant movement

deficits are seen with lesions of primary motor cortex.

Physiological recordings from the arcuate concavity have demonstrated that this

area has visual and motor-related signals. A large proportion of these units (∼45%)

have visual receptive fields (Wurtz and Mohler, 1976; Schall, 1997). Suzuki and

Azuma (1983) showed maps of visual receptive field (RF) eccentricity in this area

(Fig. 1.6I), in which RF size increased from lateral to medial, and caudal to rostral.

The activity of visual cells in the arcuate concavity is enhanced if the stimulus is the

target for a saccade (Goldberg and Bushnell, 1981).

The motoric signals in the arcuate concavity are diverse. Neurons in this area

respond in the context of saccades, smooth pursuit, fixation (Bizzi, 1968; Bizzi and

Schiller, 1970), vergence, divergence, and ocular accommodation (Gamlin and Yoon,

2000). Passing current into this area of cortex elicits saccades, ordered topographically

by saccade amplitude (Beevor, 1888). Fig. 1.6 (from Schall, 1997, Fig. 7) summarizes

over a century of research describing the visual and motoric properties of the arcuate

concavity.

After Bruce et al.’s redefinition of FEF as the region of the arcuate concavity

in which eye movements could be stimulated with less than 50 µA of current, most

studies of this area focused on this region of core FEF (primarily within the arcuate

sulcus), excluding the region on the arcuate gyrus, 8Ar.

In addition to differences in stimulating current needed to evoke eye movements,

several lines of evidence speak against 8Ar being part of FEF. First, saccades are

elicited with lower thresholds from sites on the rostral bank of the arcuate sulcus,

in the region corresponding to areas 8Ac, 45a and 45b; higher thresholds are needed

for sites in the anterior convexity in the region corresponding to 8Ar (Bruce et al.,

1985; Stanton et al., 1989; Schall et al., 1995b). Second, there is a higher density of

large layer V pyramidal cells that project to the superior colliculus and brainstem in

FEF than in 8Ar (Stanton et al., 1989). Compared to FEF, 8Ar also has a thicker,

more clearly defined granular layer/layer 4 (Gerbella et al., 2007) and lacks the thick

fascicles of 8Ac (which is included in FEF) (Schall, 1997), and has more conspicuous

bands of Baillarger and radial fibers (Schall, 1995). Finally, 8Ar differs from FEF in

its strength of connections with LIP (Schall, 1997).

17

Despite these findings, two recent studies indicate that the part of the arcuate

cortex directly involved in eye movements occupies a region larger than the classic,

electrical stimulation-defined, core frontal eye field. Using voltage-sensitive dye, Sei-

demann et al. (2002) demonstrated that the area of prearcuate cortex activated by eye

movements extended beyond the region of core FEF and onto the gyrus around area

8Ar. Similarly, Moschovakis et al. (2004) showed that the area in which metabolic

activity is increased around the time of an eye movement extends beyond the smaller

zone corresponding to FEF based on electrical stimulation. The region they describe

includes not only a large part of the anterior bank of the arcuate sulcus, but also

the caudal prearcuate concavity and part of premotor cortex in posterior bank of

the arcuate sulcus (Moschovakis et al., 2004; Seidemann et al., 2002). Are FEF and

8Ar both part of the same oculomotor area, or are they distinct? Barbas’s studies

of projections show that projections to FEF from 8Ar are relatively rare (Barbas

and Mesulam, 1981). This would be more consistent with FEF being the same area,

rather than a different one (Barbas and Mesulam, 1981). These studies, along with

pre-1985 FEF recordings, support a role for both FEF and 8Ar in processing eye

movements. However, to what extent these two areas overlap in their functional roles

is still an open topic of research.

1.5.2 FEF

Since Bruce et al. (1985), the region from which saccadic eye movements can be

evoked with currents of less than 50 µA has been used as the functional definition of

FEF (Tehovnik et al., 2000). Much smaller than the general arcuate concavity, this

area is confined to the anterior bank of the arcuate sulcus (Funahashi et al., 1991).

Earlier work showed that FEF generates signals sufficient but not necessary to

generate saccadic eye movements (reviewed by Schall and Thompson, 1999). However,

more recent studies have shown that reversible inactivation of the FEF impairs the

ability of monkeys to make saccades (Dias et al 1995, Sommer & Tehovnik 1997). This

is consistent with previous observations that initial saccade impairments following

FEF ablation recovered over time (Schall and Thompson, 1999).

Physiological studies show that, similarly to neurons in the broader arcuate con-

cavity, FEF neurons discharge in relation to saccadic eye movements and have visual

responses (Bruce and Goldberg, 1985; Schall et al., 1995a). Roughly half of the

18

neurons respond to visual stimuli, and FEF response fields (RFs) are large and gen-

erally emphasize the contralateral hemifield (Bruce and Goldberg, 1985; Schall et al.,

1995a). Visual responses are also modulated by the presence of stimuli outside their

receptive field and by animals’ readiness to make a saccade (Schall et al., 1995a).

For motoric FEF neurons, responses before saccade initiation reflected whether

or not the stimulus in the response field was the target for the eye movement. Schall

et al. (1995a) demonstrated significant variation in the level of activity in the 50

ms immediately preceding a saccade to the search target, in 96% of their sample of

neurons. This movement-related activity reflected the saccade produced, regardless

of the visual stimulus. This suggests that this activity represents the outcome of the

subjects’ decision process (Funahashi et al., 1991; Schall et al., 1995a).

Saccade amplitude is mapped topographically mapped in the FEF (Schall and

Thompson, 1999). Bruce et al. (1985) found that not only is saccade amplitude

orderly arranged, but also that saccade direction varied with depth of stimulation in

the arcuate sulcus (Tehovnik et al., 2000). Both for stimulation-evoked and natural

saccades, shorter saccades are represented ventrolaterally, and increasingly larger-

amplitude saccades represented dorsomedially (Bruce et al., 1985). Maps of visual

RF size and eccentricity in the rostral bank of the arcuate roughly correspond to the

map of saccade amplitude (Schall, 1997).

Connections of FEF with extrastriate visual cortex and with superior colliculus are

topographically organized (Schall, 1997; Schall and Thompson, 1999). FEF receives

afferents from most prestriate visual cortical areas and most areas in PFC, including

SEF, areas 46 and 12, and anterior cingulate area 24 (Barbas and Mesulam, 1981;

Stanton et al., 1993; Schall et al., 1995a). Projecting to oculomotor structures in-

cluding the caudate nucleus, the deep layers of the superior colliculus and brainstem

pre-oculomotor nuclei (Schall et al., 1995a), place FEF in a position to directly trigger

saccades.

The current definition of FEF includes an area much smaller than the general

arcuate concavity, and corresponds to areas 8Ac, 8Am, 45a and 45b (Preuss and

Goldman-Rakic, 1991; Schall, 1995). FEF lies within the rostral bank of the posterior

curve of the arcuate in frontal cortex of macaques. It extends anteriorly over the lip of

the arcuate a few millimeters onto the surface of the prearcuate gyrus, and posteriorly

a few millimeters onto the caudal bank (Bruce et al., 1985; Tehovnik et al., 2000).

19

Mediolaterally, it extends about 10 mm, centered approximately on the arcuate’s

midpoint as defined by the intersection of the straight extension of the principal

sulcus with the arcuate. However, the ventral and dorsal boundaries of FEF are less

clear (Schall et al., 1995b). The overall area of the FEF is approximately 100 mm2

(Tehovnik et al., 2000).

In terms of cytoarchitecture, area 8 extends only to the lip of the arcuate sulcus,

which excludes part of FEF included in the functional definition (Tehovnik et al.,

2000). However, Stanton et al. (1989) showed that though the location of FEF does

not line up with previously mapped cytoarchitectonic areas, it does coincide with

an area of increased concentration of large layer V pyramidal cells. Along the lip of

the prearcuate gyrus and at dorsomedial and ventrolateral locations outside of the

FEF, there is a marked decrease in number of large pyramidal cells in layer V. Given

that the layer V pyramidal cells are thought to be the major conveyors of oculomotor

signals from the FEF to the superior colliculus and the pons (Tehovnik et al., 2000),

this is consistent with lower thresholds for saccade initiation being found here.

1.5.3 8Ar

Much of what we know about the area immediately anterior to the FEF (current

definition) comes from pre-1985 studies of the arcuate concavity, as well as studies

of the PFC that recorded from this area (e.g., Zaksas and Pasternak, 2006; Kim and

Shadlen, 1999). Two recent studies have also reinforced the evidence that this area

is part of the frontal lobe oculomotor region (Seidemann et al., 2002; Moschovakis et

al., 2004).

8Ar is a transition zone located in the caudalmost prearcuate concavity and is

distinct from the FEF and the caudal part of area 46 (Gerbella et al., 2007). This

area includes the cortical surface between the rostral boundary of 8Ac and area 45a,

and the caudal boundary of area 46 (Schall, 1997). Preuss and Goldman-Rakic (1991)

defined this myeloarchitectonic area 8Ar as shown in Figs. 1.4 and 1.8. Gerbella et

al. (2007) describe 8r as an architectonic area just rostral to the FEF, correspond-

ing to Preuss & Goldman-Rakic’s area 8Ar. Like FEF, this area has high prefrontal

connectivity (88% of labeled cells). However, unlike FEF, this area’s only robust

extrafrontal connection is with dorsal and ventral LIP (Medalla and Barbas, 2006).

20

Figure 1.7: Eye movements evoked by microstimulation along the bank of the arcuatesulcus (from Bruce et al., 1985, with permission).

Less-robust connections are also found with areas MT, FST and V4, however (Ger-

bella et al., 2009). 8Ar also receives input from V2, V3 (very light), V4 (moderate),

TEO (heavy) (Schall, 1995).

As described above, several lines of evidence suggest that 8Ar and FEF are distinct

areas. However, despite differences in microstimulation thresholds, connectivity, and

21

a

b

Figure 1.8: FEF connectivity schematic, from Schall et al. (2002). (a) Frontal lobeoculomotor areas. (b) Relationships between input and output in FEF, SEF andACC. (Reprinted from Neuron, Vol. 36, Schall et al., Monitoring and control of actionby the frontal lobes, p. 309-22, Copyright 2002, with permission from Elsevier.)

neuronal density, much less is known about physiological differences between these

regions.

In general, this region has been shown to play a role in visuospatial functions and in

spatial working memory (e.g., see Levy and Goldman-Rakic, 2000; Romanski, 2004).

Zaksas and Pasternak (2006) described here a class of direction-selective visually

22

responsive neurons, which play an executive role while monkeys performed a direction

discrimination task in which the sample and test stimuli were separated by a brief

memory delay.

Kim and Shadlen (1999) also showed this area to be involved in decision-making.

They recorded from FEF and the posterior third of principal sulcus region (roughly

corresponding to areas 8Ar & 46), that responded selectively when the monkey

planned a saccade to within neurons’ RFs.

Gamlin and Yoon (2000) also showed evidence that this region plays a role in

vergence and ocular accommodation. The recent studies by Seidemann et al. (2002)

and Moschovakis et al. (2004) demonstrated a clear role for this area in oculomotor

preparation. Within the caudal prearcuate concavity, Gerbella et al. (2007) also found

that, similarly to 45B, this area was activated by execution of saccadic eye movements.

Looking across these studies, we see that within this arcuate region, there are areas

specialized for all classes of voluntary eye movements: saccades, smooth pursuit, and

vergence. This area also represents complex cognitive and motor signals.

The bottom line is that without microstimulation or histology, it is difficult if not

impossible to determine whether recordings are from FEF or 8Ar. What is clear is

that both areas contain signals tightly connected to eye movements, along with other

cognitive signals.

a b

Figure 1.9: Oculomotor connectivity schematic (from Schall, 1997, with permission).(a) Projections connecting oculomotor areas across the brain. (b) Schematic ofoculomotor functional connections.

23

1.6 Linking movement preparation to behavior

As we described in the previous section, several brain areas in the frontal lobe are

involved in the preparation and control of somatic and eye movements. In addition,

over the past few decades, much progress has been made in understanding how the

brain uses sensory input to guide behavior. Studies of neural circuitry in monkeys have

revealed a number of interconnected cortical and sub-cortical areas that participate

in sensory-guided decisions and movement planning (Shadlen and Newsome, 1996;

Horwitz and Newsome, 1999; Kim and Shadlen, 1999; Schall, 2001; Coe et al., 2002).

As a monkey receives sensory signals, neural activity in different areas of the brain

reflects aspects of the upcoming action, such as weighing sensory evidence (Newsome

et al., 1989; Shadlen and Newsome, 2001; Romo et al., 2004), comparing the value

of different actions (Barraclough et al., 2004; Sugrue et al., 2004; McCoy and Platt,

2005; Louie and Glimcher, 2010, etc.), or preparing movements (Evarts and Tanji,

1974; Tanji and Evarts, 1976; Wise, 1985). However, comparatively little is known

about how the dynamics of these computations play out across neural circuits or

neural populations.

In the beginning of this chapter, we discussed work supporting the idea that

voluntary movements are prepared before they are executed, and that neurons in

several premotor areas reflect this process. In the next two sections, we focus more

closely on how the state of movement preparation is reflected in neural activity.

1.6.1 Models and paradigms for studying movement prepa-

ration in the oculomotor system

A number of studies addressing the relationship between single-unit neurophysiol-

ogy and reaction time have explained variability in reaction time based on features

of individual neurons’ responses (Hanes and Schall, 1996; Dorris et al., 1997; Schall

and Thompson, 1999). These studies have been conducted both in the context of

instructed delay and RT tasks (introduced in Section 1.2). In RT tasks the processes

of movement preparation and initiation are difficult to distinguish. Therefore, the

studies employing RT tasks generally focus on the process of movement initiation

by measuring neural activity in the few hundred milliseconds preceding a saccade or

reach. In the context of RT tasks, neurons’ firing rates increase rapidly preceding

24

a given movement in motoric cells in many areas of the brain. In many models of

movement preparation, sets of cells must increase their activity toward their indi-

vidual thresholds in order to trigger a movement. These firing rate thresholds are

idiosyncratic to each neuron. Hanes and Schall (1996) discuss two classes of models

that could account for variability in RT given this rise of firing rate to a threshold.

In one class of models, the level of the threshold varies from trial to trial, causing

the increase in firing rate to trigger movements at different times depending on the

trigger’s level. In another class of model, the threshold is fixed and variations in the

speed of firing rate increase account for RT variability. For the FEF neurons they

recorded from, Hanes and Schall’s experiments supported the second class of mod-

els. In this thesis, we will refer to such fixed-threshold, variable-increase models as

rise-to-threshold.

In Hanes and Schall’s formulation, rise-to-threshold models describe the relation-

ship between RT and neural activity correlated with movement initiation. However,

variations of a rise-to-threshold hypothesis have been extended to investigate the

relationship between neural activity and behavior in many different contexts. This

extended class of rise-to-threshold hypotheses has been used to describe RT variabil-

ity in terms of individual cells’ variations in firing rate in many oculomotor areas of

the brain (e.g., Dorris et al., 1997; Roitman and Shadlen, 2002).

In instructed delay tasks, an extended version of the rise-to-threshold hypothesis

predicts that trials with short RTs should correspond to trials in which neurons’

firing rates were higher at the time of the go cue, the subject’s instruction to move.

In the reach system, several groups have also associated higher firing rates with faster

RTs (Lecas et al., 1986; Riehle and Requin, 1993; Bastian et al., 1998, 2003), even

for neurons that were not directionally selective (Bastian et al., 2003). A study

in humans describes a similar finding, demonstrating the relationship between the

latency of subjects’ response and the magnitude of the lateralized readiness potential,

a signal from the scalp that precedes movements (Kutas and Donchin, 1974). This

body of work supports the idea that by measuring neural activity during movement

preparation, we can gain insight into the processes that control timing of behavioral

output. Understanding how temporal agreement about when to move is coordinated

across areas is an open question of interest to many.

Many neurons in many brain areas contribute to each movement (Georgopoulos,

25

1996; Schall and Thompson, 1999). This suggests that the activity of any single neu-

ron is not necessary for movement production. Although Hanes and Schall (1996)

showed that using parameters from a single neuron a simple linear rise to threshold

simulation could accurately predict saccadic reaction time, it is unclear how this re-

lationship holds up for neurons less directly involved in movement initiation, or how

it extends to populations of neurons. Also in the context of a RT task, Thompson et

al. (1996) describe how presaccadic activity in FEF coincides with similar activation

throughout the oculomotor system in a winner-take-all race, and that the ultimate

saccade is made into the movement field of the set of neurons in which the movement

activity first reaches threshold. However, this assumes that each neuron is well de-

scribed by a rise-to-threshold model (Fig. 1.10). The Thompson model describes how

a population of motoric neurons increases its activity right before a saccade. Suppose

that all of the individual neurons did in fact act like rise-to-threshold neurons directly

preceding a saccade. However, it is not at all obvious that we could gain any insight

into movement preparation by studying this activity related to movement initiation

(Churchland et al., 2010). What is the best way to relate behavior to movement

preparatory responses in populations of neurons that may have different tuning and

response properties? This is one of the central questions of our research.

1.6.2 Extending single-neuron models to a population

Though much of the research about the neural basis of movement preparation has

been done in the oculomotor system, it has been in the context of single-electrode

recordings, where the properties of the stimulus were tailored to maximize individual

neurons’ responses. However, it is not clear whether our conceptual understanding of

movement initiation derived from single unit recordings will work when population ac-

tivity related to movement preparation is considered. By performing our experiments

in an oculomotor area, but recording across a population of neurons with different

tuning properties, we can build on previous research to develop more complete models

of movement preparation.

26

go cuetargets on

Ac

tiv

ity

(s

pik

es

/se

c)

Ac

tiv

ity

(s

pik

es

/se

c)

RT

FR - FR0

Extended rise-to-threshold hypothesis

Higher firing rates -> faster RTs

Figure 1.10: The rise-to-threshold hypothesis predicts that the higher the firing rateat the time of the go cue, the faster a saccade can be triggered.

1.7 Dynamics of movement preparation

1.7.1 Motivation to study dynamics

Whether making a decision about noisy sensory input, or trying to use a snapshot

of activity in the brain to control a prosthesis, we want to be able to understand the

dynamics of movement preparation on single trials. Though this work focuses on basic

science, our motivation for studying the dynamics of movement preparation is also

to help validate some of the tools which are important for prosthetics development.

Our overall goal is to understand how movement preparation unfolds on the timescale

relevant for behavior. The brain takes in input from many neurons at once, and uses

whatever signals are available on a given trial to produce a movement. If there is

uncertainty about where to move, the brain needs to find a way to continue planning

the movement, though perhaps leaving the plan more flexible. In the context of

neural prosthetics, understanding how plans are formed on single trials is particularly

important; averaging across trials is not an option.

Individual neurons in different areas of the brain reflect movement preparation in

different ways, but little is known about how these diverse responses fit together as

part of a larger dynamical system. To study the dynamics of movement preparation

at a fine temporal scale, we need to examine how the underlying signals of the neu-

ronal population evolve from moment to moment. Difficulty in interpreting single-cell

responses is often solved by averaging neural activity across many trials; however, this

would possibly obscure idiosyncrasies of neural dynamics that vary from trial to trial.

27

The way we are studying the dynamics underlying movement preparation is to look

at many cells on single trials.

The reason we want to take this approach is that it gives us a richer look at

the system than looking at single units. As the monkey performs his task, there

is a state the system is in. Multielectrode recording gives us a better look at this

hypothesized state, and provides a rich look at the system on a trial-by-trial basis.

This gives us the leverage we need for looking at how the state of the system evolves

on a given trial. Our goal is to trace the state of the underlying system. Is the state

of the system related to eye-movement parameters? In order to convince ourselves

that the state reflects underlying dynamics, we need to confirm that it relates to

behavior. What we want to know is whether the state of the system is tightly linked

to the trial-by-trial parameters of eye movements. This has been looked at in FEF and

superior colliculus in the context of single-electrode recording, but not with resolution

multielectrode recording now offers. Additionally, much of the previous work in the

oculomotor system was framed in the context of studying movement initiation, rather

than movement preparation.

Churchland et al. (2006) explored this in PMd, evaluating how an extension of

the rise-to-threshold hypothesis applies to populations of neurons. They found that

shorter RTs corresponded not to trials when neurons’ firing rates were highest, but

rather, when cells’ firing rates were closest to their cross-trial mean. In the state-

space model they propose, moving close to the cross-trial mean is represented by the

population activity entering an “optimal subspace” (Fig. 1.11). On trials when the

monkey has sufficient time to prepare his movement, the population activity enters

the optimal subspace, and RTs are shorter on these trials.

Afshar et al. (2010) extended the optimal subspace hypothesis, demonstrating that

while RTs are fastest on trials when PMd population activity is close to its mean,

that there is also structure within this optimal subspace. In the optimal subspace

hypothesis, the only factor that affects RT on a given trial is how close the neural

activity is to the heart of the optimal subspace (Fig. 1.11). In contrast, in the extended

rise-to-threshold hypothesis, the only relevant factor is how quickly cells’ firing rates

increase, or in some versions, how high the firing rate is before an upcoming saccade.

Afshar et al. draw on both of these concepts, putting forth their “initial condition

hypothesis.” This hypothesis predicts that within an optimal subspace RTs are even

28

Figure 1.11: Optimal subspace hypothesis (from Churchland et al., 2006, with permis-sion). Each axis in this state space corresponds to a neuron’s firing rate. As neurons’firing rates evolve over the course of a trial, this activity is represented as a trajectorymoving through the state space. Distinct regions of the state space correspond toplans for different movements (illustrated by shaded regions). RTs are shortest whenneural trajectories reach these optimal subspaces by the time the go cue is given.

faster on trials when trajectories are closest to the direction they need to move during

movement initiation. Here, we bring the initial condition hypothesis to the oculomotor

system. We build on this model to explore the relationship between RT and individual

neurons of the population in greater detail.

1.7.2 Open questions: using state-space approaches in the

oculomotor system

The initial condition hypothesis is not incompatible with the rise-to-threshold hy-

pothesis, but rather it aims to describe the neural basis of movement preparation at

the system level. Though the activity of particular neurons may be explained from

single-neuron frameworks like rise-to-threshold, intuition about the dynamics of the

system cannot be built up from individual neurons. In our work, we will also be able

to compare our results to Afshar’s recent study in PMd, in the context of planning

arm movements (Afshar, 2008). This will allow us to test whether the dynamics of

motor planning generalize to many movement preparation systems in the brain.

29

1.8 Questions addressed here

In this chapter, we gave an overview of behavioral and physiological evidence for

movement preparation. We focused on a premotor area related to arm movements

and a premotor area related to eye movements, and asked how movement preparation

is reflected in neural activity of these areas. Discussing some of the models used to

describe movement preparation, we speculated about mechanisms underlying this

process.

In this thesis, we focus our oculomotor studies on prearcuate cortex, an area

involved in eye movement preparation and decision-making. Combining a rich behav-

ioral paradigm with heterogenous populations of neurons recorded on our arrays gave

us the opportunity to: (1) explore this area’s role in movement preparation, which

has been implicated but not addressed directly, (2) look at how movement prepara-

tion is represented in a oculomotor population, (3) compare responses in dots epoch

from population to single cells elsewhere, and (4) compare systems-level descriptions

of movement preparation in prearcuate and PMd.

Chapters 2 and 3 focus on the oculomotor system, but also compare some results

to the reach system.

Chapter 2 describes basic response properties from prearcuate cortex, and covers

the period of the trial preceding the saccade, in which the neurons need to integrate

sensory information in order to decide where to saccade. It also explores the effect

of past events on the neurons’ firing rates, both within a trial and into subsequent

trials.

Chapter 3 describes in detail how we can evaluate the behavioral relevance of the

neural dynamics we see in Chapter 2.

Chapter 4 reviews some of the experiments we proposed in the reach system, and

approaches to including some of the behavioral elements (e.g. decision-making) that

are part of the oculomotor studies.

Chapter 2

Response properties of prearcuate

cortex

2.1 Background & Motivation

Prefrontal cortex (PFC) plays a significant role in the organization of movement and

has been described as “the peak of the hierarchy of anterior neural structures dedi-

cated to the execution of action” (Fuster, 1993). Neurons in PFC reflect a wide range

of processes, including decisions in the context of sensory discrimination (Kim and

Shadlen, 1999), working memory (Miller et al., 1996; Zaksas and Pasternak, 2006),

and signals related to eye movements, attention and executive control (reviewed by

Miller, 2000). The majority of research in the PFC has focused on relating activ-

ity from individual neurons to behavior. However, much less is known about the

dynamic processes driving this neural activity. To learn more about the processes

underlying visually-guided decision-making and movement preparation, we need to

better understand how activity of simultaneously recorded PFC populations relates

to task properties and behavior. We draw on the body of literature describing these

relationships for individual neurons.

Kim and Shadlen (1999) recorded from individual neurons in the PFC (focusing on

several areas, including prearcuate cortex) while their monkeys performed a random

dots discrimination task. They found that most neurons responded to target onset

in their response field, and firing rates reflected the direction and strength of the dot

movement, as well as the direction of the upcoming saccade. While monkeys were

30

31

trying to assess the direction of motion of the dots, neurons reflected the strength of

the motion. By the end of the dots epoch, many neurons reflected only the monkeys’

target choice. This work demonstrated that PFC neurons encode the outcome of

sensory processing, but also reflect the process of converting these signals into a

binary decision variable.

We would like to look at these processes of action selection with finer resolution,

inspecting how trial-to-trial differences in the state of the system relate to the mon-

key’s behavior on a given trial. Spiking noise makes it difficult to infer single-trial

estimates of the system’s state from individual neurons. However, combining simul-

taneous signals from many neurons on a given trial gives a better estimate of the

system’s state. Our ultimate goal is to use these single-trial estimates to learn about

the processes underlying decision-making and movement preparation. Before relat-

ing single-trial population activity to behavior, a useful first step is to use the same

methods to simply characterize the average dynamics of the population in the context

of the monkey’s task. This intermediate approach was recently taken by Machens et

al. (2010) to demonstrate the population encoding of task variables in PFC.

In this chapter our goal is to characterize response properties of a collection of PFC

neurons in the prearcuate region, from array recordings. To characterize responses

from ensembles of neurons in prearcuate cortex, we ask: (1) What are individual

cells’ responses, and how do they vary for different parts of the trial and in different

contexts? How similar are these cells to those in other oculomotor areas like LIP

or FEF? (2) Combining information from individual cells, what can we learn about

the dynamics of the population response? (3) What can we learn from population

recordings that was not possible with single-unit recordings?

In our recordings, we find neurons that respond to visual, decision-related, and

motoric aspects of the task, as well as many that represent past choices or reward

history. To what extent can we gain insight into neural dynamics by drawing together

signals from this diverse population? While a monkey performs a direction discrimi-

nation task or plans an eye movement, how are the dynamics of his decision process

reflected in the signals we record? We begin by describing some of the basic response

properties of neurons in the prearcuate area between the principal sulcus and FEF,

building on previous work described in Chapter 1.

In each of the following sections, we will describe both single-unit responses from

32

our array recordings, as well as characterizing the population as a whole. We highlight

some of the diversity of response types from individual units by showing peri-stimulus

time histograms (PSTHs), which relate the average responses of a neuron to differ-

ent parts of the task. In many studies, the population activity is summarized by

averaging together PSTHs from individual units. However, without a certain level

of homogeneity, averaging PSTHs cancels out rather than emphasizes the activity in

which we are interested. In this chapter we will use dimensionality reduction methods

to describe the average dynamics of the population, akin to a PSTH, but in multiple

dimensions. The advantage of this approach is that it allows us to visualize how the

population activity evolves over the course of the trial, without requiring units to

have similar response properties.

2.2 Methods

2.2.1 Subjects

Three adult male rhesus monkeys, C, T and V, were trained to perform a two-

alternative, forced-choice motion discrimination task, in which saccades served as

the operant response. Daily access to fluids was controlled during training and exper-

imental periods to promote behavioral motivation. Before training, the monkeys were

prepared surgically with a head-holding device (Evarts, 1966) and a scleral search coil

for monitoring eye position (Judge et al., 1980). After initial training, we implanted

a 96-channel silicon electrode array (Blackrock Microsystems, Salt Lake City, UT),

as described in (Churchland et al., 2006; Hatsopoulos et al., 2004). Training, experi-

ments and data collection were performed by John Reppas, Jamie Sanders, and Stacy

Rosenbaum, in the lab of Dr. Bill Newsome.

All surgical, behavioral, and animal care procedures complied with National Insti-

tutes of Health guidelines and were approved by the Stanford University Institutional

Animal Care and Use Committee.

2.2.2 Task design and training

Monkeys performed a delayed saccade task and a direction discrimination task. In

the delayed saccade task, monkeys made saccadic eye movements to targets presented

33

Fixate Targets On Delay Period Go Cue Saccade

Fixate Targets On Motion On Delay Period Go Cue Saccade

a

b

Hold Period Reward

Hold Period Reward

Figure 2.1: Behavioral tasks. (a) Monkeys were trained to perform delayed saccadesto targets on a radial grid. On each trial, a single target was presented one of thesethese grid positions. (b) In the direction discrimination task, monkeys were trained tomake a perceptual discrimination about the direction of motion in a patch of randomlymoving dots, and to saccade to the target in this direction. The presentation of thedots stimulus was followed by a delay period (∼1.5 sec) before the go cue was given.

within a polar grid of potential spatial locations, as shown in Figure 2.1a. Monkeys

were rewarded for shifting their gaze inside the target window after a variable delay

period (∼500-1500 ms). In some experiments, the fixation point was placed in the cor-

ner of the monitor to allow saccade eccentricities up to 24◦ to be measured. Neurons

in prearcuate cortex can have response fields for saccades greater than 24◦ (Suzuki

and Azuma, 1983; see Fig. 1.6h). However, physical limitations of our monitor limited

measurement of saccade eccentricities greater than 24◦.

Figure 2.1a illustrates the sequence of events comprising a typical trial of the

delayed saccade task. From left to right, trials began with the onset of a small dot

that the monkey was required to fixate for 150 ms. Next, one saccade target in the

radial grid (hollow gray circle) appeared for ∼500-1500 ms, in one of the locations

drawn in Fig. 2.1a. Following this delay, the fixation point disappeared, cueing the

monkey to saccade to the target. After completing the saccade, the monkey was

required to hold fixation at the target during a second delay period after which his

reward was delivered.

Figure 2.1b illustrates the sequence of events comprising a typical trial of the

motion discrimination task. On each behavioral trial the monkeys observed a noisy

random-dot motion stimulus and reported which of two possible directions of motion

were present by making a saccadic eye movement to one of two targets. We refer

34

to these targets as ‘T1’ and ‘T2’, which were placed at different locations across

experimental sessions. From left to right in Figure 2.1b, trials began with the onset

of a small dot that the monkey was required to fixate for 150 ms. Next, two saccade

targets (hollow gray circles) appeared for 250 ms. The two targets were 8-16 degrees

eccentric from the visual fixation point and either placed on opposite sides of the

fixation point, forming a line centered on the fixation point (contra-ipsi configuration),

or separated by 90 degrees in the contralateral hemifield, with the fixation point

forming the vertex of a right triangle (contra-contra configuration). The targets were

positioned in-line with the axis of motion being discriminated.

The targets were visible for 250 ms before onset of the visual motion stimulus,

which appeared for 800 ms, centered on the fixation point. The motion stimulus was

composed of dynamic random dots, viewed through a circular aperture on a dark

computer screen. On each trial a variable proportion of the dots moved coherently

in one of two directions while the remaining dots were flashed transiently at random

locations and times (for a detailed description see Britten et al. 1992, 1993; Bair et

al. 2001). The difficulty of the discrimination was varied parametrically from trial-to-

trial by adjusting the percentage of dots in coherent motion: the task was easy if most

of the dots moved coherently (e.g., 50% or 100% coherence), but became progressively

more difficult as the coherence decreased. Typically, the animals viewed a range of

signed coherences spanning their psychophysical threshold. The animals were always

rewarded for indicating the correct direction of motion, except at 0% coherence where

they were rewarded randomly (50% probability) irrespective of their choice.

Following offset of the motion stimulus, the monkey was required to maintain fix-

ation for a variable delay period (300-900 ms for monkey V, 500-1400 ms for monkey

T), after which the fixation point disappeared, cueing the monkey to report his deci-

sion with a saccade to the target corresponding to the perceived direction of motion.

If the monkey chose the correct direction of motion, he received a reward after holding

his gaze at the target location during the second delay period.

In both tasks, fixation was enforced throughout the trial by requiring the monkey

to maintain his eye position within an electronic window (1.5◦ radius) centered on the

fixation point. Inappropriate breaks of fixation were punished by aborting the trial

and enforcing a time-out period before onset of the following trial. Psychophysical

decisions were identified by detecting the time of arrival of the monkey’s eye in one of

35

two electronic windows (3-5◦ radius, scaled linearly with target eccentricity) centered

on the two choice targets (T1 and T2). All trials were presented pseudo-randomly in

block-randomized order.

2.2.3 Procedures

During both training and experimental sessions monkeys sat in a primate chair at

a viewing distance of 57 cm from a CRT (cathode ray tube) color monitor. Visual

stimuli were presented on the monitor under computer control. The monkeys’ heads

were positioned stably using the headholding device, and eye position was monitored

throughout all experimental sessions by means of a magnetic search coil apparatus

(0.1◦ resolution; CNC Engineering, Seattle, WA). Juice was delivered on correct trials

via a tube attached to the monkey’s chair.

Behavioral control and data acquisition were managed by a PC-compatible com-

puter running the REX software environment (Hays et al., 1982) and QNX Software

System’s (Ottawa, Canada) real-time operating system. Visual stimuli were gen-

erated using a VSG graphics card (Cambridge Graphics, UK) and presented on a

CRT display. Neural signals were amplified and collected along with digitized task

events and eye position traces using the Cerebus system (Blackrock Microsystems,

Salt Lake City, UT) operating in conjunction with REX. Thresholds for spiking ac-

tivity were set as 3.75 times the root mean square of the signal amplitude of each

channel, and were recalculated at regular intervals throughout an experimental ses-

sion. Snippets of waveforms were saved at the time of each threshold-crossing, from

0.3 ms before the threshold-crossing until 1.3 ms after and digitized at 30 kHz. All

data were subsequently analyzed offline with custom scripts written in the MATLAB

(The MathWorks, Inc., Natick, Massachusetts) programming language.

2.2.4 Array locations and neural recordings

Arrays were implanted into the surface of cortex, in the arcuate concavity between the

caudal aspect of the principal sulcus and the anterior bank of the arcuate sulcus, as

estimated visually from local anatomical landmarks (Fig. 2.2). Figure 2.3 shows the

placement of arrays with respect to the arcuate and principal sulci for all 3 monkeys.

36

PMd

anterior bank of arcuate

(prearcuate cortex)

as

ps

Figure 2.2: Array location. Neural activity was recorded from an array of 96 electrodesimplanted in the anterior bank of the arcuate, or in dorsal premotor cortex of monkeys(for PMd comparison figures). PMd array is from monkey H; prearcuate array isfrom monkey V. AS, arcuate sulcus; CS, central sulcus; PCD, pre-central dimple; PS,principal sulcus.

Monkey V

Left Hemisphere

Monkey T

Left Hemisphere

Monkey C

Right Hemisphere

a b c

PS

AS

PS

AS

PS

AS

A P A P P A

Figure 2.3: Array placement, with respect to arcuate and principal sulci. Anterior(A) and posterior (P) directions are indicated with labels and gray arrows, pointinganterior. Exact placement varied across the three monkeys with arrays implanted inthis region. Differences in location of the array likely influenced which subpopulationsof cells were recorded, which may explain some of the variation in results seen acrossmonkeys. (a) Monkey V. Units on this array were tended to be more motoric andless driven during the direction discrimination part of the task. (b) Monkey T. Unitswere more generally modulated by visual stimuli and the decision task than MonkeyV. (c) Monkey C. Responses on this array were overall much weaker and also lessstable than those from the other monkeys.

37

Neural recordings were collected using silicon microelectrode arrays, with 96 1.5-

mm electrodes (in prearcuate arrays; 1.0 mm electrodes in PMd), and read out using

the Cerebus recording system. Spikes from each electrode were sorted and clustered

offline, based on a principal component analysis of the resulting voltage waveforms

using the Plexon Offline Sorter (Plexon Inc., Dallas, Texas). This process initially

overestimated the number of clusters; custom Matlab scripts were used to automat-

ically remove recording artifacts (transient events evoking rapid changes in firing

rate across the array), post-process and group clusters based on waveform shape,

inter-spike interval, and firing rate. Daily recordings yielded ∼100-200 single and

mulit-units (not distinguished here, but see Reppas et al., 2010), collectively referred

to as ‘units’. As in Yu et al. (2009), our analyses do not depend on having single

units, so we refer collectively to the single- and multi-units on our array as simply

‘units’.

Units were active to varying degrees during the target onset period (visually-

modulated units), the delay period (movement preparation units), and/or after the

saccade/reach had been executed (post-choice units) (see Reppas et al., 2010). Though

target locations were constant during an experiment, response fields of the neurons

were spatially scattered. Thus, targets were in the center of some units’ RFs, but

only in the periphery for others. Therefore, we developed a criterion for minimal

tuning that we employed to select units for analysis. To study motor preparation, we

included units with firing rates that were significantly modulated relative to baseline

for at least one target (p <0.05; t-test) during a 300 ms window preceding saccade

initiation, and that had an average firing rate of at least 1 spike/second higher for

one of the two target locations during this window. Units with tuned activity in this

epoch comprised roughly a third of all units on the array. Units’ preferred directions

were estimated as the target location which elicited the greatest activity during this

window. This weak test of tuning was used in order to increase the size of the pop-

ulation we sampled. However, the exact threshold used did not change the nature

of our results. We did not typically explore response fields systematically; hence,

our operational assignment of preferred direction may be off considerably from the

real preferred direction. Units with no modulation during this epoch were excluded

from this analysis. Number of units included for each dataset were as follows: 34-113,

38

SNR: 11.28 SNR: 6.46

SNR: 3.92 SNR: 1.85

1

0

-1

1

0

-1

1

0

-1

1

0

-1

A.U

.A

.U.

1.6 ms

Figure 2.4: Diversity of waveform signal-to-noise ratios. The signal-to-noise ratio(SNR) described by Suner et al., 2005 gives a quantitative measure of reliability ofthe waveforms. The four examples shown here are representative of neurons includedin our population. Arbitrary units on the vertical axis are determined by the RMSnoise across channels and are the same across all panels.

mean: 69 (monkey T) and 41-124, mean: 81 (monkey V). Of those, the following num-

bers were tuned for a given target location: 9-96, mean: 35 (monkey T) and 6-107,

mean: 40 (monkey V).

Example waveforms are illustrated in Figure 2.4 along with a signal-to-noise ratio

(SNR), used to measure across-trial reliability. This SNR is defined in Suner et al.

(2005) as: SNR = A2 ∗ SDnoise

, where A, the amplitude, is the peak-to-peak voltage

of the mean waveform, and SDnoise is the standard deviation of the noise. Neurons

with SNRs ≤ 1.75 were excluded from our analysis.

Inter-spike intervals (ISIs) for these waveforms are shown in Figure 2.5. Waveform

SNRs and interspike interval statistics tend to be correlated; units with high waveform

SNRs also tend to have ISIs with less evidence of contamination (spikes within the

refractory period) (Suner et al., 2005).

39

0

100

200

300

0

100

200

300

0 50 1000

50

100

150

200

Interval (ms)

0 50 1000

500

1000

1500

Interval (ms)

Co

un

tC

ou

nt

Figure 2.5: Diversity of inter-spike interval statistics. The inter-spike interval (ISI)characterizes response statistics, and was calculated for each unit as part of the spike-sorting process. ISIs were used as a signature of unit identity, and used to differentiatebetween different units on the same electrode.

2.2.5 Dimensionality reduction of neural data

From a dynamic systems point of view, our goal is to extract low-dimensional tra-

jectories that describe the evolution of this neural system over time during single

trials. These trajectories provide a compact summary of the activity of the system

during behavior, and may offer insight into the underlying biology of the dynamical

system. In order to achieve this low-dimensional representation, we apply Sensible

Principal Component Analysis (SPCA), (Roweis, 1998) and Gaussian Process Factor

Analysis (GPFA), (Yu et al., 2009) dimensionality reduction techniques to data from

our multielectrode recordings.

GPFA aims to uncover latent dimensions that provide a compact description of the

variance in a higher dimensional space, distinguishing between variance independent

to individual neurons and variance shared across the population. This shared variance

represents the fluctuations common to many neurons in the population, which should

give us information about the state of the underlying neural process driving the

population.

40

Using dimensionality reduction techniques to find neural trajectories through this

state space allows us to estimate the time evolution of this underlying neural process,

which we can visualize on single trials, or averaged together across trials (in the same

spirit as a population PSTH). Reducing the dimensionality of the population data

also allows us to combine data from many cells, regardless of individual response

profiles, and to reduce noise by emphasizing the most salient features of ensemble

response properties.

GPFA is different than PCA-like methods in a few important ways. First, rather

than filtering all data with a filter with one time constant, instead GPFA finds the

time constants appropriate for each dimension separately. Second, it distinguishes

between sources of noise particular to individual cells (private variance) and cross-

trial variation that is correlated across many cells (shared variance).

Though SPCA and GPFA generally yielded roughly similar results, SPCA was

more robust to some types of noise in the data (e.g., situations like the Heywood case

in Factor Analysis, Kolenikov et al., 2006). Therefore, SPCA was used for preliminary

assessment of datasets, and was preferable in some cases for rapid visualization. How-

ever, in GPFA time constants for each dimension are determined separately, which

allowed us to achieve finer resolution of trajectories than with SPCA.

Dimensionality reduction was performed on data collected during the time win-

dow corresponding to the epoch of interest. For SPCA, data were preprocessed by

convolving each cell’s spike train with a Gaussian (acausal, not time-shifted) kernel

with width of 30 ms to produce a vector of firing rate varying continuously in time

(the parameters used in preprocessing do not have a significant influence on results—

see Afshar, 2008 for detailed controls). These vectors were then downsampled by a

factor of 10, representing firing rates every 10 ms, and placed in a T x N array, where

T = number of timesteps and N = number of neurons. SPCA was then performed

on this array.

To apply GPFA to our data, we organized our data into an N x M array of spike

times in 1 msec bins, where N is the number of simultaneous units which met our

criteria for inclusion, and M is the total number of timesteps in the epoch of interest

multiplied by the number of trials. The square root transform was applied to binned

spike counts, which served to stabilize spiking noise variance. Parameters of the GPFA

model were fit using the Expectation-Maximization (EM) algorithm as described in

41

Yu et al. (2009). Neural trajectories were then extracted from this model. These

trajectories were orthonormalized to order the dimensions in terms of the covariance

explained.

Since the covariance of the population varies across different epochs of the trial,

the choice of time window upon which GPFA is run will affect the neural trajectories

generated. The smaller the time window, the more closely the parameters of GPFA

will fit covariance in the epoch of interest. Therefore, different epochs of time can be

chosen to emphasize different aspects of the data. For instance, performing GPFA on

the entire trial is useful for visualizing how the neural state evolves through different

parts of the task, but will yield worse estimates of covariance during the perisaccadic

epoch than if GPFA had been run on that epoch alone.

2.2.6 Computing the ROC predictive index

To compute the cells’ receiver operating characteristic (ROC) curves, we followed the

methods described in Shadlen and Newsome (2001). We advanced a 200 ms window

by 10 ms intervals across each neuron’s firing rate during the dots epoch (300 ms

before dots onset, 1000 ms after dots offset) and saccade epoch (600 ms before and

after the saccade onset) to measure whether a neuron’s firing rate was greater than

a criterion κ for trials of a given choice and coherence level.

We used 42 of our best-tuned prearcuate neurons within one dataset to compute

predictive indices based on these ROC curves. Since some neurons were tuned for

T1 saccades and others were tuned for T2 saccades, the neurons tuned for T1 had

positive predictive indices and T2 neurons had negative predictive indices. To average

across T1 and T2-preferring neurons, we flipped the sign of the ROC predictive index

for T2-preferring neurons to create a population ROC curve based on neurons tuned

for either target location.

2.3 Results

2.3.1 Tuning properties of neurons / cell classification

FEF contains cells with visual and movement-related activity (e.g., Bruce and Gold-

berg, 1985; reviewed by Schall, 1997). In Bruce and Goldberg’s recordings, 40% of

42

the neurons showed visual, but no movement activity, 20% responded before cued

saccades, but showed little or no visual response, and the remaining 40% had visual

and motoric response properties. For a subset of these neurons, they calculated in-

dices for comparing movement and visual activity. The neural activity used for this

calculation was found by taking the 100 ms interval of the task with the highest firing

rate and subtracting the background rate. The movement activity index was defined

as the ratio of a cell’s activity in a learned-saccade task, in which the monkey did

not receive a visual target cue, to the cell’s activity in a visually-guided saccade task.

Similarly, the visual activity index was the ratio of a cell’s activity in a purely visual

task (no saccade) to the cell’s activity in the visually-guided saccade task. Overall,

there is a continuum along these visual and movement axes, indicating that there are

not two distinct classes of neurons represented. However, a slight correlation (r =

-0.26) shows a weak tendency for visuo-movement neurons to be more visual or more

motoric. We compared these response properties with data we recorded in prearcuate

cortex.

Across hundreds of units in prearcuate cortex (from 3 monkeys performing saccade

tasks), we found subsets of units that were modulated by various combinations of

visual stimuli, the dots task, during the perimovement period, and during the hold

period before the monkey received his reward. As in Bruce and Goldberg (1985),

our data also shows a diversity of responses with no clear clustering based on visual,

motoric, or decision-related signals (data not shown). Detailed description of neural

responses in prearcuate can be found in Reppas et al. (2010).

Figure 2.6 shows PSTHs for an example neuron from one of our arrays during

the delayed saccade task. The PSTHs are spatially arranged to correspond to the

delayed saccade target locations. Each PSTH shows this neuron’s response at the

particular target location, aligned in time to target onset and movement onset (left

and right panels, respectively). The polar tuning curve in the center summarizes

tuning strength for the visual response to target onset. The direction and strength

of tuning are computed for each neuron, during the target onset epoch, but also

during the delay period before the go cue and in the epoch before the saccade is

executed. Figure 2.7 shows tuning across the array for these three epochs, for one

example dataset for each monkey. For all three monkeys, units tended to be tuned

for contraversive saccades (Fig. 2.7), and tuning was weakest during the delay period

43

Monkey V

Tuning direction (θ): 82

S/N: 7.6

1 sec

Figure 2.6: Visual responses to the outer-most ring of targets from Figure 2.1, at12 degrees eccentric. Each PSTH is the average response across trials for the targetat that location, aligned in time to target onset and movement onset (left and rightpanels, respectively). The polar tuning curve is calculated from the time windowshown in red, just after targets appear. Mean and SEM are shown by the datapoints and error bars. We fit a circular Gaussian function to this data, shown in thecontinuous, elliptical plot overlaid on the data points. From this, the direction oftuning (theta) is estimated as the peak of the fitted Gaussian. For this neuron, thetawas 82 degrees, very near vertical. A signal-to-noise (S/N) ratio for tuning strengthis also computed, which is the peak response over the baseline or pre-target response.An S/N ratio of almost 8, shown here, is very strong tuning.

of the trial, and strongest at target onset and right before a saccade.

44

Target Onset Delay Period Pre-Saccade

a

b

c

log S/N ratio

Figure 2.7: Tuning during delayed saccade task. All units with target-onset tuningcurve fits with a S/N ratio > 2 (i.e. peak is at least twice baseline) are shown herefor one example experiment for each monkey. Plots show the preferred direction oftuning (theta) versus log-S/N ratio (radius). Each point is data from one neuron, andpanels show tuning during three epochs of the task: around target onset (50-350 msafter target onset), during the delay period (on average 650-950 ms after target onset),and before the saccade (250 ms before saccade onset). (a) - (c) show responses fromMonkeys T, V, and C, respectively. Arrays in Monkeys T and V were implanted inthe left hemisphere; Monkey C’s array was in his right hemisphere (Fig. 2.3). Acrossall three task epochs and all three monkeys, tuning is biased towards the contralateraldirection. Fewest neurons are tuned in the delay period, and when they are, theirS/N ratio tends to be lower.

45

2.3.2 Diversity of PSTHs

Once chronically implanted, non-moveable arrays are implanted, experimenters have

no control over the cells they sample. Therefore, recording from even well-characterized

cortical areas like FEF, array recordings could well yield a very different subpopula-

tion of neurons than recording with moveable electrodes. Recording with moveable

electrodes, experiments have the ability to select neurons that are responsive during

a particular task. Though the geometry of the arrays introduces a laminar bias—

neurons recorded likely come from the same layer or two, but there is no selection

bias as in single-unit recording. Not filtering for particular response properties, we

see diverse responses from the units on our arrays. These units are diverse in their

response properties and in their RF position with respect to saccade targets (see

Chapter 3 for a more extensive discussion of response diversity).

2.3.3 Visualizing population activity

In cortical areas where cells’ responses are fairly stereotyped, averaging PSTHs of

many cells reduces noise and gives an estimate of the population response. How-

ever, if neurons are tuned for different locations or have different timecourses of their

response, averaging together PSTHs will cancel signals from individual cells. Dimen-

sionality reduction offers us a better way to combine diverse responses. Reducing the

dimensionality of our data, we find the dimensions corresponding to shared fluctua-

tions of the population, onto which we can project the population activity. Though

we ultimately want to leverage this method to explore the population response on in-

dividual trials, we can also use this method to gain insight into the average dynamics

of the population. Here, we simply use this method to combine signals from di-

verse sets of neurons to make use of the information represented in the heterogeneous

population, rather than excluding it by averaging.

Figure 2.8 shows the average path the neural population follows through a low-

dimensional state space during single trials of the motion discrimination task. The

blue trace represents trials when the monkey ultimately made a saccade to the target

in his contralateral hemifield (T1) (red traces: same, for ipsilateral (T2) saccades).

As activity of the population settles following fixation, this is reflected in the trajec-

tories’ movement through this state space (Fig. 2.8a). Approximately 100 ms after

46

the targets turn on, the trajectories bend, corresponding to the visual transient seen

in many individual neurons. When the dots turn on (Fig. 2.8b), activity of the

population turns again, and follows the same path for T1 and T2 choices for ap-

proximately 200 ms, after which it moves in the direction of the upcoming saccade.

These low-dimensional representations of average population activity offer a simple

way to visualize activity of heterogeneous populations of neurons, compared with

other methods of combining the diverse responses of single neurons in prearcuate.

These low-dimensional visualizations can be used for rapid assessment of data and

for forming and testing hypotheses.

Fixate

Targets On

Motion On

x2

x1

Motion On

Saccade to T1

Saccade to T2

x2

x1

a b

Figure 2.8: Low-dimensional representation of population data. Applying SPCA(Roweis, 1998) to population data from individual trials, trajectories through thislow-dimensional space were computed and averaged with other trials correspondingto the same eye movement. Blue/red trajectories correspond to trials when the mon-key ultimately made a saccade to T1/T2. Dots on trajectories are spaced at 50 msintervals. (a) Beginning of trial, from fixation through dots on. Movement of thetrajectories during this part of the trial is likely driven by cells such as the ones inFig. 3.2b and Fig. 3.2c. (b) Middle of the trial, from dots on through the time of sac-cade initiation. These trajectories show the average path along which the populationevolves as the monkey responds during different task epochs.

2.3.4 Decision-related activity

As in Kim and Shadlen (1999), we find individual neurons in this area that respond

differentially during the presentation of the dots stimulus and represent planning to

an upcoming saccade target. PSTHs in Figure 2.9 highlight the diversity of some of

the decision-related responses in prearcuate. These neurons passed criteria for tuning

47

0 0.5 10

45

Dots On

−0.5 0 0.5

Saccade

0

45

0

15

Fir

ing

Ra

te (

sp

ike

s/s

)

0 0.5 10

80

Fir

ing

Ra

te (

sp

ike

s/s

)

Dots On

−0.5 0 0.5

Saccade

0

14

0

30

Fir

ing

Ra

te (

sp

ike

s/s

)40%20%10%5%0%

Unit 305

Unit 272Unit 280

Unit 278

Unit 285Unit 217

Figure 2.9: Peri-stimulus time histograms (PSTHs) for prearcuate units simultane-ously recorded during the dots task. Solid (dashed) lines correspond to eventual T1(T2) choices. The strength of the motion signal is represented by line color, as definedin the legend. Responses are aligned to time of dots onset (left panel) and saccadeonset (right panel). Individual units exhibit a wide range of response dynamics duringthe presentation of the dots stimulus, and leading up to the saccade.

in the dots epoch of the task, though not necessarily for having responses that distin-

guished between different strengths of motion signals or coherences. Some neurons

look very similar to LIP neurons, distinguishing between different coherences around

200 ms. Others, like Unit 272, are more unusual, showing a coherence-dependent

response at the same time but which disappears by the time the dots turn off. This

48

0.4

0.5

0.6

0.7

0.8

0.9

1

Pre

dic

tiv

e In

de

x

0 0.5 1

0.4

0.5

0.6

0.7

0.8

0.9

1

Pre

dic

tiv

e In

de

x

0

0.4

0.5

0.6

0.7

0.8

0.9

1

Pre

dic

tiv

e In

de

x

0 0.5 1 0

Dots On SaccadeDots On Saccade

Unit 305

Unit 272Unit 280

Unit 278

Unit 285Unit 217 40%20%10%5%0%

Figure 2.10: ROC predictive index for prearcuate units simultaneously recorded dur-ing the dots task. The strength of the motion signal is represented by line color, asshown in the legend. Individual units exhibit a wide range of ROC predictive indexvalues, which match coherence effects in PSTHs shown in Fig. 2.9. Note the oddreversal of coherence sign during the delay period in Unit 305.

type of neuron seems to reflect sensory discrimination, but not to be storing variables

related to the monkey’s decision. Figure 2.10 shows predictive indices computed from

cells’ receiver operating characteristic (ROC) curves, following the methods described

above and in Shadlen and Newsome (2001). The timecourse of the ROC predictive

indices roughly matches that of the cells’ PSTHs. For Unit 272, the predictive index

peaks around 300 ms, and then tapers off, consistent with that cell’s PSTH.

49

For comparison with responses to the same task in LIP, Figure 2.11 shows LIP

results from Shadlen and Newsome (2001) next to the average ROC predictive index

from one day of our experiments in prearcuate. Note the difference in vertical scales—

the magnitude of ROC predictive indices is greater in LIP. The dynamics in the two

areas are similar, though the absolute values of the predictive index are smaller for

prearcuate. Also note that the shape is slightly different, which is likely influenced by

some of the idiosyncratic ROC predictive indices from prearcuate neurons. Kim and

Shadlen looked at the ROC predictive index in a PFC population, including neurons

from prearcuate cortex (see Kim and Shadlen, 1999, Figure 5). Their results were

similar to both the LIP and prearcuate predictive indices in terms of dynamics of

predictability, with respect to coherence. The magnitude of their predictive index

was closer to that in LIP, perhaps due to stronger selection criteria about which cells

to include in their population. Taking note of the diversity of the individual ROC

predictive indices in Figure 2.10 emphasizes that for our population, averaging is

likely not the best way to combine these signals. Therefore, we sought a better way

to assess how much predictive information is captured by the population.

How can we combine the predictive information in the neural population without

averaging? We appeal to the low-dimensional visualizations to gain intuition about

how the strength of motion signals is represented in the population activity.

2.3.4.1 Visualizing the effect of coherence on population activity

Figure 2.12 shows a low-dimensional representation of the population activity during

the motion discrimination epoch of the task. Trajectories from individual trials are

averaged together based on eventual saccade direction (solid lines = T1, dotted lines

= T2) and strength of dot motion (indicated by color). Dots on the trajectory

are placed at 50 ms intervals. When the dots turn on, the trajectories start from

approximately the same place. As the monkey integrates the motion information in

the dots stimulus, the trajectories deviate toward the region of space representing a

movement plan for T1 or T2. Rotating this plot in three dimensions, it is apparent

that the mean trajectories toward these T1 and T2 plan regions separate earlier

on high-coherence trials. This is consistent with the dynamics seen in the ROC

analysis seen in prearcuate and in LIP in Figure 2.11, that decision-predicting neural

activity is evident sooner on high coherence trials. This is notable because it allows

50

LIP population

Dots on Saccade Dots on Saccade

Prearcuate population

500 ms

0.45

0.5

0.55

0.6

0.65

0.7

0.75

Pre

dic

tiv

e In

de

x (

me

an

)a b

40%20%10%5%0%

0.5

0.6

0.7

0.8

0.9

500 ms

51.2%25.6%12.8%6.4%0%

Pre

dic

tiv

e In

de

x (

me

an

)Figure 2.11: ROC predictive index comparison between prearcuate and LIP (fromShadlen and Newsome, 2001, with permission). (a) Average ROC indices acrossprearcuate neurons recorded. Note that averaging ROC curves with diverse time-courses like Fig. 2.10, Unit 278 and Fig. 2.10, Unit 272 will cause destructive in-terference, decreasing the overall amplitude of the predictive index. (b) AverageROC predictive index around the time of dots presentation and saccade in LIP (fromShadlen and Newsome, 2001). Though the absolute value of the predictive index islower in prearcuate, it should be noted that the stimuli were not optimized for theneurons recorded on the array and targets were not necessarily within neurons’ RFs.Nevertheless, the dynamics of the responses and the coherence effects were similar inboth areas.

us to visualize dynamics of decision-making within a more diverse population. This

coherence-based separation of the T1 and T2 trajectories in the state space can be

quantified by measuring the distance between mean trajectories, as a function of

coherence, as illustrated in Figure 2.13.

The analysis in Figure 2.13 can be performed in this low-dimensional space used

for visualization (as shown), but also in the high-dimensional space of firing rates

from each neuron in the population. Figure 2.13b shows that before the dots turn on,

the distance between the mean T1 and T2 trajectories is similar across coherences,

as would be expected since the monkey has not yet been given any information about

his eventual saccade target. After the dots turn on, the distance between the mean

T1 and T2 trajectories begins to increase, reflecting that the neurons are integrating

51

40%20%10%5%0%

dots on -> dots off, all coherences

t1

t2

x1

x2

x3

dots off

dots off

dots on

time

Figure 2.12: Low-dimensional representation of population activity from dots onsetto when the dots were extinguished 800 ms later. Using GPFA to project populationactivity into 3 dimensions, the average trajectories to each target are shown as afunction of coherence (see legend for colors). Solid (dotted) lines represent averagetrajectories on trials resulting in a T1 (T2) saccade. Velocity can be estimated bylooking at distance between dots on trajectories, which are drawn at 50 ms intervals.

the motion information to allow the monkey to make a decision about which way to

look. The distance between the high-coherence T1 and T2 trajectories increases faster

than that between the low-coherence T1 and T2 trajectories, presumably reflecting

differences in the timecourse of deciding which way to saccade, as seen in the ROC

curves. By the time the dots turn off, the distance between the mean trajectories is

nearly the same for all coherences, suggesting that neurons in prearcuate reflect the

decision-making process, but that this process is completed by the time the dots turn

off.

Figure 2.13a illustrates how the distance between mean trajectories is measured,

and Figure 2.13b presents results of applying the method in (a) to one dataset. This

trend is seen across multiple datasets in one monkey (monkey T). The neural popu-

lation recorded in monkey V was not as responsive during the dots epoch, though a

weak effect appears to be present in this monkey as well.

We repeated this analysis within the high-dimensional space (42 dimensions in

52

this dataset) and found similar results. Data shown in Figure 2.13 was prepared by

reducing the dimensionality to 8 dimensions using GPFA. Reducing the dimension-

ality to 3, 5, or 16 dimensions using either GPFA and SPCA also produced similar

results. Finding similar dynamics of the coherence-based separation of mean T1 and

T2 trajectories across different dimensions and using different methods demonstrates

that this result is not sensitive to the exact parameters of our analysis method.

40%20%10%5%0%

t1

t2

t i+1

x1

x2

x3

dots on -> dots off, all coherences

distance at t i

Euclidean distance between mean T1 and T2 trajectories

0 0.5 1

0

0.2

0.4

0.6

0.8

1

Dots on

Time (s)

% M

axim

um

Sep

arat

ion in 4

2-D

Spac

e

a b

Dots off

Figure 2.13: Measuring distance between mean trajectories as a function of coherenceand time. (a) To quantify the effect of coherence on neural dynamics during motiondiscrimination, we calculated the Euclidean distance between the average T1 and T2trajectories (in low or high-dimensional space) at each time point, as a function ofcoherence. (b) We normalized these distances by the maximum distance, and plottedthe normalized distance between T1 and T2 trajectories as a function of time. Thisplot shows very similar dynamics to the ROC predictive index curves, but unlikethe ROC curves, does not require averaging or selective processing based on units’tuning. Rather than the strict pre-selection of neurons that are typically used in ROCanalyses, this distance metric implicitly includes responses from all neurons.

2.3.4.2 Dimensionality of data

The low-dimensional representation of the population trajectories is compelling, but

how well does it capture coordinated activity of the population? Using GPFA, we can

distinguish between private variance, which is the variance particular to individual

neurons (e.g., from spiking noise), and shared variance, which describes trial-to-trial

covariation across the population. Of the total variance accounted for across the whole

trial the majority was private variance, with shared variance accounting for 13.9%

53

of the total variance in the population (Fig. 2.14, left panel). After this separation,

we can look at how much of the shared variance in the population is captured by

each of the latent dimensions. Figure 2.14 breaks down the shared variance in three

different epochs of the trial by how much variance is accounted for by each of 15

latent dimensions. During the dots epoch (800 ms, from dots onset to dots offset), 7

dimensions are needed to explain 75% of the shared variance. However, during the

saccade (150 ms before/after saccade) and reward epochs (the second hold period),

only 5 dimensions are needed. Visualizing neural trajectories in three dimensions

will give us a more accurate depiction of underlying dynamics when the first three

three dimensions capture a larger proportion of the shared variance. Knowing the

underlying dimensionality of the neural activity during particular epochs can guide us

in choosing how many dimensions to use in our analyses, such as the one described in

Figure 2.13. Future work may elucidate behavioral or task-related variables to which

these dimensions correspond.

Whole trial Dots epoch Saccade epoch Reward epoch

x1

x2

x3

x4

x5

x6

x7

x8

x9

x10

x11

x12

x13

x14

x15

private

shared

source of

variance

variance

accounted for

by dimension x

Figure 2.14: Relative proportion of shared variance explained varies across epochs.Left panel: Shared variance accounts for 13.9% of the total variance of the population.Right panels: Relative contribution of dimensions to shared variance during dots,saccade and reward epochs of the task.

54

2.3.5 Effects of choice and reward history: this trial

After monkeys completed the direction discrimination part of the task and made their

saccade to the appropriate target location, they were required to hold their gaze at

this target for a second delay period, until the reward was given. This epoch of the

task, the ‘hold period’, is highlighted in Figure 2.15. Responses during this part of

the task were typically stronger than the visual, saccadic, or decision-related signals,

as described in Reppas et al. (2010). This suggests that neurons in our populations

are more tightly connected to circuits involved in evaluating the outcome of previous

actions than to circuits involved in taking in sensory input to select and generate

upcoming behavior.

Fixate Targets On Motion On Delay Period Go Cue Saccade Hold Period Reward

Figure 2.15: Behavioral task. As in Figure 2.1b, monkeys were trained to makea perceptual discrimination about the direction of motion in a patch of randomlymoving dots, and to saccade to the target in this direction. After the monkey’ssaccade, he was required to hold his gaze at the target during a second variable delayperiod before the reward was given.

Hold-period responses were robust both in the delayed saccade and in the dots

tasks. What do these responses reflect? Many cells that responded vigorously during

the hold period did not modulate their firing rates nearly as much during the other

task epochs (target onset, dots presentation, etc.). This suggests that the hold-period

response is not likely explained by low-level visual signals. We also wondered whether

the hold-period responses were being driven by eye position. To address this question,

we performed a control in which targets were placed at two positions along a horizontal

line. After collecting several hundred trials of the dots task in this configuration, the

targets were displaced horizontally such that the target that had been on the right

side stayed in place (becoming the left target), and the other target moved further to

the right (becoming the right target). In this configuration, the eyes were held at the

same position for saccades to the right target in the first block as the left target in the

second block. If neurons’ hold-period responses reflected eye position, the responses

55

a

b

Figure 2.16: Hold period activity represents previous action in some neurons (s) andprevious choice in others (b). Blue (red) lines correspond to T1 (T2) saccades on thecurrent trial.

to this target should be the same in both blocks. However, we found that neurons’

hold-period responses reflected the direction of eye movement in both blocks, despite

the differences in absolute eye position. Thus, the hold-period responses represent

the direction of the just-completed eye movement.

Some neurons responded similarly during the hold period in both the delayed-

saccade and dots tasks, suggesting that hold period activity reflected the previous

saccade, independently of task context (e.g., Fig. 2.16a). However, other neurons’

hold period activity was selective during the dots task but not after a delayed sac-

cade, indicating that some cells reflected the previous saccade only when it was the

consequence of a choice, rather than an instruction (Fig. 2.16b).

56

Saccade

x1

x3

x2

Saccade

Saccade

+ 700 ms

a b

x1

x3

x2

Saccade

+ 700 ms

Figure 2.17: Hold period effect in population activity. (a) Average hold periodtrajectories for T1 and T2. Dots on trajectories are placed at 50 ms intervals. (b)Individual and average hold trajectories for T1 and T2. 20 of ∼500 trials are shownfor each target (for ease of viewing). Shading on mean trajectories represents SEM.

In the dots task, we show two state-space views of the population activity during

the hold period. Figure 2.17a shows the average trajectories corresponding to T1 and

T2 choices, from the time of saccade initiation to 700 ms after, the minimum length

of the delay period. Figure 2.17b also shows the mean trajectories, with the standard

error of the mean indicated with shading. 20 single trial trajectories are also shown,

which show trial-to-trial variability. Presumably, this variation is due to a number

of factors which may include coherence of the dots (correlated with confidence that

a reward will be delivered), choice and reward history, among other factors such as

confidence a reward will be delivered, fatigue, the monkey’s degree of satiety and

overall motivation level.

2.3.6 Effects of choice and reward history: subsequent trials

The effects seen during the hold period often extend beyond the end of the trial, and

continue to influence neurons’ firing rates into subsequent trials. Figure 2.18b shows

this effect in the PSTH from a single unit. Responses in this PSTH are split into 4

groups based on the monkey’s saccade direction on the current and previous trials.

Both shades of blue (red) correspond to T1 (T2) choices on the current trial. Light

blue (red) lines correspond to trials in which the monkey’s choice on the current trial

was the same as the previous trial (e.g., light blue (red): T1 (T2) current / T1 (T2)

57

previous). Dark blue (red) lines correspond to trials in which the monkey’s choice on

the current trial was the opposite of the previous trial (e.g., dark blue (red): T1 (T2)

current / T2 (T1) previous).

Before the monkey fixates, this neuron’s responses reflect only the monkey’s choice

on the previous trial. At this point in the trial the monkey has no information

about his upcoming movement, so neurons’ responses should not reflect a movement

plan to either target. However, somewhat surprisingly, neurons such as the one in

Figure 2.18b reflect the outcome of the previous trial fairly robustly. This is notable

because the dots task does not require, or provide any advantage to track previous

trials. This effect persists up until the time the dots turn on, shortly after which the

neuron’s response reflects both the monkey’s previous saccade as well as his upcoming

choice. This effect of choice history persists throughout the trial, though to different

degrees in across the population.

Figure 2.18 shows this effect on a population level. For clarity, only T1 choices

are included in this figure, though the same effects are present for both targets. The

trajectories begin 1000 ms before the fixation point turns on, and represent choices to

T1 (light blue) or T2 (dark blue) targets on the previous trial. When the monkey’s

gaze is focused on the fixation point, the eye position signals should in theory be the

same, regardless of trial history. However, we see that the ellipses which represent

the SEM of the trajectories at the time of fixation are clearly separated, indicating

that saccade history in fact does influence the activity in this population.

Similar effects are seen when trials are sorted by whether the monkey was rewarded

or not on the previous trial. These effects of choice and reward history suggest that

prearcuate cortex is involved not only in choosing and generating eye movements, but

also in tracking previous actions in order to inform upcoming behavior.

58

Fixate

Fixation - 1000 ms(T1 chosen on prev. trial)

PC 1

Fixation - 1000 ms(T2 chosen on prev. trial)

Targets On

Motion On

x2

x1

T1 / T1 choices

Target choice on

current / previous trial

T1 / T2 choices

T2 / T2 choices

T2 / T1 choices

a

b

Figure 2.18: History effects. (a) PSTH for one unit. Responses are broken downby target choice on current and previous trials. Blue/red represent T1/T2 choiceson current trial. Bright color indicates choice on previous trial was the same ascurrent; dark color indicates previous trial choice was to the opposite target. (b)Low-dimensional representation (using SPCA) of the population for T1 choices onthe current trial, where the previous choice was T1 (light blue) or T2 (dark blue).Trajectories begin 1000 ms before fixation was acquired (which sometimes extendsinto hold period of previous trial) and continue until the dots are presented. Ellipsesrepresent SEM at trial events. Trajectories are significantly influenced by targetchoice on previous trial up to the time the dots are presented and sometimes throughthe saccade and hold period of the trial (not shown in population, but visible inPSTH).

59

2.4 Discussion

2.4.1 Bias in population measures of choice predictivity

When recording from single electrodes, experimenters implicitly choose cells based

on their response characteristics. When recording from arrays, this is not an option.

Fixed in place, the electrodes on the arrays pick up signals from subpopulations of

neurons close to the array. These multielectrode recordings yield many cells for which

the target is not in their response field, for which responses to stimuli are weak or

absent, or cells with unusual patterns of response which would typically be excluded

from single-electrode recordings. However, in some contexts it is more important to

have signals from many neurons at once than to have a smaller number of high-quality

single-units (for instance, in neural prosthetics). In this chapter, we show that we can

see many of the same characteristics other groups have demonstrated in single units

in our population, despite the fact that the stimuli are not optimized for the neurons

recorded.

When comparing response properties from multielectrode recordings during the

dots task with the single-electrode recordings described in the literature, should we

screen the neurons, and only include ones with strong predictive indices or which

distinguish between dots of different coherences? This approach would give us the

best circumstances for comparing our neurons with those of LIP (e.g. Shadlen and

Newsome, 2001). In Figure 2.11, this is what we did.

Using the method described in Section 2.2.6, the population ROC predictive

index shows similar dynamics to that of Shadlen and Newsome (2001) in LIP: in

both prearcuate and LIP, decisions are made more rapidly when the motion signal

is stronger. However, Figure 2.13 includes all neurons that differentially modulate

their firing rates during the dots period. In LIP, Shadlen and Newsome recorded

cells for which the target was placed in their response field. Here, the target was not

necessarily in cells’ preferred or null direction, nor did we adjust our analysis based

on individual cells’ response properties or tuning. However, looking at the ROC pre-

dictive index curves in Figure 2.10, it is apparent that averaging might not be the

best way to combine the signals from these neurons. Not only are the signs of these

curves different for T1- and T2-tuned neurons, but the timecourses are different as

well. In other cortical areas where this analysis has been applied (such as LIP), the

60

dynamics of individual neurons’ responses are more similar to each other, and aver-

aging the ROC curves should provide a reasonable view of the average dynamics of

the predictive signal. However, if the goal is to know how well the decision can be

predicted based on population data, averaging ROC curves is not the best approach;

the data should be combined in an intelligent way before finding the ROC curve.

Using a state-space analysis provides a straightforward way to combine the choice

predictive signals from the population, as described in Section 2.3.4. Though our goal

was not to directly compare the numbers from this analysis with the predictive indices

in Figure 2.11, the dynamics look appealingly similar. In order to understand how

these predictive indices relate to distances in this firing rate space, we would like to

see LIP data worked up in the same way. Since this analysis does not require single-

trial data, this should be possible, and would provide a useful basis for comparison

with our state-space methods.

Like Kim and Shadlen (1999), many of the neurons we recorded showed decision-

related activity, that is, their responses during the dots epoch were predictive of

the monkey’s upcoming choice. Our responses were similar to those found in LIP,

which makes sense since the two areas are interconnected (Medalla and Barbas, 2006).

We showed here that we can take a population of all-comers and, with very little

preprocessing or biased selection, we can reproduce some of the same dynamics seen

using conventional methods such as ROC analysis in other areas as well as in this

area. This result shows that our tools allow us to reproduce previous findings using

different techniques, even though our cells are much less stereotyped and were not

selected based on their response properties. This opens the door for other population

studies. The more we develop generalizable analysis methods that do not require

individually-selected neurons, the more intuition we can gain about natural ensembles

of neurons.

The next steps for this experiment would be to (1) focus on single-trial analyses,

to investigate the dynamics of decision-making on single trials, building upon the

state space analyses that we performed with the average trajectory in Section 2.3.4.1,

(2) to look at data from other better characterized areas in this way (e.g., Shadlen

and Newsome, 2001), and in future experiments, (3) to stimulate neuron(s) while

the monkey is making a decision about the direction of motion of the dots, to test

hypotheses about how decision-making evolves through neural state space on single

61

trials.

2.4.2 Dimensionality vs. epoch

During the direction-discrimination task populations of neurons in prearcuate cortex

represent different aspects of the task, during which the underlying dynamics may

be described by different numbers of latent dimensions (e.g., Yu et al., 2009, Figure

2). We wanted to compare how many dimensions were needed to describe responses

during different epochs of the task. Given different firing rates and response proper-

ties during different epochs of the task, the axes of this space describing the latent

dimensions corresponding to the shared variability will also be different. Therefore,

to compare the dimensionality of the data across different epochs we need to measure

the shared variability during different epochs within the same space. To do this, we

ran GPFA on the window of the trial from target onset to reward, which included

the epochs of interest, and used the resulting dimensions to generate the pie charts

in Figure 2.14.

Figure 2.14 shows that the dimensionality/shared variance is different during dif-

ferent epochs of the trial. This is likely is related to distinct patterns of firing rate

variation over different epochs of the trial. We need more dimensions to explain the

same amount of the variance in the hold and saccade epochs of the task than during

the dots period.

Across the prearcuate recordings, we see very small amounts of shared variability.

Why is this? Prearcuate appears to have both lower dimensionality and less shared

variability than PMd. This might be due to more cognitive influences, lower firing

rates, or more heterogeneity in what cells in the prearcuate population represent. Our

assessment of the underlying variability in prearcuate is consistent with the results of

Machens et al. (2010), who, in the context of a different task (and serial recordings),

found that shared variability in PFC is captured by about 6 dimensions.

One assumption we have made in employing dimensionality reduction to investi-

gate the underlying dynamics of decision-making and movement preparation is that

the neurons we record are all participating in the same process. It could be the case

that we are sampling from two distinct populations of neurons involved in movement

preparation, and by combining them we obscure our ability to learn about the pro-

cess driving either subpopulation. An important future analysis will be to repeat the

62

host of analyses described in this chapter, after splitting up prearcuate neurons into

subpopulations, based on anatomy and response properties. In the context of a tac-

tile frequency-discrimination task, Machens et al. (2010) found that while individual

neurons response properties did not cluster into distinct groups, the dimensions of

variance in the population corresponded to two task parameters, elapsed time and

frequency of the first stimulus. In the dots task in prearcuate, do different dimensions

also represent different task variables (e.g., eye movements or choice predictivity)?

At this point, our results in prearcuate are intriguing, but still introduce more

questions than they answer. Future work will help us interpret the significance of the

different dimensionalities across tasks, actions, and systems.

2.4.3 History effects: rethinking the idea of ‘baseline’

Many studies show evidence for an effect by comparing cells’ firing rates during their

epoch of interest with firing rates during the ‘baseline’ period, after fixation but

before target onset. However, in Section 2.3.6, we provided evidence that activity

throughout the trial can carry strong influences of previous choices (or rewards, data

not shown). Our task does not require the monkey to remember what he has done

in the past, nor would this information help him with the task. Nevertheless, the

population carries a strong influence of what happened in the past. Similarly, Dorris

et al. (2000) demonstrated that neural activity of superior colliculus neurons can be

significantly influenced by previous saccades. In PFC, Genovesio et al. (2006) found

that many neurons in PFC encode elapsed time in the context of a delayed saccade

task. It is plausible that neurons that track time within a trial may also reflect

the passage of time on longer timescales. Our results are also consistent with those

of Hasegawa et al. (2000), who showed that activity of prefrontal neurons reflected

past or future performance much more than performance on the current trial of their

oculomotor task.

Our data showed that choice and reward history can have a significant influence

on the next trial. Likewise, assessments of baseline activity in this area are likely to be

biased. In this experiment, subjects have no reason to keep track of what happened

in the past, yet neural activity and behavior show influences of previous actions and

rewards. This is consistent with this region of cortex being involved in short-term or

working memory, as described in Fecteau and Munoz (2003).

63

One challenge we faced was that our experiment included many different condi-

tions (e.g., two possible target locations, 5-7 different coherence levels), which made

it difficult to see significant effects after sorting trials by each of these conditions.

Future experiments could use a smaller number of task conditions to more fully ex-

plore effects of history. However, some of these effects might only come about in more

cognitively engaging contexts. Further work is needed.

2.5 Summary

At the beginning of this chapter, we set out to characterize response properties from

an ensemble of prearcuate neurons, both on the single neuron and population levels,

and averaged across trials as well as on single trials.

We demonstrated that neurons in prearcuate have diverse responses, represent-

ing visual input, decision-related signals, movement preparation, traces of where the

monkey directed his saccade both in the same trial and on the previous trial, and

also whether or not he received a reward on the trial before. These history effects are

present at the population level for both the delayed saccade and the decision-making

task. We compared the choice predictivity of neurons in prearcuate with Shadlen

and Newsome’s measurements of choice predictivity in LIP (Shadlen and Newsome,

2001). Although individual neurons in prearcuate showed a wider range of response

profiles in their PSTHs and ROC predictive index timecourses, the dynamics of choice

predictivity in the populations as a whole looked encouragingly similar.

Our second goal was to combine information from individual cells to investigate

the dynamics of the population response. We used dimensionality reduction to vi-

sualize the average neural trajectories in a state space constructed from single-trial

population responses. During the dots presentation, we found that the average trajec-

tories corresponding to eventual T1/T2 choices separated sooner for easier decisions,

consistent with what we saw in the ROC analysis. We also compared the dimensional-

ity of different epochs of the task, showing that different numbers of latent dimensions

are needed to capture the majority of the shared variance of the population. With

respect to task history, we demonstrated that the single-neuron effects we see are very

strongly represented within the population.

Finally, we wanted to learn what we could discover about neural dynamics that

64

was not possible with single-unit recording. The methods we used in this chapter

confirm that state-space approaches allow us to visualize neural dynamics, on average.

In Chapter 3, we look at neural dynamics on single trials during the perimovement

epoch of the task.

Chapter 3

Population activity in prearcuate

cortex accounts for variance in

saccadic latencies

3.1 Chapter Intro

In Chapter 2, we used dimensionality reduction to visualize dynamics and to examine

properties of these dynamics on single trials. We found that various metrics of these

trajectories differed depending on stimulus properties, or how hard the monkey’s task

was. In this chapter, we investigate the relationship between these trajectories and

behavior. The specific behavior we have examined is saccadic reaction time, and our

goal is to determine whether movement preparation activity in the prearcuate gyrus

predicts reaction times. The relationship between neural activity and reaction times

has been examined extensively in FEF, where activity of individual cells has been

shown to correlate with velocity, reaction time, and other saccade metrics. Whether

activity in prearcuate is also tightly tied to these parameters is not known. Within

the rich dataset described in Chapter 2, this chapter will focus on the preparatory

neural activity around the time of the go cue.

65

66

3.2 Background & Motivation

In order to make an eye (or arm) movement, neural circuits first prepare and then

initiate the movement. In an instructed delay task, the preparatory activity can be

studied in isolation between the times the target appears and the presentation of a

go cue. After the go cue, the preparatory activity changes so as to initiate the move-

ment. When no instructed delay period is present, it is believed that neural circuits

still briefly prepare the movement and then initiate the movement (e.g., Crammond

and Kalaska, 2000). A target for an eye (or arm) movement is simply presented,

and the movement can begin as soon as the subject is ready. This type of reaction

time task has been used in the oculomotor system by Hanes and Schall (1996) and

others to study the neural correlates of movement initiation. In this chapter, we use

an instructed delay task to focus on movement preparation, the period just before

movement initiation.

Movement preparation allows the rapid and accurate execution of voluntary move-

ments, and can be influenced by factors that may change from moment to moment,

such as attention and differences in stimulus properties. Consequently, movement

preparation unfolds differently across many different repetitions of the same move-

ment. By looking at individual trials, our goal is to investigate the effect of prepara-

tory neural activity on saccade execution.

Simultaneous recording from populations of neurons allows dynamics of movement

preparation to be estimated on single trials. During behavior, the brain presumably

combines input from populations of cells with wide ranges of response field positions

relative to the upcoming saccade. However, in the oculomotor system this relationship

has only been studied in individual cells with saccade targets in their response field.

To study the dynamics of movement preparation, we asked: how does variation in

reaction times relate to trial-to-trial variation in ensemble activity, from neurons with

heterogeneous response fields?

Prearcuate cortex is the cortical area we investigated to address this question.

We define prearcuate cortex as the cortical surface on the rostral bank of the arcuate

sulcus, between the frontal eye field (FEF) and the principal sulcus (Fig. 2.2). This

may include any part of FEF on the surface of the cortex, as well as area 8Ar (called

area 8Ar by Schall 1997; Preuss and Goldman-Rakic 1991; 8r or r8 by Gerbella et al.

2007; Medalla and Barbas 2006) and possibly area 46. This area lies between FEF, a

67

patently oculomotor area, and the principal sulcus, which contains areas involved in

visuomotor memory (Levy and Goldman-Rakic, 2000; Romanski, 2004), and cognitive

processing (Fuster, 1973; Goldman-Rakic, 1987; Funahashi et al., 1990; Zaksas and

Pasternak, 2006). Prearcuate cortex has fewer projections to brainstem nuclei than

FEF (Gerbella et al., 2007; Schall, 1995); however, neural responses in this area are

clearly involved in movement preparation (Seidemann et al., 2002; Moschovakis et al.,

2004).

We chose prearcuate cortex for (1) its saccade-related activity and the ability to

evoke eye movements with stimulating current (Seidemann et al., 2002), (2) its role

in cognitive processing and visuomotor memory, and tight correlation with behavior

(Kim and Shadlen, 1999), (3) its anatomical connections to LIP and other visual

areas (V2, V3, V4, MT, TEO) (Schall, 1995), and (4) its location on the surface of

the brain, which allows us to use multielectrode arrays to record from many cells at

once.

However, despite the wealth of signal types in prearcuate, the eye-movement re-

lated activity itself has not been well-studied. Is the state of movement preparation

in prearcuate cortex, as reflected by ensemble activity, tightly linked to parameters

of eye movements?

To ask this, we recorded from a heterogeneous population of prearcuate neurons

while monkeys performed a task in which saccades served as the operant response.

We analyzed these recordings in the context of a state-space analysis, where ensemble

activity was represented as a trajectory evolving through a high-dimensional space,

corresponding to cells’ firing rates. Finding a trial-by-trial relationship between vari-

ance in behavior and ensemble activity within this state-space demonstrates how such

analyses can be used to gain leverage on understanding the dynamics of movement

preparation.

Our results demonstrate that movement preparatory neural activity from a di-

verse population can fruitfully predict reaction time. We find that the trial-by-trial

evolution of the system’s state is related to the monkey’s saccadic reaction time, and

that this relationship is stronger when making use of the simultaneity in diverse pop-

ulation of neurons than by looking at correlation between individual cells’ activity

and behavior. These results are consistent with recent findings in PMd (Afshar et

al., 2010), which show similar correlation coefficients between population activity and

68

reach reaction times.

Parts of this work have been published previously in abstract form (Kalmar et al.,

2010).

3.3 Methods

3.3.1 Subjects

Two adult male rhesus monkeys, T and V, were trained to perform a delayed-saccade

task and monkeys T and V also performed a two-alternative, forced-choice motion

discrimination task, in which saccades served as the operant response. After initial

training, we implanted a 96-channel silicon electrode array (Blackrock Microsystems,

Salt Lake City, UT), as described in Churchland et al., 2006; Hatsopoulos et al., 2004.

More details can be found in Section 2.2.1.

3.3.2 Task design and training

Monkeys were trained on the direction discrimination task described in Section 2.2.2,

but the discrimination per se will not be important for the analyses described in

this chapter. We used these trials as a source of eye movement behavioral data and

neural data related to eye movement planning. This task and the epoch of interest

are illustrated in Figure 3.1.

For the data reported in this chapter, we focused on the window of time around

the go cue and saccade. The full data set analyzed in this chapter consists of 45

experiment sessions from monkey T and 35 sessions from monkey V.

3.3.3 Procedures

Monkeys’ behavior was closely monitored and enforced by computer control. Behav-

ioral and neural signals were acquired by REX and Cerebus systems. Spikes were

sorted and analyzed offline (see Section 2.2.3 for details).

69

Fixate Targets On Motion On Delay Period Go Cue Saccade

Figure 3.1: Behavioral task. Monkeys were trained to make a perceptual discrimina-tion about the direction of motion in a patch of randomly moving dots, and to saccadeto the target in this direction. The presentation of the dots stimulus was followedby a delay period (∼1.5 sec) before the go cue was given. We analyzed perisaccadicactivity 500 ms before and after the go cue. This epoch included the monkeys’ sac-cades, as shown by the highlighted region. Saccade latencies were typically 150 - 300ms.

3.3.4 Array locations and neural recordings

Arrays were implanted into the surface of cortex, in the arcuate concavity between

the caudal aspect of the principal sulcus and the anterior bank of the arcuate sulcus,

as estimated visually from local anatomical landmarks (Fig. 2.2). Spike-sorting and

tuning criteria are described in Section 2.2.4.

3.3.5 Datasets

We analyzed 90/70 datasets from 2 monkeys (subjects T and V), from 45/35 days of

recording (monkey T/V). Comparison figures from PMd (Afshar et al., 2010) were

drawn from 5 days of recording in monkey G and 2 days of recording in monkey H.

Data were kindly supplied by Afshar et al. (2010). Our recordings sampled similar

subsets of units day-to-day; however, populations of units were not assumed to be

independent nor identical across days (Chestek et al., 2007; Batista et al., 2007).

Use of different task parameters on consecutive days of recording reduced day-to-day

redundancy of signals from similar populations of units. Data here are sampled from

datasets which represent experimental sessions with the highest trial counts and best

signals from the array.

70

3.3.6 Behavior

We measured monkeys’ latency to saccade, or reaction time (RT) as the time between

the delivery of the go cue and saccade initiation. Saccade start and end times were

calculated as the time the derivative of the analog eye position first exceeded a velocity

threshold of 15◦/sec and decreased below 10◦/sec. Trials on which saccade velocity

profiles were not Gaussian within a reasonable goodness of fit were excluded.

Monkeys’ RTs were brisk, typically between 150 and 300ms, and varied with the

length of the delay period This is consistent with what would be expected if the

subjects were planning during the delay period (e.g. Figure 1.1, and Churchland et

al. 2006). Trials with RTs greater than two standard deviations from the mean were

excluded. This comprised a very small percentage of trials. Since we are using brisk

RTs as an indicator that the monkey was planning and our objective was to study

neural activity related to this process, we did not want to include trials where there

was any behavioral evidence that the monkey was not planning.

3.3.7 Dimensionality reduction of neural data

GPFA methods are described briefly in Section 2.2.5 and in detail in Yu et al. (2009).

Figure 3.3 shows neural trajectories in two dimensions, focusing on the time between

when the monkey is given the go cue (green dots) and when the monkey initiates (blue

dots) and executes his saccades. To focus in on the epoch most related to movement

preparation, neural data were aligned to the time of the go cue and the trajectories

plotted as a function of time in each of the orthonormalized dimensions (Fig. 3.11).

3.3.8 Analysis and presentation

Unless otherwise noted, all data visualization was done by reducing the dimensionality

of the data, and all calculations were done in the full-dimensional space.

71

3.4 Results

3.4.1 Diverse neural responses in prearcuate

Our units’ response profiles were diverse in two ways: (1) location of response fields

with respect to the target location, and (2) timecourses and patterns of response,

compared to typical eye-movement responsive cells in FEF or LIP.

Location of RFs. Because we we could not move our electrodes, we had access

to only the population of units under our array. The majority of units’ response

fields were in the contralateral hemifield, but we could not choose target locations to

match all units’ response fields. Though we only included in this analysis units with

stronger responses for one of the two targets present, the target is not likely in the

center of most of the units’ response fields. From day to day, we varied our target

locations to sample different subpopulations of units. However, on any given day,

we recorded from a population of units with varied response field locations, relative

to the target. Units’ preferred directions tended to be in the contralateral hemifield,

though target and saccade-related responses were usually seen for ipsiversive saccades

as well (though often weaker), as illustrated in the previous chapter (Figure 2.7). A

minority of units preferred the ipsilateral hemifield (data not shown). However, given

that we did not optimize target locations based on individual units’ responses, these

mappings of preferred direction are very broad.

Description of responses. In Figure 3.2 we show an example of the diversity of

the average responses collected simultaneously on the array, during one recording

session. We see units with various combinations of activity that is modulated during

target onset (visually-modulated), the dots presentation and subsequent delay period

(decision-related), around the time of the eye movement (saccade-related), and during

the hold period before the delivery of the reward (choice/action-related). For example,

the top right panel in Figure 3.2 shows a unit with a brisk response to the target onset,

but modulates its responses much less until right after the monkey completes saccades

to either target. In contrast, the unit in the upper left panel is more modulated during

the dots epoch of the trial, and its perisaccadic response is much more gradual. This

unit appears to be more involved with the decision about the direction of dot motion

than the visual or saccadic response per se. Many units in this region respond most

vigorously in the epoch of the trial after the saccade to the target and before receiving

72

the reward, like the unit in the center panel of Fig. 3.2 (Reppas et al., 2010).

Diversity of PSTHs. In addition to recording from units whose RFs were not

lined up with the target location, we also recorded from units with a variety of

target-RF alignments, with corresponding diversity in PSTHs. Some units increased

their firing rates briskly directly preceding a saccade, similarly to motoric cells in FEF

(e.g. top right panel, Fig. 3.2). Other units increased their activity before a saccade

more gradually, like the top two left-hand panels in Fig. 3.2. Still another group of

units decreased their activity before a saccade, but differentially for different target

locations, as in the center and right panels in the bottom row of Fig. 3.2. Many trends

were seen during the perisaccadic epoch. We included all units that were tuned in

the 300 ms epoch before a saccade (as defined in Methods), which included units of

each type mentioned here (units in Fig. 3.2 were chosen to illustrate diversity of cells

recorded on the array, but were not necessarily included in perimovement analyses).

Each of these units show different relationships with behavior, despite the fact that

they were recorded simultaneously. How can we describe the relationship between

this heterogeneous population and behavior?

3.4.2 Neural trajectories evolve along a stereotyped path

Reducing the dimensionality of the data using GPFA, we can visualize how the en-

semble activity evolves over the course of individual trials. Figure 3.3 shows neural

trajectories during the perimovement epoch of the trial, in 2 dimensions. Figure 3.3a

shows data from many single trials to the same target in prearcuate, from a monkey

performing the saccade task; Figure 3.3b shows data from PMd in a different monkey,

performing a delayed reach task (e.g., Fig. 4.10). In this figure, each gray line is a

neural trajectory for a different trial, all representing saccades/reaches to the same

target. These trajectories illustrate the evolution of the population state from 100 ms

before the go cue (green dots) until 100 ms after the end of the saccade (Fig. 3.3a)

or reach (Fig. 3.3b) (movement onset shown with blue dots). While individual trials

evolve differently, they tend to follow a stereotyped path as the subject prepares and

executes his movement. The stereotyped path has less trial-to-trial variability in PMd

than in prearcuate. Possible reasons for this will be discussed later. However, the

primary effect is similar across systems: for each target location, neural trajectories

follow a stereotyped path.

73

2

22

10

40

5

25

5

45

0

14

Fir

ing

Ra

te (

sp

ike

s/s

)

−0.5 0 0.5 1

5

45

−0.5 0 −0.5 0 0.5 1 −0.5 0 −0.5 0 0.5 1

6

22

−0.5 0

dots dots dotssaccade saccade saccade

targets

on Time (s)

T1 choices

T2 choices

5

30

6

24

Figure 3.2: Peri-stimulus time histograms (PSTHs) for prearcuate neurons simulta-neously recorded during the dots task. Examples are representative of diversity ofneural responses recorded on array. Activity is shown during target onset and dotsepoch to allow comparison with PSTHs from other oculomotor areas. Visuomotor andmotoric neurons show response profiles similar to those of similar neurons in FEF.Note that some neurons increase their firing rates leading up to a saccade; othersdecrease their firing rates leading up to a saccade.

Visualizing movement preparation trajectories allows us to gain intuition about

the dynamics of the population on single trials. Figure 3.3 suggests that there is a

common path along which the neural state moves in order to initiate and execute

movements. In addition to being a convenient visualization, do these trajectories

actually reflect dynamics in the underlying system relevant to movement preparation?

The best way we have of answering this question is to try to relate the dynamics of

these trajectories to behavior. On single trials, does the state of the population tell

us about how quickly the animal will execute its movement? One might consider the

stereotyped path in Figure 3.3 as being like a race track, with the neural trajectories

as runners trying to get from the starting block (go cue) to the finish line (movement

74

Go cueMovement onset

x1

x2

x1

x2

a b

Figure 3.3: Neural trajectories for a single saccade/reach target in (a) prearcuatecortex and (b) PMd evolve along a stereotyped path. For this visualization, only theperimovement epoch of the trial is shown (100 ms before the go cue, to 100 ms afterthe end of movement).

onset). In this analogy, trying to predict subjects’ RT based on features of the

population trajectories would be akin to predicting runners’ race times based on

their starting position relative to the starting block. We want to understand how a

runner’s race time corresponds to how, when, and where he moves on the race track.

To what extent does trial-by-trial variability in race paths or trajectories correspond

to trial-by-trial differences in behavior?

3.4.3 Trial-by-trial relationship between neural dynamics and

RT

Afshar et al. (2010) studied the relationship between neural trajectories within a

high-dimensional state-space and RT, in PMd. They found that the further along the

mean trajectory the ensemble firing rate was on a given trial, the faster the subject’s

reaction time. In the foot race example, this would be akin to runners’ race times

being faster on races when their starting point was closer to where they needed to

run. Presumably there is a limit to how far along this path the neural state can be

without triggering a movement, just as a runner wants to get as close as possible

to the starting line without crossing it. We hypothesize that there is a single-trial

relationship between responses from the neural population in prearcuate and saccadic

initiation times, as has been shown in PMd. Using the same approach in prearcuate

cortex, we investigated the relationship between RT and dynamics of the population

response.

75

x1

x2

x1

x2mean trajectoryacross trials

reference(go cue + t)

mean at

ktrial at reference

projection

go vector

x1

x2

a b cgo cue

go cue

vector

G

M

Figure 3.4: Measuring distance along the mean trajectory. Schematic diagrams showthe mean trajectory in blue/green; green dots represent the state of the system atthe time of the go cue on individual trials. (a) Trajectory for trial i shown in gray.The black arrow represents the difference between the average state of the system atthe go cue and on trial i. On this trial, the state is less far along the mean path thanaverage. (b) Example trajectory for trial j. At the time of the go cue on this trial,the system has progressed further along the mean path than on the average trial,illustrated by the black arrow. (c) Schematic of projection computation. The vectorG from mean go cue to trial k go cue is projected onto the mean path, estimated bya reference vector between the mean go cue and a reference point, vector M. Notethat this schematic is drawn in two dimensions, but is easily extended (though less-easily visualized) to an arbitrary number of dimensions. Unless otherwise noted, theanalyses done in this chapter were done on the full dimensionality of the data.

Figure 3.4 illustrates how we measure the evolution of movement preparation in

the population response. Though this schematic is depicted in two dimensions, the

actual neural trajectories reflect the ensemble activity of 100 units, within a 100-

dimensional space. In Figure 3.3 and in the schematic for Figure 3.4, employing

dimensionality reduction allows us to visualize these high-dimensional neural trajec-

tories within a low-dimensional space. However, the following analysis was done in

the full N-dimensional space, where N is the number of tuned units recorded on a

given day. Unless otherwise noted, all data visualization was done by reducing the

dimensionality of the data, and all calculations were done in the full-dimensional

space.

To evaluate the progress of movement preparation at the time of the go cue,

we assumed that the stereotyped shape of the trajectories in Figure 3.3 represents

the path along which movement preparation progresses. In the same way that a

focused runner will be as close as possible to his ideal starting position when the

starting gun is fired, on trials in which movement preparation is more complete the

state of the system should be further along this path when the go cue is delivered. We

76

approximated this path by computing the mean trajectory across trials with the same

saccade direction. Figure 3.4 describes the process by which we measured distance

along this path. We chose a reference point on the mean trajectory 140 ms after the

go cue, and defined the reference vector as connecting the time of the go cue and the

reference time on the mean trajectory. In Figure 3.4, the hollow arrow represents the

mean trajectory, where the hollow green circle represents the mean population state

at the time of the go cue. The green part of this trajectory is before the go cue,

the blue part is after. We hypothesized that the state of more complete movement

preparation corresponds to being further along the reference vector at the time of the

go cue. For each trial, we then measured the progress of movement preparation by

finding the projection of the trajectory at the time of the go cue onto this reference

vector. Figure 3.4a illustrates a trial where the neural trajectory is behind the mean at

the time of the go cue, corresponding to a long RT. Figure 3.4b shows a trial ahead of

the mean at the time of the go cue, corresponding to a fast RT. Figure 3.4c illustrates

how we measured this. If this hypothesis is supported by our data, we expect to see

that trials with greater projections are also the trials with faster reaction times.

Figure 3.5 supports this hypothesis, demonstrating a systematic relationship be-

tween projection along the mean trajectory and reaction time. The top row of Fig. 3.5

shows these results for one example dataset for each of 4 monkeys. Each panel shows

RT as a function of projection, with the best fit line (using linear regression). The cor-

relation coefficient (inset) and p-value summarize the relationship for each dataset.

The bottom row of panels in Fig. 3.5 shows histograms of these correlation coef-

ficients across datasets. Each point corresponds to one target location during one

experimental session. Dark bars represent correlation coefficients significant with p

<0.05. There is variation across days, some of which we account for in the next

section. Despite this variation, the mean correlation coefficient (monkey T: -1.05e-1

± 9.67e-3, monkey V: -8.41e-2 ± 1.42e-2) is significantly smaller than zero for each

monkey (t-test, p <0.001). Having a large number of datasets was important for

uncovering this effect, as individual datasets showed a wide range of correlation co-

efficients. Our result is consistent across both prearcuate and both PMd monkeys

(data from Afshar et al., 2010).

As a control, we asked how much of the relationship between our distance metric

and RT was due to chance. We shuffled distance along the mean trajectory and

77

150

200

250

300

150

200

250

300

−40 0 40150

200

250

300Reaction time (ms)

−100 0 100150

200

250

300

Projection onto mean path (A.U.)

a b c d

e

r = -0.360

p < 0.001

r = -0.365

p < 0.001

−100 0 100 200−50 0 50

r = -0.471

p < 0.001

r = -0.413

p < 0.001

2

4

6

0

4

8

12

−0.5 0 0.50

−0.5 0 0.5−0.5 0 0.50

4

8

12

Correlation coefficient (r)

Number of datasets

−0.5 0 0.50

4

8

12

16

f g h

*

** *

Figure 3.5: Projection along the mean path is correlated with reaction time. (a-d)Reaction times are faster on trials when the state of the system at the go cue isfurther along the mean trajectory. (a, b) Example prearcuate datasets, monkey T(r = -0.360, p <0.001), monkey V (r = -0.365, p <0.001). (c, d) Example PMddatasets, monkey H (r = -0.471, p <0.001), monkey G (r = -0.413, p <0.001). (e-h)Histograms of the correlation coefficient across multiple datasets and target locations.Dark bars represent correlation coefficients significantly different than zero (p <0.05).The means of these histograms are less than zero (p <0.001, all datasets). Arrowsindicate population mean, asteriks correspond to examples in (a-d). (e)monkey T (N= 90 datasets/conditions), (f) monkey V (N = 70 datasets/conditions), (g) monkeyG (N = 78 datasets/conditions), (h) monkey H (N = 24 datasets/conditions)

reaction times with respect to one another, and recalculated the correlation coefficient.

This was repeated 100 times for each target location. Mean correlation coefficients

were 6.30e-4 ± 5.07e-4 (monkey T) and 5.65e-4 ± 8.80e-4 (monkey V), which are

significantly smaller than the average non-shuffled correlation coefficients (t-test, p

<0.001 for both monkeys). This shows that the neural data explains more reaction

time variance than chance.

These results were calculated using a reference point 140 ms after the go cue,

though a wide range of reference points gave similar results. Figure 3.6 shows cor-

relation coefficients calculated from offsets from 250 ms before the go cue to 250 ms

after. Thin gray lines show results for individual datasets and saccade direction of

the correlation coefficient for each choice of reference point offset. The average and

78

−200 −100 0 100 200−0.4

−0.2

0

0.2

0.4

Reference point offset from go cue (ms)

Co

rre

lati

on

co

eff

icie

nt

(r)

−200 −100 0 100 200

a b

Figure 3.6: Correlation coefficients as a function of reference offsets. Thin graylines correspond to datasets/target locations; thick gray lines show the average andstandard error of the mean. Purple lines show the average and standard error ofcorrelation coefficients significant with p <0.05. (a, b) Monkeys T, V. Analyses inthis chapter were performed with a reference point of 140 ms after the go cue. Resultswere similar within reference points chosen across a range.

standard error of the mean across all saccade directions and experiments is shown

by the thick gray line. The purple line shows the same, for correlation coefficients

that were statistically significant with p <0.05. Arrows mark the reference point

used in Figure 3.5. For offsets around the time of the go cue, the reference vector

captures the direction of the mean trajectory poorly, resulting in a noisier estimate of

distance along the mean path and a smaller correlation coefficient. Note that positive

correlation coefficients for negative reference points show the same effect as negative

correlation coefficients for positive reference points. When the slope of the RT vs.

distance plot is positive, this supports the interpretation that further the neural tra-

jectory has traveled away from the direction of movement initiation, the longer the

RT should be. As in Figure 3.5, the results in Figure 3.6 show that there is a lot of

variability across datasets. However, the relationship between RT and distance along

the mean path is fairly robust to the choice of reference point offset.

We wondered to what extent our effect was driven by increases in firing rate.

Does the mean path simply correspond to each unit increasing its firing rate in its

preferred direction? We performed the following control to address this question. In

the analyses represented in Figure 3.5, the reference vector approximated the direction

in which the neural state needed to move in order to initiate a saccade. The reference

79

vector might correspond to a simultaneous increase in each unit’s firing rate, or to

half the units modulating their firing rates up and the other half modulating them

down. In this analysis, we defined the “rise” vector to correspond to an increase in

the firing rate of each unit tuned for the upcoming saccade. By projecting each trial’s

neural trajectory at the time of the go cue onto this rise vector, we could similarly

measure whether projection in this direction was related to movement preparation.

Following the same steps outlined above, we found correlation coefficients for each

dataset, summarizing how much variance in RT was explained by projection onto the

rise vector. These correlation coefficients supported a significant relationship between

the projection onto the rise vector and RT (t-test, p <0.001 for both monkeys; mean

correlation coefficients, monkey T: -5.48e-2 ± 1.15e-2, monkey V: -5.78e-2 ± 1.35e-2).

The magnitude of this effect was significantly smaller than that of projection onto the

reference vector for monkey T (t-test: p <0.005). For monkey V, this effect was also

smaller than for projections onto the reference vector, but not significantly so (t-test,

p = 0.18). However, projecting neural activity onto the mean path rather than the

rise vector yielded better correlation coefficients in both monkeys. This suggests that

though increasing firing rates do explain part of this effect, focusing on this increase

in firing rates alone would leave out a significant part of the story.

3.4.4 RT correlation varies with a number of factors

What is the source of the variation in the RT correlation coefficients in Figure 3.5?

Across experimental sessions, various factors changed, some under the monkey’s con-

trol (e.g., number of trials), some under our control (e.g., location of the targets),

and some due to sources of variance beyond our control (e.g., signal quality from

the electrode array). Even if we could hold all variables constant, some amount of

variation would still be expected due to intrinsic noisiness in the system. Figure 3.7

breaks down how much of the variation in the correlation coefficients in Figure 3.5

can be accounted for by some of these factors.

On the array, the subpopulation we recorded fluctuated from day-to-day. Since

we could not choose target locations to match response fields of each unit within a

recording session, we varied the location of the target positions across recording ses-

sions. As we moved the targets, this influenced the number of units on the array that

were tuned for the given target. Consequently, the number of units that were tuned

80

and included in this analysis varied from day to day. Fig. 3.7a shows how the correla-

tion coefficient varied when saccade targets were placed at different locations, with 0

degrees representing the ipsilateral hemifield and 180 representing the contralateral.

Unsurprisingly, correlation coefficients tend to be stronger when the target is in the

contralateral hemifield. Fig. 3.7b illustrates that for datasets with greater numbers

of tuned units, the correlation coefficient also tends to be greater in magnitude. The

number of trials performed on a given day also has a weak influence on the correlation

coefficient (Fig. 3.7c), but likely this matters most in having enough data points to

achieve statistical significance for the RT vs. projection plot.

The precise shape of the neural trajectories varies from trial to trial, as shown

in Fig. 3.3. Many factors can influence this shape, including the number of units in

the population, their firing rates and how they are modulated through time, and the

location of the target positions. When the shape of the mean trajectory becomes more

complex (less like a line), evaluating the progress of movement preparation using the

projection method described above becomes more difficult. This is due to the way we

are evaluating progress along the mean path; approximating the mean path with a

vector will be less accurate when the mean path is less linear. To assess how this shape

influenced our ability to estimate progress of movement preparation and its relation

to RT, we calculated the curvature of the mean trajectory. We defined curvature

as the distance between the mean go cue and the reference point along the mean

path, divided by this distance in Euclidean space (Fig. 3.8). Figure 3.7d supports

this interpretation, that correlation coefficients are weaker for mean trajectories that

are more curved.

3.4.5 Does simultaneity matter? Comparing results for single

cells and populations

How important is the simultaneity of the data? We compare the correlation coeffi-

cients for the population trajectories (e.g., Fig. 3.9a) with those of the population

of individual units. For each individual unit, we found its projection along its own

mean firing rate, and plotted RT as a function of those (Fig. 3.9b). As with the

population trajectory, we summarized this relationship by finding the correlation co-

efficient for the units’ projections and the monkey’s RTs. The histogram in Fig. 3.9c

81

0 50 100 150

−0.3

0

0.3

Target location (theta)

Correlation coefficient (r)

0 1 2 3 4

Curvature of mean path

0 1000 2000

Number of trials

0 40 80 120

Number of units

r = 0.317

p < 0.001

r = -0.283

p < 0.001

r = -0.070

p = 0.378

r = 0.215

p < 0.01

a b c d

Figure 3.7: Correlation coefficients of datasets depend on multiple factors: (a) Targetlocation (r = 0.317, p <0.001), (b) Number of tuned units recorded (r = -0.283, p<0.001), (c) Number of trials recorded (r = -0.070, p = 0.378) and (d) Curvature ofmean trajectory (r = 0.215, p <0.01). Trends are similar in both monkeys, combinedhere for greater statistical power.

x1

x2

x1

x2

a b

x1

x2

c

Low curvature Medium curvature High curvature

Figure 3.8: Measuring curvature. (a) Example of a mean trajectory with low cur-vature. (b) Example of a mean trajectory with high curvature. (c) Curvature wascalculated by dividing the distance along the path (gray) by the Euclidean distancebetween the mean trajectory at the time of the go cue and at the reference time (reddashed line).

shows the distribution of correlation coefficients for the population of individual units.

The correlation coefficient of the simultaneous population projection from Fig. 3.9a

is indicated with the dashed line. In this dataset, using the population trajectory

explained more of the RT variance (r = -0.365) than the mean of the individual units

(r = -0.038), and more than the best unit (r = -0.312). Repeating this analysis using

simultaneous responses from the 33 best individual units, the correlation coefficient

was -0.201, which is better than the mean, but still not as good as the full population,

or two of the best units individually. This indicates that including responses of many

82

−40 0 40 80150

200

250

300

−100 0 100150

200

250

300

−0.2 0 0.20

5

10

15

20

25

30

Re

ac

tio

n t

ime

(m

s)

Nu

mb

er

of

un

its

Correlation coefficient (r) Projection (A.U.)

−50 0 50 100150

200

250

300

a b

c d

r = -0.365

p < 0.001r = -0.098

p < 0.05

r = -0.201

p < 0.001

All Units Unit 236

Best Units

Projection (A.U.)Projection (A.U.)

Re

ac

tio

n t

ime

(m

s)

Figure 3.9: Does simultaneity matter? Trials with larger projections along the meantrajectory have faster RTs, for the population and for individual cells. Figure showsexample data from one dataset from monkey V. (a) Projection vs. RT, using all cellssimultaneously to estimate state of system and mean trajectory. (b) Projection vs.RT, using one example cell’s firing rate to estimate the state of the system and themean trajectory. Correlation coefficients were collected for each cell individually. Ahistogram of these values is shown in (c). Blue bars show correlation coefficients sig-nificantly different than zero (p <0.05). Dashed line: correlation coefficient from (a).(d) The analysis was repeated for the subset of neurons with correlation coefficientsin shaded blue region, taken simultaneously. The relationship between neural activityand reaction time was strongest when using all cells simultaneously.

individual units is better than just looking at a subset of the best ones.

For each dataset and target location, we calculated the relative number of individ-

ual units with correlation coefficients greater than the population. For most datasets

in both monkeys, the correlation coefficient from the population was better than most

of the units. Figure 3.10 shows the percent of individual units in each dataset with

correlation coefficients better than the population (for populations with significant

correlation coefficients). In Subject T, at most 12% of individual units did better

than the population. In Subject V, no more than 5% of units did better than the

population, with two exceptions. In these two datasets, the mean trajectories have a

83

0 10 200

4

8

12

16

% of units/dataset that outperform the population

Nu

mb

er

of

da

tas

ets

0 100

5

10

15

90 100/ /

a b

Figure 3.10: Simultaneity improves correlation with RT. Histograms show per-centage of neurons with stronger correlation coefficient than population fordatasets/conditions with significant correlation coefficients (p <0.05). (a) monkeyT (N = 49 datasets/conditions), (b) monkey V (N = 26 datasets/conditions).

high amount of curvature (see Fig. 3.7d and Fig. 3.8) when a reference offset of 140

ms is used. When reference offsets are optimized for the shape of the mean trajectory

in each dataset, such artifacts due to curvature can more often be avoided.

3.4.6 Dimensionality of neural data

In the previous section, we described how the correlation between population activity

and RT compares to that of individual units, and how many units contribute to this

result. Another way to look at what is driving this relationship is to look at the

contributions of factors, rather than individual units. In Figure 3.3, we visualized

how activity of the population formed a neural trajectory as it evolved through 3

dimensions of state space. To gain intuition about how activity of the population

evolves in each of the latent dimensions found using GPFA, we can plot population

activity projected into each latent dimension individually. In Figure 3.11, each panel

shows the timecourse of GPFA projections in a different latent dimension, for T1

choices. As in Figure 3.3, green dots correspond to the time the go cue was given,

and blue dots represent the state of the system at the time of saccade initiation.

(For illustrative purposes, only 10 trials are shown. These trials span the recording

session.) In contrast to (Yu et al., 2009), the structure in the timecourses of these

factors is visible for only the first 5 dimensions, unlike the 10 dimensions in Yu et al.

We used the same raw data as used in Yu et al. and repeated our analyses using the

84

same amount of data as in prearcuate (500 ms before/after the go cue) and found the

same results. Thus, differences in the dimensionality of PMd and prearcuate cannot

be explained by differences in the amount of data.

The trajectories in Figure 3.11 represent only part of the variability captured by

GPFA. GPFA distinguishes between private variability, which is unique to individual

units and includes factors such as spiking noise, and shared variability, which describes

the covariations of the population. The goal of GPFA is to detect shared variance

in the population activity and describe factors that capture the shared variance.

The power of GPFA is limited by how much shared variance actually exists in the

population we examine. The shared variability is the aspect that reflects the state of

the neural system. Fig. 3.12a shows, for each example dataset and across all target

conditions, the proportion of the total variance in the population captured by private

and shared variance. Fig. 3.12b shows what proportion of the shared variance shown

in a is accounted for by each factor. As shown in Fig. 3.11, the first 2 dimensions

account for the majority of the shared variance. This supports the observation made

in Figure 3.11, that the dimensionality of movement preparation is smaller than in

PMd.

3.5 Discussion

Individual neurons in different areas of the brain reflect movement preparation in

different ways, but little is known about how these diverse responses fit together as

part of a larger dynamical system. In the rise to threshold hypothesis of Hanes and

Schall (1996), the rate at which neural activity increases during movement initiation

(after the go cue in a reaction time task) predicts how quickly subjects initiate a

saccade. In instructed delay tasks, movement preparation and movement initiation

are separated by a go cue, allowing these processes to be studied independently. A

natural extension of the rise to threshold hypothesis for movement preparation would

be that the higher a neuron’s firing rate is at the time of the go cue, the faster a subject

initiates his movement. In the context of arm movement tasks, Riehle and Requin

(1993) and Bastian et al. (2003) found evidence that neurons in M1 and premotor

cortex support this hypothesis (but see Crammond and Kalaska, 2000; Churchland

et al., 2006).

85

x 2, :

x 7, :

x 12 , :

x 3, :

x 8, :

x 13 , :

x 4, :

x 9, :

x 14 , :

x 5, :

x 10, :

x 15 , :

500 ms

−4

0

4x 1, :

−4

0

4x 6, :

−4

0

4x 11 , :

La

ten

t d

ime

ns

ion

Prearcuate (Monkey T)

Figure 3.11: Time-course of GPFA projections for T1 choices, ordered by relativecontribution. In contrast to Yu et al., 2009, only the first 4 dimensions are needed toexplain 75% of the variance.

Figure 3.13a illustrates this extended version of the rise to threshold hypothesis,

where the axes correspond to firing rates of individual neurons (3 axes shown; actual

dimensionality would be higher, corresponding to all relevant neurons). In this figure,

firing rates for each neuron increase during the delay period, before the go cue (green

part of the trajectory; green circle is the go cue). After the go cue, firing rates continue

increasing until the movement is initiated (blue part of trajectory; blue circle is the

average time of movement initiation). In this extended rise to threshold hypothesis,

trials with higher population firing rates correspond to short RTs as illustrated in

Figure 3.13a.

Also using an instructed delay reaching task, Churchland et al. (2006) examined

the relationship between firing rates in PMd and monkeys’ RTs. They did not find

a consistent relationship between higher firing rates and shorter RTs. Rather, they

found that RTs were shortest when firing rates were close to their mean, falling

within an ‘optimal subspace’ (as illustrated in Figure 3.13b). In this figure, the mean

86

a

b

Prearcuate PMd

Monkey T Monkey V Monkey G Monkey H

x1

x2

x3

x4

x5

x6

x7

x8

x9

x10

x11

x12

x13

x14

x15

private

shared

source of

variance

variance

accounted for

by dimension x

Figure 3.12: (a) Relative proportions of private variance (gray) vs. shared variance(black) captured by GPFA. The average percent shared variance captured by GPFAfor each monkey was (from left to right): 12.3, 13.2, 37.9 and 36.2%. (b) Relativeproportions of shared variance accounted for by each dimension. One example datasetis shown for each monkey, which includes data at all target locations.

trajectory is arbitrarily depicted as curving downward, but any shape of the mean

trajectory would be compatible with this hypothesis. The important distinction from

the extended rise to threshold hypothesis is that the optimal subspace hypothesis

predicts that trials in which firing rates are closest to their mean, rather than trials

with high firing rates, have shorter RTs.

Building on the optimal subspace hypothesis of Churchland et al. (2006), Afshar et

al. (2010) also described a relationship between firing rates and RT in which elevated

firing rates do not necessarily predict a short RT. Afshar et al. (2010) demonstrated

that while RTs are fastest on trials when PMd population activity is close to its

mean, that there is also structure within this optimal subspace. In Churchland et al.’s

optimal subspace hypothesis, the only factor that affects RT on a given trial is how

close the neural trajectory is to the heart of the optimal subspace. In contrast, in the

extended rise-to-threshold hypothesis, the only relevant factor is how quickly units’

firing rates increase before an upcoming saccade. Afshar et al.’s initial condition

hypothesis (Fig. 3.13c) draws on both of these concepts, and demonstrate that in

general RTs are faster on trials when the population activity is close to its cross-trial

87

a

Rise to threshold

hypothesis (extended)

b c

Optimal subspace

hypothesis

Initial condition

hypothesis

Long RTs

Short RTs

Long RTs

Short RTsLong RTs

Short RTs

x1

x2

x3

x1

x2

x3

x1

x2

x3

Figure 3.13: Hypotheses relating neural preparatory activity to reaction time. Thethree axes represent firing rates of three (of many) neurons, and the a schematicof the average neural trajectory is depicted by the green/blue arrow. Green (blue)circles represent the average neural trajectory at the time of the go cue (averagetime of movement initiation), and the arrow indicates the direction corresponding tomovement execution. Green (red) dots represent the state of the neural trajectoryat the time of the go cue on trials with short (long) RTs. (a) The rise to thresholdhypothesis, extended to preparatory activity in population activity. This hypothesispredicts that trials in which neural trajectories are further along the mean path (in thedirection of increasing firing rates) at the time of the go cue will have correspondinglyshort RTs. (b) Optimal subspace hypothesis, from Churchland et al. (2006). Thishypothesis predicts that trials in which neural trajectories are close to their averageat the time of the go cue will correspond to trials with the shortest RTs. In thishypothesis, the path of the mean trajectory need not correspond to increasing firingrates. (c) Initial condition hypothesis, from Afshar et al. (2010). This hypothesispredicts that trials in which neural trajectories are furthest along the mean path atthe time of the go cue will have correspondingly short RTs. This hypothesis is similarto the extended version of the rise to threshold hypothesis shown here, except thedirection of the mean trajectory need not correspond to increasing firing rates.

mean. However, within the optimal subspace, short RTs occur on trials in which

firing rates are closest to the region of the state space corresponding to movement

initiation. As in Figure 3.13b, the path of the mean trajectory within this space is

unimportant. The relevant factor is how far in the direction of movement preparation

neural activity has progressed, at the time of the go cue.

Here, we bring the initial condition hypothesis to the oculomotor system. How

does neural activity in prearcuate cortex during saccades fit into this framework?

Our results suggest that, despite differences between the reach system and data from

PMd, that the optimal-subspace model and Afshar et al.’s extension of it also explain

88

the dynamics of movement preparation in prearcuate cortex. Similar to Afshar et al.,

we were able to explain a significant amount of the across-trial variance in subjects’

RTs with trial-to-trial differences in ensemble activity. However, here we also: (1)

recorded from an oculomotor area with heterogeneous activity with respect to task

variables neurons encode, including cognitive factors, (2) characterized an area that

was much less explored, (3) identified several factors explaining variance in corre-

lation coefficients, (4) showed how population results compared to individual units,

(5) studied the dimensionality of response, and (6) compared results and visualized

trajectories across areas.

It is enticing, and perhaps unexpected, that the initial condition hypothesis ex-

plains dynamics of movement preparation in prearcuate cortex. However, caution

should be used in interpreting these results in the context of other models and stud-

ies in this area, especially in its implications for the validity of the rise-to-threshold

hypothesis. In confirming that the initial condition hypothesis describes our data,

this does not exclude the possibility that the rise-to-threshold hypothesis also ex-

plains movement preparation in prearcuate. The primary distinction between the

rise-to-threshold and initial condition hypotheses is that in the former, short RTs

always coincide with trials in which neurons have elevated firing rates at the time

of the go cue. In the initial condition hypothesis, short RTs correspond to trials in

which neurons’ firing rates are also extreme with respect to their averages, but the

sign may be positive or negative, depending on whether the cell increases or decreases

its firing rate before a saccade (e.g., units in Fig. 3.2).

On a neuron-by-neuron level, we can also inspect our units with respect to saccade

initiation, as in Hanes and Schall’s version of the rise-to-threshold hypothesis (with

the caveat that task structure is different). Doing this, in fact we do see many

individual units in this area that look very similar to those reported in (Hanes and

Schall, 1996) (e.g., Fig. 3.14). Are these units, as a population, better described by

the extended rise-to-threshold or the initial condition hypothesis?

We compared whether the extended rise-to-threshold or initial condition hypoth-

esis better explained the relationship between RT and trial-to-trial activity in our

population. We found that when we projected ensemble activity at the time of the go

cue onto the rise vector (described in Section 3.4.3), there was a significant correlation

with RT. However, these correlation coefficients were weaker than when projecting

89

160 200 240 280

0.2

0.3

0.4

0.5

Reaction time (ms)

Ra

te o

f g

row

th (

sp

ike

s/s

)

Fir

ing

Ra

te (

sp

ike

s/s

)

Time from saccade onset (ms)

10

35

−300 −200 −100 0 100

2

a b

c d

Figure 3.14: (a) Example FEF neuron. (from Hanes and Schall, 1996. Reprintedwith permission from AAAS.) (b) Slope of firing rate increase vs. RT. Slope of fit toactivation functions: 0.6 - 1.8 spikes/sec2 (r2 = 0.87 - 0.99). Mean slope = 1.25 ±

0.1 spikes/sec2. Rate of growth of the activation function decreased significantly withincreasing reaction time. 22/25 cells showed the same trend. (c) Example prearcuateneuron (V2007-08-08, unit 202). (d) Slope of fit to activation functions: 0.20 - 0.48spikes/sec2 (r2 = 0.72 - 0.98). (slope = -0.26 - 1.01 spikes/sec2 on individual trials).21/27 cells with significant trends were negative.

this activity onto the reference vector. This control demonstrated that though our

data could be explained by the extended rise-to-threshold hypothesis, they were better

explained by the initial condition hypothesis.

3.5.1 Saccadic preparation in prearcuate

A number of studies have explored the cortical surface between the principal sulcus

and the arcuate sulcus (reviewed by Schall, 1997), but little physiology has been

done to systematically characterize the eye-movement responses in the area of the

arcuate concavity rostral to the frontal eye field (FEF). The higher density of layer V

pyramidal neurons (Stanton et al., 1989) and lower current needed to evoke saccadic

eye movements from FEF (Bruce et al., 1985) have motivated researchers to focus on

the caudal aspect of the arcuate concavity. However, recent studies by Moschovakis

et al. (2004) and Seidemann et al. (2002) suggest that the oculomotor region of the

90

arcuate concavity extends beyond the region functionally defined as FEF, including

the area defined by Preuss and Goldman-Rakic (1991) as 8Ar.

Our objective was to investigate how the state of a system involved in eye move-

ments relates to behavior, on single trials. Approximating the system’s state on single

trials requires simultaneous recording from many units. In theory, this simultaneous

recording could be done in any oculomotor area, including FEF. In practice, technical

limitations currently restrict this kind of recording to the cortical surface. By target-

ing the anterior lip of the arcuate sulcus and the adjacent area on the arcuate gyrus,

we were able to record from a population of units in the oculomotor region described

by Moschovakis et al. (2004) and Seidemann et al. (2002), straddling the border of

FEF and 8Ar. Due to the nature of our electrode arrays, we were not able to pass

stimulating current to elicit eye movements. Based on comparison with anatomical

and microstimulation studies, we estimate that our arrays were on the boundary of

FEF and 8Ar.

Our recordings from prearcuate yielded impressive heterogeneity in response types,

as illustrated in Fig. 3.2. Given the nature of array recordings, stimuli and behav-

ioral tasks were not tailored for individual neurons, but rather chosen so as to yield

reasonable responses from as many neurons as possible. Rather than catalogue and

characterize the units individually, we chose to analyze the responses at a popula-

tion level. We wanted to see what the system was doing, rather than each of the

components individually. What we found was that we can make better predictions

about behavior by using the simultaneity of the neural data. Drawing from a pop-

ulation of units that individually are quite variable in their ability to predict RT

better accounts for RT variance than including only units which individually predict

RT variance well. These results support a role for this area in saccade preparation.

Also, these results suggest that unlike areas such as superior colliculus, where indi-

vidual neurons often account for RT better than the population, in prearcuate cortex,

movement preparation is best accounted for by ensemble activity.

3.5.2 Comparison between dynamics in prearcuate and PMd

Reducing the dimensionality of our data allows us to look at single-trial evolution

of population activity. While individual trials evolve differently, they tend to follow

a stereotyped path. This path has more trial-to-trial variability in prearcuate than

91

in PMd. Why are trajectories noisier in the prearcuate data? Several things could

account for this. First, the prearcuate units we record are less tuned during the peri-

movement period than the PMd units (differences in mean firing rate for the most and

least responsive target location in prearcuate were at least 1 spike/second, whereas in

PMd they were at least 4 spikes/second). Second, the shared variance in prearcuate

is lower than in PMd (Fig. 3.12), which may be due to greater heterogeneity of re-

sponse type in prearcuate. The dimensionality of perimovement signals is also lower

(Fig. 3.11), possibly corresponding to a lower degree of complexity in controlling the

effector. Third, firing rates are generally lower in prearcuate than in PMd. With

lower firing rates, the prearcuate data are more sensitive to noise, and may indicate

that our prearcuate population is not being as strongly driven by eye movements as

PMd is by arm movements. Finally, the saccade tasks did not pressure the monkey

to consistently prepare saccades during the delay. In the saccade tasks used, delay

periods were much longer and monkeys were not penalized for extremely long RTs.

Therefore, the monkey could wait until later in the delay period, or even after the go

cue, to begin preparing his eye movement. If the monkey prepared eye movements

on some trials but not others, this might lead to the type of trial-to-trial variation in

trajectories that we see.

Despite all of these differences in the prearcuate and PMd data, our ability to

relate the underlying dynamics of the system is roughly comparable. This suggests

that, despite potential differences in mechanism, some aspects of the neural strategy

underlying movement preparation may be common to both the oculomotor and reach

systems.

3.5.3 Summary

In this chapter, we studied how single-trial dynamics of populations of units in

prearcuate reflect subjects’ level of preparation for an upcoming movement, and com-

pared this to the relationship between single-trial dynamics and movement prepara-

tion in PMd. Using a heterogeneous population of units, we were able to explain

a significant amount of the across-trial variance in subjects’ RTs with trial-to-trial

differences in ensemble activity. This is noteworthy because it (1) extends the class

of reaction-time prediction models to a population of units with different RF loca-

tions and different response properties, (2) validates that the dynamics of movement

92

preparation that we visualize as low-dimensional trajectories do in fact reflect behav-

ior, (3) provides physiological evidence that this area of prearcuate cortex is tightly

linked to eye movements, and (4) demonstrates that despite differences in the oculo-

motor and reach systems, movement preparation can be described in the same way in

both systems. This framework for analyzing neural population activity and dynamics

should permit further comparisons of movement preparation within and across motor

systems.

Chapter 4

PMd experiments

In Chapters 2 and 3, our goal was to connect the neural state of the oculomotor system

to decision and movement preparation processes. One of the aims of our research is to

understand how these processes compare for eye movements and for arm movements.

This chapter outlines experiments related to movement preparation and decisions in

the reach system, connecting plan dynamics in PMd to arm movements.

In a natural environment, we must form plans even if we have incomplete infor-

mation about where to move. To make a movement quickly and accurately, one must

start preparing a movement, but be able to change plans according to incoming sen-

sory input. In this chapter, we describe two sets of experiments that address forming

and switching plans, and one that addresses the relationship between plan signals

and muscle activation. The first set of experiments probes how quickly plans can

be formed and changed. Second, we address how plans are formed in situations of

uncertainty. The final set of experiments explores how planning dynamics relate to

muscle activity.

4.1 How quickly can movement plans be switched?

4.1.1 The prosthetic cursor task

Neural prosthetic systems, capable of translating cortical activity into control signals

for guiding prostheses, must operate quickly and accurately in order to be clinically

viable. In communication prostheses, it is important to be able to directly estimate

desired cursor positions from the delay (plan) activity present before, or even without,

93

94

Chain Position 1

Chain Position 3

Chain Position 2

360o��

0o��

Pre

f D

ir

200 ms

H

E

Touch Hold Trial #1 Trial #3Trial #2 Trial #4 'GO' Cue Real Reach

Figure 4.1: The chain task. (adapted from Santhanam et al. (2006) by permissionfrom Macmillan Publishers Ltd: Nature.) In this task the monkey was trained toperform two types of trials: real reach trials, where the monkey planned and executeda movement, and plan-only trials, where the monkey planned a movement, but wasnot given the go signal to execute the reach. For the plan-only trials, the monkey’smovement plan was read out and used to select the planned reach location. If thelocation read out correctly matched the target location, another plan-only trial waspresented. The plan-only trials were presented either until the decode algorithmselected the wrong target location or until a chain, or sequence, of N plan-only trialswas completed. (N = 3 for Monkey H but was arbitrarily long for Monkey G.) Theend of each chain was followed by a real reach trial.

arm movements. Our lab has reported that such systems can achieve high perfor-

mance (Santhanam et al., 2006). However, achieving this high performance in new or

different paradigms requires understanding how neural activity changes in these con-

texts? Here we asked how premotor cortex (PMd) neurons respond during planning

to a rapid succession of targets.

We are able to plan movements on very fast timescales. Behavioral studies have

revealed temporal limits of planning, demonstrating that following movement instruc-

tion it takes at least ∼200 ms to execute a reaching movement (Rosenbaum, 1980;

Riehle and Requin, 1989; Crammond and Kalaska, 2000). However, less is known

about the neural basis of this lower limit.

The study by Santhanam et al. (2006) demonstrated that not only are monkeys

capable of planning reaching movements at very fast speeds (reach targets were pre-

sented at ∼3 Hz), but that this plan activity in neurons in PMd could be decoded

95

by a computer algorithm and used to control a prosthetic device. Monkeys were en-

couraged to plan reaches to targets as soon as they appeared by withholding rewards

if reaction times were too slow. To read out a monkey’s planned reach, Santhanam

used a maximum likelihood (ML) decoder to determine to which target location the

monkey was planning to reach. While their algorithm was able to decode a sequence

of planned reach locations in their prosthetic cursor task, the accuracy of the decoder

decreased for targets presented later in a sequence. That is, plan activity to a given

reach goal changed depending on whether the plan was generated first in a sequence

of plans, or later on in this sequence. In this section, we ask: how do movement

plans change as a function of sequence position? To address this question, we inves-

tigated how changes in the location of a reach goal influence plan activity in PMd,

and how this depends on the speed with which a monkey must switch his plan within

a sequence.

In the prosthetic cursor experiment of Santhanam et al. described above, we hy-

pothesized that the drop-off in decoder performance for movements planned in rapid

succession would be due either to a change in tuning of the neurons recorded, or to

a change in the firing rate of these neurons (Kalmar et al., 2005). To test this hy-

pothesis, we analyzed the neural data from the prosthetic cursor task of Santhanam

et al. (2006). In the prosthetic cursor task, a monkey was given a sequence of poten-

tial reach targets, but only instructed to execute a reach to the last target position

(Fig. 4.1). Since the go cue could be given at any point in the sequence, the monkey

needed to plan reaches to each of the potential targets, to ensure a short enough RT

once the go cue was given. During delay periods between presentation of a target and

either the go cue or the next target, neural activity was recorded and a ML decoder

was used to determine the monkey’s plan. At the end of each delay period, either the

target icon flashed to indicate that the decoder had read out a plan to the correct

target location, or the fixation and touch points were extinguished, which was the

monkey’s cue to reach to the target on the screen. Sequences of ‘plan-only’ target pre-

sentations were interleaved with cued reaches. Successful decoding of planned reach

direction allowed us to confirm that the monkey was planning on each trial. However,

the accuracy of the decoder decreased for plans deeper into a sequence (Gilja et al.,

2005). This suggested that something about movement plans changed as a function

of sequence position.

96

4.1.1.1 Tuning curves are modulated by chain position

Chain Position

1 2 3

30

70

110

150

190

230

310

350

Ta

rge

t D

ire

cti

on

(d

eg

)

10

30

70

110

150

190

230

310

350

25

2

15F

iring

rate

(sp

ike

s/s

)

1 2 3

0

1

0

1

Not normalized Normalized

Figure 4.2: Tuning does not depend on position in chain. Left: Average firing ratesfor each target direction and chain position are shown in grayscale, for two exampleneurons. Right: Same as figures on left, except within the tuning grid each columnis normalized by the peak firing rate for that chain position.

To visualize changes in tuning or firing rate across chain positions, we plotted 2D

tuning curves for both neurons (Fig. 4.2, left column). In these tuning curves, each

column shows relative levels of neural activity for each target direction—at a given

chain position, and each row shows the relative change in firing rate for a given target

location—across chain positions. The target direction eliciting the highest firing rate

from a cell was considered that neuron’s preferred plan direction. The majority of

neurons maintained the same preferred plan direction across chain positions; only

these neurons were included in the rest of this analysis. (The neurons that did not

maintain the same preferred plan direction across chain positions generally did not

show clear trends. Many of these neurons had low firing rates, and changes in their

preferred direction were not statistically significant.) The consistency in tuning is

97

Monkey H Monkey G

Chain Position

1 2 3 1 2 3 1 2 3

Chain Position

1 2 3 1 2 3 1 2 3

30

70

110

150

190

230

310

350

30

70

110

150

190

230

310

350

30

70

110

150

190

230

310

350

30

70

110

150

190

230

310

350

30

70

110

150

190

230

310

350

30

70

110

150

190

230

310

350

Ta

rge

t D

ire

cti

on

(d

eg

)

Firin

g ra

te (s

pik

es

/s)

16

2

11

4

14

4

14

4

10

6

25

5

8

1

11

5

25

10

45

10

35

10

45

10

28

10

16

2

34

18

14

2

26

6

35

10

Figure 4.3: For each neuron, data were grouped by target location and position inthe chain. Tuning curves were often modulated up or down for sequential chain po-sitions, though few changes in tuning direction were observed. Interestingly, MonkeyG tended to have positive gains whereas Monkey H tended to have more negativegain cells.

easier to see when each column of the 2D tuning curve is scaled by its maximum

value, as in Figure 4.2 (right column). Using these 2D tuning curves, we found that

though cells’ tuning remained relatively constant, the tuning curves were often scaled

up or down for sequential chain positions. Figure 4.3 illustrates a representative

sample of neurons’ tuning at different chain positions. Many neurons recorded on our

arrays had similar preferred directions.

We summarized the modulation of tuning curves by finding the gain for popula-

tions of neurons in two monkeys. The gain was computed by regressing a line to the

preferred direction’s average firing rate across chain positions. Interestingly, Monkey

G tended to have positive gains (Fig. 4.4, right) whereas Monkey H tended to have

more negative gain cells (Fig. 4.4, left), though this could be due to slight differences

in task structure. If gains were only negative, this would be consistent with the idea

that the dynamics driving movement planning cannot change fast enough for cells to

98

100

80

60

40

20

0 -12 -10 -8 -6 -4 -2 0 2 4

60

50

40

30

20

10

0 -8 -6 -4 -2 0 2 4 6 8

25th percentile: -0.6676

50th percentile: -0.1143

75th percentile: 0.2961

p<0.05

25th percentile: -0.1932

50th percentile: 0.6655

75th percentile: 1.7227

p<0.05

GainGain

Nu

mb

er

of

ne

uro

ns

Nu

mb

er

of

ne

uro

ns

Monkey H Monkey G

Figure 4.4: Gain histograms. For each neuron, gain was calculated by regressing aline of average firing rates calculated during the delay period (∼300-450 ms) vs. chainposition, treating position in the sequence as a continuous variable. Histograms showtuning curve gains for neurons combined across multiple experiments. The histogramof tuning curve gain is shifted to the left for Monkey H, but slightly to the right forMonkey G.

maintain the same firing rates for tuning curves deeper in the chain. Sequential chain

positions would then have increasingly scaled down versions of the tuning curve at

the first chain position. Since we saw cells with both positive and negative gains in

the same recordings, this suggests that the data cannot be explained by an inability

to produce enough spikes repeatedly. What is causing this modulation of firing rates?

We did not see evidence of this modulation in the standard delayed reach task, which

was used to train the decoding algorithm. This suggests that modulation of overall

firing rates is likely related to the rapid switching of planned movements, shorter

delay periods in the prosthetic cursor task, or possibly to the level of mental effort

exerted by the monkey.

4.1.1.2 Inter-monkey differences

Why are the gain effects different for the two monkeys? One possibility is that even

though both arrays were positioned similarly in PMd, slight differences in placement

or electrode depth might have caused subsampling of different populations of neurons.

If, as in Andersen et al. (1985a), an external factor is modulating tuning of these

neurons, this factor might be acting differently on the two arrays. Figure 4.5 shows

placement of the arrays with respect to sulcal landmarks for the two monkeys. Though

the arrays are very nearly in the same location in both monkeys, slight differences in

the populations of neurons recorded may account for some of the differences in gain.

99

Monkey H

Monkey G

Figure 4.5: Array differences: Placement of electrode arrays. Arrays were placed indorsal aspect of premotor cortex (PMd) in both monkeys. Though array positionswere quite similar, small differences in placement could have caused a different sub-population of neurons to be recorded. Intraoperative photographs of arrays with sulciindicated. Sulcal landmarks were drawn by hand on magnified, full-color, full-contrastversions of these photographs, then pasted in register on these images. Ce.S., centralsulcus; S. Pc. D, superior precentral dimple; Sp. A. S., spur of the arcuate sulcus; A.S., arcuate sulcus. Figure modified from Batista et al. (2007), with permission.

Both monkeys performed the reaching task with the hand contralateral to the array.

Another possibility is that differences in trial structure and reward schedule (Fig. 4.6)

caused the monkeys to adopt different strategies, which translated into differences in

neural representation of the task. Monkey H was rewarded for real reaches only, and

chains of prosthetic cursor trials were limited to 3-target sequences. For monkey G,

chains continued until the decoding algorithm failed, after which a real reach was

always cued. These chains were up to 12 trials long, but for this monkey each trial

was rewarded. It is unclear to what extent differences in neurons’ tuning gain re-

late to these task differences. The experiment in Section 4.1.2 describes an improved

experiment designed to address the neural limits of fast planning.

100

Monkey Hreach

planonly

no reward reward

Monkey G... reachplan

only

reward

Figure 4.6: Trial structure and reward schedule. For Monkey H, chains of prostheticcursor trials were up to three trials long and the monkey was rewarded on the realreach trial at the end of each chain. For Monkey G, chains of prosthetic cursor trialswere variable in length and the monkey was rewarded on each trial.

4.1.1.3 Gain field analogy

Figures 4.7 and 4.8 illustrate two compatible hypotheses about this effect. Figure 4.7

shows Andersen et al.’s model of gain effects, applied to a posterior parietal cortex

neuron. In their model, individual cells’ retinotopic tuning is modulated by the posi-

tion of the eyes. Using this approach to describe our data, we notice that something is

happening differently at different chain positions that modulates the neurons’ tuning

curves up or down. What is modulating neural activity in the prosthetic cursor task?

The source of the modulation in firing rate across chain positions is not understood,

but may be due to attention, reward expectation or intrinsic cortical dynamics.

Retinotopic postition (degrees)

Re

sp

on

se

(s

pik

es

/se

co

nd

)

Gain modulation in PPC (from ref. 2)

Eye postition * Retinotopic tuning

Figure 4.7: The change in firing rate and lack of tuning shift in PMd responses resem-ble gain fields seen in other cortical areas, such as posterior parietal cortex (Andersenet al., 1985b), where an animal’s eye position modulates the gain of retinotopic tuningcurves.

101

4.1.1.4 State-space model for investigating cortical differences

Figure 4.8 shows a way to think about this task in neural state space (e.g., Yu et al.,

2009, Figure 1). As a monkey plans to three sequential targets, population activity

moves through the appropriate regions of state space—that is, given enough time

to plan (blue trajectory). However, as the plan instructions get faster, the intrinsic

dynamics of cortical processing are not fast enough to bring ensemble activity all

the way to the appropriate regions (red trajectory), and the population trajectory

falls short of the optimal region within this state space. Since this state space is

comprised of many neurons tuned for different directions, reaching the appropriate

region within this space does not necessarily correspond to an increase in firing rate.

Similarly, falling short of this subregion need not necessarily correspond to a decrease

in population firing rates. To bring the system into the appropriate state, individual

cells may modulate their firing rates up or down. These results were reported in

Kalmar et al. (2005). To test how the gain effect seen in individual neurons relates

to this ‘base-cutting’ picture, we need to be able to systematically vary the speed

of the target presentation. The next section describes an experiment we designed to

address this explicitly.

neuron 1

(firing rate)

neuron 2

neuron 3

Reach

target 1

Reach

target 2

Reach

target 3

One “slow” target

presentation chain

One “fast” target

presentation chain

Figure 4.8: The gain effect seen during rapid movement preparation could be dueto a variety of influences, including attention, reward expectation (e.g., Roesch andOlson, 2003) or intrinsic cortical dynamics.

102

4.1.2 Future directions

One limitation of the prosthetic cursor task is that the speed of planning and rapid

plan-switching were not varied systematically. Future work can build on the obser-

vations from Section 4.1.1 to explicitly investigate the effects of planning speed and

sequence position on neural activity, and to identify the neural basis for the speed

limits imposed on plan-switching. Figure 4.9 describes a behavioral task that could

be used to measure how quickly neural plan activity in PMd can signal a change in

the direction of a reach goal, and how this relates to a monkey’s reaction time in a

task requiring rapid switching of reaching plans. This experiment would give us an

estimate of how quickly movement plans can be modified in a task where the monkey

is instructed where to reach.

p = 0.75 for another plan cue

p = 0.25 for the go cue (left target)

p = 0.25 for the go cue (right target)

Figure 4.9: Schematic of a proposed task for investigating neural dynamics of rapidplan-switching. As in the delayed reach task, each trial of the plan-switching taskwould begin with hand and eye fixation. Next, two isoluminant squares, one blue andone yellow, would be presented. Either color square may appear in either of the twolocations. The squares are presented for a variable delay period of 30-430 ms. Thisdelay period would be used for all target presentations in a chain, and a new delayperiod drawn after the monkey executed a reach, which marks the end of each trial.At the end of the delay period, either the blue and the yellow square switch places(with probability = 0.75), or the go cue would be given (p = 0.25). The monkey wouldbe trained to always ignore the blue square (the distractor) and to reach to the yellowsquare (the target) when the go cue is given. He would be rewarded for brisk andaccurate movements, according to the same criteria described for the delayed reachtask. Typically, 1000 - 1500 trials of this task would be recorded, each consisting ofchains 1-8 targets long.

103

4.2 Planning with uncertainty

Providing the monkey with explicit cues about where to reach is a straightforward way

to study movement preparation. Though we can learn many basic principles about

the motor system using simple tasks, some properties of movement preparation may

only be revealed in more ethologically-relevant contexts. In the real world, there is

often uncertainty about which actions will best achieve our goals. In this section, we

will explore how reach plans are represented when there is uncertainty about where

to reach.

We need to be able to anticipate events in the environment in order to respond

quickly. Often, though, the appropriate response is not immediately apparent. In

order to act rapidly, our motor system needs to be able to accommodate uncertainty

about the environment, and incorporate these statistics into movement plans. A num-

ber of studies have demonstrated a strong behavioral correlation between subjects’

reaction times and the degree of uncertainty in a directed motor response task (Pos-

ner et al., 1980; Rosenbaum, 1980; Favilla et al., 1989; Pellizzer and Hedges, 2003).

To explore the neural basis of this relationship, several studies have used similar

tasks while recording patterns of neural activity in various motor-related brain areas.

Basso and Wurtz (1998) found that superior colliculus neurons modulate their activ-

ity in relation to the number of possible saccade targets (irrespective of number of

visual stimuli), thus encoding uncertainty about upcoming eye movements. Similarly,

Bastian et al. (1998) demonstrated that the amount of prior information about the

direction of an upcoming reach is reflected in the activity of single neurons in motor

cortex. Expanding on this, Cisek and Kalaska (2002) proposed that when presented

with two potential reach targets, individual neurons in premotor cortex simultane-

ously represented movement plans to both target locations. However, Horwitz and

Newsome (2001), using a different instructed-delay task, found suggestive evidence

that superior colliculus neurons alternated their movement plan between two targets

on individual trials, rather than representing both plans simultaneously. Plan ac-

tivity from individual neurons on single trials is noisy, making it is difficult, if not

impossible, to disambiguate the plan strategies proposed by Cisek and Kalaska, and

Horwitz and Newsome with existing data.

To study the dynamics of movement planning at a fine temporal scale, we need to

examine how the underlying signals of the neuronal population evolve from moment

104

to moment. Difficulty in interpreting single-cell responses is often solved by averaging

neural activity across many trials; however, this would possibly obscure the dynamics

of planning we want to study. A solution to this problem is to record from many

cells simultaneously, as described throughout this thesis. This single-trial approach

is necessary for prostheses, which cannot afford to average across ‘trials’ to determine

an upcoming movement, and will allow us to study dynamics of movement planning

on a behaviorally-relevant timescale. In this section, we use multi-electrode recording

techniques to visualize single-trial movement plans of a population of PMd neurons,

and to characterize how plan dynamics are influenced by uncertainty in our task.

Recent evidence by Cisek and Kalaska (2002) suggests that neurons in movement

preparatory areas are capable of simultaneously preparing reaches to two spatially

distinct target locations when there is ambiguity about which target will be cued.

Pellizzer and Hedges (2003) found behavioral evidence supporting this finding. A

pilot study in our lab attempted to reproduce these experiments, that is, to determine

whether we found evidence of co-coding—simultaneous representation of mutually

exclusive movement plans—on single trials. If the monkey instead changed his plan

back and forth between the two potential target locations on each trial (flipping), but

did so with different timing across trials, the across-trial average activity during the

plan period would look as if the monkey was planning to both target locations on each

trial. With access to multiple neurons recorded simultaneously, we were able to look

at single-trial plan activity in greater detail. The results in this section came from a

collaboration with Zuley Rivera Alvidrez, which was reported in Rivera Alvidrez et

al. (2007) and Rivera Alvidrez et al. (2008).

To disambiguate the two hypotheses (simultaneous plans to competing targets vs.

rapidly alternating plans to single targets), we used a color cue task. This task is

a variation of the standard delayed reach task described above. After fixation, blue

cues were presented simultaneously at two of 7 possible target locations (Fig. 4.10,

bottom panel). These potential reach targets were illuminated for a 400 ms delay

period. At the end of this delay one of the possible targets turned yellow, indicating

that this was the reach target. The other possible reach target, the distractor, did not

change color but remained on the screen. At the end of a second (generally shorter)

delay (30-800 ms), the monkey was given the go cue and rewarded for touching the

yellow target within a fixed time window.

105

Acquire touch

Acquire fix

Possible targets

Delay Period

Target

disambiguation

Movement period

Target acquired

Distractor task with ambigious period

Go cue

Distractor task

Acquire touch

Acquire fix

Target cue

Delay period

Movement period

Target acquiredGo cue

Delayed reach

Acquire touch

Acquire fix

Target cue

Delay period

Movement period

Target acquiredGo cue

Figure 4.10: Delayed reach and distractor tasks. Monkeys were trained to reach totargets on a screen when the go cue appeared. The target (yellow) could appear atthe beginning of a delay period by itself, together with a distractor (blue), or at theend of an ambiguous delay period. These tasks allow controlled analysis of neuralactivity during the planning and execution of reaching movements, during periodswhere the reach direction is completely or incompletely specified.

To assess how ambiguity about the upcoming reach cue affects the monkey’s reach

plan, we measured how plan activity in PMd evolves during the first delay period,

on single trials. We analyzed our results within a state space, where movement plans

are represented by the firing rates of a population of neurons (Fig. 1.11) (Churchland

et al., 2006). This state space analysis has been discussed extensively in Chapters 2

and 3.

Recent work by other members of the Shenoy lab has focused on different methods

of dimensionality reduction for state-space analyses. While Yu et al. (2006, 2009)

describe a low-dimensional nonlinear dynamical systems model by which dynamics of

106

F3

F2

F1 F1F2

F3

a b

Figure 4.11: (a) Trajectories during the delay period to two different targets(Fig. 4.10, delayed-reach task), and (b) with a distractor presented simultaneouslyat the opposing location (Fig. 4.10, distractor task).

plan activity can be studied, simpler linear approaches, such as Principal Components

Analysis (PCA), and Factor Analysis (FA) have also been explored. The analyses in

this section were performed before GPFA was developed. Fig. 4.11 shows a state-

space view of plan trajectories to two different targets’ optimal subspaces (green/blue

dots), using factor analysis to reduce the responses of ∼250 neurons to their three

largest dimensions of variability. To visualize plans to different targets (with different

optimal subspaces) within the same state space, we perform dimensionality reduction

on data collected from all reach directions. We define the optimal subspaces by fitting

a Gaussian to points at the end of (sufficiently long) delay periods in the standard

delayed reach task (or in other instructed tasks).

In the distractor task (Fig. 4.10, middle row), we used factor analysis (FA) to

visualize the neural population’s plan activity. If PMd were co-coding plans to both

targets, we would expect to see the plan trajectory fall between the subspaces for

the two targets (Fig. 4.12a). If PMd flipped plans between the two possible targets,

we expected to see the plan trajectory move between the two subspaces (Fig. 4.12b).

Fig. 4.13 illustrates several example plan trajectories during the delay period. Though

there is evidence for flipping on some trials, there is a considerable amount of variation

between different trajectories. Some of this variation is likely due to the presence of

a second delay period before the go cue, which gave the monkey less incentive to

begin planning during the first delay. To demonstrate that the plan trajectories were

actually related to movement preparation, a control was performed in which data from

107

Leftreachtarget

Rightreachtarget

x1

x2

x3

Both targets(non-executable)

Leftreachtarget

Rightreachtarget

x1

x2

x3 Co-coding Flipping

a b

Figure 4.12: (a) Co-coding hypothesis, as described in Cisek and Kalaska, 2002. (b)Plan-flipping hypothesis, similar to proposed mechanism from Horwitz and Newsome,2001.

each neuron were shuffled across trials. For trials of the ambiguous distractor task

(Fig. 4.10, bottom row), firing rates from each neuron during the ambiguous delay

period were randomly permuted across trials, as illustrated in Figure 4.14a. On each

shuffled trial, firing rates of each neuron were drawn randomly from that neuron’s set

of firing rates from trials in which the ambiguous distractor was presented.

Breaking simultaneity of recording with this shuffle means that any structure in

these trajectories would likely be due to chance (Fig. 4.14c). Though it is clear

that there is more structure in the plan period in Fig. 4.14c than the control in

Fig. 4.14b, without access to other behavioral or task variables, few conclusions can

be drawn from this result. A supporting piece of evidence that this structure relates

to planning comes from the monkey’s behavioral bias. On some days, the monkey’s

default plan during the ambiguous period was heavily biased toward one target (e.g.

Fig. 4.15). Such biases in planning are reflected in longer RTs for the target to which

the monkey was not biased, in comparison with RTs for the standard delayed reach

task. Figure 4.15a shows neural plan trajectories during the ambiguous period, for

trials when the monkey’s bias was the same as the target to which he was ultimately

cued (in this figure, the left target). Figure 4.15b illustrates trials in which the

monkey’s bias is obvious. On these trials, the monkey planned to the left target

during the ambiguous delay period, and then switched his plans to the right target

once the cue was given for that target. This further supports the hypothesis that

neural activity in the plan period of the ambiguous distractor task corresponds to

108

F1

F2

F3

F1

F2F3

a b

c d

Figure 4.13: Sample trajectories from the ambiguous period of the distractor task.Neural trajectory endpoints to two targets (blue and green dots) are from delayedreach trials (no distractor). Neural trajectories from the ambiguous distractor taskare projected into this space. Red dots mark starting points of the trajectories atthe time of target presentation. Crosses represent the mean of clusters, and trajecto-ries show neural activity during the whole ambiguous delay period. (a) Example ofactivity during the ambiguous period that stays near the baseline state, presumablycorresponding to a trial in which the monkey was not planning during the delay pe-riod. (b) shows a trajectory with evidence of flipping—it moves toward the subspacefor the blue target before moving to the green. (c) and (d) show more examples offlipping. The 3-dimensional space in which the data was projected was found usingfactor analysis on the delay period of the delay-reach trials and the ambiguous periodof the distractor trials for the same two opposing targets.

the monkey’s upcoming plan. However, to better understand what drives changes

in planning during this ambiguous period we may gain leverage by systematically

varying parameters of this task.

In this pilot study, we asked how uncertainty influences plan activity when the

monkey had no information about to which of the two targets he should plan. Future

experiments could clarify interpretation of these data by (1) enforcing planning by

excluding the second delay period, thus pressuring the monkey to begin planning

109

0.015 0.045

T1 T2N1

T1 T2 TnN2

Tn T39 T17N1

T8 T72 T27N2

T2

0

180

a

b c

Figure 4.14: Occupancy rate and shuffling test. In the ambiguous distractor task delayperiod, this test measures what percentage of the time the trajectories occupy a targetregion of the state space corresponding to planning, as opposed to the baseline region,which does not. If these trajectories represent planning, firing rates of all neurons willbe driven by the same underlying system. Breaking the simultaneity of the neural databy shuffling firing rates of individual neurons across trials should eliminate structurein the trajectories related to planning. Neural activity was shuffled across trials as acontrol as shown in (a). (b) shows shows unshuffled neural trajectories during theambiguous distractor delay period, where purple dots mark the endpoint of the plantrajectory in the non-ambiguous distractor task. The black lines separating targetclusters from baseline were found using regression. The portion of the trajectoriescrossing these lines into the target region of the state space are shown in magenta.The occupancy rate for the trajectories in this target region for this set of targetswas 13.1%. (c) is same as (b), except the shuffled data were used. The occupancyrate computed for the shuffled data was 2.6%. The figure inset shows a histogramof the occupancy rates obtained after 1000 random shuffles of the neural data. Themean of the distribution was 2.58% with 0.47% standard deviation. These analysessupport our interpretation that flipping trajectories, during the ambiguous distractordelay period represent not noise unrelated to planning, but changing plan states.

during the ambiguous period, and (2) systematically increasing the likelihood the

monkey will be cued to reach to a given target location. This would allow us to

quantify how movement plans change as a function of how much information about

the upcoming reach direction is available during the delay period, and to what extent

110

a b

Figure 4.15: Illustration of strong bias towards left targets during planning in theambiguous distractor task. (a) shows trajectories during the ambiguous period fortwo opposing targets, with highlighted trajectories showing planning towards the lefttarget (blue dots). Note that there is no bias in terms of the ultimate saccades made(i.e. roughly equal numbers of green and blue dots), but that the bias is apparentduring the ambiguous period of the trial, in which the monkey seems to form a defaultplan to the blue target. (b) is the same as (a), except both the ambiguous periodand the delay period are shown for the trials in which the right target was ultimatelycued. As in (a), the neural trajectories during the ambiguous delay period movetoward the region corresponding with planning reaches to the blue target. However,during the second delay period when the correct target is cued, these default plansare then switched to the correct location (indicated by the purple dots). Highlightedtrajectories show apparent flipping in the motor plan during the delay period.

we see evidence for co-coding or flipping.

Our pilot study suggested that on some trials monkeys alternated planning to the

two possible reach goals, planned reaches to only one of the targets, or formed plans

not directly corresponding to any of the targets. To our knowledge this is the first

evidence that in a task with two target options, a variety of strategies are used on

different trials. Future experiments will be necessary to determine how the degree of

uncertainty in a task influences which strategies are employed, ultimately allowing us

to understand the dynamics of planning in ethologically-relevant contexts.

111

4.3 Neural dynamics and muscle activity

In the previous sections, we endeavored to understand how features of external stim-

uli influence how populations of neurons represent movement plans. This section

describes a collaboration with Zuley Rivera Alvidrez, in which we investigated how

neural representations of movement planning relate to muscle signals. More details

about this experiment can be found in Rivera Alvidrez et al. (2010).

How does activity in the motor cortex relate to muscle activation? Understanding

how ensembles of PMd and M1 neurons correspond to muscle activity is of both

fundamental scientific interest, as well as important for designing neural prostheses.

The state-space models described throughout this thesis can provide us with a

way to denoise, visualize (if 3-D or less), and think of the evolution of the neural

population in a few meaningful dimensions (illustrated in Fig. 4.16a). In Chapters 2

and 3 we used this state-space to help us gain intuition about the neural population,

and to relate the neural dynamics within this space to sensory encoding, decision-

making, and trial-by-trial differences in behavior. However, our understanding of

the underlying system would be further enhanced by relating these orthogonal axes,

which explain features of the neural population in state-space (S1, S2 and S3 in

Fig. 4.16a), to task variables. These state-space axes are likely to be a combination

of the behavioral signals explained by the neural data, rather than clearly relating to

any one behavioral signal in particular. Nevertheless, these axes may be recombined to

allow a potentially more useful and intuitive representation of the data (e.g. Machens

et al., 2010).

A body of literature relates activity in individual neurons in PMd and M1 to sub-

jects’ spatial goals, hand motion, joint motion, force output, and electromyographic

(EMG) activity (Scott, 2003). However, characterizing the relationship between in-

dividual neurons and various task variables has not added up to a comprehensive

understanding of how these movement areas translate neural activity into motor out-

put. We hypothesize that there is a systematic representation of motor output within

ensemble activity in PMd/M1. Further, we propose that we can rederive neural axes

such that axes in the state space correspond to muscle activity (see schematic in

Fig. 4.16b). Within this space, we predict that a neural trajectory moving along

an axis, say muscle1, will activate or deactivate muscle1, wheareas one moving along

muscle2 will change the activity of muscle2. Moreover, in this idealized view the axis

112

Figure 4.16: Low-dimensional neural state-space. (a) shows the hypothetical evolu-tion of the neural population activity (neural trajectory) in the statespace definedby the axes S1, S2, S3. t1 and t2 are two different time steps corresponding todifferent points in the neural trajectory. (b) shows an alternative space defined bybehaviorally-relevant axes. Figure from Rivera Alvidrez et al., 2010.

corresponding to a particular muscle would reflect the underlying relationship of the

neural data to that muscle. If this space accurately captures the relationship between

activity of the neural population and muscle activation, we expect that it should gen-

eralize across all tasks and behavioral contexts. To find this relationship, we asked:

how do average neural trajectories for one reach target relate to the corresponding

mean EMG?

We recorded simultaneous neural activity from motor cortices (M1/PMd) of rhesus

monkeys performing an arm-reaching task while recording EMG from arm muscles.

Reducing the dimensionality of the neural data, we asked whether any aspects of

the low-dimensional neural activity strongly correlated with the EMG signals, as

schematized in Figure 4.16b. We explored how a host of parameters of the low-

dimensional projections related to EMG signals. Of these, we found that the curvature

of the low-dimensional trajectories (as in Fig. 3.8) was tightly related to the timecourse

of muscle activity. To determine this, we found neural axes based on reaches to

one reach target (<5% of the data). Figure 4.17 shows how we defined axes of

neural curvature, and the corresponding curvature shown as a function of time. By

finding the curvature of the neural trajectory as a function of time, we produced 1-D

timeseries that could be compared with the mean EMG signals for each target.

In Figure 4.18 (right column), we show these curvature timeseries for four differ-

ent muscles. In the delayed-reach task used in this experiment, monkeys reached to

a large number of different target locations. Neural curvature axis (NCA) projec-

tion timeseries corresponding to reaches to different target positions are shown using

113

Figure 4.17: Illustration of neural curvature axis computation. (a) 3-D neural trajec-tories (circles correspond to turns). We defined the neural curvature axis (NCA) as anaxis perpendicular to the neural trajectory at time t (perpendicular to the black linesegment in (a, inset)) and lies in the same plane as the neural trajectory segmentdefined by [t-10ms, t+10ms]. Picking an axis perpendicular to the neural trajectoryat the reference time point ensured that the projection of the neural trajectory ontothe axis would have a derivative equal to zero at that point, capturing the turn orpoint of high curvature in the neural trajectory. Picking an axis that lies in the samelocal plane as the neural data ensured that the axis would be relevant (explained localneural variance) to the trajectory locally. This plane was found by computing the toptwo principal components of the neural trajectory segment. The neural trajectorieswere projected onto the this axis to obtain 1-D timeseries for each target (b). Figurefrom Rivera Alvidrez et al., 2010.

different line colors and widths. The left column of Figure 4.18 shows mean EMG

signals for each target, with reaches to different targets represented using the same

colors as in the right column.

A significant correlation across the right (neural curvature) and left (EMG) columns

support the hypothesis that the neural curvature metric is a good indicator of muscle

activity. On average, these neural axes could explain EMG for reaches across multiple

targets (average R2 = 0.65).

These results suggest that there are fundamental directions in the neural space

along which the neural population moves to activate particular muscles. Further, our

neural population included a mixture of single neurons and multi-units of unidentified

type. That our results were found using a heterogeneous neural population suggests

that the tight relationship between ensembles of cortical neurons and muscle activity

114

is not limited to corticomotorneuronal cells, and suggests that any transformation of

the cortical signals made by the spinal cord circuits is likely to be minor. Overall, our

ability to relate neural activity to muscle activity with few parameters and using very

little training data makes this a promising framework in which to study motor control

both from a scientific perspective and for the development of prosthetic systems with

highly generalizable performance.

Figure 4.18: NCA projections predict EMG features. Right column shows the NCAprojections (see text for details) for all targets (superimposed and shown in differentcolors) associated with wrist extensor (a), wrist flexor (b), trapezius (c), and biceps(d). The correlations R2 between the NCA projections and the mean EMG actvityfor the same muscles (left column) were 0.71 (extensor), 0.6 (flexor), 0.73 (trapezius),and 0.55 (biceps). Figure from Rivera Alvidrez et al., 2010.

115

4.4 Summary

This chapter focused on the reach system, describing the relationship between neural

dynamics in PMd and behavior in three different contexts. First, we investigated

how neural dynamics reflect rapid, sequential planning. In the context of a rapid

sequential-planning task, we showed that the firing rates of PMd neurons were mod-

ulated not only by the direction of a planned reach, but also by the order in which

a reach target was cued. This effect of sequence position was observed only in when

plans were cued rapidly (several target presentations per second), suggesting that

there are limits to how quickly neural plan dynamics can change. Second, we vi-

sualized neural trajectories during an ambiguous planning period, preceding final

specification of a reach target. While subjects waited for one of two possible reach

targets to be cued, neural trajectories reflected evidence of plan-switching more often

than predicted by chance. Finally, we demonstrated that population activity in PMd

reflects signatures of muscle activity, and that these neural signatures generalize ro-

bustly across muscles and reach directions. Taken together, these results extend our

understanding of how neural dynamics of movement planning change when the system

is pushed to plan quickly, how plan dynamics unfold in the context of uncertainty,

and how these underlying neural dynamics relate to motor output.

Chapter 5

Conclusion

5.1 Summary & Contributions

As we take in sensory input to decide where and when to move, neural circuits

in the brain evaluate and choose between multiple motor plans. On millisecond

timescales, neurons within and across circuits must coordinate their activity to allow

us to produce fast, accurate movements. Understanding the dynamics of this process

will provide insight into the neural basis of behavior and may facilitate development

of neurally-controlled prosthetic devices.

Studies of movement preparation in the arm-movement system have demonstrated

that the longer subjects have to prepare, the faster they can initiate arm movements.

Across many cortical and subcortical areas, researchers have demonstrated a clear

neural correlate for this effect (discussed in Snyder et al., 2006; Churchland et al.,

2006). Recent work has begun to describe the neural dynamics underlying movement

preparation and to describe relationships between these dynamics and behavior (e.g.,

Churchland et al., 2006; Afshar, 2008; Afshar et al., 2010).

In the oculomotor system, a large body of literature has described the relationship

between subjects’ reaction times and neural activity reflecting movement initiation

in eye-movement areas such as FEF, SC, and LIP (e.g., Wurtz and Goldberg, 1972;

Hanes and Schall, 1996). However, considerably less research in the oculomotor sys-

tem has focused on movement preparation—activity involved in planning a movement

before an instructed go cue. Recording from prearcuate cortex, an oculomotor region

on the surface of the frontal lobe, we used a rich behavioral paradigm that allowed us

116

117

to study responses related to movement preparation and to decision-making. Though

responses in prearcuate cortex have been characterized in the context of its broader

cortical neighborhood (Section 1.5.1), few studies have explicitly focused on this area

of the cortical surface between the arcuate and principal sulci. Utilizing multielectrode

recording techniques in prearcuate cortex, we were well positioned to ask (1) how the

population responses in this area reflect movement preparation and decision-making,

(2) how these responses compare to those recorded in other oculomotor areas, (3)

how dynamics of movement preparation unfold, and (4) how these dynamics compare

to those described in the arm-movement system.

In Chapter 2, we characterized response properties of a heterogeneous popula-

tion of prearcuate neurons, using both traditional metrics and novel state-space ap-

proaches. Dimensionality reduction methods allowed us to visualize average dynamics

of the population response, which would be obscured by averaging across the diverse

PSTHs of individual neurons. During the presentation of the dots stimulus, we found

neurons that reflected choice predictivity, similarly to those described by Kim and

Shadlen (1999) in this region of prefrontal cortex and to those of Shadlen and New-

some (2001) in LIP. Applying state-space methods to either the full dimensionality

of the data or to the dimensionality-reduced responses, we characterized dynamics of

choice-predictive activity in the population. These dynamics reflected a similar time-

course of dot motion integration to those we obtained using ROC analysis. During

the perimovement epoch of the task, our recordings contained neurons that responded

before, during and after saccades, some of which exhibited similar characteristics to

FEF neurons (e.g., Schall, 1997).

In Chapter 3, we drew upon visualizations of the neural trajectories during the

perimovement epoch to test hypotheses about movement preparation within the state

space. Including all neurons with saccade-related responses, we were able to explain

a significant amount of variability in monkeys’ reaction times based on the position

of neural trajectories at the time of the go cue on individual trials. This is the first

time, to our knowledge, that the dynamics of movement planning derived from the

activity of multiple, simultaneously recorded neurons in the oculomotor system have

been related to reaction time on a trial-by-trial basis.

We also used dimensionality reduction methods to determine how many dimen-

sions were necessary to describe the shared variance of the population responses in

118

prearcuate and in PMd. Overall, less of the variability in prearcuate responses was

shared across the population. This may be due to individual neurons in the prearcuate

population receiving more disparate cognitive influences. Though the shared variabil-

ity in prearcuate was lower, fewer dimensions were needed to describe the variability

that was shared across the population. Thus, of the dimensions that did capture

covariation, these dimensions explained a large amount of variance.

Why is the dimensionality of PMd greater than that of prearcuate? A body of

work in the arm-movement literature describes complex movements that are elicited

when passing current into primary motor cortex (e.g., Graziano et al., 2002). How-

ever, passing even large amounts of current into oculomotor areas elicits only saccades

(Robinson and Fuchs, 1969). Eye and arm movements differ in many ways (e.g., ef-

fector speed, weight, number of muscles required), so one might expect oculomotor

and reach systems to utilize different strategies for motor preparation. Arm move-

ment has more degrees of freedom than eye movement, and involves more muscles

as well. As far as physical constraints, eye and arm movement differ as well, both

in terms of forces (e.g., gravity), and in terms of obstacle avoidance during move-

ment. Given these differences in biomechanical aspects of the arm and the eye, it

would be unsurprising if the strategies required to control arm movement were more

complex or higher-dimensional. Continued application of dimensionality reduction

approaches across different movement systems will further our understanding of how

the underlying processes driving movement preparation compare across systems.

In Chapter 4, we found that neural plan activity in PMd was influenced by prior

target cues within a rapid sequential-planning task, in which multiple targets were

presented each second. On short timescales, the amplitude of neurons’ responses was

modulated not only by the location of a target cue, but also by the order in which

the target was presented. However, this effect of sequence position on neurons’ firing

rates was not seen when a sequence of target cues was presented slowly (> 1 second

between target presentations). One interpretation of this result is that the dynamics

of movement planning cannot change quickly enough to meet the demands of the

rapid plan-switching task. This inability to fully change plans on fast timescales may

be a consequence of limits of the neural dynamics of planning. Future experiments

will allow us to visualize dynamics in such tasks, which may aid in understanding the

source of constraints on rapid planning and plan-switching.

119

In the rapid plan-switching task, reach plans on previous trials influence neural

activity on the current trial. In prearcuate cortex we found an analogous effect on

longer timescales (several seconds per trial), as described in Chapter 2. In these

longer-timescale effects, the monkey’s prior action was reflected in neural activity,

both during the hold period after his saccade and often well into the next trial. Such

effects in PFC have been hypothesized to maintain a history trace of subjects’ actions

for the purpose of short-term working memory (reviewed by Fecteau and Munoz,

2003). The mechanisms underlying these shorter- and longer-timescale effects of

previous actions in PMd and prearcuate cortex may be similar. Insight into these

mechanisms could be gained by further characterization of the dynamical systems

driving both saccadic and reach planning.

Finally, the latter sections of Chapter 4 describe collaborative work with Zuley

Rivera Alvidrez in which we further characterize the neural dynamics of planning arm

movements. In Section 4.2, we used dimensionality reduction to visualize complex

plan dynamics in an instructed delay task with an ambiguous period. Section 4.3

also describes plan dynamics, but focuses on relating them to muscle output. Using

population recording in PMd/M1 and EMG recording, our results suggest that there

are fundamental directions in the neural state space along which the neural population

moves to activate particular muscles.

Our results in prearcuate cortex and in PMd bring us several steps closer to

understanding the neural dynamics of choosing and preparing movements, and more

generally, how the process of movement preparation is reflected in different systems

in the brain.

5.2 Array recording

5.2.1 Pros and cons of array recording

5.2.1.1 Advantages

The multielectrode recording arrays we used for our experiments offer both advan-

tages and disadvantages. Some of these advantages have been described throughout

this thesis. In particular, arrays enable simultaneous recording of populations of neu-

rons and the ability to relate activity of neural populations to behavior on single

120

trials. This simultaneous recording allows time-varying influences common to the

population (such as cognitive factors) to be either factored out or directly investi-

gated. Another advantage of using arrays is that the recording yield is higher. For

each behavioral experiment, many more cells can be recorded. Likewise, it is possible

to collect a higher number of trials for each neuron, assuming relative stability of

the array. The fixed position of the array also allows across-day comparisons of the

population response, which would be of particular use in studies of learning. Having

unmovable electrodes also prevents selection bias based on cells’ response properties

and implicitly creates variety in the relationship between cells’ response fields and

target locations.

5.2.1.2 Challenges of array placement

Though these advantages are substantial, there are also significant disadvantages of

chronic electrode array recording. First, determining correct placement of arrays

can be challenging. Our experience with misplaced arrays has underscored the im-

portance of confirming array locations early. We have learned that sulcal anatomy,

though a good general landmark, is not a reliable predictor of physiological responses.

Across and even within a subject (across hemispheres), the exact location of the arm

representation in PMd can vary by several millimeters.

Additionally, we learned that neural responses can be task-modulated (though

not well-tuned) even if the array is in the wrong place for the behavioral task. Ta-

bles 5.1 and 5.2 provide an overview of the recording utility of the arrays implanted in

prearcuate and PMd over the course of the experiments described in this thesis. The

problems with array placement noted in these tables have led us to explore various

techniques for targeting the appropriate cortical area.

To target the appropriate area for array placement, several groups have tried

recording or microstimulating before array implantation. Both of these approaches

carry different advantages and complications. Using single-electrode recording from

the cortical area of interest provides detailed spatial resolution about the response

properties in that region. However, the depth of the electrodes on the array may not

match the depth at which the neural signals of interest were recorded. Depending

on the density of the neurons of interest (e.g., Betz cells in M1), the population

recorded with single electrodes may not be well-sampled by the array. Additionally,

121

dural adhesion can occur in areas of single-electrode recording, which can complicate

array insertion surgeries (Romo, unpublished observation). Though dural adhesion

was initially a concern, members from our group have recorded from PMd and M1

prior to successful array insertions. However, we have not yet ruled out the possibility

that prior recording diminishes the longevity of high-quality neural signals obtainable

from the array.

Microstimulating the region of interest prior to array implantation can carry less of

a risk of dural adhesion, since it can be done intraoperatively or in a smaller number of

experimental sessions. If microstimulation is to be done in the context of behavior, an

extra surgery is necessary in order to access the tissue. If done interoperatively, effects

of anesthesia must be taken into consideration. Some anesthetics inhibit movement,

and the anesthesia must be kept very light in order to see effects of microstimulation.

We explored this possibility extensively, though ultimately determined that the risk to

the monkeys and potential problems with the approach were too significant. One such

problem of microstimulation, both intraoperative and in the context of behavioral

experiments, is that passing current into the region of interest may influence a broader

population of neurons than desired, which could lead to incorrect placement of the

array.

Microstimulating the region of interest after array implantation is a fairly reli-

able way to determine how neural responses correspond to limb representation. The

disadvantages of this method are that it requires a full array implantation surgery,

and that microstimulation through the array has previously been shown to damage

electrodes or the neural tissue (Shenoy lab, unpublished observation). However, array

technology has improved over the past few years, and passing current through the

Table 5.1: PMd arrays

Array SignalsGeorge good signalsHam I good signalsHam II array lost signals after 1 weekHam III array misplaced (leg representation)Larry I weak signalsLarry II array misplaced (leg representation)Larry III array misplaced (finger representation), no plan activityIsaac good signals

122

Table 5.2: Prearcuate arrays

Array Overall Dots Epoch Perimove Epoch Hold EpochVito I array misplaced

(M1)N/A N/A N/A

Vito II good signals fair signals good signals good signalsNorris no neural signals N/A N/A N/ACody array instability,

poor tuningpoor signals poor signals good signals

Tex good signals good signals good signals good signals

array to evoke movements now carries less of a risk of array or tissue damage.

Despite these challenges involved in appropriately positioning electrode arrays,

our experience has allowed us to improve our ability to place arrays correctly and to

more rapidly identify when an array is misplaced.

5.2.1.3 Other challenges of array recording

The second key disadvantage of using multielectrode arrays is that the signal quality

can be quite variable, and ultimately the ability to record single units disappears

entirely. This process of array ‘roll off’ may occur either gradually or abruptly, and

can begin as soon as 4 months (Shenoy Lab, Monkey L, array I) or as late as 3 years

(Shenoy Lab, Monkey D, array I) after array implantation. The average duration of

high-quality recordings from the arrays in the Shenoy and Newsome labs is approx-

imately 9 months. Given the inherent surgical risk of implanting arrays, the length

of time required to train a monkey, and the uncertainty in determining appropriate

array placement, having access to high-quality signals for as long as possible is obvi-

ously important. The limited duration of high-quality signals from array recordings

is one of their most fundamental drawbacks.

A third disadvantage of using chronically implanted arrays is that they do not

allow targeting of specific neural populations, or neurons with particular response

properties. Though this yields population recordings less sensitive to biases in se-

lecting neurons based on responses, our recordings are instead biased toward cells in

the same or nearby layers. If the cells sampled are responsive to the task, this may

not be a problem. However, if cells are not responsive to the task or have complex

123

response properties, this may complicate analysis. Also, the depth of tuning of a neu-

ron on the array may be worse than that of a single-electrode-recorded neuron, for

which target properties can be more readily optimized. Neurons’ firing rates in array

recordings may be lower due to this lack of target optimization. Firing rates may also

appear lower due to spike-sorting issues, which may be exacerbated by suboptimal

positioning of electrodes with respect to cell bodies.

Recording from collections of neurons with varied response properties may be

advantageous in some situations. However, a significant difficulty with including in

our analyses nearly every cell in a “population” is that we do not know whether

these cells actually belong together functionally. The cells we record from on the

array might belong to 3 or 4 different functional assemblies, and our state space

trajectories may therefore represent a mongrel assemblage of these systems that does

not actually reflect the neural state of any functional system in the brain.

A fourth challenge of using fixed electrode arrays is that signals from different units

recorded on the same electrode can be difficult to separate. Using movable electrodes,

experimenters can adjust the position of electrodes to achieve clean isolation of units’

activity. Without this ability, assigning spikes to neurons relies solely upon spike-

sorting methods, none of which achieve the same signal quality as possible with

single-unit recording. Therefore, for analyses that require a collection of well-isolated

neurons (e.g., determining differences in response properties between anatomical cell

types, such as putative interneurons and pyramidal cells), recording with moveable

electrodes is preferable to fixed multielectrode array recording. Our ability to reliably

determine neurons’ identity in array recordings is limited by spike-sorting methods,

each of which has a different set of limitations.

Neural signals from PMd data in this thesis were sorted using in-house methods

(“RRR”) described in Santhanam et al. (2004). Neural signals from prearcuate data

were sorted using the Plexon Offline Sorter. When applying the RRR method to

prearcuate data for the purpose of comparison, we found that RRR “underclustered”

the data, finding only one multi-unit cluster (Fig. 5.1, upper left panel). In contrast,

the Plexon Offline Sorter tended to “overcluster” the data, splitting signals from one

cell into multiple clusters (Fig. 5.1, upper right panel). The correct solution is likely

somewhere in between. Though automated scripts were written to iteratively modify

spike-sorting performance of the Plexon Offline Sorter, the quality of the automatic

124

Vo

lta

ge

(µ

V)

Time relative to trough (ms)

Vo

lta

ge

(µ

V)

Time relative to trough (ms)

RRR sort, Channel 96, 1 unit Plexon sort, Channel 96, 8 units

0

40

-40

0

50

-50

0-0.2 0.2 0.4 0.6 0-0.2 0.2 0.4 0.6 -10 -5 0 5

0

6

-6

PC1

PC

2

a b c

Figure 5.1: Comparison of waveforms found using (a) RRR and (b) Plexon OfflineSorter. (c) shows the first two principal components (PC1 and PC2) of raw datafor this recording channel, using RRR. Blue circles correspond to centers of clustersof events assigned to Unit 1 (blue waveform in (a)); the pink square corresponds tounassigned events (pink waveform in (a)).

spike sorts is still not comparable to hand-sorted spikes. For this reason, and for

the intractability of hand-sorting spikes from hundreds of neurons from the number

of data sets (∼100) surveyed in this thesis, we did not focus our analyses on any

questions that required single units. We hope that in the future, better automated

spike-sorting methods will allow better resolution of the neural populations recorded

on unmovable electrode arrays.

Finally, using fixed electrode arrays presents a challenge in determining the func-

tional area recorded. Areas such as FEF and 8Ar are distinguished by the microstim-

ulation current needed to evoke an eye movement. Stimulating through the array is

possible, but it is unlikely that the current thresholds would be directly comparable

with those reported in the literature (e.g., different materials are used for electrodes;

array may be encapsulated by glia). Some of these limitations of multielectrode

recording may be addressed with future improvements in recording technology.

5.2.2 What can and cannot be addressed by multielectrode

recording and state-space approaches?

Single-electrode recordings have shown that prearcuate cortex particpates in sensory-

guided decisions and movement planning (Boch and Goldberg, 1989; Kim and Shadlen,

1999; Zaksas and Pasternak, 2006). However, less is known about how the dynamics

125

of these computations play out across neural circuits or neural populations. Though a

stimulus may initiate a process that unfolds differently on different trials, traditional

trial-averaging methods will not be able to explain what happened on individual

trials. State-space approaches as described in this thesis and in Churchland et al.

(2007) and Yu et al. (2009) provide the ability to visualize influences that may not

be the same across many repetitions of a trial. These methods allow investigation

of dynamics beyond those driven directly by a stimulus, such as volitional changes

of mind or occasional, unpredictable lapses in attention. Recording from individual

neurons, spiking noise limits the viability of finding concrete signatures of this type

of cognitive processing. To investigate changes of mind that are linked to external

stimuli, single neurons’ responses can be aligned to these stimuli to look at average

responses. However, it is not possible to time-lock rasters to internal events like

changes of mind, whose exact times vary from trial to trial and are not externally

accessible. Without averaging responses across trials, the signals available in individ-

ual neurons on single trials are typically too noisy to interpret directly. Even if it

were it possible to interpret these signals, another problem is that individual neurons

might not reflect the underlying dynamics of the overall system. Therefore, simul-

taneous recording and state-space approaches are necessary to visualize or otherwise

characterize internally-driven cognitive dynamics.

We used multielectrode recording for the experiments in this thesis because, de-

spite the aforementioned limitations, this approach offers the best way to characterize

the dynamics of neural populations. In this thesis, array recording and state-space

approaches allowed us to describe relationships between subjects’ behavior and de-

cision variables and movement preparation on a trial-by-trial basis. Such methods

are important not only for studying the single-trial dynamics of movement prepara-

tion on behaviorally-relevant timescales, but are also necessary for neurally-controlled

prostheses, which have access to signals from only a very limited amount of time.

5.3 Open questions & Future directions

In Chapter 3 we discussed different hypotheses of movement preparation and how they

would be represented in populations of motoric neurons. The extent to which these

hypotheses describe neural data may be influenced by the criteria used in selecting

126

the neurons to include in the analysis.

Our characterization of cortical areas is heavily influenced by researchers’ choices

of which cells to record. How would our understanding of FEF or LIP differ if pre-

vious studies had relaxed their selection criteria to include recordings from all cells

encountered, as on the array? Alternatively, if our population had comprised of neu-

rons typical of core FEF (e.g., Bruce et al., 1985; Hanes and Schall, 1996), how would

our conclusions differ? Individual neurons in FEF described in the literature typically

have higher firing rates than the prearcuate neurons we recorded here. Higher firing

rates of the population would create larger excursions of neural trajectories through

state-space, perhaps creating more stereotyped trajectories (as in PMd). Though we

can speculate how some features of FEF trajectories may differ from our state-space

descriptions of prearcuate populations, future work is needed to determine the influ-

ence of individual neurons’ response properties on our description of neural dynamics.

Additionally, further exploration of cell selection biases on state-space representations

across cortical areas may provide a more complete understanding of neural dynamics

than would be possible using sets of neurons selected in single-electrode recording.

Another important avenue of future research is to investigate differential charac-

teristics and contributions of subpopulations of neurons. Determining which sets of

neurons to include in population analyses (as described in Section 5.2.1) may signif-

icantly influence our understanding of neural dynamics. Future studies could sepa-

rate populations of neurons based on functional properties. For instance, population

analyses of hypothetical FEF recordings may yield different outcomes depending on

whether visual, visuo-motoric, and motoric neurons are combined or separated. To

target the appropriate subpopulations, we need to understand where different neurons

project. In the future, optogenetic techniques in concert with array recording may

provide the ability to (1) selectively stimulate specified sub-populations of neurons,

(2) simultaneously stimulate and record from neural ensembles, and (3) stimulate

incoming projections from specific areas (e.g., projections to FEF or 8Ar from LIP).

In Chapter 4 we described a result in which the dynamics of movement planning

in PMd appeared not to be able to keep up with the pace of the behavioral task. After

determining the appropriate populations of neurons in which to investigate dynamics

of movement planning, we can return to the question: what are the fundamental

speed limits of changes in neural plan dynamics? How do plan dynamics differ when

127

planning is being driven by an external stimulus, rather than by internally-driven

changes of mind? Future experiments may address these questions, as well as how

speed limits of neural dynamics compare in the oculomotor and reach systems.

The trial-to-trial relationship between population activity and reaction time in

Chapter 3 provides evidence that neural trajectories reflect dynamics of the underlying

system that drive behavior. If we could stimulate (optically or electrically) neurons on

the array during decision-making, how would this be exhibited in neural trajectories

and the state space representation? We can test our understanding by stimulating

selected neural ensembles to understand how the activity of these neurons influences

the dynamical system represented by the larger population.

5.4 Conclusions

The central finding of this work is the demonstration that neural dynamics of hetero-

geneous populations of oculomotor neurons can be described using state-space meth-

ods, and that these dynamics relate to monkeys’ behavior on a trial-by-trial basis.

Recording from neural ensembles in prearcuate cortex, a relatively uncharacterized

oculomotor area, we demonstrated this effect in the context of movement preparation.

Our results also support findings from single-unit studies in this region of the frontal

lobe. Using both traditional and state-space approaches, we found that prearcuate

cortex reflects decision-making in a random dots discrimination task, similar to Kim

& Shadlen’s results (Kim and Shadlen, 1999). We also found that the responses of a

heterogeneous population of neurons in prearcuate cortex contain signatures of pre-

vious actions. Finally, state-space and multielectrode recording methods allowed us

to compare systems-level descriptions of movement preparation an oculomotor area

(prearcuate) and an arm-movement area (PMd). Despite differences across the two

systems and across effectors, we found that the same framework could be used to

describe the dynamics of movement preparation.

Taken together, the experiments described in this thesis provide a deeper under-

standing of planning activity in the motor system and how plan dynamics change with

different task constraints. Moreover, relating the dynamics of movement planning to

behavior may ultimately aid in the development of neurally-controlled prostheses.

Bibliography

Achtman N, Afshar A, Santhanam G, Yu BM, Ryu SI, Shenoy KV (2007) Free-paced

high-performance brain-computer interfaces. J Neural Eng 4:336–47.

Afshar A, Santhanam G, Yu B, Ryu S, Sahani M, Shenoy K (2010) Single-trial neural

correlates of arm movement preparation (submitted).

Afshar A (2008) Neural mechanisms of motor preparation and applications to pros-

theses. Ph.D. dissertation., Stanford University.

Andersen RA, Asanuma C, Cowan WM (1985a) Callosal and prefrontal associa-

tional projecting cell populations in area 7a of the macaque monkey: a study using

retrogradely transported fluorescent dyes. J Comp Neurol 232:443–55.

Andersen RA, Essick GK, Siegel RM (1985b) Encoding of spatial location by posterior

parietal neurons. Science 230:456–8.

Andersen RA, Hwang EJ, Mulliken GH (2010) Cognitive neural prosthetics. Annu

Rev Psychol 61:169–90, C1–3.

Bair W, Zohary E, Newsome WT (2001) Correlated firing in macaque visual area

MT: time scales and relationship to behavior. J Neurosci 21:1676–97.

Barbas H, Mesulam MM (1981) Organization of afferent input to subdivisions of area

8 in the rhesus monkey. J Comp Neurol 200:407–31.

Barraclough DJ, Conroy ML, Lee D (2004) Prefrontal cortex and decision making in

a mixed-strategy game. Nat Neurosci 7:404–10.

Basso MA, Wurtz RH (1998) Modulation of neuronal activity in superior colliculus

by changes in target probability. J Neurosci 18:7519–34.

128

129

Bastian A, Riehle A, Erlhagen W, Schoner G (1998) Prior information preshapes

the population representation of movement direction in motor cortex. Neurore-

port 9:315–9.

Bastian A, Schoner G, Riehle A (2003) Preshaping and continuous evolution

of motor cortical representations during movement preparation. Eur J Neu-

rosci 18:2047–2058.

Batista AP, Santhanam G, Yu BM, Ryu SI, Afshar A, Shenoy KV (2007) Refer-

ence frames for reach planning in macaque dorsal premotor cortex. J Neurophys-

iol 98:966–83.

Beevor C (1888) A further minute analysis by electric stimulation of the so-called

motor region of the cortex cerebri in the monkey (macacus sinicus). Philosophical

Transactions of the Royal Society of London. B 179:205–256.

Bizzi E (1967) Discharge of frontal eye field neurons during eye movements in unanes-

thetized monkeys. Science 157:1588–90.

Bizzi E (1968) Discharge of frontal eye field neurons during saccadic and following

eye movements in unanesthetized monkeys. Exp Brain Res 6:69–80.

Bizzi E, Schiller PH (1970) Single unit activity in the frontal eye fields of unanes-

thetized monkeys during eye and head movement. Exp Brain Res 10:150–8.

Boch RA, Goldberg ME (1989) Participation of prefrontal neurons in the preparation

of visually guided eye movements in the rhesus monkey. J Neurophysiol 61:1064–84.

Britten KH, Shadlen MN, Newsome WT, Movshon JA (1992) The analysis of vi-

sual motion: a comparison of neuronal and psychophysical performance. J Neu-

rosci 12:4745–65.

Britten KH, Shadlen MN, Newsome WT, Movshon JA (1993) Responses of neurons

in macaque MT to stochastic motion signals. Vis Neurosci 10:1157–69.

Bruce CJ, Goldberg ME (1985) Primate frontal eye fields. I. Single neurons discharg-

ing before saccades. J Neurophysiol 53:603–35.

130

Bruce CJ, Goldberg ME, Bushnell MC, Stanton GB (1985) Primate frontal eye fields.

II. Physiological and anatomical correlates of electrically evoked eye movements. J

Neurophysiol 54:714–34.

Chestek CA, Batista AP, Santhanam G, Yu BM, Afshar A, Cunningham JP, Gilja

V, Ryu SI, Churchland MM, Shenoy KV (2007) Single-neuron stability during

repeated reaching in macaque premotor cortex. J Neurosci 27:10742–50.

Churchland MM, Santhanam G, Shenoy KV (2006) Preparatory activity in pre-

motor and motor cortex reflects the speed of the upcoming reach. J Neurophys-

iol 96:3130–46.

Churchland MM, Shenoy KV (2007) Delay of movement caused by disruption of

cortical preparatory activity. J Neurophysiol 97:348–59.

Churchland MM, Yu BM, Ryu SI, Santhanam G, Shenoy KV (2006) Neural vari-

ability in premotor cortex provides a signature of motor preparation. J Neu-

rosci 26:3697–3712.

Churchland MM, Yu BM, Sahani M, Shenoy KV (2007) Techniques for extracting

single-trial activity patterns from large-scale neural recordings. Curr Opin Neuro-

biol 17:609–618.

Churchland M, Cunningham J, Kaufman M, Ryu S, Shenoy K (2010) Re-examining

the preferred direction during movement preparation. Neuron (submitted).

Cisek P, Kalaska JF (2002) Simultaneous encoding of multiple potential reach direc-

tions in dorsal premotor cortex. J Neurophysiol 87:1149–54.

Coe B, Tomihara K, Matsuzawa M, Hikosaka O (2002) Visual and anticipatory bias

in three cortical eye fields of the monkey during an adaptive decision-making task.

J Neurosci 22:5081–90.

Collin NG, Cowey A, Latto R, Marzi C (1982) The role of frontal eye-fields and

superior colliculi in visual search and non-visual search in rhesus monkeys. Behav

Brain Res 4:177–93.

131

Crammond DJ, Kalaska JF (1989) Neuronal activity in primate parietal cortex area

5 varies with intended movement direction during an instructed-delay period. Exp

Brain Res 76:458–62.

Crammond DJ, Kalaska JF (2000) Prior information in motor and premotor cortex:

activity during the delay period and effect on pre-movement activity. J Neurophys-

iol 84:986–1005.

Crosby E, Yoss R, Henderson J (1952) The mammalian midbrain and isthmus regions.

Part II. The fiber connections. D. The pattern for eye movements on the frontal

eye field and the discharge of specific portions of this field to and through midbrain

levels. The Journal of comparative neurology 97:357.

Donoghue JP (2002) Connecting cortex to machines: recent advances in brain inter-

faces. Nat Neurosci 5 Suppl:1085–8.

Dorris MC, Pare M, Munoz DP (1997) Neuronal activity in monkey superior colliculus

related to the initiation of saccadic eye movements. J Neurosci 17:8566–79.

Dorris MC, Pare M, Munoz DP (2000) Immediate neural plasticity shapes motor

performance. J Neurosci 20:RC52.

Dum RP, Strick PL (2002) Motor areas in the frontal lobe of the primate. Physiol

Behav 77:677–82.

Evarts EV (1966) Pyramidal tract activity associated with a conditioned hand move-

ment in the monkey. J Neurophysiol 29:1011–27.

Evarts EV (1968) Relation of pyramidal tract activity to force exerted during volun-

tary movement. J Neurophysiol 31:14–27.

Evarts EV, Tanji J (1974) Gating of motor cortex reflexes by prior instruction. Brain

Res 71:479–94.

Everling S, Munoz D (2000) Neuronal correlates for preparatory set associated with

pro-saccades and anti-saccades in the primate frontal eye field. J Neurosci 20:387.

132

Favilla M, Hening W, Ghez C (1989) Trajectory control in targeted force impulses.

VI. Independent specification of response amplitude and direction. Exp Brain

Res 75:280–94.

Fecteau JH, Munoz DP (2003) Exploring the consequences of the previous trial. Nat

Rev Neurosci 4:435–43.

Ferrier D (1874) On the localisation of the functions of the brain. British Medical

Journal 2:766.

Ferrier D (1875) Experiments on the brain of monkeys. No. I. Proceedings of the

Royal Society of London 23:409–430.

Funahashi S, Bruce CJ, Goldman-Rakic PS (1989) Mnemonic coding of visual space

in the monkey’s dorsolateral prefrontal cortex. J Neurophysiol 61:331–49.

Funahashi S, Bruce CJ, Goldman-Rakic PS (1990) Visuospatial coding in primate

prefrontal neurons revealed by oculomotor paradigms. J Neurophysiol 63:814–31.

Funahashi S, Bruce CJ, Goldman-Rakic PS (1991) Neuronal activity related to sac-

cadic eye movements in the monkey’s dorsolateral prefrontal cortex. J Neurophys-

iol 65:1464–83.

Fuster JM (1973) Unit activity in prefrontal cortex during delayed-response perfor-

mance: neuronal correlates of transient memory. J Neurophysiol 36:61–78.

Fuster JM (1993) Frontal lobes. Current Opinion in Neurobiology 3:160–165.

Gamlin PD, Yoon K (2000) An area for vergence eye movement in primate frontal

cortex. Nature 407:1003–7.

Genovesio A, Tsujimoto S, Wise SP (2006) Neuronal activity related to elapsed time

in prefrontal cortex. J Neurophysiol 95:3281–5.

Georgopoulos AP (1996) Arm movements in monkeys: behavior and neurophysiology.

J Comp Physiol A 179:603–12.

Georgopoulos AP, Kalaska JF, Caminiti R, Massey JT (1982) On the relations be-

tween the direction of two-dimensional arm movements and cell discharge in pri-

mate motor cortex. J Neurosci 2:1527–37.

133

Gerbella M, Belmalih A, Borra E, Rozzi S, Luppino G (2007) Multimodal archi-

tectonic subdivision of the caudal ventrolateral prefrontal cortex of the macaque

monkey. Brain Struct Funct 212:269–301.

Gerbella M, Belmalih A, Borra E, Rozzi S, Luppino G (2009) Cortical connections

of the macaque caudal ventrolateral prefrontal areas 45a and 45b. Cereb Cortex .

Gilja V, Kalmar R, Santhanam G, Ryu S, Yu B, Afshar A, Shenoy K (2005) Trial-by-

trial mean normalization improves plan period reach target decoding. In Abstract

Viewer/Itinerary Planner. Program No. 519.18., Washington, DC. Society for Neu-

roscience.

Godschalk M, Lemon RN, Kuypers HG, van der Steen J (1985) The involvement of

monkey premotor cortex neurones in preparation of visually cued arm movements.

Behav Brain Res 18:143–57.

Goldberg ME, Bushnell MC (1981) Behavioral enhancement of visual responses in

monkey cerebral cortex. II. Modulation in frontal eye fields specifically related to

saccades. J Neurophysiol 46:773–87.

Goldman-Rakic PS (1987) Circuitry of primate prefrontal cortex and the regulation

of behavior by representational memory In Handbook of Physiology. The Nervous

System., Vol. V, pp. p. 373–417. Bethesda, MD: Am. Physiol. Soc.

Goldman-Rakic PS (1988) Topography of cognition: parallel distributed networks in

primate association cortex. Annu Rev Neurosci 11:137–56.

Gottlieb JP, Bruce CJ, MacAvoy MG (1993) Smooth eye movements elicited by

microstimulation in the primate frontal eye field. J Neurophysiol 69:786–99.

Gottlieb JP, MacAvoy MG, Bruce CJ (1994) Neural responses related to smooth-

pursuit eye movements and their correspondence with electrically elicited smooth

eye movements in the primate frontal eye field. J Neurophysiol 72:1634–53.

Graziano M, Taylor C, Moore T (2002) Complex movements evoked by microstimu-

lation of precentral cortex. Neuron 34:841–851.

Hanes DP, Schall JD (1996) Neural control of voluntary movement initiation. Sci-

ence 274:427–430.

134

Hasegawa RP, Blitz AM, Geller NL, Goldberg ME (2000) Neurons in monkey pre-

frontal cortex that track past or predict future performance. Science 290:1786–9.

Hatsopoulos N, Joshi J, O’Leary J (2004) Decoding continuous and discrete motor

behaviors using motor and premotor cortical ensembles. J Neurophysiol 92:1165.

Hays AJ, Richmond B, Optican L (1982) Unix-based multiple-process system, for

real-time data acquisition and control. Electron Conventions, El Segundo, CA.

Hochberg LR, Serruya MD, Friehs GM, Mukand JA, Saleh M, Caplan AH, Branner

A, Chen D, Penn RD, Donoghue JP (2006) Neuronal ensemble control of prosthetic

devices by a human with tetraplegia. Nature 442:164–71.

Horwitz GD, Newsome WT (1999) Separate signals for target selection and movement

specification in the superior colliculus. Science 284:1158–61.

Horwitz GD, Newsome WT (2001) Target selection for saccadic eye movements:

prelude activity in the superior colliculus during a direction-discrimination task. J

Neurophysiol 86:2543–58.

Judge SJ, Richmond BJ, Chu FC (1980) Implantation of magnetic search coils for

measurement of eye position: an improved method. Vision Res 20:535–8.

Kalaska JF, Scott SH, Cisek P, Sergio LE (1997) Cortical control of reaching move-

ments. Curr Opin Neurobiol 7:849–59.

Kalaska J (1991) What parameters of reaching are encoded by discharges of cortical

cells? In Motor control: concepts and issues: report of the Dahlem Workshop on

Motor Control: Concepts and Issues, Berlin, 1989, December 3-8, p. 307. John

Wiley & Sons.

Kalmar R, Gilja V, Santhanam G, Ryu S, Yu B, Afshar A, Shenoy K (2005) PMd de-

lay activity during rapid sequential movement plans. In Abstract Viewer/Itinerary

Planner. Program No. 519.17., Washington, DC. Society for Neuroscience.

Kalmar R, Reppas J, Ryu S, Shenoy K, Newsome WW (2010) Ensemble activity un-

derlying movement preparation in prearcuate cortex In Computational and Systems

Neuroscience (COSYNE) meeting, Salt Lake City, UT.

135

Keele S (1968) Movement control in skilled motor performance. Psychological Bul-

letin 70:387–403.

Kim JN, Shadlen MN (1999) Neural correlates of a decision in the dorsolateral

prefrontal cortex of the macaque. Nat Neurosci 2:176–85.

Kolenikov S, Bollen K, Savalei V (2006) Specification tests with Heywood cases In

Proceedings of the American Statistical Association, Social Statistics Section [CD-

ROM], Alexandria, VA. American Statistical Association.

Kutas M, Donchin E (1974) Studies of squeezing: handedness, responding hand,

response force, and asymmetry of readiness potential. Science 186:545–8.

Lecas JC, Requin J, Anger C, Vitton N (1986) Changes in neuronal activity of

the monkey precentral cortex during preparation for movement. J Neurophys-

iol 56:1680–1702.

Levinsohn G (1909) Uber die beziehungen der grosshirnrinde beim affen zu den

bewegungen des auges. Graefe’s Archive for Clinical and Experimental Ophthal-

mology 71:313–378.

Levy R, Goldman-Rakic PS (2000) Segregation of working memory functions within

the dorsolateral prefrontal cortex. Exp Brain Res 133:23–32.

Louie K, Glimcher PW (2010) Separating value from choice: delay discounting ac-

tivity in the lateral intraparietal area. J Neurosci 30:5498–507.

Machens CK, Romo R, Brody CD (2010) Functional, but not anatomical, separation

of “what” and “when” in prefrontal cortex. J Neurosci 30:350–60.

McCoy AN, Platt ML (2005) Expectations and outcomes: decision-making in the pri-

mate brain. J Comp Physiol A Neuroethol Sens Neural Behav Physiol 191:201–11.

Medalla M, Barbas H (2006) Diversity of laminar connections linking periarcuate and

lateral intraparietal areas depends on cortical structure. Eur J Neurosci 23:161–79.

Messier J, Kalaska JF (2000) Covariation of primate dorsal premotor cell activity

with direction and amplitude during a memorized-delay reaching task. J Neuro-

physiol 84:152–65.

136

Miller EK, Erickson CA, Desimone R (1996) Neural mechanisms of visual working

memory in prefrontal cortex of the macaque. J Neurosci 16:5154–67.

Miller E (2000) The prefrontal cortex and cognitive control. Nature Reviews Neuro-

science 1:59–65.

Moschovakis AK, Gregoriou GG, Ugolini G, Doldan M, Graf W, Guldin W, Had-

jidimitrakis K, Savaki HE (2004) Oculomotor areas of the primate frontal lobes: a

transneuronal transfer of rabies virus and [14c]-2-deoxyglucose functional imaging

study. J Neurosci 24:5726–40.

Mott F, Schaefer E (1890) On associated eye-movements produced by cortical

faradization of the monkey’s brain. Brain 13:165.

Mulliken GH, Musallam S, Andersen RA (2008) Decoding trajectories from posterior

parietal cortex ensembles. J Neurosci 28:12913–26.

Musallam S, Corneil BD, Greger B, Scherberger H, Andersen RA (2004) Cognitive

control signals for neural prosthetics. Science 305:258–62.

Newsome WT, Britten KH, Movshon JA (1989) Neuronal correlates of a perceptual

decision. Nature 341:52–4.

Nicolelis MA (2001) Actions from thoughts. Nature 409:403–7.

Padoa-Schioppa C, Li CSR, Bizzi E (2002) Neuronal correlates of kinematics-to-

dynamics transformation in the supplementary motor area. Neuron 36:751–65.

Pare M, Munoz DP (1996) Saccadic reaction time in the monkey: Advanced prepara-

tion of oculomotor programs is primarily responsible for express saccade occurrence.

Journal of Neurophysiology 76.

Parthasarathy HB, Schall JD, Graybiel AM (1992) Distributed but convergent or-

dering of corticostriatal projections: analysis of the frontal eye field and the sup-

plementary eye field in the macaque monkey. J Neurosci 12:4468–88.

Pellizzer G, Hedges JH (2003) Motor planning: effect of directional uncertainty with

discrete spatial cues. Exp Brain Res 150:276–89.

137

Petrides M (1994) Frontal lobes and behaviour. Curr Opin Neurobiol 4:207–11.

Posner MI, Snyder CR, Davidson BJ (1980) Attention and the detection of signals.

J Exp Psychol 109:160–74.

Preuss T (2007) Evolutionary specializations of primate brain systems. Primate

Origins: Adaptations and Evolution pp. 625–675.

Preuss TM, Goldman-Rakic PS (1991) Myelo- and cytoarchitecture of the granular

frontal cortex and surrounding regions in the strepsirhine primate galago and the

anthropoid primate macaca. J Comp Neurol 310:429–474.

Reppas J, Kalmar R, Ryu S, Shenoy K, Newsome W (2010) Characterizing response

properties in prearcuate cortex (in preparation) .

Reppas J, Ryu S, Newsome W (2010) Hold-period activity in prearcuate cortex. (in

preparation) .

Riehle A, Requin J (1989) Monkey primary motor and premotor cortex: single-cell

activity related to prior information about direction and extent of an intended

movement. J Neurophysiol 61:534–49.

Riehle A, Requin J (1993) The predictive value for performance speed of preparatory

changes in neuronal activity of the monkey motor and premotor cortex. Behav

Brain Res 53:35–49.

Rivera Alvidrez Z, Kalmar R, Afshar A, Santhanam G, Yu B, Ryu S, Shenoy K (2007)

Single-trial representation of uncertainty about reach goals in macaque pmd. In

Society for Neuroscience annual meeting.

Rivera Alvidrez Z, Kalmar R, Afshar A, Santhanam G, Yu B, Ryu S, Shenoy K (2008)

Single-trial representation of uncertainty about reach goals in macaque pmd. In

Computational and Systems Neuroscience (COSYNE) meeting, Salt Lake City, UT.

III-52.

Rivera Alvidrez Z, Kalmar R, Ryu S, Shenoy K (2010) Low-dimensional neural fea-

tures predict muscle EMG signals In Proceedings of the 32nd Annual International

Conference of the IEEE EMBS, Buenos Aires, Argentina.

138

Robinson DA, Fuchs AF (1969) Eye movements evoked by stimulation of frontal eye

fields. J Neurophysiol 32:637–48.

Roesch MR, Olson CR (2003) Impact of expected reward on neuronal activity in

prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J

Neurophysiol 90:1766–89.

Roitman JD, Shadlen MN (2002) Response of neurons in the lateral intraparietal area

during a combined visual discrimination reaction time task. J Neurosci 22:9475–89.

Romanski LM (2004) Domain specificity in the primate prefrontal cortex. Cogn Affect

Behav Neurosci 4:421–9.

Romo R, Hernandez A, Zainos A (2004) Neuronal correlates of a perceptual decision

in ventral premotor cortex. Neuron 41:165–73.

Rosenbaum DA (1980) Human movement initiation: specification of arm, direction,

and extent. J Exp Psychol Gen 109:444–74.

Rosenkilde CE (1979) Functional heterogeneity of the prefrontal cortex in the monkey:

a review. Behav Neural Biol 25:301–45.

Roweis S (1998) EM algorithms for PCA and SPCA. Advances in Neural Information

Processing Systems 10 (626).

Santhanam G, Ryu S, Yu B, Afshar A, Shenoy K (2006) A high-performance brain-

computer interface. Nature 442:195–198.

Santhanam G, Sahani M, Ryu S, Shenoy K (2004) An extensible infrastructure for

fully automated spike sorting during online experiments. Conf Proc IEEE Eng Med

Biol Soc 6:4380–4.

Schall JD (1991) Neuronal activity related to visually guided saccadic eye movements

in the supplementary motor area of rhesus monkeys. J Neurophysiol 66:530–58.

Schall JD (1995) Neural basis of saccade target selection. Rev Neurosci 6:63–85.

Schall JD (2001) Neural basis of deciding, choosing and acting. Nat Rev Neu-

rosci 2:33–42.

139

Schall JD, Hanes DP, Thompson KG, King DJ (1995a) Saccade target selection

in frontal eye field of macaque. i. visual and premovement activation. J Neu-

rosci 15:6905–6918.

Schall JD, Morel A, King DJ, Bullier J (1995b) Topography of visual cortex connec-

tions with frontal eye field in macaque: convergence and segregation of processing

streams. J Neurosci 15:4464–4487.

Schall JD, Thompson KG (1999) Neural selection and control of visually guided eye

movements. Annu Rev Neurosci 22:241–259.

Schall J, Hanes D (1993) Saccade latency in context: Regulation of gaze behavior by

supplementary eye field. Behavioral and Brain Sciences 16:588–589.

Schall JD (1997) Visuomotor areas of the frontal lobe In et al. R, editor, Cerebral

Cortex, Vol. 12, chapter 13, pp. 527–638. New York, Plenum Press.

Schall JD, Stuphorn V, Brown JW (2002) Monitoring and control of action by the

frontal lobes. Neuron 36:309–22.

Schiller PH, True SD, Conway JL (1979) Paired stimulation of the frontal eye fields

and the superior colliculus of the rhesus monkey. Brain Res 179:162–4.

Schiller PH, True SD, Conway JL (1980) Deficits in eye movements following frontal

eye-field and superior colliculus ablations. J Neurophysiol 44:1175–89.

Schlag J, Schlag-Rey M (1987) Evidence for a supplementary eye field. J Neurophys-

iol 57:179–200.

Schwartz AB (2004) Cortical neural prosthetics. Annu Rev Neurosci 27:487–507.

Scott S (2003) The role of primary motor cortex in goal-directed movements: in-

sights from neurophysiological studies on non-human primates. Curr Opin Neuro-

biol 13:671–677.

Seidemann E, Arieli A, Grinvald A, Slovin H (2002) Dynamics of depolarization and

hyperpolarization in the frontal cortex and saccade goal. Science 295:862–5.

Shadlen MN, Newsome WT (1996) Motion perception: seeing and deciding. Proc

Natl Acad Sci USA 93:628–33.

140

Shadlen MN, Newsome WT (2001) Neural basis of a perceptual decision in the

parietal cortex (area LIP) of the rhesus monkey. J Neurophysiol 86:1916–36.

Smith W (1940) Electrically responsive cortex within the sulci of the frontal lobe.

Anat. Rec 76:75–76.

Snyder LH, Dickinson AR, Calton JL (2006) Preparatory delay activity in the monkey

parietal reach region predicts reach reaction times. J Neurosci 26:10091–9.

Stanton GB, Bruce CJ, Goldberg ME (1993) Topography of projections to the frontal

lobe from the macaque frontal eye fields. J Comp Neurol 330:286–301.

Stanton G, Deng S, Goldberg M, McMullen N (1989) Cytoarchitectural characteristic

of the frontal eye fields in macaque monkeys. J Comp Neurol 282:415.

Stoet G, Snyder LH (2003) Task preparation in macaque monkeys (macaca mulatta).

Anim Cogn 6:121–30.

Sugrue LP, Corrado GS, Newsome WT (2004) Matching behavior and the represen-

tation of value in the parietal cortex. Science 304:1782–7.

Suner S, Fellows M, Vargas-Irwin C, Nakata G, Donoghue J (2005) Reliability of

signals from a chronically implanted, silicon-based electrode array in non-human

primate primary motor cortex. IEEE Transactions on Neural Systems and Reha-

bilitation Engineering 13:524–541.

Suzuki H, Azuma M (1983) Topographic studies on visual neurons in the dorsolateral

prefrontal cortex of the monkey. Exp Brain Res 53:47–58.

Tanji J, Evarts EV (1976) Anticipatory activity of motor cortex neurons in relation

to direction of an intended movement. J Neurophysiol 39:1062–8.

Taylor CSR, Gross CG (2003) Twitches versus movements: a story of motor cortex.

Neuroscientist 9:332–42.

Tehovnik EJ, Sommer MA, Chou IH, Slocum WM, Schiller PH (2000) Eye fields in

the frontal lobes of primates. Brain Res Brain Res Rev 32:413–48.

141

Thompson KG, Hanes DP, Bichot NP, Schall JD (1996) Perceptual and motor pro-

cessing stages identified in the activity of macaque frontal eye field neurons during

visual search. J Neurophysiol 76:4040–4055.

Wagman I, Krieger HP, Papatheodorou CA, Bender MB (1961) Eye movements

elicited by surface and depth stimulation of the frontal lobe of macaque mulatta.

J Comp Neurol 117:179–88.

Weinrich M, Wise SP (1982) The premotor cortex of the monkey. J Neu-

rosci 2:1329–45.

Weinrich M, Wise SP, Mauritz KH (1984) A neurophysiological study of the premotor

cortex in the rhesus monkey. Brain 107 (Pt. 2):385–414.

Wise SP (1985) The primate premotor cortex: past, present, and preparatory. Annu

Rev Neurosci 8:1–19.

Wise SP, Boussaoud D, Johnson PB, Caminiti R (1997) Premotor and parietal cor-

tex: corticocortical connectivity and combinatorial computations. Annu Rev Neu-

rosci 20:25–42.

Wurtz RH, Goldberg ME (1972) Activity of superior colliculus in behaving monkey.

3. Cells discharging before eye movements. J Neurophysiol 35:575–86.

Wurtz RH, Mohler CW (1976) Enhancement of visual responses in monkey striate

cortex and frontal eye fields. J Neurophysiol 39:766–72.

Yu B, Afshar A, Santhanam G, Ryu S, Shenoy K, Sahani M (2006) Extracting

dynamical structure embedded in neural activity. In Weiss Y, Scholkopf B, Platt

J, editors, Neural Information Processing Society (NIPS), Vol. 18, pp. 1545–1552,

Cambridge, MA. MIT Press.

Yu BM, Cunningham JP, Santhanam G, Ryu SI, Shenoy KV, Sahani M (2009)

Gaussian-process factor analysis for low-dimensional single-trial analysis of neu-

ral population activity. J Neurophysiol 102:614–35.

Zaksas D, Pasternak T (2006) Directional signals in the prefrontal cortex and in area

MT during a working memory for visual motion task. J Neurosci 26:11726–42.