Author's personal copy · Introduction While behaving in a constantly changing environment, the visual ... increase of the amplitude of this memory-related ERP component (labeled

Author's personal copy

Distinguishing between lateralized and nonlateralized brain activity associated withvisual short-term memory: fMRI, MEG, and EEG evidence from the same observers

Nicolas Robitaille a,⁎, René Marois b, Jay Todd b, Stephan Grimault a,c, Douglas Cheyne d, Pierre Jolicœur a

a Centre de recherche en neuropsychologie et cognition, Département de psychologie, Université de Montréal, Montréal, Québec, Canadab Vanderbilt Vision Research Center, Department of Psychology, Vanderbilt University, 530 Wilson Hall, Nashville, Tennessee 37203, USAc Centre national de la recherche scientifique, Franced Program in Neurosciences and Mental Health, Hospital for Sick Children, Toronto, Ontario, Canada

a b s t r a c ta r t i c l e i n f o

Article history:Received 5 May 2010Revised 8 July 2010Accepted 12 July 2010Available online 16 July 2010

Previous functional neuroimaging studies have shown that maintenance of centrally presented objects invisual short-term memory (VSTM) leads to bilateral increases of BOLD activations in IPS/IOS cortex, whileprior electrophysiological work suggests that maintaining stimuli encoded from a single hemifield leads to asustained posterior contralateral negativity (SPCN) in electrophysiology and magnetoencephalography.These two findings have never been investigated using the same physiological measures. We recorded theBOLD response using fMRI, magnetoencephalography (MEG), and electrophysiology (EEG), while subjectsencoded visual stimuli from a single hemifield of a balanced display. The EEG showed an SPCN. However, noSPCN-like activation was observed in the BOLD signals. The BOLD response in parietal cortex remainedbilateral, even after unilateral encoding of the stimuli, but MEG showed both bilateral and contralateralactivations, each likely reflecting a sub portion of the neuronal populations participating in the maintenanceof information in VSTM. Contrary to the assumption that BOLD, EEG, and MEG responses – that were eachlinked to the maintenance of information in VSTM – are markers of the same neuronal processes, ourfindings suggest that each technique reveals a somewhat distinct but overlapping neural signature of themechanisms supporting visual short-term memory.

© 2010 Elsevier Inc. All rights reserved.

Introduction

While behaving in a constantly changing environment, the visualsystemmustmaintain, in a readily available form, a portion ofwhatwasseen; a process supported by visual short-term memory (VSTM).Recently, important insights about the neural representation of VSTMwere obtained following the identification of several new physiologicalmarkers of VSTM. Researchers have argued that the maintenance ofinformation in VSTM is likely supported by the intra-parietal and intra-occipital cortex (IPS/IOS), because activity in these cerebral regions isstrongly correlated with the amount of information held in memory(Todd and Marois, 2004). Conversely, lateralized visual stimuli, to beencoded andmaintained for a brief period of time (e.g., 1 or 2 s), lead to

sustained neural activity over the posterior regions of the cerebralcortex, contralateral to the stimuli tobe encoded (Klaver et al., 1999). Anincrease of the amplitude of this memory-related ERP component(labeled SPCN, for Sustained Posterior Contralateral Negativity) as thenumber of items remembered increased was found (Brisson et al.,2008), and was subsequently used in several investigations of VSTM(Brisson and Jolicoeur, 2007; Jolicoeur et al., 2008; Robitaille andJolicoeur, 2006; Robitaille et al., 2007).

These two physiological markers of VSTM (the BOLD response, andthe SPCN) have several features in common. The topographicaldistribution of the SPCN (Brisson and Jolicoeur, 2007, 2008; Jolicoeur etal., 2008; McCollough et al., 2007; Perron et al., 2009; Robitaille andJolicoeur, 2006; Robitaille et al., 2007) is very similar to that of the N2pc,for which parietal sources were identified(Hopf et al., 2000). Theamplitudes of the electrophysiological and hemodynamic markersincrease monotonically with the number of items presented, but reacha maximum at the subject's maximal VSTM capacity (e.g., calculatedusing Cowan's k formula (Cowan, 2001; Pashler, 1988)), creating aplateau for higher number of items.Moreover, bothmarkerswere linkedto individual differences inVSTMcapacity (Todd andMarois, 2005; Vogeland Machizawa, 2004). The most prominent difference between theSPCN and the BOLD activation in IPS/IOS is the encoding fieldmanipulation used to isolate the SPCN. Indeed, the SPCN, as other ERP

NeuroImage 53 (2010) 1334–1345

Abbreviations: ER-SAM, event-related SAM; fMRI, functionnal magnetic resonnanceimaging; IPS/IOS, intra-parietal/intra-occipital cortex; MEG, magnetoencephalogra-phy; MEM, maximum entropy of the mean; MNE, minimum-norm estimates; SAM,synthetic aperture magnetometry; SPCM, sustained posterior contralateral magneticfield; SPCN, sustained posterior contralateral negativity; VSTM, visual short-termmemory.⁎ Corresponding author. BRAMS, Suite 0-120, Pavillon 1420 boul. Mont-Royal,

Université de Montreal, C.P. 6128, Station Centre Ville, Montreal (QC), Canada H3C 3J7.Fax: +1 514 343 2175.

E-mail address: [email protected] (N. Robitaille).

1053-8119/$ – see front matter © 2010 Elsevier Inc. All rights reserved.doi:10.1016/j.neuroimage.2010.07.027

Contents lists available at ScienceDirect

NeuroImage

j ourna l homepage: www.e lsev ie r.com/ locate /yn img


components like the N2pc and the LRP, is based on a “contralateral–ipsilateral” difference to isolate the lateralized portion of the brainresponse, where the ipsilateral side of the brain is used as a control“condition,”oras a control activation(Gratton, 1998) for the contralateralactivation. This manipulation is intended to remove the effect of anyactivity that is not lateralized according to the stimulus presentation side(or response-button side, for LRP). Studies ofVSTMusing fMRI so far haveused bilateral stimulus presentations and found bilateral activation inIPS/IOS.

The goal of the present study was to observe, directly, therelationship between the BOLD activation in IPS/IOS, the electrophys-iological (SPCN) component, and the magnetoencephalographical(SPCM) marker of the maintenance of information in VSTM. We testedthe same subjects both with fMRI and MEG – with EEG recordedsimultaneously with MEG – in very similar experiments designed toallowcomparisons across brain imagingmodalities. To allow theuse of aregression analysis on the number of items accurately held in memory(Todd and Marois, 2005), we presented 1, 2, 4, or 6 visual objects. Weused bilateral stimulus presentations, with an arrow indicating whichstimuli (on the left or right side offixation) had to be encoded (Grimaultet al., 2009; Robitaille et al., 2009; Vogel and Machizawa, 2004). Thisallowed us to compute both load-related activity (by collapsing trialswith left-encoding and right-encoding) and an SPNC-like activation(using the “contralateralminus ipsilateral”measure)with the data fromall imaging modalities. We recently coined the term SPCM, a magneticequivalent of the SPCN (labeled SPCM for Sustained PosteriorContralateral Magnetic field) (Robitaille et al., 2009). Sensors showingthe SPCMwere located on two separate clusters of sensors, over parietalcortex. A critical finding of that study was that for different sensorclusters (different from the SPCM), we found an increase in magneticfield amplitude with the increase in the number of items held inmemory that was independent from the encoding hemifield (i.e., nointeraction between hemifield and the increase in activation as afunction of memory load). This led us to conclude that a more complexnetwork of neural generators was active during the retention periodthanwhat was isolated as the SPCM. However, this previous study onlyused two loads, preventing the use of a parametric analysis based onestimatedmemory capacity across loads (e.g., regression using Cowan'sk (Todd and Marois, 2004)). Furthermore, anatomical MRIs wereavailable for only 5 participants, which limited the possibility of sourcelocalization. These limitations were overcome here because ananatomical MRI was acquired for every subject and we used a broaderrange of memory loads.

The specific hypothesis we will test is that both physiologicalmarkers (BOLD activation in IPS/IOS and the SPCN/M) reflect the sameunderlying neural processes. In others words, the generators of theSPCN/M would be the left and right IPS/IOS; each of them wouldincrease in activation level more for stimuli encoded from thecontralateral side of space, relative to activation for stimuli encodedfrom the ipsilateral side. When stimuli are encoded from both sides ofthe screen simultaneously, the result would be a bilateral activation, asfound in fMRI and suggested by the results of Klaver et al. (1999). Weconsider that this is commonly assumed, as both papers (Todd andMarois, 2004; Vogel andMachizawa, 2004) are often cited as though theSPCN/M and BOLD responses are different manifestations of the sameunderlying brain functions.

Methods

Subject

13 subjects were recorded in this experiment. One subject wasexcluded for a failure to maintain fixation during the task. The twelveremaining subjects (7 females) were between 19 and 31 years old(average 23.3), reported having no neurological problem and wereable to easily discriminate the colors used in the memory task. For the

first six subjects we counterbalanced the order of MEG and fMRIsequences. However, the three subjects who performed the fMRI firstshowed strong artifact in their MEG signal. To avoid furthercontamination of the MEG signal (of magnitude around 3e-14 Tesla)following the ~75 min exposure to the 3 Tesla magnetic field of theMRI, the remaining subjects did the MEG experiment first. An ICAartifact removal procedure (see below) successfully cleaned the MEGsignals of the three subjects tested first with fMRI, so their resultscould be included in the analyses.

MEG and EEG

MEG and EEG proceduresStimuli were presented on a back-projected translucent screen,

located 75 cm in front of the subject. The area containing all thepossible stimuli subtended 14° (width) by 7° (height) of visual anglecentered within the display. Each trial started with the presentation,for 200 ms, of two arrowheads directly above and below the fixationpoint (see Fig. 1), with the arrowheads pointing to the left or the rightof the screen. The fixation cross was then presented alone for 600 to700 ms (varied randomly across trials). The random values wereadded so activity related to the arrows would not systematicallyoverlap activity related to the memory array. On each side of thescreen, 1, 2, 4, or 6 colored diskswere presented for 200 ms (always anequal number on each side), at randomly selected positions within a3×4 imaginary grid. Colors were selected among 8 highly discrimi-nable colors (black, dark blue, green, light blue, pink, red, white, andyellow). A color was never repeated on one side of the screen, butselectionwas independent across sides. The retentionperiodwas 1000to 1100 ms (randomly selected from a rectangular distribution),followed by the test display. The test display consisted of a colored disk(one on each side of the screen), located at the position of one diskpresented for encoding. This display was presented for 1500 ms. On50% of the trials, the test disk had the same color as the one previouslypresented at this location; otherwise it was of one of the 7 remainingcolors. Subjects had 1500 ms to answer by pressing one of two keys onan optically-coupled response pad (right index for “same,” rightmiddle finger for “different”). A colored disk was always presentedsimultaneously on the other side of the screen, with color and positionvaried in the same way as for the test disk, but independently.Feedback was provided after each trial by changing the fixation crossto a+ or— sign, for a correct or an incorrect answer, respectively. Thefeedback was presented for 600 to 900 ms, chosen on the basis of theprevious random interval to create an average interval of 4400 ms(range: 4350 to 4450, selected from a rectangular distribution)between the onset of each trial. Trials were presented in 20 blocks of40 trials. Subjects initiated the block manually, allowing for a restperiod as needed. Trial order was counterbalanced.

The amount of information maintained in VSTMwas assessed usingCowan's k formula (Cowan, 2001) based on the behavioral results:(proportion of hits−proportion of false alarms)⁎the number of itemspresented. This formula is useful because it corrects for possible biases inthe propensity to respond ‘same’ or ‘different’ (see also Pashler, 1988).

MEG and EEG recordingsA CTF-VSM whole-head 275-sensor MEG system in a magnetically

shielded room was used for the recordings. Data were filtered with a150 Hz low-pass filter and digitalized at 600 Hz during the recording.Bad MEG channels (3 or 4, depending on the subject) were excludedfrom the recording. EEG (PO7, PO8, right mastoid) was also recordedwith reference to the left mastoid, and later algebraically re-referencedto the average of themastoids. Bipolar EOG (electrodes placed at the leftand right canthi for horizontal EOG and above and below the left eye forvertical EOG) was recorded in order to monitor eye blinks and eyemovements. Bipolar ECG was also recorded. Trials with a correct or anincorrect response were included in the brain signal analyses.

1335N. Robitaille et al. / NeuroImage 53 (2010) 1334–1345


The ERP and MEG analyses were done using CTF software, EEGLAB(Delorme and Makeig, 2004), Fieldtrip, Brainvisa, Brainstorm, AFNI,and custom programs.

EEG analysisThe data were screened in order to remove trials containing eye

blinks or eye movements, or artifacts in the electric signals. Trials werebaseline-corrected based on the mean activity during the pre-encodingperiod (−200 to 0 ms, time relative to the onset of the memory array)and averaged. Amplitude of the SPCN was measured as the averagevoltage during the retention period (400 to 1200 ms). The SPCN wascalculated as usual (amplitude for the contralateral sensor (PO7 forencode-right trials and PO8 for encode-left) minus amplitude for theipsilateral sensor), for each load. A multiple regression was thenperformed for these values. The regression matrix contains a singlepredictor of interest (the behavioral k, centered), and dummy coding toremove the overall mean for each subject. The effect of load was alsocalculated by averaging the amplitude for both electrodes and stimuluslocation, for each load, andwas submitted to the same typeof regression.

MEG analysisTrials with eye movements were removed because they could have

been systematically correlated with the task (i.e., left movement forencode-left trials and right movement for encode-right trials). For eachsubject, we then performed an independent components analysis (ICA)of the entire data set. Components isolating activity from eye blinks,cardiac, or respiratory activity, were selected based on their topogra-phies, their time-course, and their frequency signatures. Datawere thenback-transformed in signal-space (without these components), base-line-corrected (−200 to 0 ms) and averaged by conditions, producingevent-related fields (ERFs). Statistical analyses were performed on theretention period on a sensor-by-sensor basis using the multiple linearregression analysis described earlier.

Beamformer analyses (ER-SAM)were performed using the raw data(prior to ICA), again with trials where subjects failed to keep fixationremoved. This was deemed appropriate as beamformers can effectivelyreduce spatially stable noise, and data in which the artifacts wereremoved by ICA cannot be effectively used in the usual beamformercalculations due to a reduction of the rank of the data matrix. For eachsubject, a single weight matrix was calculated on the raw data of theentire experiment. Activation images were then produced for eachconditions, averaged over the entire retention period. Use of a singleweight matrix prevented any differences across conditions beingattributed to differences in the weights with changing noise conditions.For ER-SAM,weusedbroadbandactivity (DC to150 Hz) and theweightsmatrixwas calculated from the onset of thememory array stimuli to theend of the retention period (0 to 1400 ms), after baseline correction(−200 to 0 ms). Images were calculated (spatial sampling resolution of3 mm)at every timepointof the retentionperiodbeforebeingaveraged.

Sources of the evoked magnetic fields were also estimated usingcortically constrainedweightedminimumnorm (MNE; Brainstorm, andMEEG software tools fromLENA-CNRS-UPR640, CognitiveNeuroscience

andCerebral imaging Laboratory) andMaximunof Entropyon theMean(MEM (Amblard et al., 2004; Grova et al., 2006)). The cortical surfacewas extracted from the anatomical MRI scan using BrainVisa software.Approximately 8000 sources, oriented perpendicularly relative to thecortical surface, were distributed over the cortical surface, and theselocal sources were used in distributed source localization analyses.Event-relatedfield for eachcondition, after being cleanedwith ICA,wereused as input for these analyses. MNE surfaces were computed with aTikhonov parameter value of 10. MEM surfaces used activity during thebaseline (−200 to 0) for a Multivariate source pre-localization (MSP).Imageswere computed for every timepoint in the retention period (400to 1200 ms) before being averaged. Resulting imageswere interpolatedback in the anatomical MRI space (voxels size: 4 mm3) of each subject,transformed in Talairach space using AFNI (Cox, 1996) and spatiallysmoothed (FWHM 12mm). For ROI analysis, activation within a 7-mmradius sphere centered on the coordinate of interest was averaged ineach map.

fMRI

fMRI procedureThe procedure was identical as for the MEG recordings except for

two manipulations added to account for the overlap of the BOLDresponse across trials. First, random blank intervals were addedbetween trials (52% without interval, 26% with one TR, 13% with twoTRs, 6% with three, 3% with four), which allowed deconvolutionanalysis. Second, two blank conditions were added (arrow pointing tothe left or right, but without stimuli or test afterward), for which time-course will also be extracted and subtracted from the time-course ofexperimental conditions. Each run consisted of 102 trials to allowcounterbalancing of the order of the 10 trial types (2 sides×4 loads,+2 blanks), with a supplemental trial at the beginning and the end toensure that all analyzed trials are preceded and followed by a trial ofeach possible type. Anatomical 3D high-resolution images wereacquired using conventional parameters. Stimuli were back-projectedon a translucent screen, visible via amirror fixed onto the antenna. T2-weighted EPI images were acquired in AC–PC orientation,TR=2200 ms, TE=30 ms, FOV 24 cm, Flip Angle 70°, 28 axial slicesof 64×64 voxels, 5 mm thick without slice gap, interleaved. Eachsubject performed 4 functional runs, followed by a high-resolution 3Danatomical scan. Data acquisition was performed with a 3 T TrioSiemens scanner at l'Unité de Neuroimagerie Fonctionnelle del'Institut de Gériaterie de l'Université de Montréal. The first 8 subjectswere tested with an 8-channel antenna on the Trio platform, the 4 lastwith a 12-channel antenna, on the Trio TIM platform after scannerupgrades.

fMRI analysisAnalyses were performed using AFNI (Cox, 1996) and in-house

Matlab routines. Preprocessing consisted of slice-timing alignment,motion-correction, 8 mm FWHM spatial smoothing and within-runnormalization. For each subject and condition, an SPM map was

Fig. 1. Sequence of events in each trial. Stars refer to random intervals of 0–100 ms, which were compensated as needed after the response production to have a total duration of4400 ms. A load 4, encode-left trial, requiring the answer “same” is illustrated, which received a positive feedback.

1336 N. Robitaille et al. / NeuroImage 53 (2010) 1334–1345


created using multiple regression analysis, with regressors defined foreach trial type and convolvedwith a canonical hemodynamic function(1 parameter gamma function (Cohen, 1997)). Influence of within-run trends (linear and higher-order) was removed by includingregressors following Legendre polynomial trends (degree 5). Thiscreated 96 maps (12 subjects×8 conditions), which were trans-formed into Talairach space before being submitted to a multipleregression analysis with regressors defined for k (balanced), for theSide factor, and for the interaction of k by Side, with a random-effectmodel. A false discovery rate (FDR) threshold of qb .001 was used tocontrol for multiple comparisons in the Load maps, and of qb .05 forthe Side map. We used a more liberal statistical threshold for the Sidemap because this is the first investigation of the effect of thismanipulation in fMRI. Maxima were defined manually as the voxelswith the highest activations in the corrected statistical maps. Thetime-course of the activation was extracted from a sphere(radius=5 mm) centered on the maxima, using deconvolution. Theactivation time-course for the trials without stimuli to encode wassubtracted from the time-course for experimental trials.

Results

Behavior

The number of items effectively encoded (calculated with Cowan'sk formula (Cowan, 2001)) varied across set size and reached a plateaubetween 4 and 6 items, as shown in Fig. 2A, F(3,33)=9.66, pb .0001.All others factors (presentation side, first or second day of recording,fMRI or MEG recording) did not have a significant effect on k, all Fsb1.The group k values for each memory set size, averaged across bothfMRI andMEG recording sessions (weighted by the number of trials ineach), was used as the estimate of VSTM capacity at each load. Thesevalues were then used to create the load regressor, which will be usedin the subsequent analyses of MEG, ERP, and fMRI data.

ERP — event-related potentials

The SPCN waveform amplitude increased during the retentionperiod as load increased from 1 to 4, and then reached a plateau(Fig. 2C). The SPCN is calculated by subtracting the activation at theipsilateral electrode (i.e., PO8 when the arrows pointed to the rightand PO7 when the arrows pointed to the left) from the activation atthe contralateral electrode (i.e., the contralateral minus ipsilateraldifference). Note that the same type of calculation will be performedwith all the physiological signals used in the study, namely we willcompute an SPCM for the magnetic evoked field and an SPCN-likeactivation for the BOLD signal to quantify the degree to which thecontralateral brain response is greater than the ipsilateral response.The amplitude of the SPCN during the retention period (400 to1200 ms after the onset of the display), is plotted with SEM in Fig. 2B(green dashed curve), was significantly predicted by k, F(1,35)=7.94,pb .008. In order to compare to the fMRI analysis below, we alsoexamined the overall effect of load in the average of the ipsilateral andcontralateral ERP waveforms (rather than in the subtraction of thesewaves) (Fig. 2D). Robust initial visual components following the onsetof the stimuli were present— this is because these waveforms are notsubtraction waveforms. However, the amplitude of the ERP was notconsistent through the retention period, and not significantlypredicted by k, F(1,35)=.034, pN .85; see blue curve in Fig. 2D. Theunsubtracted ‘load’ results probably reflect the contribution ofmultiple generators (e.g., sustained posterior negativity, on the onehand, and P3, on the other), and highlight the usefulness andimportance of using the SPCN (contralateral minus ipsilateraldifference waves) in the analysis of the EEG data, in the study ofVSTM (Brisson and Jolicoeur, 2007; Mazza et al., 2007; McCollough et

al., 2007; Robitaille and Jolicoeur, 2006; Vogel and Machizawa, 2004;Vogel et al., 2005).

ERF — event-related fields

Load-sensitive channels (Fig. 3A) were identified with multipleregression of the MEG evoked fields. The two clusters of channels, eachchannel indicated with a bold dark circle, had a p-value inferior to .001(uncorrected, channels evaluated independently). The left clustercontained 21 contiguous sensors with primarily ingoing (negative)fields during the sustained response and the right cluster of 20contiguous sensors with primarily outgoing (positive) fields duringthe sustained response (the polarity of signals depends on the positionand orientation of the neural generators relative to the sensors). TheERFs for the left cluster (Figs. 3B and C) showed evoked responsesfollowed by sustained activity after the onset of the encoding-display.During the retention period, there was a clear differentiation of thewaveforms according to the memory load conditions; the higher loadshowed higher amplitude from load 1 up to load 4, and no furtherincrease from load 4 to load 6. The ERFs for the right cluster (Figs. 3E andF) showed again a clear differentiation as a function of memory load;higher load led to higher amplitude up to load 4 and a plateau acrossloads 4 to 6. For comparison between left and right sensors we invertedthe polarity of left cluster channels. Combined activity of thesewaveforms during the retention period was significantly predicted byk, F(1,35)=27.34, pb .0001, (Fig. 3D). However, the SPCMs calculatedfrom these waveforms were not significantly predicted by k, F(1,35)=2.02, pN .16.

Side-sensitive channels were also identified using multiple linearregressions (Fig. 3G). The posterior midline cluster contained 28sensors. To avoid cancellation of the ERFs, we computed the averageactivity separately for the left and the right sensors within this cluster

Fig. 2. (A) The number of items maintained in VSTM (k) increased from load 1 up toload 4 and then decreased slightly at load 6. The amplitude of the SPCN waveform (C)during the retention period, 400–1200 ms, followed k (green dashed line in B), but notthe waveforms averaged for each load not taking side into account, shown in D (bluedashed line in B). SEM illustrated with vertical bars in A and B.



(midline sensors were ignored). The 14 sensors in the left portion ofthe cluster had higher amplitude for the contralateral trials (Fig. 3H)than for the ipsilateral trials (Fig. 3I). Conversely, the 11 sensors in theright portion of the cluster showed higher activity for the contralateraltrials (Fig. 3L) than for the ipsilateral trials (Fig. 3K). Combinedactivity (again here we transformed the left sensors to be negative-going) of these waveforms during the retention period (Fig. 3J, solidblue line) indicated a modulation by the load, for the overall loadeffect F(1,35)=15.56, pb .0001. In order to determine if we couldreplicate here the SPCN effect, we also calculated the SPCM from thesesensors (Fig. 3J, dashed green line). The amplitude of the SPCM acrossthe load was significantly modulated by k, F(1,35)=12.31, pb .002.This indicates that there are cerebrals regions (to be localized moreprecisely with the source localization analyses reported below) thatemit a higher magnetic field when there is an increase in the numberof items encoded from their contralateral hemifield.

Activity in the two anterior-most clusters was not significantlymodulated by Load (both Fsb3.9, psN .05) or by the Load×Sideinteraction (both Fsb1.4, psN .05). Given that activity in these clusterswas not affected by the memory load manipulation, we did notanalyze them further.

These analyses of the ERFs indicated that the effects we wereinterested inwere visible at the sensor level, and this justified the sourcelocalization analyses we present in subsequent sections of the article.

fMRI — functional magnetic resonance imaging

Results in Fig. 4 show load-related activity found using aregression on k on the fMRI BOLD. As expected from previousresearch we isolated a pair of symmetric posterior clusters containingthe IPS/IOS coordinates (Todd andMarois, 2004). Within each cluster,three maxima were visible. For each bilateral pair of maxima (we

Fig. 3. The two posterior clusters of MEG sensors significantly modulated by k (A) showed robust activation throughout the retention period (B–C and E–F). The average waveformsfor these sensors (with polarity adjusted) show an activation pattern during the retention period that was predicted by k (blue solid line in D), but they did not have a systematicmodulation by side, which led to a flat amplitude across load for the SPCM (green dash line in D). The posterior cluster of MEG sensors significantly modulated by the side (G) wasdivided in a left (H–I) and a right (K–L) cluster. The averages waveform for these sensors (with polarity adjusted) show an activation during the retention period that was predictedby k (blue solid line in J), as did the SPCM calculated from these sensors (dashed green line in J).



defined maxima as a sphere of 5 mm surrounding the position of thevoxel having the highest local activation), we extracted the time-course using deconvolution (see Fig. 5 for an example time-course).To examine more closely the effect of load, we averaged together theactivity of the left and right maxima, for the encode-left and encode-right conditions (blue curves on the middle column of Fig. 4), at time6.6 and 8.8 s. This activation was submitted to a multiple regression

analysis, with k as the regressor of interest. This second analysis wasintended as a direct test of the effect of mnemonic load for a specificpair of maxima located in homologous areas of the brain. It is indeedredundant in the present case (because these voxels were shown tofollow k in the previous regression analysis), but it will allow a directcomparison of the effect of mnemonic load for different brain areasand most importantly for the SPCN-like BOLD signal described later in

Fig. 4. Bilateral maxima found in the k-regressionmap in fMRI (first column) were the superior IPS (A), the IOS (B), and VO (C). The sidemap revealed themiddle occipital cortex (D)and the inferior occipital (IO) cortex (E). The average amplitude of every maxima pair was predicted by k, either for the fMRI BOLD (blue curve) or the ER-SAM source analysis of theMEG signal (green curve). However, the SPCN-like activations calculated from these maxima were only significantly predicted by k for the ER-SAM in superior IPS (row A) or thefMRI BOLD in IO cortex (row E).



this paragraph. The number of items was also included as a covariateto remove anymodulation of the response by the actual visual display.The superior IPS maxima (Fig. 4, row A) showed a clear modulationcorrelated with k, F(1,34)=5.11, pb .03. The IOS maxima (Fig. 4, rowB) were also significantly predicted by k, F(1,34)=12.01, pb .002; aswere the ventral–occipital (VO) maxima (Fig. 4, row C), F(1,34)=6.56, pb .02. These results were expected because these maxima wereidentified on the load effect map. In order to compare these resultswith the EEG and MEG results, an SPCN-like BOLD activation was alsocalculated (blue curves of the third column of Fig. 4) by subtractingthe activation of the ipsilateral maximum (left maximum for leftstimuli and right maximum for the right stimuli) from thecontralateral one. Although we found increases in BOLD signalstrength that were larger on the contralateral side, none of themhad a clear plateau between load 4 and load 6, and hence none wasmodulated by k, all F(1,34)b1, all pN .6 (the number of items was alsoincluded as a covariate to remove any modulation of the response bythe actual visual display). Overall, we found clear load effects thatfollowed k when we averaged over left and right hemisphere maxima(replicating previous results (Todd and Marois, 2004)). However, wedid not find an SPCN-like BOLD response that followed behavioralload effects, as estimated by k.

Superior IPS activation is shown in details for Fig. 5A. For bothhemifield and maxima, the increase from load 1 to load 4, followed bya plateau for load 4 and 6 is visible. Furthermore, Fig. 5B recombinesthese data to create an ipsilateral (left maxima for stimuli on the leftand right maxima for stimuli on the right) and a contralateral (rightmaxima for stimuli on the left and left maxima for stimuli on theright) time-course of activation. Either when the stimuli to beencoded are in the ipsilateral hemifield of the cerebral hemispherewe are recording, or in the contralateral hemifield, we observe the

same pattern of activation. Thus, encoding visual stimuli from a singlehemifield lead to a bilateral activation in superior IPS.

We also determined which brain areas showed a BOLD responsesignificantly different across the encode-left and encode-right condi-tions, independently of the number of items encoded. This “side map”(Figs. 4D and E) showed clusters of significant activation (correctedfor multiple comparisons using False Discovery Rate: qb .05). Theinversion of polarity across hemisphere indicates a systematicmodulation according to the position of the stimulus to be encoded:activation was higher for stimuli encoded from the contralateralhemifield. We isolated three pairs of maxima in these maps. The firstpair of maxima was located in the middle occipital gyrus, inBroadmann area 18; slightly more central and posterior than theIOS maxima (Fig. 4D; see Table 1 for the Talairach coordinates). Theiractivity was significantly predicted by k, F(1,34)=5.39, pb .03. Theactivation in the second pair of maxima, located in the inferioroccipital (IO) gyrus, was also significantly predicted by k, F(1,34)=8.84, pb .006. Note that the arrows on Fig. 4E indicate these maxima.The medial activation in the left hemisphere did not have ahomologous area to be compared with, so we did not include thisregion in the analysis. The third pair of maxima corresponded to theVO maxima described in the load maps. The SPNC-like BOLDactivation, unlike what was found in any of the previously describedmaxima pairs, was significantly modulated by k for the IO maxima, F(1,34)=5.36, pb .03. The time-course of the BOLD response, estimat-ed by deconvolution for this pair of maxima, is shown in Fig. 6. For theleft IO (top row of Fig. 6), the Load response was modulated by theposition of the stimuli, leading to a significant Load×Side interaction,F(1,80)=11.92, pb .001. The right IO BOLD Load response, on thecontrary, was not influenced by the position of the stimuli, F(1,80)=.19, pN .65. The SPNC-like activation for the two others pairs ofmaxima (MOG and VO) was not modulated by Load, F(1,34)b1.

The interaction (Load×Side) SPM map did not exhibit anysignificant activation, either using FDR or cluster threshold (Formanet al., 1995) when correcting for multiple comparisons. Thus, amongthe significant activations found in the fMRI data, the only candidatecerebral region for creating a load-related asymmetry similar to whatis commonly found in EEG and MEG was the left IO.

An increase of activation for contralateral trials relative toipsilateral trials, proportional to the number of items presented(rather than on the number of items encoded, as measured with k),was visible for several of the maxima pairs in parietal and occipitalcortex. Multiple regressions indicated a significant correlation withthe number of items for all the maxima pairs, all FsN5.5, all psb .025,except for the superior IPS, F(1,35)=1.64, pN .20. Consequently, thereare three occipital maxima pairs, IOS, middle occipital, and VO, thatdid not show a correlation with k, but that showed one with thenumber of items presented for encoding. Given that the same numberof items were presented in both attended and ignored hemifield; thiseffect is likely attributable to attention or memory-related factors.However, the absence of a plateau when the capacity of VSTM wasexceeded indicates a processing step that is distinct from a pureworking memory load effect and most likely corresponds to anattentional involvement in VSTM.

Fig. 5. Time-course of the BOLD activation in superior IPS. Left and right superior IPSshowed the increase of activation for loads 1 to 4, and a plateau for loads 4 and 6. Thesimilarity of the data in the ipsilateral and contralateral time-course (row B) indicates abilateral activation.

Table 1Talairach coordinate of regions investigated.

RH LH

Superior IPS 16, −67, 49 −14, −69, 46Inferior IPS (Xu and Chun, 2006) 26, −65, 34 −25, −70, 29IOS 35, −85, 14 −31, −81, 15Middle occipital 24, −86, 16 −21, −86, 18IO (inferior occipital) 39, −72, −7 −40, −72, −7VO 34, −69, −16 −26, −75, −15



ER-SAM–SPM on the source localization of the MEG signal

For each condition and subject, activation volumes were createdusing three source localization analyses (ER-SAM (Cheyne et al.,2006), MEM (Grova et al., 2006), and MNE (Hamalainen andIlmoniemi, 1994)). The ER-SAM (event-related synthetic aperturemagnetometry) analysis is a beamformer-based localization of thesource of evoked field. Beamformer analyses use the covariance acrossthe sensors in the raw data (i.e., trial-by-trial, before creating ERF) tomaximize the activation at the estimated sources and minimizeactivation from others sources. The ER-SAM beamformer maximizesthe signal time-locked to an event— here the apparition of the stimulito encodes. The two other methods (MNE and MEM) take advantagesof the anatomical information from each subject: an ensemble of pre-determined sources oriented perpendicular to the cortical surface(actually, the interface between the white and grey matter of thecortex)was used as the possible generator of the signal. The activationpattern of the sources leading to the recorded ERF is then estimated.This solution is constrained by minimizing the norm (MNE) ormaximizing the entropy (MEM) of the solution. In the interest ofspace constraints, we show only the ER-SAM analysis here, but itshould be noted that we found good agreement with MEM and MNEanalyses. These methods are based on very different constraints andmethods for source localization, and thus their convergence to thesame sources is a good indication of their reliability (see Fig. 7, bottomrow). The resulting 96 maps (12 subjects×8 conditions) were thentransformed in a normalized space (Talairach), and submitted to thesame multiple linear regressions approach used for the fMRI data. Tovisualize the results, we overlaid them on a standard white-greyborder surface, as shown in Fig. 7.

Significant load effects (False Discovery Rate: qb .05, top row ofFig. 7) consisted of three main cerebral activations. An increase ofactivation with load for the left and the right IPS/IOS cortex wasvisible, and was confirmed with MNE and MEM. The right frontaldecrease of activation associated with the increase in mnemonic loadincluded the right infero-frontal gyrus and the right claustrum. Thisdecrease in amplitude, however, was not visible in MNE or MEM. TheER-SAM map showing the effect of side showed less extended

activations. Similar to the fMRI results, there is an inversion of theactivation values for the left and right posterior cortex, although thisinversion occurred at different horizontal planes for the left and righthemispheres and was generally superior to the fMRI foci (see areasindicated by arrows in Fig. 7, bottom row). Although the MEM andMNE maps did not reach significance, lowering the threshold in thesemaps revealed the same regions. The interaction maps (Load by Side)did not reach significance in the sources localization analyses.

In order to compare the MEG activation with the results found forfMRI, we used the previously identified BOLD maxima as ROIs. Weaveraged the activation in the ER-SAMmaps for a 7 mm radius spherearound the voxels of maximal BOLD activations (see Fig. 4, greencurves). Statistical analyses were performed with multiple regression,and significance was assessed using permutation (Anderson andLegendre, 1999). Activation in all pairs of maxima was significantlypredicted by k, all psb .009. The same analysis conducted on the MEMand the MNE maps showed similar results. Thus, for every fMRImaximum found, theMEG activation showed an increase of activationas the number of stimuli increased, up to a plateau between 4 and 6.The SPCMwas also calculated for these pairs of maxima. The SPCM forthe superior parietal cortex was significantly predicted by k, pb .022.The IO gyrus (indicated by arrows in Fig. 4E) showed a marginalSPCM, pb .07; while all other SPCM calculations were not predicted byk, psN .11. The SPCM on MNE and MEM maps did not reachedsignificance, likely resulting from a lack of sensitivity of thesemethods. The parietal activations, thus, showed a higher responsewhen the stimuli maintained in VSTM were encoded from theircontralateral hemifield. This results contrast with the absence ofSPCN-like activation in the BOLD signal for the same cerebral regions.

Literature-based inferior IPS ROI

Inferior IPS has been linked to processing of information in VSTM(Xu, 2007; Xu and Chun, 2006), but did not reveal itself in ourmaxima-based fMRI analysis. As we considered it important todescribe the behavior of this region in our task, we defined ROIsbase on Talairach coordinates from previous reports (see Table 1).They are locatedmore lateral and inferior than our superior IPS, whichis consistent with the anatomy of the IPS. The BOLD activation for thisROI is shown in Fig. 8. An increased of the BOLD activation as thenumber of items maintained in VSTM increased, thus showing amodulation by k, F(1,34)=7.87, pb .009, but the SPCN-like BOLDactivation was not modulated by k, Fb .58, pN .44. This pattern ofresults is identical to what was found for the superior IPS. The numberof items, however, showed a significant correlationwith the SPCN-likeactivation, F(1,35)=5.23, pb .03. Thus, the inferior IPS was the onlycerebral region not in occipital cortex that showed a linear increase ofactivation for contralateral presentation of stimuli to be encoded. ER-SAM activation for these regions followed the same pattern: increaseof activationwith k, pb .00001, but the SPCN-like activation of ER-SAMactivation was not predicted by k, pN .21. However, the SPCM patternfor the MNE activation in inferior IPS was significantly predicted by k,pb .02. Superior and inferior IPS are thus showing high concordance ofactivation, having a strong bilateral BOLD responses and magneticevoked field increasing with the number of items held in VSTM.However, they also showed differences of activation for the SPCN-likeBOLD response and SPCM activations, mainly a linear increase incontralateral BOLD responses that was present for inferior IPS only.

Discussion

We measured the memory-load-related activation patterns andthe SPCN-like activation patterns for various physiological markerslinked to VSTM derived from EEG, MEG, and fMRI brain imaging. Thehemifield manipulation had limited impact on BOLD signal, whereasthis manipulation was particularly useful to isolate the effect of

Fig. 6. Time-course of the BOLD activation in IO. Left IO showed a modulation ofactivation across load for contralateral trials (upper-left panel) but not for ipsilateraltrials (upper-right panel). Right IO, however, showed a modulation for both ipsilateraland contralateral trials (lower panels).



memory load in the ERPs. The evoked magnetic fields showed both abilateral load-related increase and an SPCM effect.

Role of parieto-occipital areas in supporting VSTM

The superior IPS is likely one of the more important cerebralsources of the SPCM identified in the MEG sensor data, given that it

was the only cluster showing a significant increase of activation (asestimated with ER-SAM) with load, for contralateral stimuli. SuperiorIPS also showed an increase in BOLD signal for increasing memoryload, consistent with previous study (Kawasaki et al., 2008; Song andJiang, 2006; Todd and Marois, 2004, 2005; Xu, 2007; Xu and Chun,2006). However, unlike what was expected, no SPNC-like activationpattern was found for the BOLD signal in IPS. That is, the increase inBOLD signal in the IPS was about the same in left and right IPSregardless of the side of visual space from which stimuli wereencoded. This pattern of response is quite unlike what is found in theEEG and, to a lesser extent, theMEG results. It could be argued that wedid not have the statistical power necessary to isolate such lateralizedactivity (although we did find significant lateralized activity, but thisactivity was linearly increasing with the number of items rather thanlimited by a plateau corresponding with VSTM capacity). However,the magnitude of the load-related manipulation was ten times higherthan the magnitude of the (non-significant) SPCN-like BOLD activa-tion, so even if the SPCN-like responses turned out to be statisticallysignificant, the most predominant effect would still be a bilateralincrease in activation. It is possible that the presence of ipsilateralstimuli reduced the lateralization of activity within IPS/IOS. Indeed,one may consider that ignored stimuli would also elicit a response,albeit smaller than for encoded stimuli. Using the same stimuli as fortheir VSTM experiment, two papers (Todd and Marois, 2004; Mitchelland Cusak, 2007) investigated the BOLD activation in IPS using aniconic memory task. Unfortunately, their results are not consistent:IPS showed amodulation of BOLD activation following k (as calculatedfor the VSTM task) in an iconic memory task for Cusack and Mitchell,

Fig. 7. Regression on k (top row) and on the stimulus presentation side (middle row) for the event-related beamformer source reconstruction, displayed on a template brain surface.The load activation was concentrated in the IPS/IOS area. Side-related activations (red for higher activation for encode-left trials and blue for higher activation for encore-right trials)were less extensive but visible for both left and right IPS (indicated with arrows) and in the left occipital cortex. Bottom row showed the maps for the MEM and MNE analysis. Theyconverge with the ER-SAM analysis, but uncorrected threshold had to be used in the last two maps.

Fig. 8. Activations for the inferior IPS, as identified base on Xu and Chun's (2006)coordinates. A significant effect of load was found for the fMRI BOLD activation and theER-SAM. The SPCN-like activation was not significant.



but not for Todd and Marois. Consequently, we cannot rule-out thepossibility that the BOLD response in the ipsilateral IPS reflects thenon-mnemonic activation found by Cusak and Mitchell instead of abilateral encoding of the stimuli. If the absence of an SPCN-likeresponse is caused by sensory activation of the ignored stimuli, thiswould, however, indicate that the BOLD response is sensitive to thissensory activation while the evoked magnetic field is not.

It is also possible that the mnemonic representation in IPS wasinitially unilateral, or more strongly lateralized, but progressivelybecame more bilateral as the trial progressed. While EEG and MEGwould have the temporal resolution to detect the initial difference andthe reduction in the degree of contralateral dominance of thememoryactivity, it is possible that fMRI could not resolve this transient effect,with our scanning parameters. It is possible that the initial greatercontralateral activity reflects, in part, a need to attenuate or suppressactivity related to distractors presented in the visual field opposite tothe one containing the target stimuli. This suppression would nothave to be constant through the trial because the visual field is emptyduring the retention period. In support for this idea, the contralateral–ipsilateral difference waves tend to decrease in amplitude near theend of the trial, as visible, for example, in the results of Vogel andMachizawa (2004, Fig. 2) and here (Fig. 2C). Interestingly, there wasno such reduction in a situation where distractors were presentthroughout the trials (Drew and Vogel, 2008). Although these changesare clearly visible in many EEG and MEG experiments, it is possiblethat the magnitude of these changes is sufficiently small and transientto make them more difficult to detect using standard fMRI methods.The present results do not contain strong evidence of a rapidreduction in the degree of contralateral dominance of the visualmemory representation (e.g., Figs. 3B, C, E, and F), but the retentioninterval in the present study was relatively short. Additional researchwill required to test the present hypothesis.

The lack of agreement of the results for the superior IPS acrossMEG evoked fields and the BOLD signal is surprising given the strongagreement previously found between these two measures (Arthursand Boniface, 2003; Arthurs et al., 2000; Dale et al., 2000; Logothetis etal., 2001). However, these studies used mainly the response ofprimary sensory areas, with transient and short-duration evokedfields (usually less than 100 ms) response to sensory stimuli, whilehere the evoked magnetic fields are of longer duration (1000 ms) andare related to higher cognitive functions. However, it was also shownthat different modulations of the magnetic fields (DC shift, evoked-potential, and oscillatory activity for the alpha and gamma-band)show spatio-temporal covariance with the BOLD responses (Brookeset al., 2005). In the current paradigm, we previously showed acontralateral decrease and a bilateral increase of alpha-band oscilla-tory activity originating from parietal cortex in a very similar task(Grimault et al., 2009), and it is known that the displacement ofspatial attention, as occurring here following the presentation of thearrowheads, does modulate parietal alpha-band oscillatory activity(Medendorp et al., 2007; Thut et al., 2006; Wyart and Tallon-Baudry,2008). It is therefore possible that the SPCM observed here reflects theactivity of a subpopulation of the neurons in superior IPS, but that theBOLD responses recorded from this region integrates the activity ofmore neurons, who do not all exhibit the same modulation ofactivation.

Inferior IPS showed a clear pattern here, the BOLD and ER-SAMoverall load activations followed k, but neither had SPCN-likeactivation patterns. In a retention interval of 8300 ms, inferior IPSdid not show a BOLD activation following the k-pattern, but rather alinear increase with the number of items (Experiment 3 of Xu andChun, 2006). It was also found that sequentially presented stimuli atthe center of the screen led to an equivalent BOLD response for loads 1to 4, while sequentially presented stimuli at different eccentriclocations led to an increase in activation with higher number ofstimuli and therefore of spatial location, (Experiment 4 of Xu and

Chun, 2006). Although these results suggested a dissociation betweensuperior and inferior IPS based on the spatial content of the encodedinformation, the simple spatial manipulation used here (restrictingencoding to one hemisphere) did not create differential memory-related activations between superior and inferior IPS. In addition, theIOS activations followed closely inferior IPS: increasedmnemonic loadled to an increase of the BOLD response, and of ER-SAM activations.Furthermore, a bilateral response to unilaterally-encoded stimuli wasfound for all of IPS/IOS cortex. Thus, our experimental manipulationsdid not reveal differences in BOLD activation patterns across theexamined subportions of the IPS/IOS.

The ventral–occipital cortex activation isolated here was describedearlier (Todd and Marois, 2004), but a specific role of this region forVSTM was discarded because this area, unlike the IPS/IOS cortex,showed an equivalent increase of activation with the behavioral k foran iconic memory task as for the VSTM task, and did not show asustained activity with a longer 9200 ms retention interval. Further-more, a linear increase of BOLD activation in VO following the increaseof items presented to the subject (up to 8 items, well above VSTMcapacity) was also found for an iconic memory task (Mitchell andCusak, 2008). Although we did not perform such manipulationsourselves, their interpretation should apply to our situation, and thuswe do not consider VO as paying a critical role in VSTM here. Themiddle occipital gyrus also showed a load-related activation patternin our data, but only after being identified on the side SPM maps. ROIanalyses are much more powerful than general SPMs – not beingcorrected for multiple comparisons – which could explain why it hasnot been identified previously. No SPCN-like activation was found forthese maxima, however.

A linear increase of contralateral BOLD activation was found foroccipital areas, as for the inferior IPS. Because this effect did not followthe behavioral memory pattern with a plateau following k, this likelydoes not reflect a process specific to VSTM. Our use of a balanceddisplay, however, indicates that this effect is not related to the simpleincrease in the numbers of stimuli, but rather to an effect of attention.

Spatial attention vs. spatial location

Using fMRI, Sereno et al. (2001) identified a retinotopic map ofspatial position encoded in short-term memory for IPS, a result thatwas further expanded to several maps along the IPS sulcus (Konenand Kastner, 2008). Like usual retinotopic maps, this one representscontralateral space, creating a significantly higher activation forcontralateral than ipsilateral positions. Because our visual objectswere created by the conjunction of a spatial location and a color, wewere expecting to trigger a similar contralateral hemisphere bias inparietal activation to the one identified in Sereno and colleagues'work. Furthermore, the retinotopic organization found by Sereno andcolleagues implies that different spatial positions are encoded bydifferent groups of neurons, which should lead to a high summedactivation when multiple locations are encoded. However, encodingseveral objects, each defined by the conjunction of a spatial locationand a color, did not create asymmetries proportional to the number ofitems encoded (i.e., we did not find SPCN-like pattern in the BOLDresponse from IPS), but only a bilateral increase in BOLD activation.Some differences are evident across Sereno et al.'s design and ours. Intheir work, the target for which position was encoded was presentedalone, thus possibly creating an initial stimulus-driven laterality. Thiswould hardly be creating their effect because this retinotopicmapwasnot identified with bright visual stimuli that did not require encodingin spatial short-term memory (Sereno et al., 1995). However, thepresentation of the target alone could also create an exogenous shift ofspatial attention (Posner and Cohen, 1984; Posner, 1980), which wasnot the case in the current study. The linear increase of contralateralBOLD activation that we found for inferior IPS (Fig. 8) suggested aneffect of attention proportional to the number of items presented,



which could create the results found by Sereno and colleagues.Because they used a fixed mnemonic load (1 item), they could notdistinguish between attention-related and memory-related activityusing the plateau effect as we did here.

Another critical difference between their study and ours is thepresence of a visual object to be encoded at each of the spatiallocations to be encoded. Consequently, it is possible that the creationof an object-file by the binding of different features (spatial positionand color were the only feature relevant to the task, but the stimulialso had a disk shape), which we know has strong impact on VSTMperformance (Vogel et al., 2001), would also modify the neuralpathway supporting themaintenance of spatial position, thus creatinga different pattern of activation.

Inferior occipital cortex — unexpected activation

An SPCN-like pattern of activation was found in the BOLD responsefor the inferior occipital (IO) cortex isolated in the side fMRI activationmap (Fig. 4E). The actual pattern of activation, however, was amodulation by load for the contralateral stimuli only in the left IO; theright IO had a load-related activation pattern for both encoding sides.The stimulation was equivalent for the left and right trials so thisdifference in activation can only be attributed to either spatialattention or to visual short-term memory. Spatial attention is knownto modulate cerebral activation in the contralateral occipital cortex inabsence (Kastner et al., 1999) or presence (Kastner et al., 1998) ofvisual stimuli. Shifting attention across visual field also createscontralateral activation in the occipital cortex (Kelley et al., 2007).Previous reports using a control task (Mitchell and Cusak, 2008; Toddand Marois, 2004) or a long retention period (Todd and Marois, 2004;Xu and Chun, 2006) did not investigate the specific involvement of IOin VSTM. Given that we were predicting, based on previous results,either a bilateral increase of activation or a contralateral increase ofactivation within each hemisphere for analogous cerebral regions, wedo not have an empirically-supported interpretation for this activa-tion (grounded in previous work). Further studies will be required todetermine if the functional role of IO in lateralized VSTM task reflectedmainly a spatial shift of visuospatial attention or if the mnemonicaspect of the task was crucial.

Strengths and limitations

A good concordance of source activation was visible across thedifferent evoked field localization methods, with two main excep-tions. First, the SPCN-like activation of the MNE for literature-basedinferior IPS did follow k, which was not the case for the MEM or theER-SAM. However, superior IPS did follow the SPCN-like pattern inER-SAM, so this discrepancy is likely a difference in the precision oflocalization across methods. However, the right infero-frontal gyrusand the right claustrum decrease were only visible in ER-SAM, whileother methods showed a non-significant increase of activation withthe increase of mnemonic load at this location. For this reason, we donot wish to postulate a role for the right infero-frontal gyrus and theright claustrum in VSTM, although we consider worthwhile to reportthis result. Accordingly, despite the numerous advances in sourcelocalization of MEG signals, the use of multiple methods is stilladvised, along with careful interpretation of the results. The magneticfields, however, were the only cerebral activity for which bothbilateral and contralateral activations were found.

Conclusion

We studied several physiological markers of VSTM during mainte-nance of laterally-encoded stimuli. The BOLD activation in parietalcortex showed a bilateral increase in activation, independent of thelocation of the stimuli. This is in accordance with the magnetic fields,

which also showed a bilateral increase in activation. These resultsconvergewith our previouswork showing a bilateral increase of the ERFwhen the number of items was increased from 2 to 4 (Robitaille et al.,2009). This work, however, could not isolate VSTM process from otherconcurrent activations like attention because any difficulty-linkedprocess would show a modulation of activation in these cases. Evokedfield in MEG also showed a contralateral increase in activation for IPS,which likely represents the magnetic counterpart of the SPCN found inEEG.

References

Amblard, C., Lapalme, E., Lina, J.M., 2004. Biomagnetic source detection by maximumentropy and graphical models. IEEE Trans Biomed Eng 51, 427–442.

Anderson, M.J., Legendre, P., 1999. An empirical comparison of permutation methodsfor tests of partial regression coefficients in a linear model. Journal of Statistic andcomputer simulation 62, 271–303.

Arthurs, O.J., Boniface, S.J., 2003. What aspect of the fMRI BOLD signal best reflects theunderlying electrophysiology in human somatosensory cortex? Clin Neurophysiol114, 1203–1209.

Arthurs, O.J., Williams, E.J., Carpenter, T.A., Pickard, J.D., Boniface, S.J., 2000. Linearcoupling between functional magnetic resonance imaging and evoked potentialamplitude in human somatosensory cortex. Neuroscience 101, 803–806.

Brisson, B., Jolicoeur, P., 2007. A psychological refractory period in access to visualshort-term memory and the deployment of visual–spatial attention: multitaskingprocessing deficits revealed by event-related potentials. Psychophysiology 44,323–333.

Brisson, B., Jolicoeur, P., 2008. Express Attentional Re-Engagement but Delayed Entry intoConsciousness Following Invalid Spatial Cues in Visual Search. PLoS ONE 3, e3967.

Brisson, B., Leblanc, E., Jolicoeur, P., 2008. Contingent capture of visual–spatial attentiondepends on capacity-limited central mechanisms: evidence from human electro-physiology and the psychological refractory period. Biol Psychol.

Brookes, M.J., Gibson, A.M., Hall, S.D., Furlong, P.L., Barnes, G.R., Hillebrand, A., Singh, K.D., Holliday, I.E., Francis, S.T., Morris, P.G., 2005. GLM-beamformer methoddemonstrates stationary field, alpha ERD and gamma ERS co-localisation withfMRI BOLD response in visual cortex. Neuroimage 26, 302–308.

Cheyne, D., Bakhtazad, L., Gaetz, W., 2006. Spatiotemporal mapping of cortical activityaccompanying voluntary movements using an event-related beamforming ap-proach. Hum Brain Mapp 27, 213–229.

Cohen, M.S., 1997. Parametric analysis of fMRI data using linear systems methods.Neuroimage 6, 93–103.

Cowan, N., 2001. The magical number 4 in short-term memory: a reconsideration ofmental storage capacity. Behavioral & Brain Sciences 24, 87–185 [References].

Cox, R.W., 1996. AFNI: software for analysis and visualization of functional magneticresonance neuroimages. Comput Biomed Res 29, 162–173.

Dale, A.M., Liu, A.K., Fischl, B.R., Buckner, R.L., Belliveau, J.W., Lewine, J.D., Halgren, E.,2000. Dynamic statistical parametric mapping: combining fMRI and MEG for high-resolution imaging of cortical activity. Neuron 26, 55–67.

Delorme, A., Makeig, S., 2004. EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis. J Neurosci Methods134, 9–21.

Drew, T., Vogel, E.K., 2008. Neural measures of individual differences in selecting andtracking multiple moving objects. J Neurosci 28, 4183–4191.

Forman, S.D., Cohen, J.D., Fitzgerald, M., Eddy, W.F., Mintun, M.A., Noll, D.C., 1995.Improved assessment of significant activation in functional magnetic resonanceimaging (fMRI): use of a cluster-size threshold. Magn Reson Med 33, 636–647.

Gratton, G., 1998. The contralateral organization of visual memory: a theoreticalconcept and a research tool. Psychophysiology 35, 638–647.

Grimault, S., Robitaille, N., Grova, C., Lina, J.-M., Dubarry, A.-S., Jolicoeur, P., 2009.Oscillatory activity in parietal and dorsolateral prefrontal cortex during retention invisual short-term memory: additive effect of spatial attention and memory load.Hum Brain Mapp 30, 3378–3392.

Grova, C., Daunizeau, J., Lina, J.M., Benar, C.G., Benali, H., Gotman, J., 2006. Evaluation ofEEG localization methods using realistic simulations of interictal spikes. Neuro-image 29, 734–753.

Hamalainen, M.S., Ilmoniemi, R.J., 1994. Interpreting magnetic fields of the brain:minimum norm estimates. Med Biol Eng Comput 32, 35–42.

Hopf, J.M., Luck, S., Girelli,M., Hagner, T., Mangun, G., Scheich, H., Heinze, H.J., 2000. Neuralsources of focused attention in visual search. Cerebral Cortex 10, 1233–1241.

Jolicoeur, P., Brisson, B., Robitaille, N., 2008. Dissociation of the N2pc and sustainedposterior contralateral negativity in a choice response task. Brain Res 1215,160–172.

Kastner, S., De Weered, P., Desimone, R., Ungerleider, L.G., 1998. Mechanisms ofdirected attention in the human extrastriate cortex as revealed by functional MRI.Science 282, 108–111.

Kastner, S., Pisnk, M., De Weerd, P., Desimone, R., Ungerleider, L., 1999. Increasedactivity in human visual cortex during directed attention in the absence of visualstimulation. Neuron 22, 751–761.

Kawasaki, M., Watanabe, M., Okuda, J., Sakagami, M., Aihara, K., 2008. Human posteriorparietal cortex maintains color, shape and motion in visual short-term memory.Brain Res 1213, 91–97.

Kelley, T., Serences, J., Giesbrecht, B., Yantis, S., 2007. Cortical mechanisms for shiftingand holding visuospatial attention. Cerebral Cortex 18, 114–125.



Klaver, P., Talsma, D., Wijers, A.A., Heinze, H.-J., Mulder, G., 1999. An event-related brainpotential correlate of visual short-term memory. Neuroreport: For RapidCommunication of Neuroscience Research 10, 2001–2005.

Konen, C.S., Kastner, S., 2008. Representation of eye movements and stimulus motion intopographically organized areas of human posterior parietal cortex. J Neurosci 28,8361–8375.

Logothetis, N.K., Pauls, J., Augath, M., Trinath, T., Oeltermann, A., 2001. Neurophysio-logical investigation of the basis of the fMRI signal. Nature 412, 150–157.

Mazza, V., Turatto, M., Umilta, C., Eimer, M., 2007. Attentional selection andidentification of visual objects are reflected by distinct electrophysiologicalresponses. Exp Brain Res 181, 531–536.

McCollough, A.W., Machizawa, M.G., Vogel, E.K., 2007. Electrophysiological measures ofmaintaining representations in visual working memory. Cortex 43, 77–94.

Medendorp, W.P., Kramer, G.F.I., Jensen, O., Oostenveld, R., Schoffelen, J.-M., Fries, P.,2007. Oscillatory activity in human parietal and occipital cortex shows hemisphericlateralization and memory effects in a delayed double-step saccade task. CerebralCortex 17, 2364–2374.

Mitchell, D., Cusak, R., 2008. Flexible, capacity-limited activity of posterior parietalcortex in perceptual as well as visual short-term memory tasks. Cerebral Cortex 18(8), 1788–1798.

Pashler, H., 1988. Familiarity and visual change detection. Perception & Psychophysics44, 369–378.

Perron, R., Lefevbre, C., Robitaille, N., Brisson, B., Gosselin, F., Arguin, M., Jolicoeur, M.,2009. Attentional and anatomical considerations for the representation of simplestimuli in visual short-term memory: Evidence from human electrophysiology.Psychological Research 73 (2), 222–232.

Posner, M.I., 1980. Orienting of attention. Quarterly Journal of Experimental PsychologyA 32, 3–25.

Posner, M.I., Cohen, Y., 1984. Components of visual orienting. In: Bouma, H., Bouwhuis, D.(Eds.), Attention and Performance, Vol. X. Erlbaum, pp. 531–556.

Robitaille, N., Jolicoeur, P., 2006. Fundamental properties of the N2pc as an index ofspatial attention: effects of masking. Can J Exp Psychol 60, 101–111.

Robitaille, N., Jolicoeur, P., Dell'Acqua, R., Sessa, P., 2007. Short-term consolidation ofvisual patterns interferes with visuo-spatial attention: converging evidence fromhuman electrophysiology. Brain Res 1185, 158–169.

Robitaille, N., Grimault, S., Jolicoeur, P., 2009. Bilateral parietal and contralateral responsesduring maintenance of unilaterally-encoded objects in visual short-term memory:evidence from magnetoencephalography. Psychophysiology 46, 1090–1099.

Sereno, M.I., Dale, A.M., Reppas, J.B., Kwong, K.K., Belliveau, J.W., Brady, T.J., Rosen, B.R.,Tootell, R.B., 1995. Borders of multiple visual areas in humans revealed byfunctional magnetic resonance imaging. Science 268, 889–893.

Sereno, M.I., Pitzalis, S., Martinez, A., 2001. Mapping of contralateral space in retinotopiccoordinates by a parietal cortical area in humans. Science 294, 1350–1354.

Song, J.-H., Jiang, Y., 2006. Visual working memory for simple and complex features: anfMRI study. Neuroimage 30, 963–972.

Thut, G., Nietzel, A., Brandt, S.A., Pascual-Leone, A., 2006. Alpha-band electroenceph-alographic activity over occipital cortex indexes visuospatial attention bias andpredicts visual target detection. J Neurosci 26, 9494–9502.

Todd, J.J., Marois, R., 2004. Capacity limit of visual short-term memory in humanposterior parietal cortex. Nature 428, 751–754.

Todd, J.J., Marois, R., 2005. Posterior parietal cortex activity predicts individual differencesin visual short-term memory capacity. Cogn Affect Behav Neurosci 5, 144–155.

Vogel, Machizawa, 2004. Neural activity predicts individual differences in visualworking memory capacity. Nature 428, 748–751.

Vogel, E.K., Woodman, G.F., Luck, S.J., 2001. Storage of features, conjunctions andobjects in visual working memory. J Exp Psychol Hum Percept Perform 27, 92–114.

Vogel, E.K., McCollough, A.W., Machizawa, M.G., 2005. Neural measures reveal individualdifferences in controlling access to working memory. Nature 438, 500–503.

Wyart, V., Tallon-Baudry, C., 2008. Neural dissociation between visual awareness andspatial attention 2667–2679.

Xu, Y., 2007. The role of the superior intraparietal sulcus in supporting visual short-term memory for multifeature objects. J. Neurosci. 27, 11676–11686.

Xu, Y., Chun, M.M., 2006. Dissociable neural mechanisms supporting visual short-termmemory for objects. Nature 440, 91–95.


Documents

Author's personal copy · Introduction While behaving in a constantly changing environment, the visual ... increase of the amplitude of this memory-related ERP component (labeled