Dale-Dynamic Statistical Parametric NeurotechniqueMapping-Combining fMRI and MEG for High-Resolution Imaging of Cortical Activity

8/22/2019 Dale-Dynamic Statistical Parametric NeurotechniqueMapping-Combining fMRI and MEG for High-Resolution Imagin

1/13

Neuron, Vol. 26, 5567, April, 2000, Copyright 2000 by Cell Press

Dynamic Statistical Parametric NeurotechniqueMapping: Combining fMRI and MEG forHigh-Resolution Imaging of Cortical Activity

exist that measure changes in these hemodynamic andelectromagnetic signals. Considered individually, these

techniques offer trade-offs between spatial and tempo-ral resolution (Churchland and Sejnowski, 1988). Hemo-dynamic assessment of brain activity is temporally lim-

Anders M. Dale,* Arthur K. Liu,* Bruce R. Fischl,*Randy L. Buckner,J ohn W. Belliveau,*J effrey D. Lewine, and Eric Halgren*

*Massachusetts General Hospital NuclearMagnetic Resonance Center

Charlest ow n, M assac hus et ts 02129 i ted by t he lat ency of t he hem odynam ic response (about1 s) but can provide millimeter spatial sampling (Belli-Department of Radiology

University of Utah veau et al., 1991; Kwong et al., 1992). Conversely, meth-ods based on direct measurement of the electric andSalt Lake City, Utah 84108

Inst it ut Nat io nal d e la Sant e et mag net ic field s p ro duc ed b y neuro nal ac tivit y c an p ro -vide temporal resolution of less than 1 ms, adequate forde la Recherche Medicale

E9926, Marseilles detecting the orchestration of complex cognitive activity(Regan, 1989).However, the spatial configuration of neu-France

Department of Psychology Anatomy, and ronal activity cannot be derived uniquely based on elec-troencephalography (EEG) and/or magnetoencephalog-Neurobiology, and Radiology

Washington University raphy (MEG) recordings alone (Nunez, 1981; Hamalainenet al., 1993). In order to make this so-called inverseSt. Louis, Missouri 63130

probl emwell posed, it is necessary to impose additionalconstraints on the solution.

One common approach is to assume that the EEG/Summary

MEG signals are generated by a relatively small numb erof fo cal sources (Sherg and VonCramon, 1985; Schmidt

Functional magnetic resonance imaging (fMRI) canet al., 1999). An additional constraint can be derived

providemaps ofbrainactivationwithmillimeterspatialfrom the assumption that the sources are temporally

resolution but is limited in its temporal resolution touncorrelated (Mosher et al., 1992). These assumptions

the order of seconds. Here, we describe a techniqueare particularly appropriate when analyzing early sen-

thatcombines structuralandfunctional MRI withmag-sory responses, where the activity might reasonably be

netoencephalography(MEG) toobtain spatiotemporalexpected to be relatively focal and constrained to a

maps of human brain activitywith millisecond tempo-few primary sensory areas. On the other hand, such

ral resolution. This new techniquewas used to obtainassumptions are less justified in higher-level cognitive

dynamic statistical parametric maps of cortical activ-experiments, which have been found by intracranial re-

ity during semantic processing of visually presentedcordings in humans to involve extensive networks of

words. An initial wave of activity was foundto spread more or less synchronously activated brain areas (Hal-rapidly from occipital visual cortex to temporal, pari-

gren et al., 1994a, 1994b, 1995a, 1995b; Baudena et al.,etal, and frontal areas within 185 ms, with a high de-

1995). Similarly, the interictal spikes characteristic ofgreeof temporaloverlapbetweendifferentareas.Rep-

partial epilepsy t ypically spread very rapidly to involveetition effects were observed in many of the same

a network extended across multiple cortical and limbicareas followingthis initialwaveofactivation,providing

regions (Chauvel et al., 1987).evidencefortheinvolvementof feedbackmechanisms

An alternative approach to analyzing EEG/MEG sig-in repetition priming.

nals is to impose constraints based on anatomical andphysiological information derived from other imaging

Introduction modalities. The anatomical constraint is based on theobservation that the main cortical generators of EEG

Thorough understanding of the functional organization and MEG signals are localized to the gray matter andof the brain requires knowledge of several aspects of oriented perpendicularly to the cortical sheet (Nunez,functional neuroanatomy, including the specific loca- 1981). Thus,once the exact shape of the cortical surfacetions of processing areas, the type of processing per-

is known (for a specific subject), this information can beformed, the time course of processing, and the nature used to greatly reduce the EEG/MEG solution spaceof the interactions between these areas. Local alter- (Dale and Sereno, 1993).ations in neuronal activity induce local changes in the The solution space can be further reduced by makingelectric and magnetic fields (Hamalainen et al., 1993; use of information derived from metabolic or hemody-Mitzdorf, 1985), cerebral metabolism, and cerebral per- namic measures of brain activity during t he same taskfusion (blood flow, blood volume, and blood oxygen- (Nenov et al., 1991; Dale and Sereno, 1993; Heinze etation)(Mazziotta et al., 1983; Fox and Raichle, 1986; Fox al., 1994; Snyder et al., 1995; Liu et al., 1998; Mangunet al., 1988; Belliveau et al., 1991; Prichard et al., 1991; et al., 1998; Ahlfors et al., 1999). This is based on theKwong et al., 1992). Several noninvasive techniques hypothesis that the synaptic currents generating the

EEG/MEG signals also impose metabolic demands,which in turn lead to a hemodynamic response measur-To whom correspondence should be addressed (e-mail: dale@

nmr.mgh.harvard.edu). able usingp ositron emission tomography(PET)(Raichle,


2/13

Neuron56

1987) or functional magnetic resonance imaging (fMRI) Results and Discussion(Belliveau et al., 1991; Kwong et al., 1992). Although the

Estimation of Spatiotemporal Activity Patternsprecise natureof the coupling between neuronal activityThe goal of the method described here is to obtainand hemodynamic signals is unknown, thereis consider-estimates of brain electrical activity with the best possi-able evidence for a strong correlation between the spa-ble spatial and temporal accuracy. Given the inherenttial patterns of hemodynamic changes and neuronaldifferencesin the spatial and temporal resolutionof non-

electrical activity over time in both animals (Grinvald et invasive imaging modalities, as discussed above, weal., 1986) and humans (Benson et al., 1996; Puce et al.,wish to combine the different measures in a way that1997).takes maximal advantage of the strengths of each tech-Here we present a general framework for integratingnique. In other words, we wish to obtain spatiotemporalinformation from different imaging modalities with a pri-activity estimates that are maximally consistent with allori anatomical and physiological information to produceavailable observables (i.e., fMRI, EEG, and/or MEG) asspatiotemporal estimates of brain activity. By normaliz-well as a priori anatomical and physiological informationing these estimates in terms of noise sensitivity at each(for a more formal discussion, see Experimental Proce-spatial location, we obtain statistical parametric mapsdures below, and Liu, 2000). This, of course, requires(SPMs) that provide information about the statistical re-that we have some idea of the coupling between electri-

liability of the estimated signal at each location in thecal activityin the brain and our noninvasiveobservables.

map with millisecond accuracy. These SPMs can thenIn the case of EEG and MEG, this is relatively straightfor-

be visualized as movies of brain activity (dynamicward, as the electric and magnetic fields generated by

SPMs). In order to quantitate the spatial resolution ofneuronal activity follows from well understood funda-

these maps, we compute the pointspread function for

mental laws of physics (Nunez, 1981; Hamalainen et al.,different locations on the cortical surface. This reflects 1993).the spatial blurring of the true activit y patt erns in our The primary generators of EEG and MEG are synapticspatiotemporal maps or movies. currents, where the current flows crossing neuronal

This new method is applied to MEG and fMRI mea- membranes act as tiny current sources or sinks for cur-surements in a task involving semantic judgments of rent outflow and inflow, respectively. Note that for eachvisually presented words. Such tasks are known to in- neuron, the net current inflow and outflow through itsvolve a large number of cortical areas (Buckner and membrane has to be zero (for conservation of charge).Koutstaal, 1998; Fiez and Petersen, 1998), but the timing Thus, in order for these fields not to cancel out at theof their involvement has not previously been determined. noninvasive sensors, there has to be some net spatialThe putative roles assigned to these structures suggest separation between the current sourcesand sinks withina possible sequence of activation during t his task, with the neurons.Of course,the fields produced by individualvisual word form processing preceding semantic asso- neurons are far too weak to be observed noninvasivelyciative activation, which in turn would precede working by EEG or MEG. Thus, to generate externally detectablememory and response mapping. It is unknown if, in fact, signals, the neurons within a volume of tissue must be

processing occurs according to this sequence. More aligned and their synaptic current flows correlated intime.The scalp-recorded EEG and MEGreflect the lineargenerally, the degree to which processing is sequentialsuperposition of the fields generated by all such synap-and modular in specialized regions versus parallel andtic currents across all neurons.distributed is unknown.

Of all the neurons in the human brain, the corticalIn addition, we investigated the effect of item repeti-pyramidal c ells are particularly well suited to generatetion on the spatiotemporal activity patterns evoked byexternally observable electric and magnetic fields due tothe task. Repetition is also known to alter the activationtheir elongated apical dendrites, systematically aligned inevoked in several cortical regions (Buckner and Kouts-a columnar fashion perpendicular t o the cortical sheet.taal, 1998; Gabrieli et al., 1998), but again, the timingInhibitory and excitatory synaptic inputs from differentof these effects is not known. One hypothesis wouldcell p opulations have characteristic laminar d istribu-suggest that processing of repeated items is facilitatedtions, resulting in characteristic spatial and temporal

beginning with relatively early perceptual stages. An al-patterns of net synaptic current flows at different depths

ternative hypothesis is that repetition effects are medi-through t he cortical sheet (Nicholson and Freeman,

ated by a top-down mechanism, originating in higher-1975; Mitzdorf, 1985; Barth and Di, 1991; Schroeder

level brain areas. One way to distinguish between these et al., 1995). These current flows are typically stronglyhypotheses would be to know the order in which differ-correlated laterally along the cortical sheet (Sukov and

ent anatomical areas show the effects of repetition. The Barth, 1998). Since the thickness of the cortical sheetarea with the earliest changes might t hen be reasonably is much smaller than the distance to the EEG and MEGsupposed to help produce the later changes noted in sensors, the current source/sink distribution within aother areas. Note that an observed decrease in metabo- small slab of cortex can be represented by a currentlism in c ortical areas that are early in the anatomical dipole oriented perpendicularly to the local cortical sur-sense (e.g., sensory-perceptual areas) might nonethe- face (Dale and Sereno, 1993), who se strength (moment)less be due to a change that occurs late in time (i.e., varies with time. Thus, we can represent the net corticallate in the p rocessing stream) due to top- down effects synaptic current flows by a scalar function s(r,t), re-from higher areas. Thus, the actual timing, as well as flecting dipole strength as a function of location r andthe location of activation, is crucial for understanding time t. The coupling between the dipole strengths and

the observed electric and magnetic fields can then bethe functional dynamics of cognitive processing.


3/13

Dynamic Statistical Parametric Mapping57

expressed simply in terms of a sum or integral across Due to the linear nature of the inverse estimation ap-all spatial locations (equations 2 and 3 in Experimental proach considered here, we can obtain a straightforwardProcedures, below). expression for the uncertainty in our activity estimates

The coupling between electrical activity and hemody- due to noise in the EEG/MEG recordings (equation 5).namic measures like fMRI is less understood . In particu- By normalizing the activity estimates at each locationlar, there is little quantitativedata on how the magni t ude in the brain by the noise sensitivity (the standard errorof the hemodynamic response varies as a function of of the estimate), we obtain statistical parametric mapsthe amplitude and duration of electrical activity. There of activity at every timepoint, with millisecond accuracyis, however, considerable evidence for a strong degree (equations 6 and 7). These dynamic statistical paramet-of spatial correlation between various measures of local ric maps can be displayed as movies of brain activityelectrical activity and local hemodynamic signals. Some over time, with each frame in these movies indicating if aof the strongest evidence for this comes from a direct statistically significant signal is present at a each corticalcomparison of maps obtained using voltage-sensitive location at t hat latency.dyes, reflecting depolarization of neuronal membranes One common and straightforward way to quantitatein superficial cortical layers, and maps derived from the spatial resolution, or degree of blurring,at differentintrinsic optical signals, reflecting changes in light ab- locations in the maps is by calculating the spatial point-sorption due to changes in blood volume and oxygen- spread function. This function is simply the image oneation (Shoham et al.,1999). Previous animal studieshave would obtain if all the activity were concentrated at aalso shown strong correlation between local field poten- given point, assuming no noise in t he EEG/MEG signals.tials, spiking activity, and voltage-sensitive dye signals Due to the time-invariant and spatially linear nature of(Arieli et al., 1996; Tsodyks et al., 1999). Furthermore, our estimation method, one can derive a simple expres-

studies in humans comparing the localization of func- sion for the pointspread at every location in the braintional activity using invasive electrical recordings and (equation 8, below). Moreover, the spatial blurring offMRI also provide evidence of c orrelation b etween t he any arbitrary spatiotemporal activation pattern can belocal electrophysiological and hemodynamic response predicted as the activation-weighted linear superposi-(Benson et al., 1996; Puce et al., 1997). This suggests tion (sum) of the pointspread for each location and timethat the local fMRI response can be used to bias the point.electrical activity estimate toward those regions that Figure 1 shows the pointspread function for three dif-show the greatest fMRI response. This can b e accom- ferent locations (indicated by green circles), when theplished by using t he fMRI response as an a priori esti- electrical activity is constrained to lie on the corticalmate of the locally integrated dipole activity over time surface. The left side shows t he pointspread functions(Dale and Sereno, 1993; Liu et al.,1998). This formulation for the commonly used minimum norm solution (Hama-results in a straightforward linear expression for theopti- lainen and Ilmoniemi, 1984), calculated assuming 122mal estimate based on theEEG and/or MEG dataat each channels of MEG recordings. This illustrates a commontime point (equation 4, below). Our previous simulation problem with distributed dipole solutions, namely theirstudies suggest that some care must be taken to avoid tendency to misattribute focal, deep activations to ex-

overconstraining the solution, as minor mismatches be- tended, superficial patterns (Hamalainen and Ilmoniemi,tween the electrical and hemodynamic signals can then 1984; Dale and Sereno, 1993). The first point (top), lo-severely distort the resulting estimates (Liu et al., 1998; cated deep in the insula, has a particularly large pointLiu, 2000). These studies further suggest that by using

spread, covering more than half the lateral extent ofthe fMRI data as a partial constraint (i.e., allowing some

the brain. The second point (middle), also located in aelectrical activity in locations with no detectable fMRI

sulcus, has a somewhat smaller pointspread, but thereresponse), one can obt ain accurate estimates of electri-

is a pronounced bias toward superficial locations, ascal activity from MEG and EEG, even in the presence

evidenced by the shift in center-of-mass away from theof some spatial mismatch between the generators of

actual location. The third point (bottom), located moreEEG/MEG data and the fMRI signals.

superficially near the crown of a gyrus, has a muchsmaller pointspread. Note that the pointspread for the

Uncertainties and Potential Errorsnoise-normalized anatomically constrained estimates,in the Activity Estimatesshown on the right in Figure 1, is of much more uniformIt is important to note that estimates of brain structure orextent for the different locations. (See Liu, 2000 for afunction based on any noninvasive imaging technology,

more extensive analysis of the pointspread propertieswhether based on MRI, CT, PET, SPECT, or EEG/MEG, of different estimation approaches.)are necessarily inexact. This imprecision is caused byThis pointis furtherillustrated in Figure2,which showsseveral factors, including (1) noise in the measured sig-

pseudo-color maps and histograms of pointspread ex-nals, resulting in errors in the estimates; (2) errors intent for all locations on t he cortical surface. Again, theour model of the coupling between the parameters ofpointspread extent (in terms of half-width-half-max) forinterest (e.g., electrical activity, tissue perfusion, bloodtheminimum norm estimator varies greatly acrossdiffer-flow, or oxygenation) and the measured signals; and (3)ent locations on the cortical surface, reaching as muchfundamental ill-posedness or ambiguity of the inverseas 100 mm or more in certain locations (shown in brightproblem, resulting in limited resolution in the estimates.yellow). The noise-normalized estimator, on the otherThus, in order to properly interpret noninvasive imaginghand, results in a much more spatially uniform point-results, it is essential to have quantitative measures ofspread extent, with values averaging around 20 mm. Athe uncertaintiesand potential errorsd ue to thesed iffer-

ent factors. further reduction in pointspread extent can be achieved


4/13

Neuron58

Figure 1. Spatial Resolution of t he Anatomi-cally Constrained Estimates for Three Mod-eled Dipole Locations

The spatial resolution of the described noise-normalized method is contrasted with that ofthe standard minimum norm approach (rightand left columns, respectively). Shown are

pointspread maps for three different dipolelocations (indicated by green dot s). Note thatthe pointspread maps for the deep dipole lo-cations (first and second rows) are muchmore extensive with the minimum norm thanwith the noise-normalized estimator. Con-versely, the pointspread map for the superfi-cial dipole location (third row) is somewhatmore focal with the minimum norm estimator.In all cases, estimates were calculated con-straining activation to the cortical surface.

by increasing thenumber of MEG sensors and by includ- activity evoked during a task involving semantic pro-cessing of visually presented words. Subjects were re-ing EEG measurements in the estimation (Liu, 2000).

Note that due to the linear nature of the estimation quired to decidewhether each word referredto anobjector animal that is usually more than one foot in sizeapproach, such pointspread maps provide a direct

quantitation of the local degree of spatial blurring ex- (in any dimension). The words were either novel (i.e.,presented only once in the task) or were repeated multi-pected in the spatiotemporal maps calculated with this

method for arbitrary activation patterns. Importantly, ple times.these pointspread functions can be used t o assess thespatial accuracy of the anatomically constrained esti-

Anatomically Constrained Estimates Basedmates obtained in cognitive experiments, as describedon MEG (aMEG)in a Single Subjectbelow.Applying the anatomically constrained noise-normal-ized estimation procedure to the event-related MEG av-Spatiotemporal Mapping of Brain Activityerages, we computed spatiotemporal activity estimatesin a Cognitive Taskfor the novel and repeated word conditions. SnapshotsIn the following, we describe the results of applying the

methods described above to spatiotemporally map the of these aMEG movies are shown in Figure 3 for four

Figure 2. Spatial Resolution of t he Anatomi-cally Constrained Estimates for all Cortical

Locations

Maps and histograms of pointspread extent,in terms of half-width-half-maximum in milli-

meters on the cortical surface, are shown forthe minimum norm and noise-normalized es-

timators (left and right side, respectively).Note that the noise-normalized estimator re-sults in much more spatially uniform andoverall lower pointspread extent than theminimum norm estimator, particularly fordeep (sulcal) locations.


5/13


Figure 3. Snapshots of Estimated Cortical Act ivity at Four Representative Latencies in a Single Subject

Statistical parametric maps calculated from M EG during a verbal size judgement task are shown of the noise-normalized anatomicallyconstrained dipole amplitudes. Activation was seen to spread rapidly from visual cortex around 80 ms to occipitotemporal, anterior temporal,

and prefrontal areas. At longer latencies, activity diverged between novel and repeated words, especially in the prefrontal cortex. Activationsare displayed on an inflatedview of the left hemisphere. Sulcal and gyral cortex areshow n in dark and light gray, respectively. The significancethreshold for the dynamic statistical parametric maps is p 0.001.

different latencies. The initial activation patterns evoked peaked at about 385 ms,thendeclined and wasreplacedwith a greater response to repeated words at a latencyby novel words were quite similar to those evoked by

repeated words. In both cases, the earliest activation of about 540 ms.was found near the foveal representation of retinotopicvisual cortices beginning about 80 ms after stimulus fMRI to Words in a Single Subject

The left panel of Figure 4 shows the areas with signifi-onset. By 185 ms, activation had spread anteriorly toventral occipitotemporal areas, the intraparietal sulcus, cantly different fMRI signal when comparing the active

task conditions described above to fixation on a station-and t he ventrolateral prefrontal cortex. Thus, withinabout 100150 ms of the initial cortical activation, a ary crosshair. Many areas were found to be involved in

this task, including cortex in the occipital pole, ventralwidespread network encompassing many areas of thebrain had been activated by both novel and repeated occipitotemporal junction, intraparietal sulcus, planum

temporale, posterior subcentral sulcus, precentral gy-words. Approximately 250 ms poststimulus, the re-sponse to novel and repeated words began to clearly rus, and ventral prefrontal areas. These results are con-

sistent with previous PET and fMRI studies of semanticdiverge. Activity elicited by repeated words rapidly de-creased in several areas, most prominently in prefrontal processing of visually presented words (Buckner and

Koutstaal, 1998; Fiez and Petersen, 1998). A subset ofcortex. The greater prefrontal activation to novel words

Figure 4. Location of fMRI Responses during Word Reading in a Single Subject

Areas in all lobes were found to produce significantly different fMRI responses between conditions (left panel). Many of these areas were alsofound t o respond differently t o novel versus repeated w ords (right p anel). The fMRI activation maps are painted onto an inflated view ofthe left hemisphere of the same subject. Sulcal and gyral cortex are shown in dark and light gray, respectively.


6/13

Neuron60

Figure 5. Snapshots of fM RI-Biased Cortical Activity Estimated from MEG at Four Different Latencies in a Single SubjectThe sequence of activity in the fMRI-biased estimates were generally similar to those constrained by anatomy alone (Figure 3), with activityspreading rapidly from visual cortex to occipitotemporal, anterior temporal, and prefrontal areas. However, the fMRI-biased estimates wereconsiderably more focal. All activations are significant for both fMRI (p 0.1) and MEG (p 0.001).

these areas (Figure 4, right panel) showed a decrease similar distribution of activated areas derived from MEG(Figure 3) as compared t o those derived fro m fMRI (Fig-in activation to repeated as compared to novel words,

including the occipitotemporal, intraparietal, and ventral ure4)in thesamesubject and task providessomeconfir-mation of the spatiotemporal estimation approach.prefrontal cortices. Note that this repetition effect was

not global in that several of the areas that were found However, the spatial accuracy of the anatomically con-strained estimates (limited by the estimator pointspread,to be activated by the task, including occipital areas,

the planum temporale, posterior subcentral sulcus, and as shown in Figures 1 and 2) may not be sufficient toanswer many questions in cognitive neuroscience.precentral gyrus were not found to be affected by repeti-

tion. Again, these results are consistent with previousstudies of the effects of verbal stimulus repetition on fMRI-Biased Estimates Based on MEG (fMEG)

in a Single SubjectfMRI or PET activation (Raichle et al., 1994; Bucknerand Koutstaal, 1998; Gabrieli et al., 1998). In order to improve the spatial resolution of the spatio-

temporal activity estimates, we used fMRI to bias theNote that the anatomically constrained activity esti-mates (in Figure3) and fMRI maps were calculated com- solution toward hemodynamically activated areas, as

described above. Snapshots of the f MEG movies arepletely independently of each other. Thus, the generally

Figure 6. Grand Average Maps of fMRI Responses to Novel and Repeated Words

fMRI responses were averaged across four subjects using a cortical surface-based morphing procedure and displayed on an averagedcurvature pattern. A widely distributed cortical network was found t o be reponsive to the task conditions, with repetition effects observed inthe activity in prefrontal, medioventral temporal, intraparietal, and supplementary motor cortices.


7/13


Figure 7. Snapshots of Grand Average fMRI-Biased Cortical Estimated from M EG at Four Different Latencies

fMRI-biased spatiotemporal activity estimates were averaged across four subjects. Activity was found to spread very rapidly from the occipitalpole at 80 ms, to the medioventral temporal lobe at 100 ms, and then to lateral inferotemporal, intraparietal, prefrontal, anterior cingularcortices at 185 ms. These areas remained active for about 400 ms. By 250 ms after stimulus onset, activity in the medial temporal lobe wasgreater to novel than to repeated words. At 385 ms, this repetition effect had spread to prefrontal and anterior cingular cortices. All activationsare significant for both fMRI (p 0.01) and MEG (p 0.001).

shownin Figure5 forthe samefourlatenciesas shownin 1999b). The SPMs for the grand averaged fMRI data,shown in Figure 6, again show that the task involvesFigure 3. Overall, the fMRI bias resulted in a substantial

focusing of the activation relative to the anatomically a large number of areas. As for the individual subjectdisplayed in Figure 5, these fMRI data were used to b iasconstrained MEG estimates. The general sequence of

activation appeared quite similar in the fMRI-biased and the MEG inverse solution in the same four subjects,and the resulting fMRI-biased estimates were averagedanatomically constrained MEG estimates, with an initial

activation in visual cortex, spreading rapidly to temporal across subject using surface alignment.Selected frames f rom t he result ing movies, shownand frontal areas. Furthermore, the repetition effects

again involve primarily occipitotemporal and prefrontal in Figure 7, confirmed the basic results already de-scribed for a single subject but also revealed additionalcortices, starting around 250 ms poststimulus onset.

Note that since the same fMRI bias was used in all details. The patterns of activation were nearly identicalfor novel and repeated words for the initial 200 ms. ThefMRI-biased estimates (for novel, repeated, and novel-

repeated), any differences between the task conditions earliest activation, by about 80 ms, was noted near theoccipital pole, and by 100 ms,activity had spread anteri-are due exclusively to the differences between the MEG

signals in these c ond itions. orly along the med iovent ral temp oral lob e to t he rhinalcortex. At 185 ms, the estimated activity in the occipitalpole had already declined, while activity in the entireAcross-Subject Averages of fMRI-Biased

Spatiotemporal Estimates extent of the medioventral temporal lobe had becomemuch stronger, and new foci of activation appearedCortical activation estimates were averaged across fo ur

subjects using a surface-based morphing procedure in in occipitotemporal, prefrontal, and anterior cingulatecortices. Occipitotemporal activation declined by 385order to achieve improved alignment of c orresponding

anatomical features across individuals (Fischl et al., ms,whereas prefrontal activation continued to increase,


8/13

Neuron62

Figure 8. Estimated Time Courses of Cortical Areas Responsive to Processing of Novel and Repeated Words

The fMRI-biased time courses estimated from MEG, normalized for noise sensitivity, were averaged across four subjects (as in Figure 7).Black lines show the response to novel words, and gray lines the response to repeated words. Waveforms are derived from single corticallocations (each one representing about 0.5 cm2). A z-score of 6 corresponds to a significance level of p 108.

at least to novel words. Note again that since the same The more rapid decline from peak activity in responseto repeated words resulted in clear repetition effectsfMRI biaswas used in all the fMRI-biased estimates, any

differences between the task conditions or at different visibleby 250 ms in theleft medioventral temporal cortex(Figure 7). These differences were considerably strongerlatencies are due exclusively to the MEG signals. By

partially constraining the inverse solution, the fMRI bias by 385 ms, when they also included prefrontal and cin-gulate areas. A second peak of repetition effects oc-helps to manifest the localizing information that is inher-

ent in thedistribution of MEG signalsacross thesensors. curred at 540 ms, when activity was greater to repeatedwords in most areas. Note that many of the occipital,Thelocation of the earliest detectableactivity at 80 ms

corresponds to early retinotopic visual areas previously parietal, and perirolandic areas showing strong and sus-tained activation to words did not show clear repetitionstudied with fMRI (Sereno et al., 1995), and the latency

of this estimated response corresponds to the onset effects. Repetition effects appeared much stronger inthe left hemisphere than in the right, although their gen-latency of the EEG and MEG responses evoked by sim-

ple visual stimuli (Regan, 1989; Portin et al., 1998; Marti- eral location was similar. Activation remained close tobaseline levels from about 670 ms poststimulus onset.nez et al., 1999). The rapid sp read of fMEG activation t o

rhinal cortex and then prefrontal cortices are consistent The snapshots of the fMRI-biased spatiotemporal es-timates show estimated activity over the entire lateralwith direct intracranial EEG (iEEG) responses to visual

words and faces (Halgren et al., 1994a, 1994b). aspect of the cortical surface of the left hemisphereat selected moments in time (Figure 7). An alternativePrevious studies have suggested that the occipito-

temporal cortex is concerned with midlevel form vision method for examining the data is to plot the estimateddipole strength at selected locations at all latencies (Fig-(Ungerleider and Mishkin, 1982), rhinal cort ex with com-

plex stimulus integration with memory and emotion ure 8). These time courses support the same generalpoints made by the movie snapshots: (1) activation be-(Amaral et al., 1992; Halgren, 1994; Halgren and Marin-

kovic, 1995; Murray and Bussey, 1999), and the ventro- gan in retinotopic visual cortex and spread very rapidlyanteriorly; (2) activation appeared to be identical forlateral prefrontal cortex with working memory (Gabrieli

et al., 1998). Thus, the current data suggest that high- novel and repeated stimuli during the initial 200260 ms,depending on the site; (3) the repetition effects do notlevel association areas are coactivated with perceptual

areas in processing an event from a relatively early involve all areas equally; and (4) the repetition effectevolves only gradually in a widespread network.stage. Other areas that are coactivated include ones

thought to be involved in lexical access (supramarginal In summary, repetition effects are widespread andoccur only after the initial activation of the entire net-and superior temporal c ortex) (Benson, 1979), atten-

tional control (intraparietal cortex)(Mesulam, 1990; Cor- work. The locations of the cortical areas found to beaffected by stimulus repetition using fMRI-biased MEGbetta, 1998), and motor response organization (medial

frontal cortex) (Picard and Strick, 1996). These results estimation are consistent with the findings of previousPET and fMRI studies (Buckner and Koutstaal, 1998).are difficult to reconcile with a discrete sequential acti-

vation of successive cognitive processing modules. In Similarly, the timing of the fMEG signals that changewith repetition resemble the N4/LPC components re-particular, these data argue against theories of prefron-

tal function that posit its involvement only following the corded in many previous EEG studies (Kutas and VanPetten, 1988; Halgren, 1990, 1994). Furthermore, bothencoding of the event by posterior association and per-

ceptual cortices. the localization and the timing of repetition-induced


9/13


changes in the fMRI-biased estimates correspond very activity over time is needed. Using fMRI and/or opticalimaging methods in humans and animals along withclosely to those of the generators of the N4/LPC, as

identified in previous intracranial EEG studies in word cortical laminar recordings, it may be possible to obtaina more precise, quantitative model of this coupling. Re-repetition and semantic priming tasks (Smithet al., 1986;

Halgren et al., 1994a, 1994b; Nobre et al., 1994; McCar- cordings with multicontact laminar electrodes and 2Dsurface grids can also provide a better understandingthy et al., 1995; Nobre and McCarthy 1995; Fernandez

et al ., 1999; Guil lem et al ., 1999). of t he spat iotem poral pat terns of synaptic c urrent f lowin the c ortex at micro- and mesoscopic scales (Nichol-In addition to this global similarity in generating struc-

tures, there is a close correspondence between the son and Freeman, 1975; Barth and MacDonald, 1996).Since the observed MEG/EEG signals are directly re-usual waveforms found in these previous iEEG studies

and the fMEG waveforms reported here. The fMRI- lated to these synaptic current flows(thedipole momentis the first nonvanishing term of the multipolarexpansionbiased estimates of the response to novel versus re-

peated stimuli showed two maxima, at about 385 and of the laminar current source density distribution), thisinformation could lead to greatly improved constraints540 ms after stimulus onset, consistent with what would

be expected in the same task for the N4 and LPC, re- on the spatiotemporal activityestimates. Ultimately, thiswould allow no ninvasive measures like EEG, MEG, andspectively. Like the fMRI-biased estimate, the repetition

effect associated with the N4/LPC extends from about fMRI to be used to test biophysical models of corticalneuronal circuitry.250 to 650 ms. The first fMEG peak was due to greater

activity evoked by novel words, and the second wasExperimental Proceduresdue to greater activity evoked by repeated words. Simi-

larly, N4 amplituded eclines and the LPCincreases when

Integration of Imaging Modalitiesa word is repeated. The similarity to intracerebral re- The goal of the current approach is to determine the spatiotemporalcordings provides an especially strong validation of the

pattern of electrical activity that is most consistent with all observ-fMRI-biased activity estimates, as intracerebral EEG re- ables (e.g., EEG, MEG, and fMRI) as well as a priori information.cordings can provide reasonably localized estimates of Formally, this can be posed as a problem of computing the solution

with the maximum a posteriori probability, given by Bayes Formula asthe time course of the net dipolar current flows withina volume of tissue around t he electrode contacts.

P(j(r,t)|x(t) & f(r,t)) P(x(t)|j(r,t))P(f(r,t)|j(r,t))P(j(r,t))

P(x(t) & f(r,t)), (1)

Limitations and Future Improvementswherej(r,t) denotes the current dipolevector, x(t) denotes the vectorAlthough the simulations and experimental data pre-of EEG/MEG recordings,and f(r,t)denotes the fMRIsignalat locationsented here involve only MEG recordings, it should ber and time point t. The term P(x(t)|j(r,t)) in equation 1 represents the

noted that the methods are equally applicable to EEG orrelationship between the spatiotemporal pattern of dipole strength

combined EEG/MEG recordings. In fact, by combining and the recorded EEG and/or MEG data, commonly referred to asboth kinds of recordings, it is possible to substantially the forward solution. In the frequency range of typical EEG/MEG

recordings (typically 1000 Hz), the forward solution hasa simple,improve the accuracy of the anatomically constrainedlinear form, given byestimates in certain situations (Dale and Sereno, 1993;

Liu, 2000). The main challenge in including EEG data in x(t) s

G(r)j(t,r)dr n(t), (2)the analysis is to obtain a sufficiently accurate forwardsolution,giventhe greater dependence of theEEG signal

where S denotes the space of (possible) dipole source locations,on the exact head shape and the conductivitiesof differ- the three columns of the matrix G(r) specify the predicted EEG/MEGent tissue types (Nunez, 1981; Hamalainen and Ilmo- recording vector for the three dipole components at location r, and

n(t) denotes additive noise. For computat ional purposes, the sourceniemi, 1984).space S is typically divided into a set of discrete elements, withIn the preceding analyses, dipole activity was as-dipole components representing the local current dipole within asumed to be limited to the cortical surface (includingsmall region. Equation 2 can then be written as

the hippocampus as well as the neocortex). While it ispossible for subcortical structures such as the lateral x(t) As(t) n(t), (3)geniculatebody to produce dipolar currentsourcedistri-

where s(t) denotes a vector of dipole component strengths, and thebutions (Schroeder et al., 1995), the MEG signals from

A denotes the resulting linear forward matrix operator (Dale andthese structures tend to be of shorter latency and dura- Sereno, 1993). The term P(f(r,t)|j(r,t)) in equation 1 similarly encodestion and smaller amplitude relative to cortical signals the coupling between the spatiotemporal pattern of electrical activ-

ity (in terms of dipole strength) and the fMRI signal. Finally, the term(Gobbele et al., 1998). In cognitive experiments, whereP(j(r,t)) encodes a priori information about the solution, i.e., theone is focusing on the later, large-amplitude signalslikelihood of different spatiotemporal patterns of dipole strength.lasting hundreds of milliseconds, it is thus unlikely thatThis provides a mechanism for incorporating knowledge of spatial

subcortically generated signals are much of a factor.correlations in electrical activity within different brain structures.

However, for sensory experiments, particularly when As discussed above, little quantitative data exist regarding theEEG signals areincluded in the analysis, it mayb e impor- coupling between electrical activity and hemodynamic response.

However, there is strong evidence for a general correlation betweentant to extend the anatomical source model to also takethe spatial patternof electricalactivity and hemodynamic measures.into account possible subcortical generator structures.This suggests that one can use the measured fMRI response toIn order to further improve the accuracy of the fMRI-spatially bias t he estimate of electrical activity over time. Specifi-

biased estimates (fMEG), a better understanding ofcally, one may assume that areas that show a strong f MRI response

the coupling between the observed hemodynamic re- are more likely to be electrically active. Assuming that the priorsponse, as measured by the blood oxygenation level information about dipole strength patterns can be expressed in

terms of a multivariate Gaussian distribution, it can be shown thatdependent (BOLD) signal, and the local current dipole


10/13

Neuron64

the maximum a posteriori probability (MAP) estimate is equivalentto the linear Wiener estimate given by qi (t)

jGi

(wj x(t))2

jGi

wj CwTj, (7)

s(t) Wx(t), where W RAT (ARAT C)1, (4)where Gi is the set of (three) dipole component indices for the ith

location. Note that under the null hypothesis, qi(t) is F distributed,where C n(t)n(t)T is the spatial (sensor) covariance matrix ofwith three degrees of freedom for the numerator. The degrees ofthe noise, and R s(t)s(t)T is the spatial covariance matrix offreedom for the denominator is typically large, again depending onthe dipole strength vector (Dale and Sereno, 1993; van Oosterom,

the number of time samples used to calculate the noise covariance1999). If one further assumes that the dipole strength variance (ormatrix C.power) over time can be expressed as a function of the local fMRI

response, this information can be encod ed in the diagonal elementsof R (Dale and Sereno, 1993). Information about the spatial covari- Analysis of Spatial Resolutionance of dipole strength (i.e., spatial smoothness) can similarly be An important property of linear estimation approaches is the exis-encoded in the off-diagonal elements of R. tence of an explicit expressionfor the spatial resolution of thedipole

One straightforward way to implement the fMRI constraint is to strength estimates at every location in the source space. Due tosimply threshold the fMRI statistical parametric map and set the a the linear nature of both the forward and inverse solutions, repre-priori variance estimate to a nonzero value only at locations ex- sented by linear operators A and W, respectively, the estimatedceeding a c ertain threshold (George et al., 1995). However, simula- dipole strength vector s(t) can be expressed as a linear operationtion studies suggest that this approach is exceedingly sensitive on the actual dipole strength vector s(t). More precisely,to minor model misspecifications, in particular to the presence of

s(t) Wx(t) WAs(t) Wn(t) WAs(t). (8)generators of EEG or MEG signals that are not detected by fMRI(Liu et al., 1998; Liu, 2000). These studies suggest t hat the distorting

Thematrix WA is commonlyreferred to as a resolution matrix,whereeffect of such potential misspecifications is greatly reduced by im-

the columns specify the pointspread for each dipole location, i.e.,posing a partial fMRI constraint or bias. This is accomplished by

the spatial pattern of estimated dipole strength for a unit of actualsetting the minimum a priori variance estimate to some finite, non-

dipole strength at a particular location (Menke, 1989; Dale and Ser-zero value (typically 10% of the maximum value).eno, 1993; Press et al., 1994; Grave de Peralta Menendez et al.,1997). The half-width-half-max measure was determined by com-puting the average distance to the dipoles at which the estimatedDynamic Statistical Parametric Mapping

Typically, MEG and EEG are modeled as resulting from the activity dipole strength was greater than half its value at the point wherethe model dipole was actually placed.of discrete equivalent current dipoles with time-varying moment

and orientation. In contrast, tomographic methods likef MRI providesignal estimates at every location within a volume. Because the Cognitive Tasknoise variance may vary greatly between voxels, fMRI activation A total of 370 concretenouns,311 letters long,representing objectsestimates are usually presented as noise-normalized statistical and animals were used as stimuli. Each word w as presented for 240parametric maps(SPMs),rathert hanas mapsof raw signalstrength. ms with a 1600 ms stimulus onset asynchrony on a back-projectionThecurrent method provides activity time courseestimates for every screen using an LCD video projector. The subject pressed a keycortical location. By normalizing these estimates by predicted esti- with his/her right hand if the object or animal that the word repre-mator noise, one can similarly obtain noise normalized SPMs for sented was usually more than a foot in its longest dimension. Theeach time point. task was performed by the subjects with each of therepeated words

Dueto the linear nature of the Wiener estimate defined in equation six times before data collection began. Stimulus blocks were pre-4, it is straightforward to compute the variance of each dipole sented in a fixed order of novel-fixation-repeated-fixation. Three

strength estimate due to the additive noise. Specifically, we have hundred sixty of the words were shown only once in one of thenovel blocks and ten words were presented repeatedly in the re-

Var(s i) (win(t))2 wi CwTi , (5) peated blocks. In the repeated blocks, each of the ten repeatedwordswas presented oncebut in a different order withineachblock.

where s i denotes the ith element of the dipole strength vector s, and Active blocks were separated by 16 s of visual fixation. Thus, awi denotes the ith row of the inverse operator W (Liu, 2000). Note given repeating word occurred once and only once in each repeatedthat in general three dipole components are required to represent block, and an average of 64 s and 19 intervening stimuli passedan arbitrary dipole orientation and strength at each location in the between successive presentations of the same word. In each ofbrain. For locations whose dipole orientation are known a priori nine runs, four blocks of repeated words and four of novel werebased on anatomical information, only the dipole strength needs to presented. Animals and objects were presented in separate runs.be determined for each location (Dale and Sereno, 1993). In this Across runs presenting animals (objects), the same ten animalscase, a noise-normalized activity estimate zi(t) can be computed for (objects) were used as repeating stimuli. Exactly the same taskseach time point t and location i as follows were used for fMRI and MEG studies, with overlapping stimulus

lists. Approximately six months passed between fMRI and MEGtesting sessions.

zi(t) wi x(t)

wi CwTi, (6)

Physiological Data Acquisition and Analysis

The same four normal, strongly right-handed, young adults (onewhere C is, again, the estimated noise covariance matrix (Dale andSereno, 1993; Liu et al., 1998). By dividing the estimated total dipole male) were studied with MRI, fMRI, and MEG after appropriate in-

formed consent. ThestructuralMRI was acquired on a 1.5T Siemensstrength estimate for each location (the numerator in equation 6)b ythe predicted standard error of the estimate due to additive noise Vision scanner using an MPRAGE sequence (TR 9.7 ms, TI 20

ms, TE 4 ms, flip angle 10, FOV 256 mm, slice thickness (the denominator in equation 6), we obtain a normalized dipolestrength zi(t) that is t distributed under the null hypothesis of no 1 mm), and the cortical surface was reconstructed as described

below.dipole activity (i.e., s(t) 0). Since the number of time samples usedto calculate the noise covariance matrix C is quite large (typically fMRI data were collected using a GE1.5 T MRI scanner retrofitted

for echop lanar imaging (Advanced NMR Systems, Wilmington, MA).more than 100), the t distribution approaches a unit normal distribu-tion (i.e., a z-score). Data were acquired from 16 approximately axial slices covering the

entire brain (7 mm slice thickness; 3.1 3.1 mm in-plane resolution)If, on the other hand, no a priori assumptions are made about the

local dipole orientation, three components are required for each using a standard GEquadrature head coil. An automated shimmingprocedure was used to improve B0 field homo geneity (Reese etlocation. A noise-normalized estimate of the local current dipole

power (sum of squared dipole component strengths) at location i al., 1995). T1-weighted inversion recovery echoplanar images wereacquired for anatomical alignment of functional images (TR 20 s,is given by


11/13


TI 1100 ms, 1.5625 mm in-plane resolution). Finally, T2*-weighted collaboration of K. Kwong and J. Klopp. This research was sup-functional images were acquired using an asymmetric spin echo ported by the Human Frontiers ScienceProgram (A. M. D. and E. H.),sequence sensitive to BOLD contrast (tau 25 ms, TE 50 ms, the NationalFoundation for FunctionalBrain Imaging (B.R., A. M. D.,TR 2000 ms, 3.125 mm in-plane resolution). The fMRI data were and E. H.), the Whitaker Foundation (A. M. D.), Office of Naval Re-analyzed using a generalized linear model, where the fMRI signal search (E. H.), and the National Institutes of Health (A. M. D.: R01-was modeled as ap iecewise constant function of experimental con- RR13609; E. H.: R01-NS18741).dition (novel, repeated, and fixation) convolved with an assumedimpulse response function (Boynton et al., 1996; Friston et al., 1998).

Received June 20, 1999; revised March 1, 2000.This model was used to obtain statistical parametric maps for themain effect of experimental condition (using an F test) as well asfor the comparison of the response to different conditions, including Referencesnovel words versus fixation and novel versus repeated words (usinga t test). Activation from the four subjects was averaged by aligning Ahlfors, S.P., Simpson, G.V., Dale, A.M., Belliveau, J.W., Liu, A.K.,each subjects sulcal pattern to the average of a group of 35 normal Korvenoja, A., Virtanen, J., Huotilainen, M., Tootell, R.B., Aronen,subjects (see above). H.J., and Ilmoniemi, R.J. (1999). Spatiotemporal activity of a cortic al

MEG signals were recorded from 0.3 to 20 Hz using a 122 channel network for processing visual motion revealed b y M EG and fMRI.planar dc-SQUID gradiometer system covering the entire scalp (Ha- J. Neurophysiol. 82, 25452555.malainen et al., 1993). Trials with eyeblinks or other artifacts were Amaral, D.G., Pri ce, J .L., Pit kanen, A., and Carm ichael, S.T. (1992).rejected from analysis. Anatomical organization of the primate amygdaloid com plex. In The

Amygdala: Neurobiological Aspects of Emotion, Memory, and Men-Cortical Surface Reconstruction tal Dysfunction, J.P. Aggleton, ed. (New York: Wiley-Liss), pp . 166.Geometrical representationsfor the corticalsurfacesof eachsubject

Arieli, A., Sterkin, A., Grinvald, A., and Aertsen, A. (1996). Dynamicswere obtained using procedures described in detail in Dale et al.

of ongoing activity: explanation of the large variability in evoked(1999)and Fischl et al. (1999a). First, high-resolution 3D T1-weighted

cortical responses. Science 273, 18681871.structural images wereacquired for eachsubject.Then,a segmenta-

Barth, D.S., and Di, S. (1991). Laminar excit ability cyc les in neocor-tion of cortical white matter was performed, and the estimated bor-tex. J. Neurophysiol. 65, 891898.der between gray and white matter was tessellated, providing a

triangular representation of the surface. This representation of the Barth, D.S., and MacDonald, K.D. (1996). Thalamic modulation offolded cortical surface was used to derive the locations and orienta- high-frequency oscillating potentials in auditory cortex. Nature 383,tions of the dipoles used in the analysis of the MEGdata (see below). 7881.Finally, t he fold ed surf ace t essellation was inflated in order to

Baudena, P., Heit, G., Clarke, J.M., and Halgren, E. (1995). Intracere-unfold cortical sulci, thereby providing a convenient format for visu-

bral potentials to rare target and distractor auditory and visual stim-alizing cortical activation patterns (Dale et al., 1999; Fischl et al.,

uli: 3. Frontal cortex. Electroencephalogr. Clin. Neurophysiol. 94,1999a). For purposes of intersubject averaging, the reconstructed

251264.surface for eachsubject was thenmorphed into anaveragespherical

Belliveau, J.W., Kennedy, D.N., McKinstry, R.C., Buchbinder, B.R.,representation, optimally aligning sulcal and gyral features acrossWeisskoff, R.M., Cohen, M.S., Vevea, J.M., Brady, T.J., and Rosen,subjects while minimizing metric distortion (Fischl et al., 1999b).B.R. (1991). Functional mapping of t he human visual cortex by mag-This nonrigid surface-based deformation procedureresultsin a sub-netic resonance imaging. Science 25 4, 716719.stantial reduction in anatomical and functional variability across

subjects relative to the more commonly used normalization ap- Benson, D.F. (1979). Aphasia, Alexia, and Agraphia (New York:proach of Talairach et al. (1967), thereby improving the anatomical Churchill Livingstone).precision of the intersubject averages. Benson, R.R., Logan, W.J., Cosgrove, G.R., Cole, A.J., Jiang, H.,

LeSueur, L.L., Buchbinder, B.R., Rosen, B.R., and Caviness, V.S.Anatomically Constrained MEG (aMEG) and fMRI-Biased MEG (1996). Functional MRI localization of language in a 9-year-old c hild.(fMEG) Solutions Can. J. Neurol. Sci. 23, 213219.For aMEGand fMEG analysis, the cortical surface was first subsam-

Boynton, G.M., Engel, S.A., Glover, G.H., and Heeger, D.J. (1996).pled to about 2500 dipole locations per hemisphere. The forwardLinear systems analysis of functional magnetic resonance imagingsolution for each of the three dipole components at each of thesein human V1. J. Neurosci. 16, 42074221.locations was calculated using a boundary element model (Oosten-Buckner, R.L., and Koutstaal, W. (1998). Functional neuroimagingdorp and Van Oosterom, 1992). The conductivity boundaries for t hisstudies of encoding, priming, and explicit memory retrieval. Proc.model were determined from the segmented MRI described above.Nat. Acad. Sci. USA 95, 891898.The activation at eachlocation on the corticalsurface wasestimated

every 5 ms using the anatomically and functionally constrained linear Chauvel, P., Buser, P., Badier, J.M., Liegeois-Chauvel, C., Marquis,estimation approach described above (see also Dale and Sereno, P., and Bancaud, J. (1987). The epileptogenic zone in humans:1993; Liu et al., 1998; Liu, 2000). For aMEG analysis, the fMRI representation of intercritical events by spatio-temporal maps. Rev.weighting was set to 0%, i.e., no fMRI constraint was used.For fMEG Neurol. (Paris) 143, 443450.analysis, the fMRI weighting was set to 90%, a value suggested by

Churchland, P.S., and Sejnowski, T.J. (1988).Perspectiveson cogni-our earlier simulation studies (Liu et al., 1998).

tive neuroscience. Science 242, 741745.The aMEG and fMEG maps (Figures 3, 5, and 7) were calculated

Corbetta, M. (1998). Frontoparietal cortical networks for directingwithout constraining the dipole orientation, with the sensitivity-nor-

attention and the eye to visual locations: identical, independent, ormalized estimates calculated according to equation 7. The regionaloverlapping neural systems? Proc.Natl.Acad. Sci.USA95, 831838.time course estimates shown in Figure 8 were calculated using

dipole orientation estimates obtained based on the cortical surface Dale, A.M., and Sereno, M.I. (1993). Improved loc alization of c orticalreconstruction in each subject and sensitivity normalized for each activityby combining EEGand MEGwith MRIcortical surface recon-subject as specified in equation 6. The time course and the ampli- struction: a linear approach. J. Cog. Neurosci. 5, 162176.tude of the resulting waveforms from adjacent locations wereusually Dale, A.M., Fischl, B., and Sereno, M.I. (1999). Cortical surface-highly similar due to t he pointspread of the estimator (see Figure 1).

based analysis I: segmentation and surface reconstruction. Neuro-However,the polarity of the waveforms commonly inverted between

image 9, 179194.nearby sites with different cortical orientations and thus should be

Fernandez,G., Effern, A.,Grunwald, T.,Pezer, N.,Lehnertz, K.,Dum-considered arbitrary.pelmann, M., Van Roost, D., and Elger, C.E. (1999). Real-timetracking of memory formation in the human rhinal cortex and hippo-Acknowledgmentscampus. Science 285, 15821585.

Fiez, J.A., and Petersen, S.E. (1998). Neuroimaging stud ies of wordWe thank S. A. Hillyard, W. C. West, and L. Anllo-Vento for helpfulcomments on the manuscript. We are also grateful for the technical reading. Proc. Nat. Acad. Sci. USA95, 914921.


12/13

Neuron66

Fischl, B., Sereno, M.I., and Dale, A.M. (1999a). Cortical surface- Hamalainen, M.S., Hari, R.,Ilmoniemi, R.J., Knuutila,J., and Lounas-maa, O.V. (1993). Magnetoencephalography theory, instrumenta-based analysis II: inflation, flattening, a surface-based coordinatetion, and applications to noninvasive studies of the working humansystem. Neuroimage 9, 195207.brain. Rev. Mod. Phys. 65, 413497.Fischl, B., Sereno, M.I., Tootell, R.B., and Dale, A.M. (1999b). High-Heinze, H.J., Mangun, G.R., Burchert, W., Hinrichs, H., Scholz, M.,resolution intersubject averaging and a coordinate system for theMunte, T.F., Gos, A., Scherg, M., Johannes, S., Hundeshagen, H.,cortical surface. Hum. Brain Mapp. 8, 272284.et al. (1994). Combined spatial and temporal imaging of brain activityFox, P.T., and Raichle, M.E. (1986). Focal physiological uncoupling

during visual selective attention in humans. Nature372

, 543546.of cerebral blood flow and oxidative metabolism during somatosen-Kutas, M., and Van Petten, C. (1988). Event-related brain potentialsory stimulation in human subjects. Proc. Nat. Acad. Sci. USA 83,studies of language. In Advances in Psychophysiology, P.K. Ackles,11401144.J.R. Jennings, and M.G.H. Coles, eds. (Greenwich, CT: JAI Press),

Fox,P.T.,Raichle, M.E., Mintun,M.A., and Dence, C. (1988).Nonoxi-pp. 139187.

dative glucose consumption during focal physiologic neural activity.Kwong, K.K., Belliveau, J.W., Chesler, D.A., Goldberg, I.E., and Weiss-Science 241, 462464.koff, R.M. (1992). Dynamic magnetic resonance imaging of human

Friston, K.J., Fletcher, P., Josephs, O., Holmes, A., Rugg, M.D.,brain activity during primary sensory stimulation. Proc. Nat. Acad.

and Turner, R. (1998). Event-related fMRI: c haracterizing differentialSci. USA 89, 56755679.

responses. Neuroimage 7, 3040.Liu, A.K. (2000). Spatiotemporal brain imaging. PhD dissertation,

Gabrieli, J., Poldrack, R., and Desmond, J. (1998). The role of leftMassachusetts Instituteof Technology,Cambridge,Massachusetts.

prefrontal cortex in language and memory. Proc. Natl. Acad. Sci.Liu, A.K., Belliveau, J.W., and Dale, A.M. (1998). Spatiot emporal

USA 95, 906913.imaging of human brain activity using fMRI constrained MEG data:

George, J., Mosher, J., Schmidt, D., Aine, C., Wood, C., Lewine, J., Monte Carlo simulations. Proc. Nat. Acad. Sci. USA 95, 89458950.Sanders, J., and Belliveau, J. (1995). Functional Neuroimaging by

Mangun, G., Buonocore, M., Girelli, M., and Jha, A. (1998). ERPCombined MRI, MEG and fMRI. Hum. Brain Mapp. S1, 89.

and fMRI measures of visual spatial selective attention. Hum. BrainGobbele, R., Buchner, H., and Curio, G. (1998). High-frequency (600

Mapp. 6, 383389.Hz) SEP activities originating in the subcortical and cortical humanMartinez, A., Anllo-Vento, L., Sereno, M.I., Frank, L.R., Buxt on, R.B.,

somatosensory system. Electroencephalogr. Clin. Neurophysiol.Dubowitz, D.J., Wong, E.C., Hinrichs, H., Heinze, H.J., and Hillyard,

108, 182189.S.A. (1999). Involvement of striate and extrastriate visual cortical

Grave de PeraltaMenendez, R.,Hauk,O., Andino, S.G., Vogt, H.,and areas in spatial attention. Nat. Neurosci. 2, 364369.Michel, C. (1997). Linear inverse solutions with optimal resolution Mazziotta, J.C., Phelps, M.E., and Halgren, E. (1983). Local cerebralkernels applied to electromagnetic tomography. Hum. Brain Mapp. glucose metabolic response to audiovisual stimulation and depriva-5, 454467. tion: studies in human subjects with positron CT. Hum. Neurobiol.Grinvald, A., Lieke, E., Frostig, R.D., Gilbert, C.D., and Wiesel, T.N. 2, 1123.(1986). Functional architecture of cortex revealed by optical imaging McCarthy, G., Nobre, A.C., Bentin, S., and Spencer, D.D. (1995).of intrinsic signals. Nature 324, 361364. Language-related field potentials in the anterior-medial temporalGuillem, F., Rougier, A., and Claverie, B. (1999). Short- and long- lobe: I. Intracranial distribution and neural generators. J. Neurosci.delay intracranial ERP repetition effects dissociate memorysystems 15, 10801089.in the human brain. J. Cog. Neurosci. 11, 437458. Menke, W. (1989). Geophysical Data Analysis: Discrete Inverse The-

ory (San Diego, CA: Academic Press).Halgren, E. (1990). Insights from evoked potentials into the neuro-

psychologicalmechanisms of reading.In Neurobiology of Cognition, Mesulam, M.M. (1990). Large-scale neurocognitive networks and

A. Scheibel and A. Weschsler, eds. (New York: Guilford), pp. distributed processing for attention, language, and memory. Ann.103150. Neurol. 28, 597613.

Halgren, E. (1994). Physiological integration of t he declarative mem- Mitzdorf, U. (1985). Current source-density method and applicationory system. In The Memory System of the Brain, J. Delacour, ed. in cat cerebral cortex: investigation of evoked potentials and EEG(New York: World Scientific), pp. 69155. phenomena. Physiol. Rev. 65, 37100.

Halgren, E., and M arinkovic, K. (1995). Neurophysiological networks Mosher, J.C., Lewis, P.S., and Leahy, R.M. (1992). Multiple dipoleintegrating human emotions. In The Cognitive Neurosciences, M. modeling and localization from spatio-temporal MEG data. IEEE

Trans. Biomed. Eng. 39, 541557.Gazzaniga, ed. (Cambridge, MA: MIT Press), pp . 11371151.

Murray, E.A., and Bussey, T.J. (1999). Perceptual-mnemonic func-Halgren, E., Baudena, P., Heit, G., Clarke, J.M., and Marinkovic, K.tions of the perirhinal cortex. Trends Cog. Sci. 3, 142151.(1994a). Spatio-temporal stages in face and word processing. 1.

Depth-recorded potentials in the human occipital, temporal and Nenov, V.I., Halgren, E., Smith, M.E., Badier, J.M., Ropchan, J.R.,parietal lobes. J. Physiol. (Paris) 88, 150. Blahd, W.H., and M andelkern, M. (1991). Localized brain m etabolic

response correlated with potentials evoked by words. Behav. BrainHalgren, E., Baudena, P., Heit, G., Clarke, J.M., M arinkovic, K., andRes. 44, 101104.Chauvel, P. (1994b). Spatio-temporal stages in face and word pro-

cessing. 2. Depth-recorded potentials in the human frontal and Ro- Nicholson, C., and Freeman, J.A. (1975). Theory of current sourcelandic cortices. J. Physiol. (Paris) 88, 5180. density analysis and determination of the conductivity tensor for

anuran cerebellum. J. Neurophysiol. 38, 356368.Halgren, E., Baud ena, P., Clarke, J.M., Heit, G., Liegeois- Chauvel,Nobre, A.C.,and McCarthy, G. (1995). Language-related field poten-C., Chauvel, P., and Musolino, A. (1995a). Intracerebral potentialstials in the anterior-medial temporal lobe: II. Effects of word typeto rare target and distractor auditory and visual stimuli: 1. Superiorand semantic priming. J. Neurosci. 15, 10901098.temporal plane and parietal lobe. Electroencephalogr. Clin. Neuro-

physiol. 94, 191220. Nobre, A., Allison, T., and M cCarthy, G. (1994). Word recognition inthe human inferior temporal lobe. Nature 372, 260263.Halgren, E., Baudena, P., Clarke, J.M., Heit, G., Marinkovic, K., De-

vaux, B., Vignal, J.P., and Biraben, A. (1995b). Intracerebral poten- Nunez, P.L. (1981). Electric Fields of the Brain (New York: OxfordUniversity).tials to rare target and distractor auditory and visual stimuli: 2. Me-

dial, lateral and posterior temporal lobe. Electroenceph. Clin. Oostendorp, T.F., and Van Oosterom, A. (1992). Source parameterNeurophysiol. 94, 229250. estimation using realistic geometry in bioelectricity and biomagnet-

ism.In Biomagnetic Localizationand 3D Modeling, J. Nenonen, H.M.Hamalainen, M.S., and Ilmoniemi, R.J. (1984). Interpreting measuredRajala, and T. Katila, eds. (Helsinki: Helsinki University of Technol-magnetic fields of the brain: estimates of current distribution (Hel-ogy, Report TKK-F-A689).sinki: University of Technology, Dept. of Technical Physics Report

TKK-F-A559). Picard, N., and Strick, P.L. (1996). Motor areas of the medial wall:


13/13


a review of their location and functional activation. Cereb. Cortex6, 342353.

Portin, K., Salenius, S., Salmelin, R., and Hari, R. (1998). Activationof the human occipital and parietal cortex by pattern and luminancestimuli: neuromagnetic measurements. Cereb. Cortex 8, 253260.

Press, W.H., Teukolsky, S.A., Vetterling, W.T., and Flannery, B.P.(1994). Numerical Recipes in C, Second Edition (Cambridge: Cam-

bridge University Press).Prichard, J., Rothman, D., Novotny, E., Petroff, O., Kuwabara, T.,Avison, M., Howseman, A., Hanstock, C., and Shulman, R. (1991).Lactate rise detected by 1H NMR in human visual cortex duringphysiologic stimulation. Proc. Natl. Acad. Sci. USA 88, 58295831.

Puce, A., Allison, T., Spencer, S.S., Spencer, D.D., and McCarthy,G. (1997). Comparison of cortical activation evoked by faces mea-sured by intracranial field potentials and functional MRI: two casestudies. Hum. Brain Mapp. 5, 298305.

Raichle, M.E. (1987). Circulatory and metabolic correlates of brainfunctionin normalhumans. In Handbook of Physiology:TheNervousSystem V, Higher Functions of the Brain, F. Plum, ed. (Bethesda,MD: Amer. Physiol. Soc.), pp. 643674.

Raichle, M.E., Fiez, J.A., Videen, T.O., MacLeod, A.-M.K., Pardo,J.V., Fox, P.T., and Petersen, S.E. (1994). Practice-related changesin human brainfunctionalanatomy during nonmotorlearning. Cereb.

Cortex 4, 826.Reese, T.G., Davis, T.L., and Weisskoff, R.M. (1995). Autom atedshimming at 1.5 Tesla using echo planar image frequency maps. J.Magn. Reson. Imaging 5, 739745.

Regan, D. (1989). Human Brain Electrophysiology (New York: El-sevier).

Schmidt, D.M., George, J.S., and Wood, C.C. (1999). Bayesian infer-ence applied to the electromagnetic inverse problem. Hum. BrainMapp. 7, 195212.

Schroeder, C.E., Steinschneider, M., Javitt, D.C., Tenke, C.E., Givre,S.J., Mehta, A.D., Simpson, G.V., Arezzo, J.C., and Vaughan, H.G.,Jr. (1995). Localization of ERP generators and identification of un-derlying neural proc esses. Electroencephalogr. Clin. Neurophysiol.Suppl. 44, 5575.

Sereno, M.I., Dale, A.M., Reppas, J.B., Kwong, K.K., Belliveau, J.W.,Brady, T.L., Rosen,B.R.,and Tootell,R.B.H. (1995).Borders of multi-

plevisualareasin human revealed by functional magnetic resonanceimaging. Science 268, 889893.

Sherg, M., and VonCramon, D. (1985). Two bilateral sources of thelate AEP as identified by a spatio-temporal dipole model. Elec-

troenceph. Clin. Neurophysiol. 62, 3244.

Shoham,D., Glaser,D.E., Arieli,A., Kenet, T., Wijnbergen, C.,Toledo,Y., Hildesheim, R., and Grinvald, A. (1999). Imaging cortical dynam-ics at high spatial and temporal resolution with novel blue voltage-

sensitive dyes. Neuron 24, 791802.

Smith, M.E., Stapleton, J.M., and Halgren, E. (1986). Human m edialtemporal lobe potentials evoked in memory and language tasks.

Electroencephalogr. Clin. Neurophysiol. 63, 145159.

Snyder, A.Z., Abdullaev, Y.G., Posner, M.I.,and Raichle, M.E. (1995).Scalp electrical potentials reflect regional cerebral blood flow re-

sponses during processing of written words. Proc. Nat. Acad. Sci.USA 92, 16891693.

Sukov, W., and Barth, D.S. (1998). Three-dimensional analysis ofspontaneous and thalamically evoked gamma oscillations in audi-

tory cortex. J. Neurophysiol. 79, 28752884.

Talairach, J., Szikla, G., Tournoux, P., Prossalentis, A., Bordas-Fer-rer, M., Covello, L., Jaco, M., and Mempel, E. (1967). Atlas d Anato-

mie Stereotaxique du Telencephale (Paris: Masson et Cie.).

Tsodyks, M., Kenet, T., Grinvald, A., and Arieli, A. (1999). Linkingspontaneous activity of single cortical neurons and the underlying

functional architecture. Science 286, 19431946.

Ungerleider, L.G., and Mishkin, M. (1982). Two cortical visual sys-tems. In Analysis of Visual Behavior, D.J. Ingle, M.A. Goodale, and

R.J.W. Mansfield, eds. (Cambridge, MA: MIT press), pp. 549586.

van Oosterom, A. (1999). The use of the spatial covariance in com-puting pericardial potentials. IEEE Trans.Biom ed. Eng.46, 778787.

Documents

Dale-Dynamic Statistical Parametric NeurotechniqueMapping-Combining fMRI and MEG for High-Resolution Imaging of Cortical Activity